GLASS. Gunfire Location and Surveillance System

Transcription

1 GLASS Gunfire Location and Surveillance System Denis Alvarado BSCpE, Zayd Babamir BSEE, Christian Kon BSEE, Luis Salazar BSCpE Department of Computer and Electrical Engineering University of Central Florida Orlando, Florida United States of America Abstract With firearms being a hot topic in today s society, law enforcement response time and crime scene investigation techniques have also become prominent areas of research to further improve upon. GLASS focuses on improving law enforcement response time and detailed record keeping of public shooting events. This project achieves these improvements by designing hardware specific to gunshot audio signal processing for triangulating gunshots and utilizing recognition software code to differentiate different gunshot signals. Keywords Gunshot; gunfire; public safety; digital signal processing; analog design; digital design; multilateration; triangulation; security system; microprocessor; recognition; wifi; embedded systems. I. INTRODUCTION With recent high profile mass shootings flooding the news, firearm safety and law enforcement response time has become a serious subject for public safety. Both pro- and anti-gun advocates can agree that any new technology which makes public shootings safer for innocent bystanders is a good technology. Many institutions have panic buttons and/or alarms which can be triggered automatically or manually. Unfortunately, when a shooting occurs outdoors, law enforcement notification relies on emergency calls. Also, unless there is a first-hand witness or clear physical evidence, determining the exact location of a shooter can be difficult. GLASS addresses these issues by providing detailed information of gunfire events real-time as they occur. GLASS detects gunshots, differentiates gunshot types (pistol, rifle, and shotgun) from each other, triangulates the locations of gunshots, and then finally outputs all of this information to a user interface where it can all be easily viewed. II. DESIGN OVERVIEW The GLASS project is comprised of two different designs. The first design is the initial/theoretical design which represents what GLASS would be as a fully functioning, commercial product. The initial design was then simplified to allow the presentation of a proof-of-concept system suitable for the time and budget constraints placed on the designers. Nevertheless, the research, requirements, specifications, and general theory remain true for both designs. Sponsored by: Boeing, Sunstone Circuits, and Maxim Integrated. A. Initial Design The initial design comprises of three modules that all function within GLASS and do not utilize any outside resources. 1) Primary Module GLASS monitors for the specific audio traces that are unique to gunshots. Once a sound is classified as a gunshot, the primary module triangulates the position of the sound source using GPS. After triangulation, GLASS uses sound correlation software to classify the signal as a particular bullet caliber. 2) Secondary Module Via Bluetooth, GLASS transfers the location and time of the gunshot to an Android device where the gunshot location is placed on a coordinate system relative to where GLASS is located. The caliber of the round fired is also attached to the location and time of the gunshot in the Android application for viewing. 3) Self-Sustaining Module Given that GLASS main functions are utilized during emergency situations, it has a self-sustaining module consisting of a uninterrupted power supply battery backup as well as its own solar panel to charge the backup if main power were to be cut. GLASS hardware is also designed to consume very little power, enabling it to sustain itself on battery backup and incoming solar power if necessary. B. Proof-of-Concept Design The current GLASS design demonstrates the team s ability to design and populate hardware, as well as program software in order to complete the necessary task. This simplified design utilizes only two of the three modules in the initial design and distributes the workload differently. 1) Primary Module GLASS monitors for a certain signal peak and calculates the difference in peaks between the microphones. This calculation is then sent via wi-fi to an Arduino Uno development board which is connected to a PC via USB. 2) Secondary Module The information sent from the Primary Module is forwarded to the PC by the development board where the information is processed and displayed. The development

2 board is just a bridge for the information between the Primary Module and the PC. The PC is programmed with the triangulation and gunshot recognition software. This is proofof-concept because pushing the computation to a PC confirms the ability to process the data the way initially intended if resources were adequate. III. REQUIREMENTS The requirements for GLASS are pretty similar between the two designs and the differences are noted. Four microphones for monitoring and audio signal triangulation Data processing unit for audio signal input and alarm/time/location output (initial design) PC for data processing and data review (proof-ofconcept design) Android enabled device to receive communication from GLASS data processing unit via Bluetooth for data review (initial design) Photovoltaic solar panel (initial design) Uninterrupted power supply battery backup A. Hardware Specifications 1) Initial Design TABLE I. IV. SPECIFICATIONS INITIAL GLASS HARDWARE SPECIFICATIONS 2) Proof-of-Concept Design TABLE II. Feature Processor Microphones Battery Backup ISM Transceiver GLASS to PC Data Bridge B. Software Specifications 1) Initial Design TABLE III. Feature Peak Detectection & Conversion to Bit Stream Triangulation & Sound Recognition GLASS Operating System CURRENT GLASS HARDWARE SPECIFICATIONS Hardware Specifications ATmega328P 8-bit AVR RISC-based microcontroller, 32KB ISP flash memory, 1024B EEPROM, 20 MHz Operating Frequency. SPM1437HM4H-B 100 Hz-10 KHz Frequency Range, -22 db ±3 94 db Sensitivity, 61.5 db Signal to Noise Ratio, MHz Sample Frequency CyberPower LCD 340 W, 600 VA, 7Ah Uninterrupted Power Supply nrf24l GHz, 250 kbps-2mbps Data Rate, 4 pin SPI configuration, V Operating Voltages Arduino Uno development kit with ATmega328P. INITIAL GLASS SOFTWARE SPECIFICATIONS Software Specifications Verilog software code within Xilinx compiler. C++ Object oriented code compiled with embedded Linux. Main processing unit runs embedded Linux. Feature Hardware Specifications Bluetooth Output Device Android enabled for information display. Processor Memory Bluetooth GPS FPGA DPSRAM Microphones Battery Backup Solar Panel Analog to Digital Converter Amplifier Android Device MCIMX6D5EYM10AC ARM Cortex A9 Dual Core, 1.0 GHz, 256k x 8 RAM size 1.0 Gb of DDR3 RAM at 1.5 V, 1333 MHz operating speed, 8 bit prefetch buffer. ENW-89841A3KF 2178 kbps Data Rate, -93 dbm Receiver Sensitivity, V Operating Voltage I2C, PCM, UART Interfaces A2200-A GHz, 48 Channels, -148 dbm Sensitivity, I2C or UART Interface. XC3S100E-4VQG100C 100K Gates, 66 I/O lines, 72K block RAM, 100 pin CY7C026A 16Kb x 16 Organization, 20ns Speed, 256 Kb Density, V Operating Voltage. SPM1437HM4H-B 100 Hz-10 KHz Frequency Range, -22 db ±3 94 db Sensitivity, 61.5 db Signal to Noise Ratio, MHz Sample Frequency CyberPower LCD 340 W, 600 VA, 7Ah Uninterrupted Power Supply Instapark 30 W, 17.5 V Max Voltage, 1.68 A Max Current, V Open Circuit Voltage ADC1210S065HN/C1:5, 12 bit resolution, 2 V input Voltage, SPI or Parallel data interface, 13.5 clock cycle latency, 65 Msps sampling rate. TL084CD, 3 MHz Unity Gain, 200 V/mV Amplification Samsung Note 8.0, Quad-core Exynos 1.6 GHz CPU, 2 GB RAM, Wi-Fi and Bluetooth enabled. 2) Proof-of-Concept Design TABLE IV. Feature Peak Detectection & Wi-fi data transfer. Triangulation, Sound Recognition, & Data Display CURRENT GLASS SOFTWARE SPECIFICATIONS Software Specifications C Object oriented code complied with Arduino software. C++ Object oriented code running on a Windows PC. C. Gunshot Condition Specifications GLASS monitors audio input signals for three conditions before declaring the sound as a gunshot: 1) Decibel Level The first condition checked is the sound must have a decibel level of at least 130 db. The general decibel level for a gunshot is between db, but the threshold was lowered to 130 db to account for different environmental and distance from GLASS variables.

3 2) Peak Frequency The second audible trace GLASS checks is the peak frequency caused by the gunpowder explosion within this chamber of the firearm. This data also provides a strong first classifier for what caliber round was fired. 3) Subsonic Frequency The final condition that must be met is proof of the supersonic frequency the round creates as it flies through the air. This frequency is unique to live gunshots and is absent from gunshot audio recordings. A. Multilateration V. RESEARCH AND THEORY Multilateration needs only one array with at least three microphones to determine location of a sound for the two dimensional case and at least four microphones in order to calculate in three dimensions. 1) 2D Multilateration Since the location of each node is known that the time of setup and the position for each microphone is of equal distance from the node s center. The relative location of the source and the exact locations of the microphones can be used to calculate the exact location of the sound source. The distance from a particular microphone to the source of the sound can be calculated with the equation below. Where t represents the time it takes for the sound wave produced by the gunshot to propagate to the particular microphone and C(T) represent the speed of sound. The distance can be represented by the magnitude of the distance vector that is drawn from the sound source to the microphone. In two dimensions where are three microphones located at points A, B, and C, and have a given x-coordinate and a y-coordinate. For the sound source, we denote its position in space as the variables x, and y. = with two equations and two unknowns which is sufficient to determine an answer. 2) 3D Multilateration This method uses hyperbolic multilateration over triangulation which is easier to implement since the array can be arranged in any manner. Naturally four hyperboloids will be required to solve this case so a fourth microphone is necessary. Care must be kept when placing the microphones, as they may not lie on the same plane. If they do, the resulting solution will have multiple solutions. For this reason, we decided to place them at equal distance from the center of the node on right angles from each other The equations for the magnitudes of the distance vectors are the same in three dimensions except that the z component of the vector must also be integrated to the equations. The addition of the fourth microphone also incorporates the third equation listed below. A. Initial Design VI. HARDWARE DESIGN The initial design for GLASS was very ambitious for the time frame and budget. The design included over 25 schematics of printed circuit board design for the project s very specific board. The initial hardware block diagram is demonstrated by Fig. 1 below. These equations require us to know the time it takes for the sound to propagate to each microphone. We can relate the magnitude of any two vectors together by realizing that their magnitudes should be equivalent with the only difference being the difference of arrival between the nodes and if we multiply by the spread of sound, we get the difference in the distance from the sound source to the microphones. The difference in time of arrival to each microphone can be easily be found by determining when the maximum value occurs for each microphone. For this example, we can simplify the mathematics by setting the origin to point A. This leaves us Fig. 1: Initial GLASS Design Block Diagram

4 1) Audio Block The audio sampling section consists of the four microphones equidistant and on separate planes connected to an analog to digital converter (ADC). The ADC then transfers the data to an FPGA which pre-processes the audio data and converts it to a bit stream. While listening for a gunshot, peak detection with thresholding determines when the recording of audio begins. Peak detection is described by the logic equation below. >Threshold Once the threshold condition is found true, a write flag remains true until a 16 bit counter counters the loop back to zero. A counter is used to determine the delay of the peak between each microphone. The FPGA stores a wavelet transformed decomposition of a fixed number of samples. The wavelet transform provides the ability to encode and down sample the signal as well as reducing high frequency noise. The data is then written to dual-port SRAM (DPSRAM) memory. A fix location in memory holds the address to this stored data. 2) Data Processing Block The overall custom printed circuit board (PCB) design was modeled after the WandBoard development board. There are many major components that are implemented on the custom printed circuit board and those components play a huge role in the design on the overall embedded board. The core components of the PCB are: an ARM Cortex A9 1GHz processor, a 1 Gb module of DDR3 DRAM, four modules of 16 x 16 bit DPSRAM, the microusb DC power input, data storage via SD card reader, and the peripheral inputs/outputs including two USB input ports, a Bluetooth input/output device, and the two 46 pin headers. The PCB design was completed within Cadence OrCAD PCB Editor Software. The DDR3 SDRAM is utilized directly by the processor for calculations, operating system resources, and GLASS software resources. The DSPRAM provides stable buffering for signal processing while allowing the processor to read the data while it is still being written. GLASS uses a GPS module to determine the time and relative location of a gunshot through the data from the audio signals. An SD Card is used as removable data storage and storing of the operating system. The schematic overview of a portion of the initial design for the GLASS data processing unit is presented in Fig. 2 at the bottom of the page. 3) Power Block GLASS s power system is a self-sustaining system designed to allow GLASS continual operation even in the event of an emergency power outage. A 1500 VA uninterrupted power supply (UPS) is used to power GLASS and is charged by both the typical 120 V wall plug as well as a 60 W solar panel. With a fully charged UPS, GLASS s low power design can run continuously for at least 14 hours before the battery needs recharged. If in the case that normal power is not restored within 14 hours, the solar panel can generate enough wattage to extend the battery life another four hours. Table V below shows the power consumption of the initial design. TABLE V. GLASS INITITAL DESIGN POWER CONSUMPTION Feature Voltage (V) Current (A) Power (W) Processor DPSRAM DDR Microphones FPGA Other Total B. Proof-of-Concept Design The current hardware design for GLASS condenses the many different complex parts in the initial design into a more manageable design. The most notable simplification is the custom board design being cut from a four layer, 25 schematic design down to a simple two layer, one schematic design. The audio and power blocks were also simplified to better fit the scope of the project timeline and resources. Fig. 2: Initial Data Processing Unit Schematic

5 Fig. 3: Current GLASS PCB Design 1) Audio Block The current audio sampling section still consists of the four microphones equidistant and on separate planes and the same peak detection equation is used to determine signal peaks. The main change to the audio block is that the custom board design has moved from the data processing block to the audio block. The audio samples are read directly by the custom GLASS PCB which now utilizes an ATmega328P processor. The ATtmega328P has built in ADCs simplifying the audio design by eliminating a separate ADC module and FPGA. After determining peak, the GLASS PCB then forwards the data via Wi-fi using the nrf24l01+ Wi-fi module. The current GLASS PCB is shown above in Fig. 3. The decision to simplify the hardware design was necessary to the completion of the project on time. Unfortunately, the simplification made GLASS lose a significant amount of processing power. Due to the loss of the more complex and powerful Cortex A9 ARM processor, the current GLASS board simply functions to capture sound, pre-process the signals, and forward the data to an outside computer device. Nevertheless, the simplification still demonstrates the ability to design and produce a functioning system. 2) Data Processing Block Once the GLASS PCB forwards the audio data over Wi-fi, the data is received by an Arduino Uno. This Arduino device acts a bridge between a computer running GLASS software and the Wi-fi module receiving the data from the GLASS PCB. For the presentation of GLASS, a laptop with Windows 7 will be running the GLASS software that determines the gunshot sounds and locations. 3) Power Block For ease of presentation and system setup, the solar panel has been removed from the power block and GLASS will run entirely off of the UPS battery at full charge. The previous time of use calculations are actually improved with the current design for the new GLASS PCB does much less processing and therefore consumes less power. Considering that the power consumption of the new system is 15% of the original design, the UPS was downgraded to 340 W still enabling GLASS to run longer than it did with the larger battery in the original design as demonstrated in Table VI below. TABLE VI. CURRENT GLASS DESIGN POWER CONSUMPTION Feature Voltage (V) Current (ma) Power (W) ATmega Arduino Uno Microphones Wi-fi Modules Total A. Initial Software Design VII. SOFTWARE DESIGN The software for the initial GLASS design was complex with programs written in an array of coding languages including C++, Verilog, Java, and embedded Linux C running over multiple devices. The initial UML design is displayed in Fig. 4 at the top of the following page. 1) Verilog in Xilinx The first program to be initialized is the sound capture and serialization of data program written in Verilog on the Spartan 3A FPGA. The software has the microphones constantly listening for audio signals. When an audio signal is received from the ADCs, the FPGA serializes the data into a bit stream and then forwards the data to the DPSRAM. 2) Embedded Linux Linux was chosen as the operating system for GLASS due to its portability, and large development and support base. Access to the source code for the operating system allows Linux to be modified to our custom hardware, and adapt existing drivers to meet GLASS needs. Specifically, Linux is already configured for a development board like the

6 Fig. 4: UML of Initial GLASS Software Design WandBoard. Ubuntu in particular, has a very large support base. Upon system start, GLASS boots from processor ROM. It is then loads the operating system and begins operation. Upon receiving the interrupt, Linux will drop into Kernel mode and execute a function to place the relevant data in memory to be processed later. Then the operating system will switch back to user mode and fork two threads; one for the gunshot recognition and another for the location detection. 3) C++ GLASS Software Recognition and Location The GLASS signal processing software is mainly written in C++ object oriented code. It performs both the recognition and location algorithms and is the largest portion of software within the project. a. Recognition The first concern in the gunshot recognition algorithm is to normalize the input signal. With varying distances and angles from the microphone array, the likelihood that the sample levels match any recorded data is unlikely. Using the wavelet Transform on the data gives GLASS the ability to find the low frequency components associated with gunfire. Also the wavelet transform shows the amount of time each frequency component occupies on the signal which helps to determine when a gunshot is between two different weapon types which may be considered closer. Special care must be taken to remove the echoes which have a larger effect on the signal. As the distance from the microphone array increases the decibel level of the echoes grows relative to the magnitude obtained from the original signal. This is due to the non-linear growth of the distance traveled by the sound C A and echoes, C B A as shown in Fig. 5. Also the distance in arrival between the two signals decreases because the length of side c becomes more and more trivial, making the normalized echo amplitude closer to the amplitude of the direct signal. Recognizing the weapon type is accomplished by finding the correlation between the signal and a stored sample. The sample which results in the highest correlation should then be assumed to be the weapon type. This tells us how alike the two signals are. Since the influence of echoes becomes more and more problematic at the max decibel level decides which pre-recorded to do the correlation against. By sorting the recordings be amplitude, GLASS reduces the number of comparisons to make as a revolver at 1 meter may have the same amplitude as a rifle at much farther, however the correlation between the two will result in a lower correlation value. Although, using just this correlation is insufficient since non weapon based sounds would incur a correlation value. For this reason the spectral components of the signal must be examined to determine that the signal did indeed come from some kind of arms fire. By applying the wavelet transform on the signal the spectral components can be matched to the requirements that characterize a gunshot. These include a supersonic signature, an initial blast from the powder, and a shock wave as the bullet travels through the air. Fig. 5: Sound as it travels from source to microphone along vector b and as the echo travels along the paths of vector a then c b. Location Determination of the source of the signal is fairly straight forward, though it requires some parameters to be taken into consideration. GLASS determines the source of a sound event by relating the change in time between the sound s arrival between two microphones in the array and the effects of temperature on the speed at which the sound can propagate through the air. For that purpose there is a digital thermometer feeding the temperature to the system. In order for the software to determine the position of the gunfire the time difference between two nodes must be determined. This is done by first finding the point in the signal

7 where the signal reaches its maximum for each microphone the time delay can then be calculated by the number of samples difference the maximum occurs from each other. Each microphone is stationary at equal distance from the point of origin, and vectors from the microphones to the origin are orthogonal. Distance vectors must then be related together in that the magnitude of one distance vector is equal to the magnitude of the other and the algebraic sum of the speed of sound multiplied by the difference in time between where the maximum value occurs in each signal. Distance vectors may then be drawn from the sound source to each microphone in the array as shown in Fig. 6. Fig. 6: Sound vectors from source as viewed by the GLASS location software. With the equations in the multilateration research section, we may construct the paraboloids necessary to solve for a given sound source. Then the point is calculated and stored to be sent to the user interface by a separate thread. 4) Java Android Application The initial GLASS design utilized an Android enabled device to receive and display the gunshot location and identification information. The device would run a GLASS application written in Java programming language and the user interface would have a map displaying all gunshot locations and an information bar with detailed data on each gunshot. The application would receive this data via Bluetooth in which the GLASS main board would send. B. Proof-of-Concept Software Design Considering that the hardware specifications have been simplified, the current design is only using two types of coding languages: C and C++. 1) C Programming on GLASS PCB and Arduino Uno With the GLASS PCB replacing the FPGA and ADCs in the audio block, it functions as the audio listener and data transfer system. The GLASS board uses a modified C programming compiled by Arduino software. This code listens for audio signatures, captures them, and then transfers the data over Wi-fi to a secondary Arduino connected to the PC. The secondary Arduino functions as a bridge between the Wi-fi. The code utilizes the same peak detection algorithm as previously mentioned in the initial design and the Wi-fi link is a simple data transfer protocol. 2) C++ GLASS Software on Windows 7 Computer In the current design, the main processing was moved from the GLASS board to a main computer to handle the triangulation and gunshot recognition software. Also, the Android app/device was removed and the computer is used for data reviewing. The GLASS software is coded in an object oriented C++ language and compiled in Microsoft Visual Studio. The first thing the software does is create a data link between the computer and the Arduino Uno via USB. The computer pulls the audio data being sent from the GLASS board to the Arduino and brings it into the GLASS software for processing. The software itself processes the data the exact same way that the initial design section describes, using the same algorithms, equations, and theory. The only difference is between the initial and current designs on this process is where the processing is occurring. After the location and gunshot are determined, the software then reports the data on the computer screen for viewing. A timestamp from the computer is associated with each gunshot and the location is output in relative terms to the location of the GLASS microphone setup. VIII. CONCLUSION GLASS is a system designed to locate, recognize, and alert gunshots. The project has undergone a massive redesign for testing and presentation purposes due to a restriction of resources. The initial design stands as a consumer end product and the current design demonstrates a proof-ofconcept for the original design. The initial design composed of two systems working together to process information to be published to a user interface on a separate device. Audio is sampled through a microphone array then the analog signal is converted into digital before being buffered by four modules of DPSRAM via FPGA. The processing occurs on a custom PCB running a dual core ARM Cortex A9. The processed data is then sent to an Android enabled device for viewing via Bluetooth. The current design utilizes a custom PCB with an ATmega328 which listens and forwards audio data via Wi-fi to a computer which will process the data in the same fashion the original GLASS board would. The output data is viewable on the computer in which does the processing in the current design as well rather than an Android device. In the processing phase of both designs, when a sample of sufficient amplitude is received it processes the information as necessary to run the location and correlation algorithms. Then the correlation and location algorithms are run to determine the weapon type and the location where the gunshot came from.

8 ACKNOWLEDGMENT At the beginning of this project, it was clear that the financial burden would not be light due to the large amount of components required and the complex technology needed to implement the GLASS design. Therefore, we would first like to thank our main financial sponsor, The Boeing Company for funding GLASS. Boeing has allotted financial support for projects in the categories of homeland and cyber security. The GLASS team would like to extend our gratitude to Sunstone Circuits for providing our hardware designers with priceless advice on printed circuit board design as well as providing us with a discount on all of our printed circuit boards through their sponsorship program. Christian Kon would personally like to thank Erik Torell from Sunstone Circuits for aiding with a double check of the GLASS board schematic. Third, we thank Maxim Integrated for sending us samples of components essential to our initial design as well as providing us with countless resources on audio signal processing related to their products and great customer service. Next, we appreciate all of the professors that guided and mentored us to success in this project. And we are very gracious to NRA Licensed Instructor and Range Safety Officer John Caballero for not only providing us with sponsored range time at the Central Florida Rifle and Pistol Club, but for also challenging our designers with great mentoring on sound and waveform analysis relating to firearm sound analysis and identification. Lastly, we extend our thanks and love to our families and friends for dealing with many sleepless nights, bad moods, and events missed necessary for GLASS to succeed. REFERENCES [1] Armada International. 2013, Aug. 02, Gunfire Location system: Acoustic Gunshot Detection Systems.[Online]. Available: [2] AudioLinks How Microphones Work. [Online]. Available: [3] BBN Technologies. Boomerang. [Online]. Available: [4] Cadavid, S., Donzier, A. 2005, Jun. Small arm fire acoustic detection and localization systems: gunfire detection system. Proc. SPIE Vol. 5778, p [5] Graves, Jordan R Audio Gunshot Detection and Localization Systems: History, Basic Design, and Future Possibilies. [Online]. Available: 1/apache_media/L2V4bGlicmlzL2R0bC9kM18xL2FwY WNoZV9tZWRpYS8xNzUzODI=.pdf [6] TSCM. Transforms/Wavelets. [Online]. Available: [7] Wired. 2010, Mar. 24, Military Helicopters May Get Gunshot Location System.[Online]. Available: BIOGRAPHIES Denis Alvarado is a graduating from the University of Central Florida with a Bachelor s of Science degree in Computer Engineering. His interests include artificial intelligence, simulations, and game design. He plans to pursue a career in software design. Zayd Babamir is a graduating from the University of Central Florida with a Bachelor s of Science degree in Electrical Engineering. His interests include power engineering, hiking, camping, and traveling. He hopes for a career in power engineering. Christian Kon is a graduating from the University of Central Florida with a Bachelor s of Science degree in Electrical Engineering. His many interests include microelectronic hardware and cyber security. He plans to pursue MSEE and MBA and a career in Critical Infrastructure Protection. Luis Salazar is a graduating from the University of Central Florida with a Bachelor s of Science degree in Computer Engineering. His interests include computer hardware design and embedded programming. He hopes to pursue a career in either hardware or software design.