First responder positioning systems

First responder positioning systems Overview of user needs and technologies Jouni Rantakokko

Outline Introduction - First responder localization in urban operations First responder needs and requirements - Overview of user needs and preliminary requirements Enabling technologies -GPS - Supporting sensors in GPS-challenged environments Summary

Outline Introduction - First responder localization in urban operations First responder needs and requirements - Overview of user needs and requirements discussion Enabling technologies -GPS - Supporting sensors in GPS-challenged environments Summary

Positioning in urban environments Positioning of vehicles is feasible - GPS-receivers, inertial sensors, vehicles own sensors (e.g. velocity, wheel turn), map matching, Positioning of civilians is feasible - Possibilities Sensors: GPS, inertial sensors, magnetometers, barometer, camera, etc Pre-installed systems: WLAN, RFID, UWB, acoustic, etc - Combination of sensors and map (building floor plans), pre-installed systems and/or user feedback solves the problem although perhaps (not yet) at the desired cost?

Positioning in urban environments First responder positioning in urban environments - High accuracy, availability and reliability requirements - General limitations No pre-installation possible Building floor-plans are not (yet) available Stringent size, weight, power and cost requirements Level of automation (no user interaction) Source: WPI

FR positioning what is available? GPS-receivers - Civil Standard Positioning Service (SPS) C/A-signal - Position accuracy between 1-5 m during favorable conditions - Indoor position error may be up to 50 m, or more Difficulties - Radio signal propagation in urban environments - Lack of reliable performance metrics in urban and indoor environments - Sensitivity towards jamming

FR positioning what is available? Back-mounted PDR-type systems (+ GPS) - Three-axis accelerometers, gyros, magnetometers and a barometer - Data sheets state the following Weight <150 grams, power consumption 0.5-1.5 W Position error horizontal: 1-2% of distance - Position error vertical : < 1.5m - Looks OK on paper the truth, the whole truth, and nothing but the truth? DRM4000 NaviSeer S-NAV

FR positioning what is available? Principle of operation - Detect a step, estimate step length, estimate step direction Why do they not behave as promised? - Step detection thresholds varies, e.g. Sneaking versus running Concrete versus sand (or mud) - Step length varies Outdoors vs. indoors, walking up or down a slope/stairs, fatigue, situation dependent - Step direction Systems assume first responder moves forward (in direction of IMU orientation), but realistic movement patterns include sidestepping, walking backwards, crawling Magnetic field disturbances in urban and indoor environments - Height estimation with barometer alone Sensitivity towards weather changes, muzzle blasts, explosions, fires (heat causes air turbulence), building pressurization

FR positioning desired capability? Positioning of first responders in all environments and for all realistic movements - Give the first responder the same functionality regardless of scenario in urban and indoor environments as well as during GPS-jamming

User needs Current advances in localization and tracking technology have the potential to develop into muchneeded tools for the saving of lives in emergency response and rescue missions, and for the safekeeping of lives in military operations The requirements of firefighters, law enforcement officers and soldiers are very stringent - positioning accuracy - robustness - Reliability/availability - weight and size of the system J. Rantakokko, P. Händel, M. Fredholm, F. Eklöf,, User Requirements for Localization and Tracking Technology - A Survey of Mission-specific Needs and Constraints, Proc. of IPIN, Zurich, Switzerland, September 2010.

Source: Räddningsverket

User needs The primary use of a positioning system - Command and control (C2) system for the mission, normally led locally in close proximity to the operation. This enables e.g. efficient command and control of the operation personnel accountability fast extraction of wounded or lost firefighters from inside buildings The positions will first be needed by the incident commanders (IC) of the operation, then, as a secondary requirement, by the actual units participating in the operation

User needs Efficient local command and control M, P, F Rescue of injured personnel M, P, F Navigation through complex buildings M, P, F Safe exit (e.g. from collapsing building) F Friendly-fire / Blue-force-tracking M, (P) Distance and heading to targets/threats M, P Health status and automatic alarm functionality M, P, F Know what rooms have been cleared (searched) M, P, (F) After-action review (de-briefing) and training analysis M, (P), F Fugitive movement pattern analysis (positions of dogs) P Free the radio resource for command and control M, P, F

User needs What needs to be estimated? - Position (x,y) - Height - Position error (and integrity monitoring) - Heading for weapon and/or body - Distance and direction to targets and threats Who needs the estimated positions? - Local command - Other units in group

User needs Additional user needs - personal alarm functionality - self-navigation aid Military and law enforcement personnel also need a means of reducing friendly-fire casualties and a robust and accurate target designation - need for robust, accurate (weapon) heading estimation

Requirements discussion What is desired of a first responder positioning system? - Seamless outdoor-indoor coverage, with consistent performance in all relevant scenarios - No dependence on pre-installed infrastructure - Robust and reliable - Light-weight, small, power efficient, low-cost (SWaP-C) - Reliable estimation of room and floor (or equivalent) - Error estimates (integrity monitoring) automatic estimation of localization errors (uncertainty) detection and warning in case of electronic attack or unintentional electromagnetic interference

Requirements discussion Requirement estimates ~ 1-3 m accuracy ~ near 100% availability (in all environments) ~ << 1 kg, incl. processing unit and visualization ~ 8 24 h battery (up to 72 h for some applications) > 30 min s in GPS-denied/challenged environments (e.g. indoors) but they also depend heavily on user group, type of operation, and usage (e.g. avoid friendly fire or efficient C2),

Requirements discussion In order to achieve a high market penetration the price of the complete positioning system should be below 1000 - cost of each sub-sensor must by necessity be kept low

Requirements discussion Constant and secure accessibility for those who need the positioning data (IC and team members) - Encrypted voice communications and data transfer Physical robustness so that the system will operate reliably even under harsh conditions, including extreme temperatures and humidity Antenna and cables should be incorporated into the individual s uniform

Requirements discussion Real-time map-building capability in the form of simultaneous localization and mapping (SLAM) approaches in unknown buildings as the team moves through it - SLAM should be automatic, without the need for the team members to aim cameras or other sensors in various directions

Requirements discussion Positioning data combined with e.g. personal health status sensor data (physiological monitoring)

Requirements discussion Presentation of positioning data to be intuitive and easy to understand, in particular for the personnel actually carrying out the operation - In armed operations, visualization system should present heading to own troops and in particular the heading of the weapon - Data for distance and direction to targets and threats should also be presented

The role of communications A robust communications system for both voice and digital data flow forms the backbone of any localization system - Most discussed user needs involves the distribution of positions Voice communications should be reserved for leading the operation, while positioning details can be transmitted as digital data C2-component of the operations team would often find its work facilitated by the ability to rapidly download critical information on the target location through its wireless communications network - for instance construction plans, data from heat and smoke detectors, and CCTV images in case of a firefighting operation

Diverse operational environments The operational environments encountered will be very diverse - Firefighter example fire-suppression in a wooden two-story family home commercial concrete multi-story buildings with tens of thousands of square-metres Users will have different stringent requirements, as well as different trade-offs (e.g. cost versus accuracy) - Compare e.g. police officers assigned to traffic duty when compared to the needs of special weapons and tactics (SWAT) operators - Same reasoning applies for soldiers performing patrol duty in rural areas with only small buildings when compared to building-clearing operations (Sarajevo sniper-alley )

Diverse operational environments Modular system is desired - user may face different challenges on different missions and occasionally have no real need for positioning data - soldiers and law enforcement officers tends to prefer a modular system, while firefighters instead expect always to carry the whole/same system

Scenarios Range of op s in which localization and tracking capability would provide an edge is wide User type Military Law enforcement Firefighting Civilian users Operations Urban combat Building clearing Ship boarding Safe navigation Hostage rescue Ship boarding Crowd control Residential and apartment building fires Complex building fires Ship fires Forest fires Subterranean Rescue Operations Detention facilities Private security guards Protection of sensitive facilities and transportation of hazardous materials Protection of civil servants Emergency response operations during humanitarian missions in crisis areas Localization of the elderly and children

Situational Awareness (SA) The notion of situational awareness systems for soldiers is a broad area, but some of the important questions it could answer for the soldier are - Where am I? - Where are my comrades? - Where is the enemy? - What is my current task? - Where am I supposed to be? - How do I get there, quickly and safely? - Is it safe to shoot in this direction, or blow out this wall/door? - Are my comrades OK?

Military operations It is noteworthy that these functions can to a large extent be performed through voice communication - Robust voice communications is the number one priority - It can be difficult and require much attention for team leaders to keep track of the positions and update team members on the situation constantly

Military Urban operations Urban warfare operations, in particular in a counterinsurgency context, pose particular challenges - Difficult to distinguish between civilians and enemy fighters, since the latter hide among the former and use civilians as cover or even as living shields - Distances tend to be short - tactical situation can and will change very rapidly - Difficult to achieve a good situational awareness (SA) without technological aids - Buildings will need to be secured and searched for enemy fighters, arms caches, or contraband - Enemy snipers may hamper operations - The layout of buildings is usually unknown before the soldiers enter them

Military Urban operations Key use of a localization and tracking system will be to enable the commander in charge of the operation to lead efficiently Integrity monitoring is of crucial importance in military operations, - especially if incorporating civilian GNSS-receivers into the system - reliable estimate of localization error perhaps more important than a high localization accuracy Size, weight automation and power (SWaP) requirements are extremely important Due to the large variety of operational scenarios, a modular system is desired Weapon heading estimation is desired

Firefighting operations An automatic accountability system that would give the incident commander information about firefighters that are lost or starts to behave erratically (e.g. move away from the other firefighters and/or the fire hose) is highly desired - The situation where a firefighter is lost at the fire ground causes multiple casualties each year in the US alone - Rapid insertion teams (RIT) are then sent in to find and extract the firefighter and the ability to quickly and decisively guide them to the firefighter in need would save lives

Firefighting operations In many countries the tactics today is that the incident commander gives out orders and directs the firefighters over radio - Information about positions of all firefighters should be relayed to the incident commander Our view is that also individual firefighters should receive SA information - radio coverage problems occur - distance measurements between RIT-team and missing firefighter should increase speed of rescue Delicate task to decide upon how the information should be visualized to a firefighter during the extremely stressful and physically exhausting situations that commonly occur

Firefighting operations Physiological monitoring, with an automatic alarm function, and also real-time map-making (SLAM) capability will be of great importance for firefighters - Searches of rooms performed by a RIT can be very slow due to the limited visibility floor-plans would increase speed of search as well as accuracy of position estimates - The use of thermal imaging cameras significantly helps speed up a search

Firefighting operations Data need to be made available to the incident commander as well as to any reinforcements or medical evacuation teams entering the premises During larger fires reinforcements from other departments occur and the person designated as being the incident commander may change several times during the incident - An automatic situation awareness system is very valuable Incident commander usually sits in a vehicle and leads the operation - More information can then potentially be visualized to the fire fighting incident commander, and larger visualization interfaces can be used - The question remains what situational awareness information should be automatically conveyed to the firefighters

Additional civilian applications Correction officers and other staff at detention facilities often experience threats and violence from inmates - A personal attack alarm function which upon activation would transmit position and an alarm message to the command center would be an extra safeguard, as would the transmission of health status data Pre-installed positioning system in combination with building floor plans could be used, for instance during prison riot - combined with mobile/autonomous back-up system to ensure operations under severe conditions

Additional civilian applications Private security guards often work alone, and a localization and tracking system could serve as a means to increase their physical security - The key benefit of having an localization and tracking system for security guards would be constant access to the physical location of the security guard and possibly his health status (personal alarm / accountability system)

Additional civilian applications Protection of sensitive facilities such as nuclear power plants, oil refineries, and chemical industries - Enhance the ability to defend and protect against attacks - Facilitate the evacuation of its staff in case of serious emergencies - Enable the personnel and emergency responders to keep track of hazardous material on the premises In installations of this kind - pre-installed positioning systems can be employed to great effect - building layouts should be available and they provide an efficient means of improving indoor localization accuracy when using dead-reckoning techniques - in case of emergency, it would be advantageous if the data can also be linked straight to the emergency response team

Different users different systems Safety-of-Life critical systems - Special forces, local/state/federal SWAT-teams, firefighters - Accuracy and availability before cost Increased safety - Soldiers, police, correction officers, security guards - Availability, accuracy and cost important Demanding consumers/applications (and first adopters ) - Alarm functionality (hospitals - social workers - immigration), interactive services, gaming, surveillance of visitors in companies, - Availability and cost important, errors accepted Regular consumers (mass market) - Positioning of emergency calls, games, interactive services, - Cost most important (e.g. when integrating pos/nav in all mobile phones)

Concluding remarks Military personnel in peace-keeping and peaceenforcement operations, law enforcement officers, and firefighters face very similar needs, despite their differences in operational scenarios - It makes sense to develop localization and tracking technologies that will serve not only one but several of these end-user groups

Concluding remarks Training facility needed for development of requirements and evaluation of existing and emerging localization and tracking systems - Perform tests in cooperation with end-users - Pre-installed high-accuracy system against which new stand-alone technologies can be evaluated - Research on human-machine-interface possible the question of what information should be conveyed to the individual soldier or firefighter, and how it should be presented, is a non-trivial task that requires significant attention

Robust multisensor positioning GPS GPS Accelerometers Accelerometers Gyro Gyro Magnetometers Magnetometers Barometric Barometric altimeter altimeter Individual sensors - GPS - IMU (foot- or back-mounted) - Magnetometer - Baro-altimeter - Other possibilities include Doppler-radar, cameras, laser, ultrasound,

GNSS OK, lots of improvements going on but what will this actually give me? - An advanced integrated GPS/Galileo receiver will have access to 50+ satellites, with multiple signals at different frequencies from each satellite -This will improve accuracy in urban (no more urban canyon problems, at least for vehicles) and forest areas improve availability indoors, and reduce position errors improve robustness towards interference/jamming - This will not give meter-level accuracies in deep indoor scenarios give desired robustness

PDR-type systems Improved Pedestrian Dead-reckoning systems expected this year - Improved motion classification - Improved sensors - But it is unlikely they will be sufficiently robust towards realistic movements

Foot-mounted INS

Foot-mounted INS Sensors -MEMS-based Small, lightweight Power efficient Low cost Principle of operation - Inertial navigation - Foot-mounting allows for regular zero-velocity updates Challenges - Initialization of position and orientation - Reliable estimation of when foot is at stand-still during all realistic types of motion - Crawling

Foot-mounted INS

Magnetometer A magnetometer measures the magnetic field intensity - Small, inexpensive three-axis magnetometers available Large-scale earth magnetic field is known at different positions - Data from three-axis magnetometer can yield estimates of attitude, acts as electrical compass Complement to gyros Challenges - Metallic objects, electrical equipment can yield large local fluctuations in magnetic field intensity - Fast variations can be expected in many indoor environments, but mean value during movement is close to earths magnetic field Must remove data corrupted from these (mostly) shortterm noise sources, and extract the measurements of the global magnetic field

Magnetic field disturbances Heading error Perturbation Afzal, Renaudin, LaChapelle, Multi-Magnetometer Based Perturbation Mitigation for Indoor Orientation Estimation, NAVIGATION: Journal of The Institute of Navigation, Vol. 58, No. 4, Winter 2011.

Perturbance detection Two closely located three-axis magnetometers - z-axis aligned - x-y axis 30 degrees offset Principle - Norm of magnetic fields are the same - Earth magnetic field hardly affected by short movement - Local perturbations can be detected Gyros can be used to compensate for user motion Afzal, Renaudin, LaChapelle, Multi-Magnetometer Based Perturbation Mitigation for Indoor Orientation Estimation, NAVIGATION: Journal of The Institute of Navigation, Vol. 58, No. 4, Winter 2011.

Magnetometer Approaches - Remove data corrupted by local magnetic disturbances - If disturbance is static, calculate change in direction from magnetometer - Create your own local magnetic disturbance? - Multi-magnetometer Perturbation detection Perturbation subtraction

Barometric altimeter Measures air pressure - Small, inexpensive barometers available Air pressure varies with altitude - Barometers can be used to estimate altitude variations Complements INS and GPS for altitude estimation Challenges - Air pressure varies with weather, wind conditions, and temperature - In indoor environments air conditioning, open windows, fires, and deliberate pressurization of buildings to remove smoke causes pressure variations - Challenge is to detect these false readings and exclude them Useful for measurements of short-term variations of air pressure, long-term errors expected if not aided by other sensors

Barometric altimeter Example: shooting indoors in small room with sensor placed on floor - Pressure difference 0.5-2 mbar (50-200 Pa) - Corresponds to height reduction of 18 m

RF-ranging/positioning: BW-limited systems Portable infrastructure systems (e.g. radios on fire trucks) - Accurate in some scenarios Deployment possibilities can be restricted High-rise buildings problematic Firefighter-to-firefighter ranging - Ranging accuracy must be improved for heavy multipath scenarios SAR multipath suppression? Pre-deployed radio-ranging infrastructure - If we can assume availability of accurate maps, then RFID-type equipment may also be feasible inexpensive, small, short-range transmitters/receivers could be deployed during building construction - Will not be present in all buildings, may hinder user acceptance - Unit cost is low, but costly deployment or calibration

Example portable infrastructure ToA/TDoA type systems - Accurate time-synchronization required 1 ns equates to 0.3 m Source: WPI Precision Personnel Locator

RF-ranging: BW-limited systems Size and weight considerations may prohibit use of separate radio system for ranging - Design combined data transfer and ranging radio system (separate waveforms in SDR)

RF-ranging: IR-UWB Example measurements with TimeDomain P220 - Very accurate range estimation (~dm) - Short range (expect few tens of meters indoors) - Accurate ranging allows for cooperation Description Range (depends on parameter settings) Outdoors, limited multipath > 70m LOS Indoors, heavy multipath >50m Plaster wall with wooden framing 12-15m (<5 walls) Reinforced concrete wall < 10m (<3 walls) Non- LOS Modern family dwelling, 2-story > 15m (up to 3 walls/floors) Modern 3-glass window (metal coated energy-saving) ~10m

RF-ranging multipath suppression Use of spatial domain - Multipath components and direct path often has different angle of arrivals - Restrictions for this application Range obtained by use of low frequencies (~400 MHz?) Limited spectrum (~5 MHz?) Size/weight Single antennas on First responder

RF-ranging multipath suppression Synthetic aperture multipath mitigation -Scenario Ranging between moving firefighter and stationary vehicle - Channel Angular difference between direct component and multipath usually larger at firefighter side - Perform angular discrimination at firefighter side Size constraints prohibits multiple antennas - Possible solution Synthetic aperture During shorter times the firefighter movement can be estimated fairly accurately Choose transmitted signal wisely Split received signal into time-chunks which then has (slightly) different positions due to firefighter movement Beamforming approaches enabled

Doppler radar Velocity measurements - Potentially powerful aid to INS Details - 24Ghz, 2-axis - 92 GHz in development - Insensitive to motion type Challenges - Power consumption? - Insensitivity to motion type? Source: GLANSER A scalable emergency responder system, WPI Workshop, 2011.

Ultrasonic sensors An ultrasonic sensor can measure distance to objects, or between sensors - Small, inexpensive, directive ultra-sound sensors available Short-range distance measurement equipment - By combining ultrasonic ranging sensors and IMU s on each foot improved performance is expected - Distance to walls can aid cameras in SLAM Challenges - Noise robustness in firefighter scenarios? -Range

Maps If current, accurate, trustworthy building layout info is available the position accuracy can be dramatically improved - Particle filters seems promising for fusion of IMU and map Complements INS efficiently Challenges - Future availability unknown, industry has this far failed to harmonize / standardize formats may require legislative actions specifying building info to be submitted to authorities - High map accuracy required - Updated when changes occur - Interior building destruction may occur during catastrophic events Movement models could further improve performance - if we knew how the firefighter does not move

Imaging sensors A camera can estimate bearing to different landmarks in the environment - Small, inexpensive cameras available Localization aid, and map-building capability Complements INS for attitude estimation Challenges - Computational complexity, environment, dynamics, range Good alternative when maps unavailable

Simultaneous localization and mapping - SLAM SLAM principle - build a map and localize yourself in an unknown environment If we knew - our trajectory, we could easily create a map - the map, we could easily determine our location - Both map and trajectory are, in many scenarios, unknown!

SLAM Blue: Foot-mounted Red: Stereo-camera SLAM principle - build a map and localize yourself in an unknown environment If we knew - our trajectory, we could easily create a map - the map, we could easily determine our location - Both map and trajectory are, in many scenarios, unknown!

Camera-aided inertial navigation - difficulties Low visibility scenarios - Due to darkness, smoke, fog, Is it possible to use e.g. thermal IR cameras (or night-vision sensors)? - Fewer distinct landmarks available? - Heat sources (and reflections) not stationary? Computational complexity - Need efficient landmark selection algorithms Dynamic environments

Remaining challenges The key challenge is to have a good instant knowledge of how reliable the sensor data is - Robust sensor fusion through sub-system integrity Automatic initialization of position and orientation - In situations without accurate GPS-receivers and with magnetic disturbances, for instance when leaving vehicle in urban environment Imaging sensors UWB Cooperative navigation for increased accuracy and availability - What information is efficient to exchange? WBAN - Robust, secure transfer of data to processing unit and radio What information should be presented to the First responder and Incident commander, and how should it be presented? - How will the FR and IC use the information during tough operations, such as large scale structural fires

Technology roadmap a quick glance Existing systems unsatisfactory in urban operations -GPS -PDR-type system First responder positioning in urban environments soon feasible - Expect less than 1-2 meter error increase per minute for realistic movements with foot-mounted INS Technology maturity time-line (draft) GPS & PDR Foot-mounted INS INS & imaging sensors 2012 2013 2014 2015 2016 PDR++ PDR & Doppler

Summary First responder localization - Reliable and accurate positioning systems are highly needed they will save the lives of fire fighters and other first responders

Summary Multi-sensor system is required, composed of e.g. - GNSS-receivers GPS + Galileo + GLONASS(?) Different services and signals from all SatNav systems are available - Inertial sensors, magnetometers and barometer Foot-mounted, back-mounted and/or co-located with imaging sensors - Imaging sensors - Doppler radar - Ranging devices (RF, acoustic, ) - Cooperative navigation HW - focus on inexpensive and lightweight sensors, e.g. MEMS-based SW of paramount importance - Multi-sensor fusion algorithms - Integrity monitoring of positioning sub-systems

Summary - Positioning in GNSSchallenged environments Position error based on individual sensor set will inevitably increase with time/movement Ranging between first responders and cooperation reduces error drift but we still need regular recalibration of positions Possibilities Radio-based ranging to anchor nodes Building floor-plans Satellite imagery and ranging to recognizable objects, image data base matching, Cooperative navigation between first responders slow down rate of error increase

- Impulse radio (UWB) for ranging and exchange of information? - What sensor data should be exchanged to enhance efficiency of cooperation? Cooperative navigation for high accuracy during long-term operations J. Rantakokko et al, Accurate and reliable soldier and first responder indoor positioning: Multi-sensor systems and cooperative localization, IEEE Wireless Communications Magazine, April 2011 First responder equipped with individual sensors for position estimation Accelerometers Accelerometers Gyro Gyro Magnetometers Magnetometers Barometric Barometricaltimeter altimeter GPS GPS IR-Camera IR-Camera IMU IMU Possible future concept for robust first responder positioning systems in urban operations