DARPA MULTI-CELL & DISMOUNTED COMMAND AND CONTROL PROGRAM

Size: px

Start display at page:

Download "DARPA MULTI-CELL & DISMOUNTED COMMAND AND CONTROL PROGRAM"

Lorena Jennings
5 years ago
Views:

1 DARPA MULTI-CELL & DISMOUNTED COMMAND AND CONTROL PROGRAM ANALYSIS TOOLS EXECUTIVE SUMMARY HIGHER HEADQUARTERS/JOINT COMMAND AND CONTROL EXPERIMENT (EXPERIMENT 7) Program Executive Office for Simulation Training and Instrumentation Program Manager Future Force (Simulation) Research Parkway Orlando, FL Distribution authorized to US Department of Defense and US DoD contractors only. Critical Technology, October Other requests for this document shall be referred to the Defense Advanced Research Projects Agency, Information Exploitation Office, 3701 N. Fairfax Drive, Arlington, VA For Official Use Only. Exempt from Mandatory Disclosure under the FOIA. Exemption 3 Applies. Warning. This report contains information subject to the international Traffic in Arms Regulations (ITAR) or the Export Administration Regulations (EAR) of 1979 which may not be exported, released, or discharged to foreign nationals inside or outside the United States without first obtaining an export license. A violation of the ITAR or EAR may be subject to a penalty of up to (10) years imprisonment and a fine of $1,000,000 under 22 U.S. C or Section 2410 of the Export Administration Regulations (EAR) of 1979.

2 This page left intentionally blank.

3 Table of Contents Table of Contents 1. Introduction... ii 2. DARTS DARTS Quads Key Event Indicators Quad Observer Tools Rapid Reduction Tool for Situational Awareness Sensor Coverage Data Products Suite Battle Tempo Detections Query Fires Query Battle Animation Future Development DARPA M&DC2 Program Experiment 7 Page i

4 Table of Contents Table of Figures Figure 1. Sample DARTS Key Indicators Quad... 3 Figure 2. Observer Collection Tool Input Form... 4 Figure 3. Rapid Reduction for SA... 6 Figure 4. The SAT Score Derivation... 7 Figure 5. Sample SAT Curves... 7 Figure 6. SA and Sensor Coverage Used Together... 8 Figure 7. Sensor Coverage Tool... 9 Figure 8. Battle Tempo Sample...12 Figure 9. Shooter Not Initiator Sample Worksheet...13 Figure 10. Sample Battle Animation Slide...14 DARPA M&DC2 Program Experiment 7 Page ii

5 Introduction 1. Introduction The Data Analysis (DA) team provides an extensive suite of data products to the Core Analytic team on the Multi-Cell and Dismount Command and Control program. This suite of products supports the Core team whose responsibility is to develop meaningful insights about the Experiment, and, ultimately, to write the final Analysis Report. The data products support this effort by distilling the megabytes of data generated during each run into recognizable patterns, trend lines, summarized data, animations and other data visualization products. These data products may be used to support the Analyst s research efforts, or used as graphical information in the Final Report, or they may simply be food for thought. The purpose of this Executive Summary is to provide a brief overview of these tools and data products: 1. Data Analysis Real Time System (DARTS) 2. Observer Data Collection Tools 3. Rapid Turnaround Situational Awareness 4. Sensor Coverage 5. Data Products Suite (more than 30 different analytical tools). For complete details on all of the data analysis products, tools, and systems, please refer to the data analysis methodology report, Data Product Methodology, for Experiment 7. DARPA M&DC2 Program, Experiment 7 Page 1

6 DARTS 2. DARTS The Data Analysis Real Time System (DARTS) is a software product developed by Applied Research Associates, Inc. (ARA) to receive live network packets from the experiment and display real-time graphs. DARTS collects the data for fires, detonations, spot reports and OTB entity-state records. In addition, the DARTS user interface provides a means to record the start and end time of the planning session, the time of the pre-run mission download, and the start and end time of the record run. DARTS populates a Microsoft Access database from which charts and graphs are displayed in real time. These products are each comprised of a set of four charts, called quads. The data for each quad refreshes every five seconds so that trends are built as the battle unfolds DARTS Quads There are five quads that cycle every 20 seconds. This can be modified, depending on how the analysts configure the displays. 1. The Assets Remaining quad - about red and blue attrition, such as total assets remaining, assets remaining by node, and blue assets remaining, categorized by air and ground vehicles 2. The First Detections quad, about blue detections of red, including first detects, first detections categorized by type of blue entity, by Sensor Type, or by Target Type 3. The Fires quad, about red and blue effects, such as fires by distance, fires by node, and a timeline chart for total fires over the run 4. The Movement quad, about red and blue movement as a function of distance 5. The Key Event Indicators quad, a highly useful analytic tool, is described below and is shown in Figure Key Event Indicators Quad The Key Event Indicators quad is described in detail in the following paragraphs due to its extensive use as an analytical tool during and immediately after a run. It was developed to provide an objective, empirical idea of possible key moments or turning points in the battle. The quad consists of the following four charts: Composite Indicators - A timeline chart showing an average trend for movement, losses, and detections, using calculations for two standard deviations, a mathematical average, a nine-minute running average, and a composite score DARPA M&DC2 Program, Experiment 7 Page 2

DARTS Movement A timeline chart showing acceleration, calculated as kilometers per hour per minute, with four trend lines for average blue ground units, average blue ground units alive, average red

7 DARTS Movement A timeline chart showing acceleration, calculated as kilometers per hour per minute, with four trend lines for average blue ground units, average blue ground units alive, average red units, and average red units alive Units Lost A timeline chart showing attrition, with three trend lines for blue ground units, blue air units, and total red units Blue Detections of Red A timeline chart showing two trend lines: one for blue first detects of red multiplied by ten for readability and ease of understanding, and a trend line for total detections. In addition to the real time display, the analytic team provides a Key Event Indicators stacked charts to the core team immediately after an experimental run. This report took the quad charts described above and placed them all on a common timeline for easy comparison. The Key Event Indicators quad is shown in Figure 1, below. Figure 1. Sample DARTS Key Indicators Quad DARPA M&DC2 Program, Experiment 7 Page 3

8 Observer Tools 3. Observer Tools For Experiment 7, the Essential Areas of Analysis were to explore decision-making and collaboration. These concepts are difficult to quantify, their nature being hidden as an aspect of the cognitive processes of the participants. While the automated data collection process may capture the hard statistics involving numbers of fires, movement rates, alerts and the like, the meaning of these is not readily apparent without some human interpretation. The purpose of the Observer Collection Tool is to allow participants on the analytic team to record their observations about the battle as it unfolds, lending their expertise to interpret meaning to the actions taken by the participants. Each element of the form directly maps to an area of analysis. The form allows the Observer to characterize the event (such as whether it was collaboration or a direct task) and some evaluation of the event as shown in the pink shaded area for situational awareness. Figure 2. Observer Collection Tool Input Form After Experiment 7 was concluded, ARA built a Microsoft Access database to allow the analysts to search, sort, and query the observations from all runs. DARPA M&DC2 Program, Experiment 7 Page 4

9 Rapid Reduction Tool for Situational Awareness 4. Rapid Reduction Tool for Situational Awareness Situational awareness (SA) is a cognitive function, and exists only within the human mind. Therefore, SA cannot be measured directly. Our approach applies a suite of surrogate measures to better understand the role of SA in decision-making and its impact on the tactical battle. We compare ground truth data to the information available through the commander s graphical battle interface. This comparison, which is scored over time throughout the battle, is called Situational Awareness Technical (SA T ). SA T = Information required by the decision-maker Information available through the system SAT Basic Equation The Rapid Reduction Situational Awareness (SA) tool was developed for two reasons: 1. The SA T Curves could be made available to the analysis team within hours of an experimental run than using the existing data production scheme. 2. Providing SA T Curves to the AAR team is an important capability. The Rapid Reduction SA tool consists of four Visual Basic Applications which loads source data from raw logs, an intermediate Event database, and the final SA Database which provides the full reduced data to the SA T Curve data product. These components are shown in Figure 3. DARPA M&DC2 Program, Experiment 7 Page 5

10 Rapid Reduction Tool for Situational Awareness DARTS DARTS Logs Logs ES Loader SA Processor CSE CSE Logs Logs CSE Loader Event DB SA DB Red Red Cmdr Cmdr Spot Spot Reports Reports Red Spot Loader SA Spreadsheet Figure 3. Rapid Reduction for SA SA Curves can be generated for a run within hours of endex provided that all logs are available, as well as latitude and longitude locations for each point in the area of interest to be plotted. The SAT score captures all information available on the system concerning the location, acquisition level, and state knowledge available to the operators. Therefore, we collect and track the following components: Spot reports along with associated information to track available identification level information Fire events to model uncertainty in state information All Human Target Recognition (HTR) reports that show when a user updated the perceived state or identification based on available information contained in images or communicated by the dismounted soldier All ground truth information to determine accuracy of perceived locations, identification, and state. The Rapid Reduction SA tool is not a change to the format or content of the SA T Curves data product. The computational engine that generates the SA T Curves from low-level simulation data and user specific parameters was in fact re-used without modification. Figure 4, below, explains the SA T formulation. DARPA M&DC2 Program, Experiment 7 Page 6

situational awareness (SA T ) for a given echelon.

11 Rapid Reduction Tool for Situational Awareness Figure 4. The SAT Score Derivation The SA Curve spreadsheet is a powerful tool enabling the user to understand the level of technical situational awareness (SA T ) for a given echelon. The package is customizable, giving users the ability to modify weighting factors, include/exclude specific entities, modify importance levels of entities and the like. A sample chart is given in Figure 5, below, using default parameters. Figure 5. Sample SAT Curves DARPA M&DC2 Program, Experiment 7 Page 7

Sensor Coverage 5. Sensor Coverage ARA developed a visualization tool showing the quality of sensor coverage over time across the battlespace.

12 Sensor Coverage 5. Sensor Coverage ARA developed a visualization tool showing the quality of sensor coverage over time across the battlespace. This enables the white cell to analyze a commander s effectiveness at using his sensor assets. It also facilitates a better understanding of the slow growth areas of the SA T curve. The advantage of the sensor coverage tool is illustrated in Figure 5. During a run, the SAT growth follows a fairly typical trend. There is rapid initial growth from the intel feed and sensors coming on line, followed by a relatively flat period where the unit remains behind the line of departure, then rapid growth as the ground forces move into enemy territory and find enemy targets at close range. The sensor coverage tool curves are shown below the SAT curve. Two curves are shown in this chart. One line represents coverage of all areas beyond the LD and the other line represents the areas most critical for mission success. Note that flat areas in SAT correspond to either no-growth regions of the coverage plot (indicating no new area being covered by sensors) or growth regions of the coverage plot (indicating coverage of new areas but no new sensor detections). Figure 6. SA and Sensor Coverage Used Together SA T and Sensor Coverage curves provide complementary data plugging the gaps of understanding the battle These charts together illuminate the events in the battle. The curves explain that after an initial surge of intelligence information, there are few new detections because no new area is covered by the sensors. As the ground forces cross the Line of Departure, a corresponding increase in sensor coverage quality occurs, followed by new information available to the commander. In addition to the curves, the tool has a Graphical User Interface (GUI) providing several key elements: It enables analysts to see the position of both red and blue assets over time It identifies which sensor asset makes detections using which sensors It identifies differences between perceived location and actual location over time It helps the analyst understand red and blue attrition over time. DARPA M&DC2 Program, Experiment 7 Page 8

13 Sensor Coverage The Sensor Coverage tool is depicted in the figure below. This tool shows the quality of sensor coverage across the battlespace along with critical information on friendly and enemy forces Data Products Suite Figure 7. Sensor Coverage Tool There are at least 30 data products available to the analysis team. Each spreadsheet usually includes several worksheets containing charts, summaries, trends, and other information. The data products were created through queries, Visual Basic code, or Excel spreadsheets. Some products use a combination of all of the above. Table 1 provides a listing and brief description of the data products. Refer to the Data Analysis Methodology appendix of the Experiment 7 final report for a complete, detailed description of each of the 30 data products. DARPA M&DC2 Program, Experiment 7 Page 9

14 Sensor Coverage Table 1. Data Products Product Name AGM Settings Alerts Battle Animation Battle Tempo Command Succession DARTS Assets Remaining Quad DARTS Fires Quad DARTS First Detections Quad DARTS Key Event Indicators Quad DARTS Key Event Indicators Stacked Charts Father of All Detects First Detect/First Engaged First Detects By Platform First Detects Over Time Kills Mother of All Fires Movement Over Time Munition Counts Observations (consolidated) - database Observations (consolidated) - spreadsheet OTB Screen Captures See First, Act First Sensor Coverage Description/Purpose/Use All active AGM settings for each unit, with threat-level settings as well Table summarizing all alerts occuring in the CSE A playback tool animating the movement of entities and sensor events over the course of a run A chart, with supporting data, graphing battle tempo using kills, collaborations, taskings, detects as input with various weights A reference table showing transfer of control of blue entities during command succession Real time chart showing the number of assets remaining as the battle progresses Real time chart showing fires as the battle progresses Real time chart showing the first detections as the battle progresses Real time chart showing individual and aggregate indicators of key events Post run publication of DARTS Key Event Indicators in a stacked format Contains detect times, health status, closest blue unit and location, perceived states as compared to ground truth, and the like Combination of detection and fire data, with visualization A bar chart, with supporting data, of the first detects for red and blue, by platform, as well as total detections, and percentage pie chart In ten-minute intervals, a chart with supporting data, of first detections for red and blue A collection of charts showing total kills of various categories Contains fire times, health status, aimpoints, closest blue unit and location, perceived states as compared to ground truth, and the like In ten-minute intervals, a chart with supporting data, of the movement in meters, for red and blue, by air/ground/troops A summation of munition counts at mission start and as expended Contains all observer recordings for all runs in a searchable database Contains all observer recordings for all runs in a spreadsheet format for those who prefer Excel PowerPoint presentations containing an image of the OTB screens at one minute intervals A set of charts illustrating the relationship between a spot report, an HTR classification, and the first fire Illustrates how well the commanders used their sensor assets. An excellent complement to the SA tool. DARPA M&DC2 Program, Experiment 7 Page 10

15 Sensor Coverage Product Name Situational Awareness (SA) Curves Spot Reports Target Reference Points Taskings Underground Sensors When Things Died Who Shot John Description/Purpose/Use For each cell, and for red of blue and blue of red, a plot of SA(T) curves Spot report spreadsheets, red of blue List of reference points for each entity during the run Contains tasks, times, purpose, initiator and so on. Input to Battle Tempo. A history of the deployment of underground sensors Which unit died and when Contains the killed unit, kill state times, action unit and the like The paragraphs below describe a few of the more complex data products: Battle Tempo Fires Query Detections Query Battle Animation 5.2. Battle Tempo The Battle Tempo spreadsheet is a complex visualization of the battle tempo for each run, with the ability for the user to alter the elements examined in the tempo. The battle tempo factors in a number of data elements to graphically portray the pace of battle for the commander. These elements include the following: Taskings Fires First detects Subsequent detects Collaboration events Units lost The resultant chart plots the tempo for the CAT commander, CAU1 and CAU2 commanders, and an overall average tempo as shown in Figure 9, below. DARPA M&DC2 Program, Experiment 7 Page 11

Sensor Coverage Figure 8. Battle Tempo Sample 5.3. Detections Query This is a highly detailed spreadsheet showing all spot reports for every entity.

16 Sensor Coverage Figure 8. Battle Tempo Sample 5.3. Detections Query This is a highly detailed spreadsheet showing all spot reports for every entity. Perceived location, health and identification is given, with ground truth location, health and identification as well as the location and health of the nearest blue ground unit to the target. The spotter entity with its location, health, and sensor used are also provided. The data was derived from blue and red spot reports and the OTB Entity-State table Fires Query This spreadsheet shows all fire information from all entities during the run. Target location, health and identification is given, with ground truth location, health and identification as well as the location of the nearest blue ground unit to the target. The shooter entity with its location, health and last spot time are also provided. The Fires data product also features a worksheet containing data where the shooter was not the same entity as the initiator of the fire; that is, one Blue entity took control of another entity s assets. A sample of this worksheet is shown below. DARPA M&DC2 Program, Experiment 7 Page 12

17 Sensor Coverage 5.5. Battle Animation Figure 9. Shooter Not Initiator Sample Worksheet This spreadsheet is a sophisticated set of tools allowing the user to play back the battle, showing detections and fires, through time over the run. The difference between this product and the OTB playback is that it shows the entities with a legend for the health status. The result is a unique snapshot of the battlefield. The user may examine a particular point in time by configuring the inputs on the Master tab of the spreadsheet. The user may select the legend colors, the echelon to examine and the cells to plot. The user may also create a movie for the run, which creates the output described previously and exports the chart to PowerPoint, with the chart then superimposed over a map of the battlespace. The user may customize the movie by specifying the start and stop times and the cycles (number of minutes) to plot. The data for the animation is built by using data compiled from various sources, combined with Visual Basic for Applications (VBA) code behind the buttons on the input form. The next figure, Figure 11, depicts a sample slide of the movie output during a run at 1342 hours. The data analyst prepared a movie using the default settings. DARPA M&DC2 Program, Experiment 7 Page 13

18 Sensor Coverage Figure 10. Sample Battle Animation Slide DARPA M&DC2 Program, Experiment 7 Page 14

19 Future Development 6. Future Development The Data Analysis team has several ideas for future new products or enhancements to existing products. 1. Extend the rapid turnaround methodology for SA to other data sources so that other data products can be produced more quickly. 2. Build a relational database of the pertinent data from the entire run complete with canned reports as well as an ad hoc query capability. 3. Create a tool to expedite the definition of the SA areas of interest. 4. Expand DARTS to include real-time SA. Successful development of these future products depends upon the availability of the necessary input files, especially in the case of real-time displays. DARPA M&DC2 Program, Experiment 7 Page 15

RAND S HIGH-RESOLUTION FORCE-ON-FORCE MODELING CAPABILITY 1

Appendix A RAND S HIGH-RESOLUTION FORCE-ON-FORCE MODELING CAPABILITY 1 OVERVIEW RAND s suite of high-resolution models, depicted in Figure A.1, provides a unique capability for high-fidelity analysis of