UK OFFICIAL. Crown copyright Published with the permission of the Defence Science and Technology Laboratory on behalf of the Controller of HMSO

Introduction Purpose: to make you think about what underlies the rules and tables in a wargame. In particular: What you need to know about the data. Where that data might come from, with examples of three different approaches used by Dstl. Making peace with the fact you will never have 100% of the data you want.

What this brief will cover A user s perspective on input data Three methods for generating that data: Trials & Experimentation Performance & System Modelling Historical Analysis Checking your data Q&A Caveat: Most of this presentation focuses on objective data. This is only a part of the data conundrum that wargamers face. A topic for later discussion.

2d6 Roll 1:5 1:4 1:3 1:2 1:1 2:1 3:1 4:1 5:1 6:1 7:1 8:1 Attacker Defender Defender Defender Defender Defender Defender Defender Defender Defender 2 Engaged Exchange Withdraws Withdraws Defeated Defeated Defeated Defeated Defeated Defeated Defeated Defeated Attacker Defender Defender Defender Defender Defender Defender Defender 3 Contact Engaged Exchange Exchange Withdraws Withdraws Defeated Defeated Defeated Defeated Defeated Defeated Attacker Attacker Defender Defender Defender Defender Defender Defender Defender 4 Contact Engaged Exchange Withdraws Withdraws Withdraws Withdraws Defeated Defeated Defeated Defeated Defeated Attacker Attacker Attacker Defender Defender Defender Defender Defender 5 Engaged Contact Exchange Contact Withdraws Withdraws Withdraws Withdraws Withdraws Defeated Defeated Defeated Data in wargames Attacker Attacker Attacker Attacker Defender Defender Defender Defender Defender Defender 6 Engaged Contact Defeated Withdraws Withdraws Withdraws Withdraws Withdraws Withdraws Withdraws Defeated Defeated Attacker Attacker Attacker Attacker Defender Defender Defender Defender Defender Defender 7 Engaged Engaged Defeated Withdraws Withdraws Withdraws Withdraws Withdraws Withdraws Withdraws Withdaws Defeated Attacker Attacker Attacker Attacker Attacker Defender Defender Defender Defender Defender 8 Engaged Contact Defeated Defeated Withdraws Withdraws Withdraws Withdraws Withdraws Withdraws Withdaws Defeated Attacker Attacker Attacker Attacker Attacker Attacker Defender Defender Defender Defender 9 Contact Contact Defeated Defeated Defeated Defeated Withdraws Withdraws Withdraws Withdraws Withdaws Withdraws Attacker Attacker Attacker Attacker Attacker Attacker Defender Defender 10 Engaged Contact Contact Contact Defeated Defeated Defeated Defeated Withdraws Withdraws Withdaws Withdraws Attacker Attacker Attacker Attacker Attacker Attacker Defender Defender Defender 11 Contact Engaged Contact Defeated Defeated Defeated Defeated Withdraws Withdraws Defeated Defeated Withdraws Attacker Attacker Attacker Defender Attacker Attacker Attacker Defender Defender 12 Exchange Engaged Contact Defeated Defeated Defeated Withdraws Defeated Defeated Defeated Defeated Defeated Most wargames use numbers and rules. Different wargames use different levels of data aggregation to produce these numbers (e.g. entity vs Coy). Unless you understand where all the numbers and rules come from all wargames include black boxes. Lots of detail means lots of potential small black boxes. Manoeuvre Medium Move 3 / 6 Artillery Power 1 (2) ISR Range 0 Artillery Range 1 Light Gun Battery Often you want black boxes. E.g. players, emergent phenomenon. Make sure you re comfortable with your black boxes. Trust Ability to review & discuss Judgement Validation & Verification Good logging Acceptable risk

Users perspective What I want to know about the data in my wargame: Where did it come from? Is it a good representation of what I want to investigate? Is it credible or is it counterintuitive? If so why? What are the uncertainties, boundaries & caveats? How will these impact the wargame? What caveats do I need to put on my outputs?

Data sources Simple Systems & Performance Modelling Historical Analysis Prob. 1d6 Of 1:1 2:1 Kill 1 2 3 Close Morale 4 Combat 5 Trials & Experimentation Judgement

Data sources In Practice Systems & Performance Modelling Historical Analysis Prob. 1d6 Of 1:1 2:1 Kill 1 2 3 Close Morale 4 Combat 5 Trials & Experimentation Judgement (incl. soft effects & emergent phenomenon)

Trials & Experimentation From the Fields to the Tables Mark Pickering Crown copyright 2015. Published with the permission of the Defence Science and Technology Laboratory on behalf of the Controller of HMSO

Contents What real world data do we need? How do we collect data? What causes variation? From data to game.

What real world data do we need? Probability of hit, or P(hit) Achievable rates of fire Lethality

Collecting data Modelling the ballistics does not represent a muddy field. Trials better real world data, but expensive.

What can be modelled? Lots of highly detailed aspects, such as: Weather Pressure Humidity Wind Manufacturing variation Etc. Propellant quality Density variation in round Barrel defects But some things can t be easily modelled, such as..

How do we measure P(hit)? With difficulty

P(hit) Miss distance What range? Limited data

P(hit) - Hit grid

From data to game Example Game Mechanism Hit Armour Save Wound To hit roll = Probability of hit or P(hit) Armour save roll = Probability of not penetrating the armour To wound roll = Probability of kill or P(kill)

To hit roll Aim Point

To hit roll

To hit roll Probable hit

Armour save roll Hit area

Armour save roll Hit area Armour View image from: War Thunder www.warthunder.com

To wound roll Hit area Lethality View image from: War Thunder www.warthunder.com

Systems & Performance Modelling Generating the rules Dan Ledwick Crown copyright 2015. Published with the permission of the Defence Science and Technology Laboratory on behalf of the Controller of HMSO

Systems and performance modelling I.e. what happens when X shoots Y? Information from trials and experimentation. P(hit). Level of damage per hit. Simulates outcomes of specific events. Reports at required level. E.g. individual bullet effects aggregated to Bde level. Faster and cheaper than full trials. Highly repeatable. Can represent potential future equipment.

Systems and performance modelling Focus on Vulnerability/Lethality Overview Example Data uses

Vulnerability/Lethality overview Determine the result of a weapon attacking a target Weapon performance Armour performance Damage to target components Resulting effect on target functionality Usually involves running a computer simulation Large number of engagements simulated

Vulnerability/Lethality example Example: brick thrown at glasshouse. Effectors Initially: From interaction: Brick Glass shards Assessment Process 1 2 3 4 6 5 7 1. Trigger: Effector generator brick. 2. Propagate brick. 3. Interaction with target component. - Trigger generation of glass shards. 4. Component response damage algorithms. 5. Propagate brick and glass shards. 6 and 7. Interaction/Component response. - Repeat as necessary

Vulnerability/Lethality data uses Identifying vulnerable areas of vehicles to prioritise protection improvements Characterising weapon performance against a target set Higher level wargames and models Computer simulations, Manual Training simulations DFWES, AWES

Wargaming Generation Historical Analysis Judgement Trials and Experimentation Wargame Generation System and Performance modelling Tabletop Algorithms Computer

Data and wargaming Historical Analysis (HA) provides real-world data grounded in reality. Increases buy-in when you can say: this actually happened before. Testing the theoretical vs the actual. Can provide data on things trials and experiments cannot or will not. Particularly in the operational and strategic spaces. However, you cannot create new historical data, you have to work with what you get. Exact real world cases are rare, so historical analogy is often a requirement.

Definition The use of mathematical, statistical and other forms of analysis to understand historical engagements, operations, campaigns and conflicts for the purpose of providing impartial analysis and sensitive decision support to policy makers. Critical to sensible and fully informed policy making.

Origins 1980s Falklands War. Field trials vs. Falklands War Combat Psychology Example: soldiers are much braver if they know they re in no real danger.

What data does HA use? Anything that will tell us what we want to know or allow us to infer an acceptable estimate thereof. Primary data sources War diaries Post Op Reports Operational Data Sources Secondary data sources Official Histories Reference Books Academic Studies

What methods does HA use? Quantitative analysis Regression Analysis, correlation etc. Stats packages R, Minitab, SPSS etc. Excel formulae, charts, graphs etc. Qualitative analysis Historical Research, Framework Analysis etc. Literature reviews Case studies

Spectrum of historical analysis The Real World Increasing Depth Large-N Studies Quantitative Statistical analysis Generalised results Potential over-abstraction Comparative Analysis Qualitative or Quantitative 5 30 cases Pattern Matching Selection Bias Increasing Abstraction Single Case Studies Qualitative In-depth analysis Contextual detail Not representative

Spectrum of historical analysis The Real World Large-N Studies Quantitative Statistical analysis Generalised results Potential over-abstraction This is similar to wargaming. Increasing Depth Different types of wargame will be Comparative Analysis appropriate depending on your question and depending on your data availability. Qualitative or Quantitative 5 30 cases Pattern Matching Increasing Abstraction Single Case Studies Qualitative In-depth analysis Contextual detail More data and material allows you to produce a greater Selection number Bias of different kinds of wargame, but one size does not fit all. Not representative

What does HA offer? Multidisciplinary approach an advantage. Data from tactical up to grand strategic level. Understanding of interaction of qualitative and quantitative factors. HA can highlight the important factors and back or disprove perceived wisdom. Particularly important when the HA goes against commonly held beliefs or perceptions. These factors can be fed into wargames or wargames can be designed to highlight the importance of these factors. Can be blended with trials and experimentation data. A mixture of from HA, trials, experimentation and judgement can help robustness, especially in the future space.

Proportion of fatalaties attributed to IEDs Example study: HA of value of training 0.9 0.8 0.7 0.6 0.5 0.4 Army Marines 0.3 0.2 0.1 0 0 5 10 15 20 25 Period Iraq US Army / USMC IED Fatalities Introduction of USMC MOJAVE VIPER Training

Proportion of fatalaties attributed to IEDs Example study: HA of value of training 0.9 However, at this time the USMC also 0.8 introduced a new protective vehicle. 0.7 Which one was more important? 0.6 0.5 0.4 Army Marines 0.3 0.2 0.1 0 0 5 10 15 20 25 Period Iraq US Army / USMC IED Fatalities Introduction of USMC MOJAVE VIPER Training

% total japanese tonnage sunk/patrol d Example study: HA of value of training 0.008 0.007 0.006 0.005 0.004 <4 patrols >4 patrols 0.003 0.002 0.001 0 A ug-41 Feb-42 Sep-42 A pr-43 Oct-43 May-44 Nov-44 Date USN Submarines - Pacific Average proportion of Japanese fleet sunk each patrol day, each month by experienced and inexperienced captains. Paired t-test p value < 0.001.

Checking Your Data A User s Peace of Mind James King

Checking your tables Comparison to historical operations. Is it a reasonable (and credible) representation of the situation you re trying to wargame? Judgement of people who have conducted similar operations. Military/SME judgement. Look over as many cases as possible, not just one example. Beware single data points.

Lots of Real World Cases Extensive Trials Trials Experimentation Modelling Historical Analogy Expert Opinion (with evidence) Expert Opinion (without evidence) Expert Opinion (little relevant experience) Data provenance Best Case Provenance Worst Case Judgement Very Rare Common Availability Very Common

And finally What you need to know about the data. Think about what underlies the rules and tables in your wargame. Log your assumptions wherever possible. When designing a wargame make sure you can get the data you will need. Where that data might come from, with examples of three different approaches used by Dstl. Trials & Experimentation, Systems & Performance Modelling, Historical Analysis. There are multiple sources of data, each with their own strengths and weaknesses. Judgement ties them all together to make a wargame. Wargames and data sources are not isolated, but can test and inform each other. Making peace with the fact you will never have 100% of the data you want. Be comfortable with uncertainties and black boxes. People are black boxes! Be sure you are happy with the data you have. Understand the relationship between the impact the data will have, and the certainty you need. Beware single data points!

Questions?