Simulation Studies of Naval Warships using the Ship Air Defence Model (SADM) Dr. S. Boinepalli; G. Brown Maritime Operations Division Defence Science and Technology Organisation, Sharada.Boinepalli@dsto.defence.gov.au; Garry.Brown@dsto.defence.gov.au Abstract. Performing simulation studies during the acquisition of major warships is an accepted practice all over the world. Simulation models are used to assess the performance of the system because a physical instance does not exist and even if representative sub-systems were available, it would be difficult to replicate the operational environment and range of scenarios in which the system is expected to operate. Maintaining records of the runs and comparing with real results, once the asset is tested and put into service, assists fine-tuning and validation of the modelling and analysis. Methodologies for two simulation studies performed at DSTO are summarised where a large warship must perform Anti Ship Missile Defence (ASMD). The commercially available Ship Air Defence Model (SADM) was configured to represent the ship and used to execute Monte Carlo simulations in a variety of scenarios. In the first study, the Hard Kill (HK) and Soft Kill (SK) integration options are assessed to inform future Command and Control system requirements for large warships. This includes the definition of a metric which assists in the comparison of different HK and SK integration options. In the second study, the ASMD performance is assessed for comparison with data from other sources. This involves some pre and post processing to augment the primary SADM model. These modelling and simulation studies enable us to: a) Analyse the effectiveness of HK and SK integration on large warships, b) Understand and test the capability requirements for ASMD, and c) Analyse the performance of SADM to inform future work plans. This paper aims to briefly describe the two simulation studies and comment on the applicability of software tools like SADM for future work. 1 INTRODUCTION The use of Modelling and Simulation (M&S) in defence acquisition is not a new practice. Mathematical models and engineering models have been used extensively to evaluate the impact of new or improved national defence systems [1, 2]. In the Australian context, M&S techniques are used throughout the life cycle of existing force elements, with varying quantity, complexity and success. With opportunities afforded by improving computer technology and a desire to reduce system integration risk, simulation is also being used during the acquisition of new force elements [3]. M&S is providing insights into the behaviour of proposed systems and capabilities that are difficult and expensive to evaluate through operational testing. In this paper we present the methodologies adopted for two simulation-based analyses of the performance of a large warship. In this context, the large naval ship is a modern surface combatant equipped with a complex set of sensors and effector systems. Typically a warship will have several radars, electronic surveillance sensors and communications links to other allied data sources. The ship is expected to project power by maintaining its position in an area for a considerable time, defending itself against hostile attacks by appropriate engagement of hostile forces with a selection of effectors. Ships tend to organise their defences in layers. The outer layer may be a call to allied ships and aircraft to deter an adversary. The next layer may be a long-range missile that is suitable against ships, aircraft and incoming missiles. A more dynamic responsive effector is used against a threat that is closer to the ship, which is followed by a selection of other defences such as large 5 guns, fully automated Close In Weapon Systems (CWIS), large calibre machine guns and various decoy and seduction systems. In general the decoy, seduction and electronic attack systems are called Softkill (SK) options, whereas the other systems, which rely on impact or close detonation, are called Hardkill (HK) options. The Royal Australian Navy (RAN) has operated a number of warship classes. The RAN currently has ships in major refit and a new class of Destroyer being built in Adelaide, South Australia. With ships in various stages of their life-cycle, it is important to be able to re-evaluate the capability of each platform and assess the implications on the total defence force. These simulation studies employed the commercially available Ship Air Defence Model (SADM) to explore aspects of the Anti Ship Missile Defence (ASMD) capability. Although the primary software tool is unclassified, the configuration parameters and results are sensitive. The focus of the following sections is to describe the approach and issues associated with modelling warship capability.
The simulations were performed as two separate studies. The two simulation environments shared many similarities but were quite different in performance metrics, with the first study focused on HK-SK integration and the second study looking at overall ASMD performance. The first study was an analysis of the HK and SK coordination aspects of ASMD capability against a variety of missile threats. This type of study is useful when performing design trade-offs and can influence the allocation of additional resources that may be needed to enhance capability. The main aim of this study is to assess the merits of performing the same task in different ways. In the second study a particular configuration was modelled to assess ASMD performance under various missile threats and environmental conditions. The scenarios for the second study were derived from capability guidance documents. This type of study is useful when quantifying overall performance and may identify where a key subsystem reaches its operational limit. The main aim is to push the system to its limits and ensure that modelling iterations cover the key challenges for the capability. 1.1 Back ground The acquisition of all major capability elements is undertaken by the Defence Materiel Organization (DMO). For a major warship this is a complex project management function involving a large number of defence contractors with expertise in areas such at sensors, weapons, combat management systems and shipbuilding. Parameters for sub-systems are sourced from commercial and government organisations to construct a suitable representation of the ship. These parameters formed the basis for the ship configuration within the SADM model with customisation of specific SK and HK systems by DSTO. Sourcing, verifying and managing all of the parameters for such a complex model is challenging. Even when there are no commercial or national security sensitivities, a systematic reference to the data sources is essential for the validity of the model and it assists when inspecting preliminary results. If there are distribution restrictions on some information, effective management of these parameters allows reuse of sub-models. 1.2 Ship Air Defence Model (SADM) The Ship Air Defence Model [4] is a versatile simulation tool used in a number of defence organizations across the world. The initial work from DSTO was significantly enhanced by BAE Systems, who continue to develop the software product and provide licences for commercial use. The tool is primarily designed to simulate missile engagements in the maritime open ocean and littoral battle space. SADM simulates the defence of a single ship or task group against one or more attacking anti-ship missiles (ASMs). It simulates softkill options using chaff, decoys and jammer models while hardkill options are missile, gun and Close-in Weapon System (CIWS) models. A recent upgrade includes a hardkill softkill integration option, executed by the Command and Control (C2) system model. The model includes a batch parameter mode which facilitates the automation of multiple runs over a set of variables without having to manage individual runs or merging of resultant data. Within each run, the model may be configured to perform many iterations so that the probabilistic variables are exercised to yield statistically significant results. This Monte Carlo method [5] allows convergence and improves confidence in the results. The model executes a scenario and computes the probability of ship survival against the incoming threats. It also computes the cost incurred in using the modelled sub-systems. 2 SIMULATION STUDY: HARDKILL SOFTKILL INTEGRATION The integrated use of various defensive systems on a naval platform is the focus of extensive research in many defence organizations [6, 7 and 8]. Coordinated usage of hardkill and softkill systems is seen as one of the milestones in the development of combat systems and ship-self-defence-systems [9, 10]. This has triggered several simulation studies to analyse the combat system performance with respect to hardkill and softkill integration [8, 11]. Blodgett et al [8] developed a rapid prototyping environment to study coordinated planning. DSTO also conducted a HK-SK integration study using a SADM representation of a generic frigate. The results and metrics developed were presented at the Association of Old Crows (AOC) Symposium [12]. 2.1 Motivation The ASMD capability of warships often involves limited integration of HK and SK systems. The SADM representation in this study was constructed to test the boundaries of HK-SK integration. In addition, this work was expected to help DSTO gain an understanding of HK-SK integration capabilities within SADM. SADM version 4.0.5.6 was used for these simulations. The warship performance was tested against a set of missile threats using: a) HK options alone, b) SK options alone, c) HK and SK options together. In SADM, HK and SK coordination options are deployed in two modes: a) Independent Softkill (ISK) - In this strategy the C2 system deploys HK and SK elements independently, b) Softkill When Saturated (SKWS) - In this strategy the C2 system deploys SK elements if there are no HK elements available at that moment (due to scheduling issues).
2.2 Methodology 2.2.1 Scenarios and environment To perform this study, we designed 5 scenarios to be executed in SADM. 1. Hardkill (HK) - In this scenario, SADM representation of the ship deploys only the HK options. HK options are always deployed by the C2 system. 2. Softkill (SK) - In this scenario, SADM representation of the ship deploys only the SK options. Generally, SK options can be deployed by the ESM lock-on or by the C2 system. Here the SK options are deployed by the ESM lockon. 3. Hardkill Softkill together (HKSK_tog) - In this case, both the HK, SK options are deployed independently of each other. HK options are deployed by the C2S and SK options by ESM lock-on. 4. Hardkill Softkill, Independent Softkill (HKSK_ISK) - Here, both the HK and SK options are deployed by the C2 system, with both treated independently. This looks similar to the HKSK_tog case, but here SK options are not deployed by the ESM lock-on. 5. Hardkill Softkill, Softkill When Saturated (HKSK_SKWS) - The HK and SK options are deployed by the C2 system, but the C2 system deploys SK options only if no HK options are available at the time of scheduling. Two types of threat configurations were considered for this study (Fig 1). In the first configuration, the raid size is varied from 1 to 4 ASM threats which approach the ship together, from the same bearing (stream raid). The angle of attack takes 8 values, resulting in a total of 32 runs. In the second configuration, the raid size varies from 1 to 3 with the ASM threats arriving with a fixed bearing spread (simultaneous time of arrival). The primary axis of attack takes 6 values, resulting in 18 run combinations. Each run is executed multiple times for statistical significance. A single type of ASM threat was used for these simulations. Threat 3 Threat 2 Threat 1 Threat 2 Threat 1 Threat 3 Ship Ship Figure 1. Two threat raid configurations 2.2.2 Comparison of results - A new metric The comparison of results between scenarios is usually done by comparing the probability of survival, values of each scenario run. A Capability Metric (CM), from associated DSTO work [12], is defined to include the cost of the resources used in achieving the given probability of survival. This metric is used to compare the scenario runs. Since the aim is to maximise the Ps while minimising the cost C, the P easiest measure would be to maximise the ratio s. C However, is considered much more important than the cost. To account for this significance, CM is defined as the ratio. This ensures that the cost is scaled log(c) down significantly compared to the probability of survival. 1 The five scenarios described in the previous section were compared using both and CM. The conclusions of the comparison are briefly discussed in the next section. 2.3 Some conclusions In threat configuration 1, the HK-alone option is expected to handle an increasing number of threats with decreasing. However, SK can tackle stream raid threats at many angles with high. This should however, be understood with caution, given the fact that 1 Since completion of the study, BAE Systems has removed the calculation of cost in SADM, citing some inconsistencies present in the model. The effect of this change on this study is considered minimal due to the lower priority of cost compared to where a logarithmic scale was used for the cost.
the due to SK is valid only for a short period of time. The high P of SK-alone leads to high P when HK and s SK are used together, irrespective of the type of coordination. By examining the CM, it is possible to get a feel for the differences between different types of coordination. The SK rounds have a low cost compared to defence missiles, and hence the CM for SK alone strategy appears very attractive. Among coordinated uses, HKSK_SKWS gives the best CM, since its cost is lower than the other strategies. In the threat configuration 2, the SK is lower than it was for the first configuration due to the different raid formation. It is not always intuitive to say whether SKalone will perform better or worse than HK options. Performance depends heavily on the direction and spread of threats. However, in general, coordinated HK and SK gives better. In most cases using HKSK_tog does not s give as good results as coordinated use with HKSK_ISK or HKSK_SKWS. The analysis brought to light several possible issues with the HK SK coordination in SADM. For instance, SK assets deployed by ESM lock-on showed different behaviour compared to the strategy where SK assets are deployed by the C2 system. Further investigation of these effects, combined with defence scheduling is required. An example of one of the graphs from configuration 2 is provided in Figure 2. This shows the types of trends that may come from a raid formation approaching at different bearings. The Ps values for SK would be affected by the RCS of the ship and the direction of the ship relative to the axis of the raid. The indicative data here also shows a ranking of relative performance including costs. Indicative Comparison using CM Metric CM 0 50 100 150 200 250 Bearing HK SK HKSK_tog HKSK_ISK HKSK_SKWS Figure 2. Example of ranking options using the Capability Metric (CM) 3 SIMULATION STUDY: ASMD PERFORMANCE The aim of this second study is to gauge the performance of the ship s Anti Ship Missile Defence (ASMD) capability in specific situations under various ASM threat conditions. 3.1 Motivation One of the benefits of this type of simulation is to get an early look into the performance of a ship that is still being constructed. In this case, the modelling scope must be developed from the aspirations of the RAN, normally in the form of an Operational Concept Document (OCD), and from requirements specifications being refined by contractors to the acquisition project. The scenarios from OCDs are often developed to inform design and test engineers of the operational context for the total ship system. Some refinement and examination of assumptions is needed before they are suitable for simulation studies. This results in a study that is complementary to formal engineering activities and provides decision makers with early indications about ship capabilities that are difficult and expensive to assess during operational testing. It is extremely important that the ASMD capability of a warship is effective, but like insurance policies, hopefully it will never be called upon. In practice, the results of these simulations are used to provide confidence in the development of tactics that account for risk management in the war fighting domain.
3.2 Methodology Scenarios were based on information from acquisition guidance documents. Eight environments were chosen to cover different areas, weather and sea states. Other options included CIWS on/off and two different values for Time on Target (ASM arrival time). Subsonic and supersonic ASM threats were considered. All of these options resulted in 64 scenario combinations. As part of the configuration of the SADM model, the generic ESSM model was replaced with the INTSIM model. DSTO constructed RCS values for the ship. This study does not include results using the high fidelity Nulka model. There were a large set of metrics that were considered important to developing an understanding of ASMD performance. Some of these needed to be generated just for verification through comparison with results from other studies. The default performance metrics provided by SADM were extended by using data from various SADM output logs. The ASMD metrics were generated by parsing log files and manipulation with Matlab scripts. The automation of this process reduced the time to prepare data and reduced the risk of errors during transcription. The resultant Microsoft Excel spreadsheets were then used for final analysis. Matlab scripts were also used to make repetitious changes to SADM scenario files and improve configuration management of the SADM runs. 3.3 Simulation Infrastructure Initially an attempt was made to use a distributed version of SADM called D-SADM. The controller tool is installed on a single computer on a network and a light weight service (a small piece of software) is installed on every other computer that a user wishes to use for computation. The D-SADM controller takes a list of SADM scenario files and tasks out the jobs across the network. Parameters and executable code are pushed out to the service machines and at the end of the task result files are moved to a nominated repository. Only one SADM licence is required which must reside on the controller machine. D-SADM also provides status messages from the distributed jobs which may be monitored from the controller. The use of D-SADM seemed highly desirable and worked on the network for the initial test cases. Unfortunately some problems were encountered with the full test cases. Some execution crashes only provided warnings that virtual memory was exceeded so a decision was made to revert to the standard installation of SADM using multiple licences on multiple machines. The SADM runs were farmed out to a set of 4 server class computers each with dual quad-core processors. These 32 cores completed execution of the 64 scenarios in approximately 1 month on the wall clock. There were some issues with execution of large scenarios and some down time occurred between runs. The raw output from a SADM run is considerable, and for the entire set of scenarios there needs to be some organisation and automation to produce consistent results in a timely fashion. The development and testing of Matlab Post Processing (PP) Scripts for this purpose was demonstrated to be a reasonable investment, particularly when rework is triggered by infrastructure failures or the need to evaluate unexpected results. The inputs and outputs to the PP Scripts are depicted in Fig 3. SADM output Crunch SADM Output Write in Matrix Form Compare results Other results MAT Files Metrics Figure 3 Flow of information around Post Processing Scripts 3.4 Some Conclusions A range of metrics were constructed using data from SADM log files and results were compared with information from other modelling activities. This enabled an evaluation of metrics such as probability of survival of the ship, probability of no damage occurring to the ship and probability of raid annihilation. The ranges at which the ASM threats were engaged and the defensive missile expenditures were also compared. Apart from the results of the analysis, the study also demonstrated the benefits of configuration managed post processing of SADM outputs and brought insight into distributing many scenarios across multiple processors. Some success was achieved with scenario distribution, but there is a need for some future work on scenarios that have a large number of missiles to be modelled. Some testing is required to characterise the SADM memory model against potential network, hardware and operating system constraints.
4 SUMMARY The motivation and methodology for two simulation studies of ship performance were described in brief. Both studies used the Ship Air Defence Model (SADM) to simulate Anti Ship Missile Defence (ASMD) engagements. The first study was primarily focused on understanding the HK-SK coordination issues. The second study was focused on generating measures of performance for ASMD capability in a range of war fighting conditions. This work provided insight into the use of SADM to model ship performance where the number of scenarios is large and execution times are long. The initial success should be followed up with testing, particularly with D- SADM, to indentify execution constraints driven by the size of scenarios or erroneous parameters. The configuration management of SADM parameters and the post processing of SADM log files were also identified as key processes in these types of simulation studies. 5 BIBLIOGRAPHY 1. Committee on Modeling and Simulation Enhancements for 21 st century Manufacturing and Defence Acquisition, National Research Council (2002). Modeling and Simulation in Manufacturing and Defense Systems Acquisition, Pathways to Success; National Academy Press, Washington. 2. Saunders D et al. Simulation Based Acquisition in Australia- A View to the Future; 2000. 4. Chapman S.J. Assessment of Ship Air Defence Performance by Modeling and Simulation; 5. Gentle J. E. Statistics and Computing; Springer Series. 2003. 6. Sewards, B. Barry. An EW perception of requirements for maritime sensor and weapon integration; Defence Research Establishment Ottawa (Ontario). 1988; 7. Martin P.F.M. van Dongen, Joost Kos. Weapon system analysis; Naval Research Logistics; Vol 42 Issue 2; 291-309; 1992. 8. D. Blodgett et al. Coordinating plans for agents performing AAW hardkill and softkill for frigates; American Association for Artificial Intelligence (2001). 9. Whitley JE Jr. An introduction to SSDS concepts and development; John Hopkins APL Technical Digest; Vol 22, No.4 (2001). 10. D.R. Ousborne. Ship self-defense against air threats; John Hopkins APL Technical Digest; vol 14, No.2 (1993). 11. Chia Hk; Masters Thesis. Simulation of a combined active and electronic warfare system for the defence of a naval ship against multiple low-altitude missiles threat; (1989). 12. Boinepalli S, Farmer D. Simulation of Electronic Warfare Systems Coordinated With Hardkill Options on Maritime Platforms; AOC Symposium, 2010. 3. Marco J; Simulation Support to Acquisition; SimTect 2003.