Saab Systems 1 (18) EXECUTIVE SUMMARY for IDEALS
Saab Systems 2 (18) Table of Contents 1 SUMMARY...3 2 OVERVIEW OF THE IDEALS PROJECT...4 2.1 Identity...4 2.2 Project Organisation...4 2.3 Project Objectives and Scope of Work...5 2.4 Project Structure...6 2.5 Commercial Aspects...6 2.6 Planning...7 3 DESCRIPTION OF WORK PACKAGES AND RESULTS...8 3.1 WP 1...8 3.2 WP 2...9 3.3 WP 3...11 3.4 WP 4...12 3.5 WP 5...13 4 DISSEMINATION...14 5 LESSONS LEARNED...16 5.1 Technical...16 5.2 Project Management...16 6 CONCLUSIONS AND RECOMMENDATIONS...18 6.1 Conclusions...18 6.2 Recommendations...18
Saab Systems 3 (18) 1 SUMMARY Starting from September 2005 and during three years, Thales Nederland B.V. and Saab AB have been working on the IDEALS project. The project was funded jointly by the industries, the Dutch Ministry of Defence and the Swedish Defence Materiel Administration (FMV). The objectives of the project were to 1. To identify, develop, test and evaluate core algorithms that are needed for an array of difficult problems in advanced single and multi sensor data processing. 2. To develop and build advanced applications in radar signal and data processing and fusion with those techniques. The framework for this development was Particle Filtering, also known as Sequential Monte Carlo. It is a technique where one, instead of seeking analytical approximations of nonlinearities, performs a kind of online simulations to find the desired results. It has the advantage of being able to handle very severe nonlinear cases, however often at the price of demanding a lot of computer resources. The application areas were Detection and tracking of dim targets in sea clutter Tracking of ground targets using a ground sensor network Significant progress was made in both application areas. Problems were solved that would have been quite difficult without the particle filtering. It was interesting to note that solutions were found that had much more generality than the original application problem. A final demonstration has been devised to show the results of the project. Given the success of IDEALS, the recommendation is to proceed with a similar project, this time however focussing on combining particle filtering techniques with higher level (contextual) information.
Saab Systems 4 (18) 2 OVERVIEW OF THE IDEALS PROJECT 2.1 Identity Program name: Program abbreviation: Integrated Detection and Estimation Algorithms Solution IDEALS Starting date: T0 = 2005-09-15 Conclusion date: T0 + 36 months Contract number: 05/101.095/20 Customer: Western European Union represented by the WEAO Research Cell. This was changed the 3 rd of March 2006 as the project was transferred to the European Defence Agency. Saab AB, represented by its business unit Saab Systems, has been the Single Lead Industrial Entity (SLIE) and was responsible for the conclusion of the main contract with the WEAO Research Cell. Saab Systems sub-contracted Thales, NL for its participation as an Industrial Entity (IE) in the program. 2.2 Project Organisation The organisation for the IDEALS project is shown in Figure 1. The countries participating in the project have been The Netherlands and Sweden. Each participating country has had representatives from their respectively Ministry of Defence, which together formed the Technical Arrangement Management Group with it s Chairman from the Swedish Defence Materiel Administration (FMV). Each IE has provided a Point of Contact for the technical work that has been performed by each IE. Saab has been work package leader for WP 3 and WP 5, and Thales for WP 2 and WP 4. The overall Project Management, WP 1 has been led by Saab as the SLIE.
Saab Systems 5 (18) European Defence Agency Attila Simon FMV SWE Håkan Hindsefelt MOD NL Wim Pelt IDEALS Project Manager Kent Magnusson Saab WP1 Project Management SLIE Kent Magnusson Saab Contract Management SLIE Johanna Back Saab WP2 Yvo Boers, Thales NL WP3 Egils Svienstins, Saab WP4 Yvo Boers, Thales NL WP5 Egils Svienstins, Saab Johanna Back Saab Mark Donders Thales NL Figure 1 Project Organisation 2.3 Project Objectives and Scope of Work The Vision once set up for the IDEALS project was summarised as By means of particle filters, significantly enhanced capability to detect and track ground and sea targets under difficult conditions. The objective of the IDEALS project has been twofold. 1. To identify, develop, test and evaluate core algorithms that are needed for an array of difficult problems in modern sensor data processing and fusion. 2. To develop and build advanced applications in radar signal and data processing and fusion with those techniques. More specifically, the objectives have been to explore the so called Sequential Monte Carlo (SMC), or Particle Filtering, techniques, both theoretically, and for two application areas: Detection and tracking of dim targets in sea clutter,
Saab Systems 6 (18) and tracking of ground targets. SMC offers great potential over conventional, Kalman filter based techniques owing to the fact that arbitrarily non-linear systems can be modelled with arbitrary accuracy, although the demands for computer power can become overwhelming. 2.4 Project Structure To achieve the objectives and goals of the study the following Project structure with five Work Packages were defined: WP 1 Management. WP 2 Development of efficient core algorithms. The goal here was to achieve advances in particle filtering technology aiming at more efficient algorithms for tracking under difficult conditions. WP 3 Algorithms for ground target surveillance. The objective was to work out realistic system solutions and algorithms for tracking ground targets over large areas using short range sensors. WP 4 Advanced Track Before Detect algorithms in a naval cluttered environment. The objective was to work out realistic algorithmic solutions for tracking and detecting dim targets in an adverse naval environment. WP 5 Demonstrator. This work package aimed at running a peace keeping operations scenario showing the most important accomplishments in WP 2, WP 3 and WP 4. 2.5 Commercial Aspects It shall be noted that the proposal for IDEALS was submitted under the EUROPA MOU, ERG no. 01, under the condition that the project should be treated as A Closed Consortium Research and Technology Project. This
Saab Systems 7 (18) implies that the project and its result will only be distributed to the Swedish and Dutch MOD representatives. The project would like to acknowledge both the WEAO Research Cell as well as EDA for all the support given during the duration of the project. The Contract itself has been useful during the performance of the project giving guidelines for most of the issues that have had to be solved. 2.6 Planning A Time Schedule was drawn up at the start of the project. As always with R&D projects, it s hard to set up milestones with a concrete content. The milestones in the Time Schedule have although been met and all deliveries as well as the Progress Review Meetings have been conducted as planned. The intention of having Project Planning Meetings every month throughout the project has also been met. The use of telephone conferences instead of face to face meeting has reduced the money spent on travelling which has given the project more resources for the R&D task.
Saab Systems 8 (18) 3 DESCRIPTION OF WORK PACKAGES AND RESULTS 3.1 WP 1 Saab Systems as the SLIE has been responsible for the management of the project and the main topics have been Organisation of kick-off meeting. Contractual work: discussing and agreeing subcontract, non-disclosure agreement etc. Management related to project plan, six-monthly progress reports, final report and executive summary. Organisation of Progress Review Meetings (and workshop meetings). Management related to issue and delivery of different reports (interim as well as final reports). Internal management like: resource allocation, day-to-day management, configuration management etc. The working relations between the IEs have been outstanding during the performance of this project and so have the relations with the Dutch and Swedish MOD s representatives been. The open relation between the IEs has created an environment where a number of joint articles and papers have been produced. This has most likely been of great advantage for not only the IE, but also for EDA as well as for the Dutch and Swedish MOD.
Saab Systems 9 (18) 3.2 WP 2 The main focuses of WP 2 have been in development of efficient core algorithms. This has been pursued mainly through three lines, in which two of them are really linked or intertwined. The lines of research within this WP have been: Parallelisation of particle filters Tracking closely spaced objects theory Tracking closely spaced objects data association Parallelisation of particle filters Parallelisation of particle filters over multiple processors is an important issue in order to distribute computational load of a particle filter. For the problem of parallelisation different architectures and approaches have been studied. At first a survey of existing work has been done and it was concluded that no systematic approach to this problem existed. Some previous work exists, but in this work the focus was merely on computational load and not so much on minimizing the computational while at the same time keeping up a performance as close as possible to the optimal (one batch) particle filter. The focus has been on maintaining a similar distribution of particles. This has been achieved by attempting to balance the sum of the weights of the local sets of particles. Different approaches have been tested and some of them provided good results on some benchmark problems. It seems that parallel implementations are feasible, running several batches of particles in parallel, using a limited amount of communication between them. Tracking closely spaced objects Tracking closely spaced objects, be it with or without particle filters, is a longstanding challenge in target tracking. In IDEALS, we have focussed on understanding the problems that lie at the hart of this challenge. One of the findings was that in a joint PF based approach the so-called mixed labelling problem occurs. This mixed labelling nicely reveals the problem, not only in the processing of the information, but also in the track extraction process. It has been shown that the best filter can be ruined by blindly applying standard track extraction methods. Alternative track extraction methods have been developed and studied. One of the major breakthroughs that has been accomplished here is the construction and proven correctness of the particle-
Saab Systems 10 (18) based MAP estimator. The most valuable lesson however, might be considered the gain in insight in the longstanding problem of tracking closely spaced targets. Tracking closely spaced objects Data association Related to the problem of tracking closely spaced object is the data association problem, i.e. the determination of which measurement in a set of unlabeled detection belongs to what physical object. In target tracking, this is a problem of outmost importance, since whatever kind of filter is used, it can only work as long as it is fed with the correct detections. In WP 2 we have developed novel data association and classification algorithms based on Monte Carlo (MC) methods. Extensive MC simulations using a number of challenging scenarios have been performed to validate the feasibility of the novel algorithms. Some scenarios are adapted to emphasize the problem of closely spaced targets. Other scenarios try to imitate a ground sensor network (GSN) setup. Some important findings from the simulation studies of the data association and classification algorithms include: - One way of improving the behaviour of data association algorithms based on particle filters - when for instance the number of particles is decreased - is to include constraints (e.g. terrain constraints in ground target tracking applications) in the algorithms. - For the joint state particle filters approach, the number of particles necessary to avoid particle degeneracy is - for some challenging scenarios with passive sensors - very large. - Attribute data is easily integrated within the particle filter framework and utilizing this can for instance prevent swapping in closely spaced objects situations.
Saab Systems 11 (18) 3.3 WP 3 Ground surveillance is becoming increasingly important in the military domain e.g. peace keeping operations and maintaining territorial integrity as well as in the civil security domain. Yet ground surveillance is a notoriously difficult area for many reasons the targets are often hard to distinguish from the background; the number of objects may be very large; and terrain, vegetation and buildings obscure the targets from the sensors. Limitations in sensor capabilities can to some extent be compensated by networking many of them, and applying advanced data processing methods. WP 3 had the long term objective to suggest realistic systems and algorithms for tracking ground targets over large areas using short range sensors. It appears evident that particle filtering, with its ability to handle nonlinearities, can provide an important part of the solution. Also important are Taking terrain information into account, especially the road network Proper modelling the typical dynamical behaviour of the targets in relation to the environment Proper modelling of the sensors and their sometimes salient shortcomings, e.g. limited dimensionality The WP 3 has come a long way towards realising the given goals. We have developed particle filter based tracking algorithms, provably capable of tracking vehicles: on road and off road with real sensors - radars, acoustic sensors and seismic sensors utilizing digital map data given in a standard format with performance superior to standard (Kalman) filter based trackers under realistic conditions.
Saab Systems 12 (18) 3.4 WP 4 Traditional tracking algorithms apply a threshold to the radar video. Consequently, dim or weak targets will not lead to detections and tracking of these targets is not possible. Track before detect (TBD) is a method to track targets using unthresholded measurements. The idea is that by integrating the energy that is received from a target over time, a target that would conventionally not lead to detections can be tracked by TBD. Under this WP new and improved particle filter based track-before-detect methods have been developed, that are able to track and detect very weak targets in heavy sea clutter. Different clutter models have been used and implemented. Also different techniques, like employing constraints on target motion as well as techniques to deal with shadowing have been explored. It was shown on realistic sea clutter data, that the proposed algorithms are capable of tracking and detecting a very small target in heavy sea clutter.
Saab Systems 13 (18) 3.5 WP 5 The aim of this work package is to demonstrate the achievements from Work Packages 2, 3 and 4. The demonstration scenario is built around a sea-part and a land-part, where a load of weapons is smuggled to shore from a ship, and further transported inland by car on road and off road. The sea-part is simulated using the Thales Track Before Detect system, while the first part of the land scenario is produced by live recordings from a ground sensor network. The last part of the land scenario is simulated by the WP 3 simulator. The three parts are then combined into one scenario where the land-part is supplied to a particle filter. The result is a demonstration to be held on October 2, 2008 in connection with the final Progress Review Meeting.
Saab Systems 14 (18) 4 DISSEMINATION Apart from the kick-off meeting, six Progress Review Meetings have been conducted with representatives from the Authorities and once with a representative from EDA. In advance of these meetings a set of Interim Technical Reports have been issued and distributed to the Technical Arrangement Management Group (TAMG) and EDA. Totally has nineteen reports been produced. A set of Final Technical Reports (five all them) together with a Final Report and an Executive Summary have been produced and distributed to TAMG and EDA before the last Progress Review Meeting. In due time before each Progress Review Meeting a Progress Report has been produced and distributed to TAMG and EDA. TAMG has asked for a presentation of the results from IDEALS for the Armed Forces in the Netherland and Sweden during the vey last part of 2008. Within the scope of IDEALS nine publications have been produced and some have also been presented at conferences, seminars and others have been published in related journal etc. according to the following table. Article 1 Ground target tracking using acoustic sensors by Mats Ekman, Kjell Davstad and Lars Sjöberg, presented at Information, Decision and Control 2007 Symposium, Adelaide, Australia 2 Ground target tracking with acoustic sensors using particle filters and statistical data association by Mats Ekman and Niclas Bergman, presented at Nonlinear Statistical Signal Processing Workshop, Cambridge, UK, 2006 3 Multiple Model Algorithms Based on Particle Filters for Ground Target Tracking by Mats Ekman and Egils Sviestins, presented at Fusion 2007, Québec City, Canada 4 Particle Filter Multi Target Tracking for Closely Spaced Targets by Mats Ekman, Egils Sviestins, Lars Sjöberg, Yvo Boers and Hans Driessen, presented at Fusion 2007, Québec, Canada
Saab Systems 15 (18) Article 5 The Mixed Labelling Problem in Multi Target Particle Filtering by Yvo Boers and Hans Driessen, presented at Fusion 2007, Québec, Canada 6 Mixed Labelling in Multi Target Particle Filtering by Yvo Boers, Egils Sviestins and Hans Driessen, submitted to IEEE Transactions on Aerospace and Electronic Systems 7 MAP Estimation in Particle Filter Tracking by Hans Driessen and Yvo Boers presented at IET target tracking event April 2008 in UK. 8 Particle Filters and Data Association for Multi-Target Tracking by Mats Ekman, presented at Fusion 2008, Cologne, Germany 9 Sequential MAP Estimation in Non-Linear Non-Gaussian Dynamic Systems by Hans Driessen and Yvo Boers. Submitted.
Saab Systems 16 (18) 5 LESSONS LEARNED 5.1 Technical Some lessons that have been learnt are: Application driven development can lead to solutions of far wider generality than what is needed by the original issues Significant progress has been made in the ground target tracking capabilities, but, owing to problem complexity there is still great potential for improvement. With more accurate state representations and sensor and target models follow a rapidly growing need for particles. The required number of particles can easily become unrealistically large. Elaborate measurement models do not always give correspondingly superior results. Recording live data from real sensors requires much planning and preparations. 5.2 Project Management The objectives for the project derives from a bottom-up perspective and. due to this approach and an early contact with the two countries MoD we went very quickly from proposal to a singed contract. The few numbers of IE involved have of course been an advantage not only during the contract negotiations but also during the performance of the project. We learned, during the contract negotiations, that the commercial representatives from WEAO Research Cell, later EDA are not only very skilled but also very helpful when interpreting the contract during the performance of the project. The management of the project has been performed via monthly Project Management Meetings. These meetings have mainly been in form of telephone conferences with the Project Manager as Chairman together with the two team leaders which has proven to be very efficient.
Saab Systems 17 (18) Even though Saab Systems and Thales, NL are competitors this project must be seen as a success as we have been able to focus on the actual tasks for the project and not been involved in business cases where we could have been in conflict with long time visions for the companies. A number of technical meetings, face to face meetings, have been conducted and most of these meetings have taken place at the same time as a conference or a seminar that we have been participating in, all to reduce extra costs for the project. We have learned to work together in a very open and positive atmosphere why we certainly are looking forward to be able to continue the cooperation not only between the companies but also together with the Dutch MoD and FMV.
Saab Systems 18 (18) 6 CONCLUSIONS AND RECOMMENDATIONS 6.1 Conclusions The objectives of IDEALS were defined as: 1. To identify, develop, test and evaluate core algorithms that are needed for an array of difficult problems in advanced single and multi sensor data processing. 2. To develop and build advanced applications in radar signal and data processing and fusion with those techniques. With regard to the first objective we have solved several problems of general interest concerning dense targets, state estimates, data association, and parallelisation. With regard to the second objective we have come a long way in two domains: dim targets in sea clutter, and ground targets with sparse sensor networks. There is no doubt that the objectives have been met. 6.2 Recommendations We have indeed made significant progress in the areas of common algorithms, and ground and sea applications. However, the studies have also revealed several topics where further studies could push the capabilities further. We consider IDEALS to be an exceptionally successful program. Among the success factors are good working conditions for the scientists, not too many parties, not too short duration, and last, but not least, continual interest and support from the recipients. Our recommendation is to continue with a similarly organised project, this time exploring the combination of particle filters with context data, for the purpose of taking detection and tracking capabilities to new levels.