Edition: Date: 29 May Main Report LEVEL 1 & 2 PERFORMANCE EVALUATION. NGSP S-Band PSR: DPR & (M)SSR: DTI529 - CBR

Transcription

1 Edition: A Date: 29 May 2015 Status: Draft Main Report LEVEL 1 & 2 PERFORMANCE EVALUATION NGSP S-Band PSR: DPR & (M)SSR: DTI529 - CBR

2 Level 1 & 2 Performance Evaluation Edition Date: 29 May 2015 Blank Page IEAP_SAMPLE_L1-L2_Main 2/14 IMS Revision 1.0: 23/05/2013

3 Level 1 & 2 Performance Evaluation Edition Date: 29 May 2015 EXECUTIVE SUMMARY As stated in EUROCONTROL (SUR.ET1.ST STD & SUR.ET1.ST STD-01-01) and ICAO (Doc 8071 Vol. III), comprehensive and continuous Radar coverage of high quality and reliability is essential for the uninterrupted provision of Radar services and the application of specific Radar separation standards. Surveillance systems require sophisticated and permanent performance monitoring in real time as well as ongoing subsystem-level Performance Evaluations. This System (Level 1) and Subsystem (Level 2) Performance Evaluation provides the results of the NGSP (CBR) ATC Radar as measured between 18th and 22nd May The CBR Radar is a co-located IE (M)SSR (DTI529) and PSR (DPR) System. The (M)SSR operates to 256NM and the PSR to 100NM. The Level 1 & 2 analysis and results can be found at Annex A and Annex B, respectively. The reference for performance analysis is bound to the IEAP Quality of Service (QoS). Similarly, the Commissioning Volume (CV) is also bound to the QoS. Therefore, analysis is conducted to 250NM for (M)SSR and 60NM for PSR. Further information pertaining to antenna back angles and flight levels assessed can be found at Annex A. As this type of analysis and report has been conducted annually, the information provided can be used to determine performance, improvements and degradations of the Radar. Further, this report acts as a reference/guide to areas that Maintenance can focus on for necessary improvement, if any. It also provides useful information and focus to OEM service personnel. As a whole, the (M)SSR radar falls within specification and performs well, with the exception of the level 1 RMS Range Statistics, albeit minor. Level 2 analysis revealed that the performance of the encoder system requires attention, as errors in azimuth exceed 500 meters to the East and North-West. This mechanical measurement has been unobtainable during several former opportunities and it is understood that the unit itself is not responsible for the corrective maintenance activity required; however, the unit and/or maintenance authority should begin pushing for corrective action. Further, jitter is also significant and will be adding unwanted strain to the interrogators ability to track aircraft. PSR performance is a safety critical system for management of Violation of Controlled Airspace (VCA) as well as aircraft separation within the coverage volume. Results detailed the overall PSR performance to be approx. 4-6% lower than that achieved during the 2013 optimisation. Admittedly not a momentous degradation, it does highlight the requirement for continuous monitoring and improvement. For lower level aircraft (below 1,500ft) the performance was 6-8% lower. Annex A details the performance in two states of configuration, of which both were satisfactory. As with any Radar System new or legacy, the key to providing Radar data of a high quality is ongoing maintenance and monitoring. Radar performance changes on a continual basis in real-time. This can be a result of a failure, degradation or the environment (i.e. weather, haze, new obstructions, other RF sources, telecommunications e.g. 4G or PCs, etc). A regime of preventative maintenance (Optimisation/Training) in conjunction with classical Radar performance improvement through Optimisation and performance assessment is essential. IEAP_SAMPLE_L1-L2_Main 3/14 IMS Revision 1.0: 23/05/2013

4 Level 1 & 2 Performance Evaluation Edition Date: 29 May 2015 DOCUMENT CHARACTERISTICS General Level 1 & 2 Performance Evaluation CBR Edition: A Edition Date: 29 May 2015 Status: Keywords: Abstract: Draft IEAP, RASS This is the report on the results of the RASS Performance Evaluation conducted on the NGSP Radar between 18th and 22nd May Contact Information Author: Reviewed: QA/Release: Contact Person: Person1 Person2 Person3 Person1 Tel: Address: support@ie-asia-pacific.com Document Control Information Document Name: IEAP_SAMPLE_L1-L2_Main_Draft A Path: Host System: Windows 7 Professional SP1 Software: Microsoft Word 2010 IEAP_SAMPLE_L1-L2_Main 4/14 IMS Revision 1.0: 23/05/2013

5 Level 1 & 2 Performance Evaluation Edition Date: 29 May 2015 DOCUMENT CHANGE RECORD Revision Date Reasons for change Pages Affected Approved by A 29/05/2015 Draft release for comment All J. De Vries IEAP_SAMPLE_L1-L2_Main 5/14 IMS Revision 1.0: 23/05/2013

6 Level 1 & 2 Performance Evaluation Edition Date: 29 May 2015 ACRONYM LIST ACP ADS-B ARP ASTERIX BLI CPI DFE DHM EDR HPD IEAP (M)SSR Pd PDD PE PRI PSR RASS RASS-R RASS-S RCD RCM REX RF RSR SOC SSR STC STDEV TCV ToD ToR U/S UTC VPD Azimuth Change/Count Pulses Automatic Dependent Surveillance - Broadcast Azimuth Reference Pulse All-purpose Structured EUROCONTROL Radar Information Exchange Bench Level Instruction Coherent Processing Interval Doppler Filter Threshold Editing Data Handling Module Extended Data Recorder Horizontal Polar Diagram IE Asia Pacific Pty Ltd Monopulse Secondary Surveillance Radar Probability of Detection Product Description Document Performance Evaluation Pulse Repetition Interval Primary Surveillance Radar Radar Analysis Support System Radar Analysis Support Systems Real-time measurements Radar Analysis Support Systems Site measurements Radar Comparator Dual Radar Comparator Mono Receiver Exciter Radio Frequency Relocatable Site Radar Standard Operating Conditions Secondary Surveillance Radar Sensitivity Time Control Standard Deviation Technical Commissioning Volume Time of Detection Time of Recording Unserviceable Universal Time Clock Vertical Polar Diagram IEAP_SAMPLE_L1-L2_Main 6/14 IMS Revision 1.0: 23/05/2013

7 Level 1 & 2 Performance Evaluation Edition Date: 29 May 2015 Blank Page IEAP_SAMPLE_L1-L2_Main 7/14 IMS Revision 1.0: 23/05/2013

8 Level 1 & 2 Performance Evaluation Edition Date: 29 May BACKGROUND The customer performs annual Radar Sensor Performance Evaluations (PE) on the customer owned Air Traffic Control (ATC) Radar Sensors, using the RASS suite of measurement tools. The purpose of these Performance Evaluations is to determine the quality of surveillance data a Radar System produces and to obtain a snapshot of the operating state of the Radar's subsystems. IE Asia Pacific Pty Ltd (IEAP) was contracted by customer to conduct the annual Performance Evaluation (PE) and report on the performance outcome of the CBR NGSP Radar Sensor between 18th and 22nd May Person1 and Person2 of IEAP performed this work, the analysis of the collected data and production of this report for the customer Current Situation Prior to the commencement of the Performance Evaluation, an onsite briefing was held with the key stakeholders to determine the performance and any outstanding issues of the Radar System. During the meeting the following topics were covered; Antenna Group o o Customer had replaced both Azimuth Drive Motors and Clutch assemblies, during the last annual maintenance visit. Local Maintainers expressed concern for the PSR performance, specifically the impact of lighting protection on the recently constructed stairs. Primary Surveillance Radar (PSR) o o o A recommendation from the previous report was to link the maps to the correct files. Local Maintainers advised this had been done and that an instruction (BLI) was put in place to ensure it was checked regularly, including after processor reboot. Local Maintainers advised that Permanent Echo #1 specifics were missing from the system. One possible replacement had been identified though not implemented. Local Maintainers were unsure that such a change was within their derestriction. IEAP advised that the SOC document detailed what level of authority the unit holds with regard to Permanent Echo adjustments. Local Maintainers advised that no optimisation had occurred since the previous PE. (Monopulse) Secondary Surveillance Radar ((M)SSR) o (M)SSR instability highlighted in the previous PE report, has been rectified through the replacement of both Interrogators to the later version. Site Communication and Data Interface (SCDI) o o Local Maintainers advised that since (M)SSR replacement, the processors are not combining all PSR targets to (M)SSR targets. The same anomaly exists for the CBR NGSP Radar. Raytheon are aware of the issue and have a solution awaiting rollout. Local Maintainers advised that instability issues have been rectified by replacing the Aurora I/O card. Plant equipment o No issues reported. IEAP_SAMPLE_L1-L2_Main 8/14 IMS Revision 1.0: 23/05/2013

9 Level 1 & 2 Performance Evaluation Edition Date: 29 May Test Methodology The tests conducted in a Performance Evaluation are detailed in the IEAP Radar Performance Evaluation Product Description Document. These tests have been identified through consultation with IE staff to provide a measure of the quality of surveillance data produced by the Radar System and a top down analysis of the (M)SSR, PSR and mechanical subsystems. Each test is performed in accordance with the procedures described in the current NGSP RASS BLI. In order to maximize the reliability of the real-time analysis conducted, certain criteria for data collections need to be met. The following table highlights the minimum criteria for data collection as set out by ICAO (Doc8071 Vol3) and EUROCONTROL (SUR.ET1.ST STD and SUR.ET1.ST STD-01-01). Minimum Duration Minimum Quantity of Data System Configuration Environment, Weather Traffic Recorded Quality High density traffic area (en-route or major TMA) 1 hour* Medium density traffic areas 2 hours* Low density traffic areas 4 hours* * Must include at least 50,000 data samples Probability of Detection 200 Chains >5 minutes per chain Accuracy Analysis 150 Chains Systematic Error Estimation 200 Chains >5 minutes duration in cover of >2 Radars Normal operational configuration for prevailing traffic and environment No Anoprop conditions or heavy Angel activity or abnormal conditions (e.g. jamming, interference) should be used to verify PSR/SSR overall performance Recordings shall be made in Peak Traffic times if possible >99% of recorded data shall be correctly recorded and available for chaining In analysing the data, a number of key factors are taken into account and eliminated to the extent possible, before finalising the results. These include, but are not limited to: 1. Coverage Volume 2. Filtering of Reference samples (e.g. ADS-B with FOM>5) 3. Environment (e.g. Barometric data) 4. Constraints of the Radar design. It is important to note, that there are a number of different root causes that can lead to an out-of-tolerance specification in the Level 1 results. It is for this reason that the Level 2 analysis is performed in order to better isolate the root cause or causes of the anomaly. IEAP_SAMPLE_L1-L2_Main 9/14 IMS Revision 1.0: 23/05/2013

10 Level 1 & 2 Performance Evaluation Edition Date: 29 May REFERENCES The following documents apply: EUROCONTROL SUR.ETI.ST STD Radar Surveillance in En-Route and Major Terminal Areas dated March 1997 EUROCONTROL SUR.ET1.ST STD Radar Sensor Performance Analysis June 1997 ICAO (Doc 8071 Vol. III) 3. ANNEXES Annex A Level 1 Performance Measurement against OEM, QoS and EUROCONTROL Specifications Annex B Level 2 Sensor Baseline Evaluation Annex C Test Equipment and Software List IEAP_SAMPLE_L1-L2_Main 10/14 IMS Revision 1.0: 23/05/2013

11 Level 1 & 2 Performance Evaluation Edition Date: 29 May CONCLUSIONS The conclusions drawn from analysis of the data collected during this Performance Evaluation are detailed as follows: Level 1 Level 2 PSR overall Pd was recorded at approx. 92.6% and meets the overall Pd specifications (OEM/QoS /EUROCONTORL). This is the reported baseline state IAW the Site Operating Conditions document (i.e. all filtering off). Slight degradation of the Pd, at all flight levels, was identified when compared to the Pd% results achieved during the 2013 optimisation, more so below 1500ft. Further, the use of circular polarisation does not adversely affect the performance. Both (M)SSR Interrogators meet the overall Pd specification (QoS/EUROCONTROL) at approx. 99.9% for both (M)SSR-A and (M)SSR-B. Both (M)SSR Interrogators meet the Systematic Error specification (QoS/EUROCONTROL) for Azimuth and Range Bias. Both Interrogators meet the Random Error specifications (QoS/EUROCONTROL) for Azimuth and Range (STDEV). Both Interrogators meet the RMS specification (OEM) for Azimuth, though not for Range. Both (M)SSR Interrogators meet the Jumps, Code Validation and False Target specifications (QoS/EUROCONTROL) across the board. The PSR Target and Weather receiver channels show sound characteristics for Gain, Sensitivity and Bandwidth. The operational Beam Switching and STC maps that were created in 2013, through optimisation, remain suitable. This is evident through the Video, Multi-Level and Pd% analysis. Small areas could be improved by adjustment and ongoing analysis, specifically around the 5NM mark and over the CBR area. Results indicate the Transmitter characteristics were satisfactory. PSR antenna horizontal beam pattern measurement shows that the 3dB, 10dB beamwidth and sidelode measurements were within tolerance. Both interrogators display satisfactory characteristics for Gain, Sensitivity and Bandwidth. Both (M)SSR Interrogators show satisfactory transmitter characteristics. The Pulse Repetition Frequency (PRF) for both interrogators was approx. 238Hz. Both Interrogators were operating a double interlace pattern of Mode 3/A C (MIP 9). Windowed Video recordings show good power levels for all opportunity aircraft and Site Monitors (SM). There were no signs of punch-through, splits or reflections. The SUM/DELTA relationship shows signs of asymmetry. Analysis of the SM data indicates both transponders are performing as expected. It is worthwhile however, monitoring the output power of each. The (M)SSR antenna demonstrates satisfactory characteristics for Uplink measurements. The Downlink measurement highlighted the SUM/DELTA asymmetry, though more specifically for peak amplitudes as the crossovers and notch depth were satisfactory. Overall performance has not changed since the previous Performance Evaluation. IEAP_SAMPLE_L1-L2_Main 11/14 IMS Revision 1.0: 23/05/2013

12 Level 1 & 2 Performance Evaluation Edition Date: 29 May 2015 Multi-level analysis of the PSR revealed the PSR/(M)SSR correlation issue discussed at the prebrief. The PSR only tracks are from a number of sources several can be tuned out without degrading performance. This will allow room for raising the PSR sensitivity. Several aircraft without transponders were recorded, emphasising the requirement for optimal PSR performance. Multi-level analysis of both Interrogators reveals that whilst level 1 specifications are met, the overall performance is not optimum, primarily due to the encoder and asymmetry of the DELTA antenna. Most importantly, the reported position of targets is not accurate in the two areas that align with the significant encoder errors. Mechanical analysis revealed both encoders had similar performance with a peak error of at approx. 340 azimuth. In between 60 and 90 azimuth, the encoders had an error of approx This is significant for traffic in those areas, and especially as range increases. There is Jitter present for the entire revolution, though most prominent where the antenna boom passes the northerly azimuth motor. The inclination of the turntable has remained the same for several years, with a peak error of approx at 230 azimuth and approx. 0.1 in the opposite direction. This result is the specification boundary. IEAP_SAMPLE_L1-L2_Main 12/14 IMS Revision 1.0: 23/05/2013

13 Level 1 & 2 Performance Evaluation Edition Date: 29 May RECOMMENDATIONS The following recommendations are made to improve the performance of the CBR NGSP Radar System: It is recommended that the unit begin seeking improvement opportunities in PSR performance. Tweaking of optimised maps and establishment of some Track Initiate Inhibit zones are two common areas that will ensure the PSR is returned to optimum performance Continue monitoring the (M)SSR Antenna (receive) performance. Any further degradation may add to unwanted Biases. Ensure OBA tables remain up to date. Ensure that the large encoder error is rectified. Due to the commonality of the recorded result across both encoders, the likely cause is the Rotary Joint. As Jitter is also present, several other mechanical components may also be contributors. In terms of minimum separation requirements, the azimuth error in some areas peaks at -0.10, particularly to the North-West. Using trigonometry to determine the error in nautical miles, we use the formula; Opp Adj = Tanθ. Rearranged we have; Tanθ Adj = Opp. i.e NM = 0. 01NM 40NM = 0.01NM (18.5 m) 200NM = 0.49NM (907.5 m) Upon rectification of the reported encoder error, it is advised that the following be completed at a minimum; o o o Turning gear measurement and analysis (Gyro and Inclinometer). Level 1 recording and analysis (ADS-B as the reference) for positional accuracy. Eccentricity file update for format converters. Investigate and rectify the inclination of the antenna pedestal. A general recommendation to all owners, operators and maintainers of Radar Systems is as follows: Optimisation should be performed on an ongoing basis in conjunction with daily system checks. Checking of on-line maps, PSR detections and Receiver noise (CFAR) are examples of things that can be checked regularly. Optimisation is preventative maintenance and ongoing efforts will assist in; o o Ensuring up-to-date and accurate maps and system parameter values are applied to the system for maximum performance. Ensuring that any PSR detection issues are known and can be addressed. This is essential for maintaining safe minimum separation standards for TMA operations. Note: that the TMD system can be utilised to support the majority of these tasks. IEAP_SAMPLE_L1-L2_Main 13/14 IMS Revision 1.0: 23/05/2013

14 Level 1 & 2 Performance Evaluation Edition Date: 29 May 2015 Blank Page IEAP_SAMPLE_L1-L2_Main 14/14 IMS Revision 1.0: 23/05/2013

15 ANNEX A LEVEL 1 PERFORMANCE MEASUREMENT AGAINST OEM, QOS AND EUROCONTROL SPECIFICATIONS NGSP S-Band PSR: DPR & MSSR: DTI529 - CBR Edition: A Date: 29 May 2015 Status: Draft

16 Annex A - Level 1 Performance Measurement Against OEM, QoS and EUROCONTROL Specifications Edition Date: 29 May 2015 Blank Page IEAP_SAMPLE_L1-L2_Annex A 2/14 IMS Revision 1.0: 23/05/2013

17 Annex A - Level 1 Performance Measurement Against OEM, QoS and EUROCONTROL Specifications Edition Date: 29 May LEVEL 1 PERFORMANCE MEASUREMENT AGAINST OEM, TPSA AND EUROCONTROL SPECIFICATIONS The following analysis results have been calculated from data that has been simultaneously recorded at various levels throughout the Radar using the RASS-R EDR recorder tool in the DHM. This enables direct comparison between different sources of data, as there is a common recording time stamp for the data. The types of data recorded were as follows: 1. ADS-B data was decoded from raw OMEGA channel video from each MSSR Interrogator 2. Combined data (MSSR and PSR) was recorded at the output of the Tracker 3. MSSR only data from the output of the Interrogator cabinet. All analysis is performed on a coverage measurement volume and has been calculated, based on the Time of Detection (ToD) of each target report. The analysis also takes into account the actual Line of Site coverage limitations for the Radar location. Figure 1 shows the coverage diagram from the NGSP Radar. In addition, the Barometric sounding files for the NGSP region (e.g Canberra Airport Observations at 12h00 18 May 2015) are downloaded into the analysis tools to ensure the most up-to-date corrections are available for the analysis. Figure 1: Coverage Diagram IEAP_SAMPLE_L1-L2_Annex A 3/14 IMS Revision 1.0: 23/05/2013

18 Annex A - Level 1 Performance Measurement Against OEM, QoS and EUROCONTROL Specifications Edition Date: 29 May 2015 Figure 2 below shows the Commissioning Volume (CV) applied in order to calculate the Probability of Detection (Pd) for PSR and MSSR as defined by the Quality of Service (QoS), with exception to the lower elevation angle for PSR. The CV for PSR only requires a +0.5 degree lower angle. For the purpose of comparing the current PSR performance (at different flight levels) with the previous optimisation results, a lower elevation angle is used i.e.-5.0 degrees (dependent on lowest aircraft detected). Figure 2: MSSR and PSR Technical Commissioning Volume Table 1 details, the quantity of data gathered to confirm minimum collection criteria set out by EUROCONTROL has been met. File Reference/s: RCD Numerous recordings from approx to 2359 (UTC) between 18th and 22nd May File Reference/s: RCM Numerous recordings from approx to 2359 (UTC) between 18th and 22nd May Table 1: Data Collection Values Reference Value EUROCONTROL Specification Data Samples RCM >125,000 on all occasions Minimum 50,000 Number of Chains >5 minutes System Configuration Environment, Weather Peak Traffic Included (Y/N) Recorded Quality RCD >157,000 on all occasions > 334 on all occasions Normal No Anoprop conditions Y 100% Pd/Systematic Errors >200 Accuracy Analysis >150 Normal operational configuration for prevailing traffic and environment No Anoprop conditions or heavy Angel activity or abnormal conditions (e.g. jamming, interference) should be used to verify PSR/SSR overall performance Recordings shall be made in Peak Traffic times if possible >99% of recorded data shall be correctly recorded and available for chaining IEAP_SAMPLE_L1-L2_Annex A 4/14 IMS Revision 1.0: 23/05/2013

19 Annex A - Level 1 Performance Measurement Against OEM, QoS and EUROCONTROL Specifications Edition Date: 29 May PSR Performance The quality parameters of the primary sensor are calculated based on continuous chains of MSSR target reports greater than 5 minutes in run length within the PSR coverage volume. These results are contained in the following sections Probability of Detection The PSR Probability of Detection (Pd) is calculated for the coverage volume of the PSR Radar by using the RASS-R RCM module. Figure 3 shows a tabular display of overall PSR Pd based on the Time of Detection (ToD), from an overnight data collection. The legend on the image indicates % Pd and in terms of the TPSA specification any black cells (below 80% Pd) represent a failure to meet specification, while Grey cells indicate there is insufficient data in those areas to conduct an analysis. As per Table 2 the PSR was recorded at approx. 92.6% probability of detection. Figure 3: PSR Pd Overall Table 2: Probability of Detection PSR Overall 18 May 2015 Parameter Probability of Detection (%) Measured Result (Time of Detection) OEM Specification QoS Specification EUROCONTROL Specification 92.6% >80 >80 >90 IEAP_SAMPLE_L1-L2_Annex A 5/14 IMS Revision 1.0: 23/05/2013

20 Annex A - Level 1 Performance Measurement Against OEM, QoS and EUROCONTROL Specifications Edition Date: 29 May 2015 For information only, Figure 4 represents the performance out to 40NM as this is generally as far as local TMA operations extend to. The result is shown at Table 3. Figure 4: PSR Pd 40NM Table 3: Probability of Detection 40NM, 18 May 2015 Parameter Probability of Detection (%) Measured Result (Time of Detection) OEM Specification QoS Specification EUROCONTROL Specification 93.4% >80 >80 >90 IEAP_SAMPLE_L1-L2_Annex A 6/14 IMS Revision 1.0: 23/05/2013

21 Annex A - Level 1 Performance Measurement Against OEM, QoS and EUROCONTROL Specifications Edition Date: 29 May 2015 Analysis was also performed at various Flight Levels. The Pd% at various Flight Levels is shown at Table 4 and includes previously reported Pd% results and alternate configurations. NOTE The number of samples used for each flight level block is only a relative proportion of the total sample recorded and can only be used as an indicative performance % against the respective specifications. Analysis was conducted using the QoS Coverage Volume. Air traffic density would be slightly different for each night of recording. Table 4: Pd% at various flight levels Flight Level 2013 IEAP Opt. DFE Off, Linear Enabled May 2015 DFE Off, Circular Enabled May 2015 DFE Off, Linear Enabled 500-1,500 ft 82.8% 75.9% 76.7% 1,500-5,000 ft 90.8% 88.1% 87.1% 5,000-10,000 ft 95.9% 91.9% 92.1% 10,000-15,000 ft 99.1% 92.7% 90.5% 15,000-25,000 ft 98.9% 93.3% 93.2% 25,000-50,000 ft Not Measured 91.1% 90.8% ,000 ft (TPSA) >96.7% 90.1% 92.6% Legend: Less than 80% = RED Greater than 90% = Green Greater than 80% and Less than 90% = Black Table 5 details the average percentage of traffic for varying Flight Levels. The information highlights the importance of optimising the PSR for all heights in the coverage volume. Table 5: Average Percentage of Traffic Flight Level Air Traffic % <500ft 2.06% 500 1,500ft 5.43% 1,500 5,000ft 12.08% 5,000 10,000ft 11.21% 10,000 15,000ft 6.98% 15,000 25,000ft 22.26% > 25,000ft 39.98% IEAP_SAMPLE_L1-L2_Annex A 7/14 IMS Revision 1.0: 23/05/2013

22 Annex A - Level 1 Performance Measurement Against OEM, QoS and EUROCONTROL Specifications Edition Date: 29 May MSSR Performance The following section contains the quality parameters for the MSSR Probability of Detection Figure 5 shows a tabular display of MSSR A Pd based on the Time of Detection (ToD), from an overnight data collection. Black cells indicate MSSR Pd below 95% while Grey cells indicate there is insufficient data in those areas to conduct an analysis. The legend on the image indicates % Pd and in terms of the TPSA specification any Red, Orange or Black cells represent a failure to meet the specification. As per Table 6, the Pd for MSSR-A is approx. 99.9%. Figure 5: MSSR Pd MSSR-A Table 6: Probability of Detection MSSR-A 18 May 2015 Parameter Probability of Detection (%) Measured Result (Time of Detection) OEM Specification QoS Specification EUROCONTROL Specification 99.9% N/A >97 >97 IEAP_SAMPLE_L1-L2_Annex A 8/14 IMS Revision 1.0: 23/05/2013

23 Annex A - Level 1 Performance Measurement Against OEM, QoS and EUROCONTROL Specifications Edition Date: 29 May 2015 Figure 6 shows a tabular display of MSSR B Pd based on the Time of Detection (ToD), from an overnight data collection. As per Table 7, the Pd for MSSR-B is approx. 99.9%. Figure 6: MSSR Pd MSSR-B Table 7: Probability of Detection MSSR-B 19 May 2015 Parameter Probability of Detection (%) Measured Result (Time of Detection) OEM Specification QoS Specification EUROCONTROL Specification 99.9% N/A >97 >97 IEAP_SAMPLE_L1-L2_Annex A 9/14 IMS Revision 1.0: 23/05/2013

24 Annex A - Level 1 Performance Measurement Against OEM, QoS and EUROCONTROL Specifications Edition Date: 29 May Positional Accuracy Analysis Systematic Errors The position of MSSR targets are checked for systematic errors (bias) with the RASS-R RCD. ADS-B target reports are used as a reference and are filtered for faulty transponders and to pass only reports with a figure of merit greater than five. The MSSR target reports are then compared against the reference trajectories to determine the range and azimuth biases. Tables 8 & 9 show the results of the analysis. An XY representation of the Azimuth and Range Error dispersion is shown in Figures 7 & 8. Figure 7: Azimuth and Range Error MSSR-A Table 8: Systematic Errors MSSR-A 18 May 2015 Parameter Azimuth Bias (degrees) Range Bias (metres) Measured Result (Time of Detection) OEM Specification QoS Specification EUROCONTROL Specification 0.02 N/A <0.1 < N/A <100 <100 IEAP_SAMPLE_L1-L2_Annex A 10/14 IMS Revision 1.0: 23/05/2013

25 Annex A - Level 1 Performance Measurement Against OEM, QoS and EUROCONTROL Specifications Edition Date: 29 May 2015 Figure 8: Azimuth and Range Error MSSR-B Table 9: Systematic Errors MSSR-B 19 May 2015 Parameter Azimuth Bias (degrees) Range Bias (metres) Measured Result (Time of Detection) OEM Specification QoS Specification EUROCONTROL Specification 0.04 N/A <0.1 < N/A <100 <100 IEAP_SAMPLE_L1-L2_Annex A 11/14 IMS Revision 1.0: 23/05/2013

26 Annex A - Level 1 Performance Measurement Against OEM, QoS and EUROCONTROL Specifications Edition Date: 29 May Random Error Analysis The residual random errors in the measured position of the MSSR target reports are calculated by the RCD as a Standard Deviation (STDEV) result. The results are displayed in Tables 10 & 11 below. Table 10: Random Errors MSSR-A 18 May 2015 Parameter Azimuth Errors (degrees STDEV) Range Errors (metres STDEV) Measured Result (Time of Detection) OEM Specification QoS Specification EUROCONTROL Specification 0.04 N/A <0.08 < N/A <70 <70 Table 11: Random Errors MSSR-B 19 May 2015 Parameter Azimuth Errors (degrees STDEV) Range Errors (metres STDEV) Measured Result (Time of Detection) OEM Specification QoS Specification EUROCONTROL Specification 0.04 N/A <0.08 < N/A <70 < RMS Errors The NGSP OEM Specification does not specify performance in the bias and random error (STDEV) format of the parameters listed in paras and Instead, it expresses the azimuth and range error performance as RMS values, which effectively combines the bias and random error results as a single result. The results as calculated by the RCD are displayed in the Tables 12 & 13 below. Table 12: RMS Errors MSSR-A 18 May 2015 Parameter Azimuth Errors (degrees RMS) Range Errors (metres RMS) Measured Result (Time of Detection) OEM Specification QoS Specification EUROCONTROL Specification 0.04 <=0.07 N/A N/A 44.0 <=37 N/A N/A Table 13: RMS Errors MSSR-B 19 May 2015 Parameter Azimuth Errors (degrees RMS) Range Errors (metres RMS) Measured Result (Time of Detection) OEM Specification QoS Specification EUROCONTROL Specification 0.05 <=0.07 N/A N/A 45.4 <=37 N/A N/A IEAP_SAMPLE_L1-L2_Annex A 12/14 IMS Revision 1.0: 23/05/2013

27 Annex A - Level 1 Performance Measurement Against OEM, QoS and EUROCONTROL Specifications Edition Date: 29 May Jumps Errors attributed as jumps are calculated in the RCM and are provided as a ratio value in Tables 14 & 15. Parameter Measured Result (Time of Detection) Table 14: Jumps MSSR-A 18 May 2015 OEM Specification QoS Specification EUROCONTROL Specification Jumps % 0.00 N/A <0.05 <0.05 Parameter Measured Result (Time of Detection) Table 15: Jumps MSSR-B 19 May 2015 OEM Specification QoS Specification EUROCONTROL Specification Jumps % 0.00 N/A <0.05 < Data Validity Analysis The data from the MSSR is processed by the RCM to determine the quality of correct code detection. Correct code means the mode A/C code value corresponds to the current correct value for the associated trajectory. Tables 16 & 17 detail the results of the valid Mode A and valid Mode C analysis. Table 16: Valid Code Detection MSSR-A 18 May 2015 Parameter Valid and Correct A Code (%) Valid and Correct C Code (%) Valid and Incorrect A Code (%) Valid and Incorrect C Code (%) Measured Result (Time of Detection) OEM Specification QoS Specification EUROCONTROL Specification N/A >98 > N/A >96 > N/A <0.1 < N/A <0.1 <0.1 Table 17: Valid Code Detection MSSR-B 19 May 2015 Parameter Valid and Correct A Code (%) Valid and Correct C Code (%) Valid and Incorrect A Code (%) Valid and Incorrect C Code (%) Measured Result (Time of Detection) OEM Specification QoS Specification EUROCONTROL Specification N/A >98 > N/A >96 > N/A <0.1 < N/A <0.1 <0.1 IEAP_SAMPLE_L1-L2_Annex A 13/14 IMS Revision 1.0: 23/05/2013

28 Annex A - Level 1 Performance Measurement Against OEM, QoS and EUROCONTROL Specifications Edition Date: 29 May False Target Analysis The RCM compares individual target reports against the reconstructed trajectory. If the report falls outside of a predetermined window the report is classified as a false target. False targets are then classified by type and are listed in Tables 18 & 19 below. Table 18: False Target Results MSSR-A 18 May 2015 Parameter Measured Result (Time of Detection) OEM Specification QoS Specification EUROCONTROL Specification Multiple Targets % 0.02 N/A <0.3% <0.3% - reflections % 0.00 N/A <0.2% <0.2% - sidelobes % 0.02 N/A <0.1% <0.1% - splits % 0.00 N/A <0.1 % <0.1% Table 19: False Target Results MSSR-B 19 May 2015 Parameter Measured Result (Time of Detection) OEM Specification QoS Specification EUROCONTROL Specification Multiple Targets % 0.01 N/A <0.3% <0.3% - reflections % 0.00 N/A <0.2% <0.2% - sidelobes % 0.01 N/A <0.1% <0.1% - splits % 0.00 N/A <0.1 % <0.1% IEAP_SAMPLE_L1-L2_Annex A 14/14 IMS Revision 1.0: 23/05/2013

29 Annex B LEVEL 2 SENSOR BASELINE EVALUATION NGSP S-Band PSR: DPR & (M)SSR: DTI529 - CBR Edition: A Date: 29 May 2015 Status: Draft

30 Blank Page IEAP_SAMPLE_L1-L2_Annex B 2/40

31 1. LEVEL 2 SENSOR BASELINE EVALUATION 1.1 PSR Performance RASS measurements were conducted on the Primary Surveillance Radar (PSR) system Receiver Sensitivity The purpose of this measurement is to determine the Receiver's performance across the entire dynamic range from saturation to noise floor. The test is equivalent to the Minimal Discernible Signal (MDS) checks, whereby a known signal is injected into the Low Noise Amplifier (LNA) or High Power Coupler. The injected signal is then sampled at the latest stage possible in the Analogue Receiver and converted to LOG video for measurement. The result is what is termed the Receiver Sensitivity or Gain curve. This curve provides important information pertaining to the receivers characteristics such as: - Gain - Dynamic Range - Saturation Point - Noise Floor (MDS) As this curve provides a correlation between input and output of the Analogue Receiver, the curve is used for further testing to assign a power level to the voltage read, and in particular, receiver video recordings. Table 1 details the results of the receiver sensitivity measurements. Figure 1 thru Figure 3 display the gain curves. Table 1: Receiver Sensitivity Curve - Results Noise Floor Saturation Dynamic Range High -109dBm -49dBm 60dB Low -109dBm -49dBm 60dB Weather -109dBm -49dBm 60dB Specification -109dBm -49dBm* 60dB *The specification listed for saturation is derived on from the Dynamic range and Noise Floor values. IEAP_SAMPLE_L1-L2_Annex B 3/40

32 Figure 1: PSR - Receiver Linearity High Beam F1 Figure 2: PSR - Receiver Linearity Low Beam F1 IEAP_SAMPLE_L1-L2_Annex B 4/40

33 Figure 3: PSR - Receiver Linearity Wx Beam F1 Comments: Results are satisfactory, no anomalies identified. IEAP_SAMPLE_L1-L2_Annex B 5/40

34 1.1.2 Receiver Bandwidth Each Receiver was swept across its bandwidth to determine frequency response. A bandwidth measurement is taken to ascertain whether or not the Receiver is tuned in frequency to avoid inadvertently accepting an echo from the corresponding Transmitter. The injection and measurement points are identical to the Receiver calibration measurements. This test provides the serviceability of the band pass filter sections at the front end of the Receiver. The RFA Transmitter is set to transmit a pulse at the specified power value, and the frequency is swept between two chosen limits. The RFA continuously samples the output of the Radar Receiver video signal and uses these values to build a dbm versus frequency table. The resulting graph can be analysed to determine the 3dB bandwidth. Figures 4 through to Figure 6 show the Receiver bandwidth for PSR-B. Table 2 details the results of bandwidth measured for each PSR receiver. High Low Weather Specification Table 2: Receiver Bandwidth 3 db Bandwidth 1.33 MHz 1.30 MHz 1.29 MHz Not Specified IEAP_SAMPLE_L1-L2_Annex B 6/40

35 Figure 4: PSR - Receiver Bandwidth - High Beam F1 Figure 5: PSR - Receiver Bandwidth - Low Beam F1 IEAP_SAMPLE_L1-L2_Annex B 7/40

36 Figure 6: PSR - Receiver Bandwidth - Wx Beam F1 Comments: Results are satisfactory, no anomalies identified. IEAP_SAMPLE_L1-L2_Annex B 8/40

37 1.1.3 Transmitter Characteristics The Transmitter characteristics are measured from the Power Monitor. Pulse shape and forward power are recorded and analysed. The results for the Power Measurement are detailed at Table 3. The output pulse recorded for each Transmitter is shown in Figure 7 and Figure 8. Table 3: PSR Power Measurements Pulse Width Forward Power PSR-B (short) 1.5 μs 16.6 KW PSR-B (long) 79.8 μs 16.6 KW Specification 1.6μS ± (short) 82μS ±5.5 (long) 16kW Figure 7: PSR-B Forward Power Short Pulse Comments: Figure 8: PSR-B Forward Power Long Pulse The forward power of the PSR Transmitter is as expected. IEAP_SAMPLE_L1-L2_Annex B 9/40

38 1.1.4 Sectorial STC Measurements Figure 9: PSR Sectorial STC A Sectorial STC Recording can provide a great deal of useful information on how the STC and Beam Combining (BC) Maps have been configured for each Receiver. It gives a 360 graphical representation of the programmed STC range/azimuth bins. STC and Beam Combining are the Analogue Receivers first line of defence against unwanted PSR returns such as hills and buildings, i.e. clutter. The Sectorial STC measurement result consists of an RF input power, versus timetable presented, versus azimuth. To be able to measure the gain versus time delay, the RFA needs to be synchronised to the interrogation signal of the Radar under test. The interrogation trigger signal is used both as a start of range and azimuth indicator. The RFA will inject pulses of fixed (selectable) power level into the Receiver. The PSR s analogue video output signal is then sampled by the RFA. This amplitude is passed through the calibration curve in order to build the STC curve: A gain versus time delay curve. Figures 9 and 10 shows the STC map and Beam Switching map respectively. Comments: These are the Optimised maps from Both maps are performing well and evidence of suitability can be seen at the Clutter and Multi- Level analysis. Figure 10: PSR Sectorial Beam Switching IEAP_SAMPLE_L1-L2_Annex B 10/40

39 1.1.5 Windowed Video Recordings After the Receiver s calibration curve is known, a windowed video (azimuth by range) recording can provide useful information on the performance of the Radar and the RF environment. The windowed video recordings are reported to demonstrate problematic areas such as Receiver saturation (red). In this view, small areas of concern can easily be identified. Additionally, windowed video can be used to demonstrate to technicians where optimisation is required. Figures 11 and 12 show a window video recording comparing the clutter amplitude between normal operation and with maps off. Figure 11: PSR Windowed Video Maps Off IEAP_SAMPLE_L1-L2_Annex B 11/40

40 Figure 12: PSR Windowed Video Normal Operation Comments: There are several small areas in coverage that have the potential for performance improvement through further analysis and ongoing optimisation. A noteworthy area of ground clutter exists to the south and has the receiver bordering saturation. This area is highlighted by the red circle and was specifically switched to low beam (2013 Opt) to better detect targets over the Gundaroo airfield. The majority of effects caused by the high levels of clutter will be taken care of by the Doppler filters and adaptive clutter map. Further, no issues were identified at data level, though ongoing monitoring is recommended. IEAP_SAMPLE_L1-L2_Annex B 12/40

41 1.1.6 Clutter Map Recordings Recording of clutter is designed to determine where Receiver saturation occurs and to assist in isolating the cause of missed PSR detections, essentially testing the effectiveness of the clutter maps. Measurement of clutter is taken at the output of the Receiver s IF Amplifier. The Receiver Sensitivity Curve is required during analysis to determine the intensity of the clutter returns. Clutter recordings are almost identical to Windowed Video recordings, except they reveal an overall view. The RASS clutter map processor cancels moving targets by integrating video over eight successive scans to identify the environmental clutter. Both tests are carried out with maps off and maps on (Normal operation). The results detail the level of attenuation used to cancel clutter and indicate whether or not the maps are optimised correctly. Results of the clutter analysis are shown in Figures 13 and 14. Figure 13: PSR - Clutter Recording Maps off Comments: The effective clutter suppression can be seen here. The Beam and STC maps are operating as expected. Minor improvements are possible however; the effect on overall performance will be negligible. Figure 14: PSR - Clutter Recording Normal Operation IEAP_SAMPLE_L1-L2_Annex B 13/40

42 A horizontal polar representation of the previous PPI clutter view is shown in Figures 15 and 16. In this view, the bearing and severity of the clutter is easier to determine. Additionally, the effect of applying too much attenuation to supress clutter can be visualised. Figure 15: PSR - Clutter Recording Maps off Figure 16: PSR - Clutter Recording Normal Operation Comments: The white circle highlights the area over Gundaroo region purposely switched to Low Beam so that targets can be detected around the Gundaroo Airfield. The red circles highlight minor areas where improvements are possible through further fine tuning of the Beam and STC maps. IEAP_SAMPLE_L1-L2_Annex B 14/40

43 PSR Antenna Measurements Many characteristics of the PSR system are determined by recording the emitted RF pulses from the Radar under test and the strength of signals at the RFA Receiver. The Uplink measurement provides Horizontal Polar Diagrams (HPD) of the PSR antenna in its operational environment by recording the pulses of the Radar. For this purpose the Radar Field Analyser is set-up in the field with no connection to the Radar. The RFA's antenna picks up the Radar signal and the Uplink software calculates the HPD antenna diagram from this data. The pulses are recorded in a condensed format (pulse mode) or in detail (scope mode). After the recording the HPD is extracted from the data through fingerprinting (stagger). This antenna measurement is completed to check whether the correct horizontal antenna pattern is being radiated as per the OEM specification. Without the correct antenna pattern for the specific Radar type, detection and positioning of aircraft will prove difficult for the Receiver PSR Uplink Measurements The PSR uplinks were recorded at the same locations as the SSR uplinks. Before each Horizontal Polar Diagram (HPD) recording was made, a scope recording was conducted to test each site for presence of significant multipath and other interference. Table 4 details the results for Beamwidth measured at each site. Table 4: Beamwidth measured *Side-lobes 3dB Beamwidth 10dB Beamwidth PRF Ainslee Hill >24dB Hz Phillip Ave >25dB Hz Gunning Rd N/A N/A N/A N/A Specification >25dB 1.4 ±0.05 (S-Band) 1.2 ±0.15 (L-Band) Not specified Site Specific *The specification >xx db means better than the specified value down from peak when referenced to the level of the main lobe. IEAP_SAMPLE_L1-L2_Annex B 15/40

44 The resulting horizontal antenna pattern is shown in Figure 17 below. Figure 17: PSR - Uplink HPD Ainslee Hill The stagger pattern recording from one of the sites can be seen below in Figure 18. Figure 18: PSR - Stagger Pattern Comments: Results are as expected, no anomalies identified. It was not necessary to measure the transmit pattern of the PSR Antenna from Gunning Rd, as the other recordings were sufficient in demonstrating the sound antenna performance. IEAP_SAMPLE_L1-L2_Annex B 16/40

45 1.2 SSR Performance Receiver Sensitivity The Receiver Sensitivity is measured by recording Receiver video levels against 1090 MHz applied at the Receiver's input. The purpose of the measurement is to verify the Receiver sensitivity, gain and linearity tracking between the three channels in each Receiver and to build a calibration table required for further RASS video recordings. The SSR Receiver Calibration includes the three extremely important baseline examinations of Sensor Functionality: 1. Amplitude (Gain) Matching 2. Tangential Sensitivity 3. Dynamic Range. The test procedure involves injecting a known signal into the high power coupler and measuring the performance at the output of the Analogue Receiver. The method of mono-pulse processing used in this type of Radar system requires accurately matched gain responses from both the SUM and DELTA channels. A mismatch of the SUM and DELTA response for a given RF input level could result in poor azimuth resolution by affecting the Interrogators de-garbling performance. During the Performance Evaluation, the gain (mv/db) of each Receiver is recorded against the position the Receiver is located and the results are detailed in Table 5. The serial number of each Receiver is annotated at Table 6. Table 5: (M)SSR Receiver Sensitivity Curve - Results Interrogator SIGMA DELTA OMEGA Dynamic Range Tangential Sensitivity (M)SSR-A 25.0 mv/db 25.0 mv/db 25.0 mv/db 82.7 db dbm (M)SSR-B 25.0 mv/db 25.3 mv/db 25.2 mv/db 82.9 db dbm Specification Unknown Unknown Unknown 80 db <-90 dbm Table 6: Receiver Amplifier Serial Numbers Interrogator SUM DELTA OMEGA (M)SSR-A (M)SSR-B IEAP_SAMPLE_L1-L2_Annex B 17/40

46 Figure 19 and Figure 20 show the Sensitivity Curve of (M)SSR-A and (M)SSR-B respectively. Figure 19: (M)SSR-A Receiver Sensitivity Curve Figure 20: (M)SSR-B Receiver Sensitivity Curve Comments: Results are as expected, no anomalies identified. IEAP_SAMPLE_L1-L2_Annex B 18/40

47 1.2.2 Receiver Bandwidth The Receiver is swept across its operating bandwidths to record their frequency response. The measurement point is identical to that of the Receiver Sensitivity measurements. The purpose of this test is to ensure that the Receivers are centred on 1090MHz with sufficient bandwidth at the 3dB point to receive aircraft responses from MHz as per ICAO requirements. Table 7 details the results whilst Figure 21 and Figure 22 show the frequency response of (M)SSR-A and (M)SSR-B. Table 7: Receiver Bandwidth 3 db Bandwidth 40 db Bandwidth 60 db Bandwidth (M)SSR-A MHz MHz MHz (M)SSR-B MHz MHz MHz Specification 1086±0.5 & 1094±0.5 Mean 1090MHz±0.5 >1078 & <1102 MHz >1065 & <1115 MHz Figure 21: (M)SSR-A Receiver Bandwidth IEAP_SAMPLE_L1-L2_Annex B 19/40

48 Figure 22: (M)SSR-B Receiver Bandwidth Comments: Results are as expected, no anomalies identified. IEAP_SAMPLE_L1-L2_Annex B 20/40

49 1.2.3 Transmitter Characteristics The purpose of this test is to check whether the Transmitted pulses are within specification. Aircraft rely on receiving correctly sized and spaced pulses for it to reply. An out of tolerance spacing for example, may cause the aircraft to reply incorrectly or disregard the interrogation. Pulse Width and Spacing are recorded and analysed and an example of the (M)SSR pulse train is shown at Figure 23. Figures 24 and 25 show the recorded pulse train for SUM channel for both (M)SSR-A and (M)SSR-B, respectively. Figure 23: Example Pulse train for P1, P3 and P2 The results obtained are shown below in Table 8 and Table 9. Table 8: (M)SSR Power and Pulse Width (M)SSR -A (M)SSR -B Power (dbw) Power (kw) Pulse Width (μs) P P P3 3/A P3 - C P P P3 3/A P3 - C Specification >32dBW >1.58KW 0.8μS ± 0.1μS Table 9: (M)SSR Pulse Spacing P2 (μs) P3-3/A (μs) P3 C (μs) (M)SSR -A (M)SSR -B Specification 2μS ± 0.1μS 8μS ± 0.1μS 21μS ± 0.1μS IEAP_SAMPLE_L1-L2_Annex B 21/40

50 Figure 24: (M)SSR-A - SUM Channel Tx characteristics Figure 25: (M)SSR-B - SUM Channel Tx characteristics Comments: Results recorded are as expected, no anomalies identified. Result accuracy for power the measurement is approx. ±0.4dB. IEAP_SAMPLE_L1-L2_Annex B 22/40

51 1.2.4 Windowed Video Recording Windowed video (azimuth by range) recordings of the log video from the online Receiver system provide useful information on the performance of the (M)SSR, the test Transponder and the RF environment. Recordings were made on several stationary targets, to provide an indication of the Radar s Mode Interlace Pattern (MIP) and the Transponder strength to each interrogation. The Pulse Repetition Frequency and Site Monitor power for each Interrogator can be found at Table 10. Figures 26 thru 28 show the Site Monitor and targets of opportunity. Table 10: Pulse Repetition Frequency PRF SM Power at Rx (M)SSR-A_ Hz dbm (M)SSR-B_ Hz dbm Specification 262 Hz (CBR) Site Specific Figure 26: (M)SSR-A - Windowed Video Site Monitor (A7757) MIP9 IEAP_SAMPLE_L1-L2_Annex B 23/40

52 Figure 27: (M)SSR-B - Windowed Video Site Monitor (A7745) MIP9 Figure 28: (M)SSR-B - Windowed Video - Opportunity Target (A3754) MIP9 Comments: The power of both site monitors is considered slightly low and whilst anomalies have not surfaced to date, monitoring for further degradation is advised. IEAP_SAMPLE_L1-L2_Annex B 24/40

53 1.2.5 Antenna Measurements Poor antenna performance means poor (M)SSR performance. The effects of reduced gain, high side lobes and reflected signals received off the ground or via nearby objects are difficult to overcome by subsequent processing in the Radar s Receiver. One of the least monitored components in an (M)SSR system is the antenna (M)SSR Uplink Measurements To generate an accurate antenna pattern of the Radar s antenna system, correct transmitter operation must first be confirmed. Transmitter operation is confirmed by examining Radar pulses recorded from the RF environment. Reliable antenna patterns also require the absence of multipath, if at all possible. This too is determined by examination of recorded RF pulses. Prior to making a HPD measurement at each site, a scope recording was made to ensure the pattern is not affected by multipath interference. The results from the measurement analysis are detailed at Table 11. Figures 29 thru 31 show the (M)SSR HPD recorded at separate sites. Table 11: Uplink Results *Side-lobes 3dB Beamwidth 10dB Beamwidth SUM/OMEGA Crossover Ainslee Hill >25dB dB/-2.52 ; dB/3.05 Phillip Ave >25dB dB/-2.94 ; dB/3.13 Gunning Rd N/A N/A N/A N/A Specification >26dB 2.35 ±0.25 <4.5 Not Specified *The specification >xx db means better than the specified value down from peak when referenced to the level of the main lobe. Figure 29: (M)SSR - Uplink HPD Ainslee Hill IEAP_SAMPLE_L1-L2_Annex B 25/40

54 Figure 30: (M)SSR - Uplink HPD Phillip Ave Comments: Figure 31: (M)SSR - Uplink HPD Gunning Rd Results are as expected. There is a small side-lobe (left of bore-sight) though not causing any issues at present. It was not necessary to measure the transmit pattern of the (M)SSR Antenna from Gunning Rd, as the other recordings were sufficient in demonstrating the antenna transmit performance, of which is satisfactory. IEAP_SAMPLE_L1-L2_Annex B 26/40

55 Downlink Measurements The downlink HPD is useful for measuring the receive-only antenna pattern of the DELTA channel. Symmetrical SUM/DELTA crossovers are important for correct mono-pulse operation of the Radar. Results detailed at Table 12, Figure 32 shows the SUM (red) and DELTA (blue) pattern for the antenna. Table 12: Downlink Results Notch Depth Peak Amp/Location SUM/DELTA Crossover Ainslee Hill -25dB -1.1dB/-1.7 ; -2.1dB/ dB/-1.06 ; -2.96dB/1.08 Specification Not Specified Not Specified Not Specified Figure 32: (M)SSR Downlink HPD Comments: The DELTA (Blue) channel shows signs of asymmetry, particularly with regard to the peak amplitudes. An asymmetrical curve will add to the inaccuracy of target resolution. The crossovers points are as expected and satisfactory. IEAP_SAMPLE_L1-L2_Annex B 27/40

56 Uplink/Downlink Locations Each location is chosen carefully and should include both low and high elevation. Another important factor is the direction from the Radar in which the measurement is taken. Selecting each location as far apart (in relative angle from the Radar) as practically possible, helps determine whether any anomalies are caused by the Antenna or the Environment. It is normal for some relatively low angles to show small signs of punchthrough. These sites are used predominantly to isolate other antenna or environmental characteristics. During the recordings, the approximate average percentage of air traffic movement through relative antenna elevation is detailed at Table 13, whilst Table 14 and Figure 33 illustrate the three locations used for this site. Table 13: Average Percentage of Traffic Elevation Air Traffic < 0.0º 27.23% 0º - 1.5º 33.15% 1.5º - 3.0º 20.98% 3.0º - 5.0º 11.50% 5.0º º 6.00% > 10.0º 1.64% Table 14: Uplink/Downlink Locations Ainslee Hill Phillip Ave Gunning Rd Latitude Longitude Elevation 171m 193m 81m Relative Elevation angle -1.45º -0.54º -0.92º Figure 33: Google Earth Locations IEAP_SAMPLE_L1-L2_Annex B 28/40

57 1.3 Multilevel Data Analysis Multi-Level analysis put simply is displaying approximately 18 hours of aircraft detections (PSR & (M)SSR), for the purpose of finding areas of concern. For example, if all aircraft in one particular area are being missed, then the issue must be environmental or Radar related. If in the same instance only one aircraft was missed, it is possible that the one aircraft was physically undetectable and not the result of, for example poor Radar performance. All images shown in the Multi-Level analysis section are a product of loading data into the RASS-S Inventory tool, to visualise any anomaly/problem and conduct further investigation into the reason or cause of the issue PSR Data Analysis Shown below is a graphical representation of actual PSR replies (Blue) overlaying (M)SSR replies (Green). Within the PSR 80NM coverage, it is expected there should be a PSR detection to each (M)SSR reply. Please note that in a vertical view the azimuth is not shown, though it still contains all detections from degrees azimuth on the z-axis. The data displayed in Figure 34 is from an overnight recording commencing on the 18 May Figure 34: PSR Linear Polarisation, Filtering disabled Comments: In a baseline configuration, with Linear Polarisation selected, the PSR system performs very well. Several minor holes in coverage still exist, though primarily due to limitations of the Radar design and location. IEAP_SAMPLE_L1-L2_Annex B 29/40

58 The data displayed in Figure 35 is from an overnight recording commencing on 18 May The data has been filtered to display all (M)SSR only tracks within the PSR coverage volume and is simply an inverted version of Figure 34 (PPI view). Figure 35: (M)SSR Only Inside PSR Coverage Comments: The majority of blue tracks shown is due to the known PSR/(M)SSR correlation problem. The issue is typically more severe past the 30NM mark and is possibly compounded by encoder performance, associated jitter and the SUM/DELTA relationship curve i.e. the further out in range targets are, the more difficult it is for the (M)SSR to resolve accurately. IEAP_SAMPLE_L1-L2_Annex B 30/40

59 Shown below at Figures 36 and 37 are graphical representations of actual PSR only replies (Blue) overlaying (M)SSR replies (Green). This is primarily shown to demonstrate the importance of PSR especially with respect to Violations of Controlled Airspace (VCA). In some instances the (M)SSR Interrogator can fail to detect or track aircraft that have serviceable Transponders. In other cases, aircraft may not have a Transponder or it has been switched off. Another purpose for displaying recorded data this way is to alert local staff to the issues the PSR is suffering from for example; concentrated areas of false tracks that require attention/optimisation. Figure 36: PSR Only Tracks Comments: The majority of PSR only tracks are caused by the known PSR/(M)SSR correlation. Once this is rectified, a fresh recording should be analysed for confirmation. A higher concentration of PSR only tracks resides over the CBD of Gundaroo and is normal, as both the PSR and (M)SSR will generally struggle in this area. Planer View is used here to further demonstrate that the known correlation issue is global. Once rectified, the PSR only tracks will be drastically reduced. Whilst not shown, there were several occasions identified where targets were not cooperative with the (M)SSR interrogators, placing emphasis on the importance of PSR performance. Figure 37: Planar view IEAP_SAMPLE_L1-L2_Annex B 31/40

60 Figure 38 is a Google Earth image overlayed with the PSR only tracks. Figure 38: Nearby False tracks Comments: General aviation traffic (without transponders) can be seen on coast line routes and also across to Rottnest Island. Approach and departure routes can be seen where the PSR track has split from the (M)SSR track due to known correlation issue. Other areas show false tracks cause from road traffic, sea clutter and wind farms. Ongoing analysis of unwanted false tracks can determine their permanency and technicians can use tools available within radar software to remove (carefully). By reducing false track count, allows the sensitivity of the PSR to be raised further. IEAP_SAMPLE_L1-L2_Annex B 32/40

61 PSR False Track Analysis The PSR false track statistics can be found at Table 15 and Figure 39 below. The recorded data was with DFE off as is normally the case at CBR. The analysis excludes VCA and Permanent Echoes. Table 15: False Track Statistics Avg. False Tracks Max Min PSR Baseline 3.3/scan 18 0 PSR Cir. Polarisation 2.4/scan 18 0 Specification N/A 20/scan N/A Figure 39: False Tracks Baseline Configuration Comments: Maintenance technicians rarely utilise filtering, however Circular Polarisation is used when poor weather prevails. The purpose of analysing this data is to demonstrate to technicians the effects of the alternate configuration. The results show that no adverse effects come from switching to circular polarisation. Other areas of PSR performance are also not effected. IEAP_SAMPLE_L1-L2_Annex B 33/40

62 Permanent Echo Analysis Figures 40 and 41 show the results of monitoring the PSR Permanent Echoes (PE) #1 thru #4. Amongst other reasons, these echoes are used for alignment of the Radar when a site monitor is not available. Table 16 details the statistics for all Permanent Echoes. Table 16: STDEV Results Azimuth Range STDEV Azimuth STDEV Range PE # NM NM PE # NM NM PE # NM NM PE # NM NM Figure 40: Multi-level Data Permanent Echo #1 and #2 Figure 41: Multi-Level Data Permanent Echo #3 and #4 Comments: Permanent Echo #1 is not an ideal echo and should be replaced. The PSR struggles to hold a solid track for this Permanent Echo due to its physical nature. IEAP_SAMPLE_L1-L2_Annex B 34/40

63 1.3.2 (M)SSR Data Analysis Site Monitor Analysis An image of the (M)SSR Site Monitor (SM) analysis can be found at Figure 42. Table 17 details the SM statistics. Table 17: STDEV Results Bearing Range STDEV Azimuth STDEV Range (M)SSR-A_ (M)SSR-B_ Specification Unknown Figure 42: Multi-level Data Site Monitor 7757 Comments: Results are as expected, no anomalies identified. IEAP_SAMPLE_L1-L2_Annex B 35/40

64 Zero Code Analysis A zero (not present) code is where the aircraft does not reply to the interrogation, whether it is a Mode 3/A or a Mode C. The other reason for a zero code is the Interrogator cannot read the reply correctly due to Phasing, Gain Matching or Encoder problems, to name a few. Table 18 details the results. Table 18: Zero codes Avg. Zero C Max Zero C Avg. Zero A Max Zero A (M)SSR-A 1.07/scan /scan 1 (M)SSR-B 1.06/scan /scan 1 Specification Unknown Unknown Unknown Unknown Comments: Results are as expected, no anomalies identified Positional Accuracy Analysis The accuracy a reported target is an integral component of the Radars performance. At Figure 43, (M)SSR targets (green) overlay ADS-B targets (Magenta). As stated at Annex A, ADS-B reports are used as the reference due to their significant accuracy derived from transmitted GPS coordinates. Bias and Accuracy results are detailed in the Level 1 RCD analysis. Figure 43: Azimuth Bias error Comments: Various areas within the (M)SSR coverage volume show that the interrogator has not positioned the target accurately. With the current state of the encoder system (detailed further on) and the asymmetry of the SUM/DELTA curve, the accuracy of the interrogators will be degraded in areas. IEAP_SAMPLE_L1-L2_Annex B 36/40

65 1.4 Mechanical Analysis RASS provides a capability to measure azimuth Encoder performance and inclination of the antenna turntable assembly. Poor Encoder performance can impact overall Radar performance and make merging of sensor data in a Multi Radar Tracking (MRT) or data fusion system difficult. Encoder performance is measured by placing a three-axis Gyroscope on the antenna under test. The Gyroscope measures the actual rotation of the antenna system while RASS video recording equipment measures the reported position of the antenna via the Radar s Antenna Change/Clock Pulses (ACP) and Antenna Reference Pulses (ARP), created by the Encoder. Turntable level is measured by analysing data from an inbuilt Inclinometer packaged within the Gyroscope hardware Encoder Performance The Encoder was tested for correct operation. In the following gyro plots, the green line represents actual antenna position with respect to constant rotation, the blue line represents the position reported by the Radar (derived from the Encoder s ARP/ACP) and the red line represents the difference or error between actual antenna position and the Encoders reported antenna position. Table 19 details the performance result of both Encoders. Figures 44 & 45 show a snapshot of the recording. Table 19: Encoder Results Peak Error Azimuth Encoder X Encoder Y Specification *<0.1 N/A *The overall azimuth error of the Radar data output should be =< 0.1 degrees. Figure 44: Mechanical Encoder Analysis ENC-X IEAP_SAMPLE_L1-L2_Annex B 37/40

66 Figure 45: Mechanical Encoder Analysis ENC-Y NOTE: Measured Error; is the error calculated by the RASS software when the North Reference Pulse (NRP) is assumed to have zero error. Hence at 0 azimuth, there is zero error. In reality, zero error is actually at the azimuth the SM is located, which is the reference for a north alignment. Calculated Error; can be difficult to comprehend but essentially, it is the error calculated after taking into account the reference used to north-align the radar system, typically the SM. The net ACP (blue line) is shifted vertically up/down until the ACP has zero error (Difference between Blue and Green lines) at the SM. Further, the calculated error is proved when overlaying (M)SSR targets with a reference (ADS-B). Comments: In this case the calculated error is significant and has surfaced in the data as an azimuth bias, specifically at bearings 74 and 340, as shown in the Multi-level analysis. The performance of each Encoder is similar to one another, hinting that the cause of the large error is the Rotary Joint as it is common to both encoders. Rectifying the encoder performance issue will lower the positional accuracy results found in the level 1 RCD analysis. At bearings 74 and 330 the error is significant, as shown in the Multi-level analysis. Jitter is severe throughout rotation, but most significant when the boom passes the northerly azimuth motor. Recent maintenance indicates that the azimuth motors and clutch assemblies are not at fault, though this statement is not certain due to the possibility of suspect spares. IEAP_SAMPLE_L1-L2_Annex B 38/40

67 1.4.2 Turntable Inclination Table 20 details the inclination of the pedestal turntable. Figure 46 shows a snapshot of the recording. Table 20: Turntable Results Peak Error Azimuth Inclination Specification <0.1 N/A Figure 46: Mechanical Turntable Inclination Comments: The levelling of the pedestal has remained untouched for a number of years. The result shows that the antenna dips down by 0.1 at approx. 230, and points up by 0.1 at approx. 45. It was noted that one of the tower legs appears to have descended into the ground. Of course this was only an observation. For reference, it is the leg closest to the side entry door to the Radar cabin. IEAP_SAMPLE_L1-L2_Annex B 39/40