Fibre-to-the-telescope: MeerKAT, the South African precursor to Square Kilometre Telescope Array (SKA)

Transcription

1 Fibre-to-the-telescope: MeerKAT, the South African precursor to Square Kilometre Telescope Array (SKA) T. B. Gibbon a*, E. K. Rotich Kipnoo a, R. R. G. Gamatham a, A. W. R. Leitch a, R. Siebrits b, R. Julie b, S. Malan b, W. Rust b, F. Kapp b, T. L. Venkatasubramani b, B. Wallace b, A. Peens- Hough b, P. Herselman b a Dept. of Physics, Nelson Mandela Metropolitan University (NMMU), South Africa; b Square Kilometre Array South Africa Organization/ MeerKAT Engineering Office. Abstract. Scientific curiosity to probe the nature of the universe is pushing boundaries of big data transport and computing for radio telescopes. MeerKAT, the South African precursor to Square Kilometre Array (SKA), has 64 antennae separated by up to 12 km. By 2018, each antenna will stream up to 160 Gbps over optical fibre to a central computing engine. The antennae digitizers require highly accurate clock signals distributed with high stability. This paper outlines requirements and key design aspects of the MeerKAT network with timing reference overlay. Fieldwork results are presented into the impact of birefringence and polarization fluctuations on clock stability. Keywords: MeerKAT, Square Kilometre Array, telescope array, optical fibre Address all correspondence to: *Tim Gibbon, Nelson Mandela Metropolitan University (NMMU), Department of Physics, Optical Fibre Research unit, Summerstrand, Port Elizabeth, South Africa, Postal Code; 6031 Tel: ; Fax: ; Tim.Gibbon@nmmu.ac.za 1 Introduction The Square Kilometre Array (SKA) is a global science project in the field of radio astronomy. The telescope array will have around 50 times the sensitivity and times the survey speed of the best current telescopes. SKA will probe the origins of the universe through the study of neutral hydrogen in the universe from the epoch of re-ionization until present. This will provide insight into when the first stars and galaxies formed, how the galaxies evolved and add to our understanding of dark energy and dark matter. SKA will further facilitate the study of fundamental forces relating to pulsars, general relativity, gravitational waves, transients and evolution of cosmic magnetism 1. Optical fibre links form the backbone of the SKA. Optical communication is required to aggregate science data from individual antennae elements to central computing engines. Optical fibre further distributes centrally generate clock signals to individual antennae elements, as well as transfer monitoring and control signals. It is estimated 1

2 by completion of SKA Phase 2, the traffic generated by SKA will be more than 110 times the current global Internet traffic 2. It is further estimated that SKA will use enough optical fibre to wrap twice around the earth. A project of this scale presents exciting challenges and opportunities to the optical communication community. The SKA telescope is split across two continents, with MeerKAT telescope South Africa and ASKAP in Australia being the precursors. The SKA will be built in two phases. Phase 1 construction is set to begin in 2017 and to be completed by SKA Phase 1 consists of three individual elements: SKA1 Mid consisting of 254 dishes (64 MeerKAT dishes plus 190 SKA dishes); SKA1 Low consisting of 911 Low Frequency Aperture Array stations; and SKA1 Survey consisting of 96 dishes (36 ASKAP dishes plus 60 SKA dishes). The Mid element will be located in the Karoo in central South Africa, while the Low and Survey elements will be located in Western Australia. Located currently at the core SKA site in South Africa is KAT7. This is a seven dish midfrequency telescope array which has been operational since KAT7 is currently being developed into MeerKAT, which consists of 64 dishes spanning a baseline of up to 12 km 3,4,5. MeerKAT is in turn a precursor that will be integrated into SKA Phase 1. In MeerKAT each dish is linked to the Karoo Array Processor Building (KAPB) by buried optical fibre. By 2018, each antenna will stream up to 160 Gbps over optical fibre to the KAPB. The KAPB houses the central computing engine, which performs beamforming and correlation of the aggregated astronomy data. The central computing engine comprises of the ROACH (Reconfigurable Open Architecture Computing Hardware), which was developed by the SKA South Africa Organization and collaborators within the CASPER project 6,7. The KAPB contains a radio frequency interference 2

3 (RFI) shielded rack room with at least 100 db shielding in the range 70 MHz to 10 GHz. The MeerKAT site itself is an astronomy reserve, a radio frequency quiet environment protected by legislation. Beyond its high bandwidth, optical fibre offers the additional advantage of not contributing to RFI. Fig. 1. (a) KAT7 telescope array in the Karoo, South Africa. (b) Construction of the MeerKAT dish assembly and pedestal integration sheds, Karoo Array Processor Building (KAPB) and power facility 2 This paper is organized as follows: In Section 2 we draw comparisons between the network architectures of conventional metro-access networks and those of telescope arrays. In Section 3 we discuss the relevance and requirements for synchronization and timing signals to telescope arrays. We conclude in Section 4 with an experimental investigation into the effects of birefringence and polarization fluctuations on the stability of such signals. 2 Network Architecture for Telescope Arrays In this section we draw comparison a between a conventional telecommunication network (Fig. 2a) and a telescope array network such as MeerKAT and SKA (Fig. 2b). Consideration of the similarities and differences between the two scenarios assists in concept generation and technology down selection for designing telescope array networks. A typical metro-access 3

4 network may span 20 km, with splitting ratio of up to 1:128 or greater. MeerKAT thus shares similarities with a passive optical network (PON), with 64 dishes instead of optical network units (ONUs) and the central office replaced by the KAPB. SKA Phase 1 mid shares similarities to a large long reach PON, with distances up to 100 km and 254 dishes. For SKA Phase 2, dishes are added beyond 1000 km in long haul type spiral arms. The data rates of MeerKAT and SKA are more similar to long haul than PON networks. MeerKAT will collect up 160 Gbps of science data per dish by For telescope array networks, high volume science traffic converges to a central computing engine. This differs from a conventional access network where downstream bit rates exceed the user upload traffic. Fig. 2. Network architecture for: (a) a typical metro-access scenario. (b) a telescope array such as MeerKAT and SKA. For access networks the trend is to house costly, sensitive equipment at the central office, while ONUs are equipped with cheap plug-and-play transceivers. Contrary to this trend, the tremendous data rates of MeerKAT and SKA demand that each dish be equipped with high-end transceivers. These transceivers may be installed at the focus of the dish or within the pedestal of a dish. They are thus subject to much harsher conditions than those of a temperature regulated central office. The transceivers should produce a low level of radio frequency interference, so as not to reduce the detection sensitivity of the astronomy signals. Telescope arrays networks are 4

5 required to transport different types of signals. These include science data from the dishes, synchronization and timing signals to the dishes, and two-way monitoring and control data for telescope operation. The science data may be either digital as in MeerKAT, or radio-over-fibre as proposed for the SKA Low Frequency Aperture Array. The distribution of synchronization and timing is discussed in more detail in Section 3 to follow. 3 Synchronization and timing for telescope arrays Synchronization and timing in telescope networks is required to meet a number of important requirements. A central reference clock system consists of a single clock or an ensemble of clocks. The clock type depends on the accuracy and stability requirements, as driven by the science requirements. Typically hydrogen maser, rubidium or cesium reference clocks are used. Distribution of the clock over fibre to each of the dishes ensures phase coherence of the entire array. Phase coherence in the order of picoseconds is required over time intervals ranging from one second to tens of minutes, depending on the nature of the observation. The clock further provides high precision timing for detection of pulsars and transients, typically 1-10 ns over ten years. Depending on the implementation details, clock tones are required to provide accurate frequency standards at the dishes for local oscillators, phase locked loops, digitization, time stamping and related functions. The clocks are further used to derive absolute time for system management, antennae pointing and time intervalling. Existing example synchronization and timing distribution over fibre for telescope arrays include the Multi-Element Radio Linked Interferometer Network (emerlin) telescope in the United Kingdom, as well as Atacama Large Millimeter/sub-millimeter Array (ALMA) in Chile. For emerlin, a RMS stability of 1 ps over 1 second, 2 ps over 1 minute, and 4 ps over 10 minutes 5

6 has been demonstrated for a GHz clock over km 8. ALMA relies on sub-picosecond stability frequencies up to 950 GHz over 15 km 9. Both emerlin and ALMA make use of high precision oscillators and phase locked loops at the dishes, as well as round-trip phase correction. In emerlin phase variations due to temperature effects are measured using round-trip phase comparison. Correction is then applied through delay compensation by the correlator. ALMA relies on similar closed loop phase monitoring and correction by means of a voltage-actuated piezoelectric fiber stretcher assembly. The approach for MeerKAT is to generate the exact required frequencies centrally at the KAPB. They are then distributed without the use of phase locked oscillators at the dishes and with no closed loop phase correction. The fibre will be buried at sufficient depth with measures in place to shield it from temperature and environmental perturbations. The phase stability could nonetheless remain susceptible to unavoidable fibre birefringence and polarization fluctuations. An optical wave with arbitrary state of polarization (SOP) propagating through a single mode fibre may be treated as the superposition of two HE11 polarization modes. In real single mode fibre the degeneracy of the modes is lifted through unavoidable stress birefringence. In a deployed single mode fibre the birefringence varies along the length of the fibre, giving rise to mode coupling. The combined effects of birefringence and mode coupling introduce a differential group delay (DGD) and associated principal states of polarization (PSPs) 10. Both the DGD and the PSPs are wavelength dependent due to mode coupling. A further complication is that the birefringence and mode coupling vary randomly with time due to changing environmental conditions 11. The combined effects of birefringence and mode coupling are referred to as polarization mode dispersion (PMD). PMD received much attention up until the 6

7 early 2000s due to its potential disruption on data transmission at 40 Gbps and beyond. With the advent of modern low-pmd fibres, advanced modulation formats and coherent detection, PMD is no longer considered as problematic for data transmission. It does however remain an important consideration for accurate clock distribution in telescope arrays such as MeerKAT and SKA. For a telescope array it is vital that the antennae elements remain coherent over the period in which a measurement is being performed. For a maximum allowable degradation of 2% in telescope sensitivity, the RMS stability of the clock signal should not exceed a phase error of 0.2 radians during an observation 12. This condition may vary according the nature of the observation, the period of the observation and the telescope array implementation. As such, we consider the required clock stability for various scenarios as in Table 1 below. 4 Experimental Results In this section we present results from a field study using installed fibre on the active KAT7 telescope. The fibre under test was dark fibre, while the telescope itself was operational. The results are valuable to provide insight into the effects of birefringence and polarization on clock stability, particularly under deployment conditions within the geographical core of MeerKAT and SKA. Figure 3(a) shows the results of a single DGD scan over the wavelength range nm. It was measured using EXFO FPMD-5600 Femtosecond PMD Analyzer located at the KAPB site, 5.1 km from one of the KAT7 dishes. The fibre was routed from the KAPB site to focus of the dish, where it was looped back to create a total link length of km 13. In Fig. 3(a), the DGD 7

8 varies with wavelength due to mode coupling and second order PMD. The DGD signature was found not to remain stable but to vary over time between consecutive measurements. Table 1 Clock frequencies for MeerKAT and SKA, with the clock stability budget for expressed in ps for a 0.2 radian phase error. The PMD penalties are calculated as 3(19.4 fs/km 1/2 ) L 1/2, where L is the fibre length and 19.4 fs/km 1/2 is the mean DGD coefficient from the experimental result in Fig. 3(b). Clock (GHz) Clock Stability Budget (ps) PMD penalty for L=12 km [MeerKAT] (ps) PMD penalty for L=100 km [SKA Phase1] (ps) PMD penalty for L=1000 km [SKA Phase2 spiral arms] (ps) MeerKAT L-Band (1.1% of budget) 0.58 (3.1% of budget) 1.84 (9.9% of budget) MeerKAT X-Band/ SKA Phase1 mid (9.2% of budget) 0.58 (26.5% of budget) 1.84 (83.8% of budget) SKA Phase (12.7% of budget) 0.58 (36.6% of budget) 1.84 (115.6% of budget) Fig. 3. (a) A DGD scan over the wavelength range nm for a km buried fibre. (b) A histogram summarizing the results of 383 DGD scans measured over a 15 hour period. Fig. 3(b) is a DGD histogram for 383 measurement scans performed over a 15 hour period. During this time the dish was fully operational, including slewing movements to observe different regions of the sky. Long term characterization over the 40 nm wavelength range provides insight into the statistical PMD under these conditions. From Fig. 3(b), the mean DGD is found to be 62.1 fs, corresponding to a PMD coefficient of 19.4 fs/km 1/2. This is a low PMD, 8

9 and indication that the fibre is of good quality and well deployed so as not to introduce extrinsic birefringence. The maximum DGD recorded was fs. From Fig. 3(b) it is calculated that there is only a 1.4% probability of three times the mean DGD being exceeded, as occurring in the tail of the distribution. The implication of this result on the stability of relevant clock frequencies is estimated in rightmost three columns of Table 1. The PMD penalties have been calculated as 3(19.4 fs/km 1/2 ) L 1/2 for link lengths (L) of 12 km, 100 km and 1000 km. These lengths correspond to MeerKAT, SKA Phase 1 and SKA Phase 2 spiral arms respectively. PMD is seen to be tolerable for a reach up to 100 km, contributing within <37% of the clock stability budget in all cases. The remaining margin can then be allocated to effects such as transmitter, amplifier and receiver noise. For SKA Phase 2, however, PMD is seen to be an issue over 1000 km baselines. For the 14.5 GHz clock the PMD margin contributes 83.8% of the total clock stability budget, while at 20 GHz the PMD margin exceeds the budget. For these cases PMD compensation or mitigation strategies are required. One such mitigation strategy could feature polarization stabilization through appropriate burying of the fibre link or other means. Clock instability depends not only on DGD, but also on the polarization stability of the lightwave carrier. The phase of the clock will drift as polarization drifts between the fast and slow birefringent axes of the fibre. The magnitude of the clock drifts will correspond to the DGD of the fibre after the point of polarization change. The rate at which the clock drifts will be related to the rate of polarization fluctuation. Fig. 4 shows results for polarization monitoring of a 1450 nm laser signal transmitted over the same buried fibre link described previously. Fig. 4(a) is for the case where the loopback occurs at the pedestal of the KAT7 dish. Fig. 4(b) is for the 9

10 case where the fibre further extends up from the pedestal, with the loopback at the index of the dish. Fig 4 Poincare sphere showing polarization monitoring over km of fibre that is (a) looped back at the pedestal of the dish (b) looped back at the index of the dish. In Fig. 4(a) the polarization is seen to remain stable over a 13 hour monitoring window, with very slow drift of less than 15 degrees in total. This demonstrates that the buried fibre is well shielded from vibrations, movement and the environment. The high polarization stability is of great benefit, since it ensures that the impact of DGD of the long length of buried fibre on the clock instability is largely negated. Fig. 4 (b) on the other hand shows significant and rapid polarization fluctuations occurring over a 15 hour monitoring window. These fluctuations originate within the riser cable that extends from the dish pedestal to the focus. This non-buried section of fibre is particularly susceptible to the environment, particularly bending and twisting stress during slewing of the dish. As a result of this, the clock signal delivered to the index will drift by the DGD of the riser cable. This effect is expected to be relatively small, since the riser cable is typically only several tens of meters in length. 10

11 5 Concluding Remarks Optical communication plays a major role in the MeerKAT and SKA telescope projects, both in transporting tremendous volumes of data and in the distribution of highly accurate clock signals. Telescope array networks and conventional telecommunication networks share several key similarities and differences. Thoughtful design and technology selection is thus important in meeting the unique requirements of MeerKAT and SKA. PMD is one of several effects that may limit telescope performance by degrading clock stability. We have shown through a KAT7 field study and related analysis PMD is not expected to limit the performance of MeerKAT. PMD however remains a challenge for SKA Phase 2, where the baseline exceeds 1000 km. Acknowledgements NMMU gratefully acknowledges support from: SKA South Africa, National Research Foundation (NRF), THRIP CSIR, African Laser Centre (ALC), Telkom, Dartcom, Ingoma Communication Services, Infinera and Cisco. References 1. S. Carilli, and S. Rawlings (Editors), Science with the square kilometer array, Elsevier Amsterdam, New Astron. Rev. 48 (2004). 2. Our journey to bring the SKA to Africa SKA South Africa Organization, May 2013, J. Jonas, MeerKAT- The South African array with composite dishes and wide-band single pixel feeds, Proc. of the IEEE, 97(8), (2009). 11

12 4. D. B. Davidson and J. Jonas, An overview of electromagnetics, signal processing, imaging and calibration for MeerKAT, Proc. Internat. Conf. on Electromagnetics in Advanced Applications, (2012). 5. D. B. Davidson, MeerKAT and SKA phase 1, Proc. 10 th International Symposium on Antennas, Propagation and EM Theory, (2012). 6. J. Manley, S. Malan and F. Kapp, Meeting MeerKAT s signal processing challenges, Proc. Internat. Conf. on Electromagnetics in Advanced Applications, (2012). 7. D. Werthimer, The CASPER collaboration for high-performance open source digital radio astronomy instrumentation, Proc. URSI General Assembly and Scientific Symposium, 1-4 (2011). 8. R. McCool, M. Bentley, M. K. Argo, R. Spencer and S. Garrington, Transfer of a MHz frequency standard over installed fibre links for local oscillator distribution with a stability of 1 picosecond, Proc. European Conf. on Opt. Comm. (ECOC) 1-2 (2008). 9. W. Shillue, W. Grammer, C. Jacques, R. Brito, J. Meadows, J. Castro, J. Banda and Y. Masui, The ALMA photonic local oscillator system, Proc. USRI General Assembly and Scientific Symposium 1-2 (2011). 10. C. D. Poole and R. E. Wagner, Phenomenological approach to polarization dispersion in long single-mode fibres, Electr. Lett. 22(19) (1986). 11. S. C. Rashleigh, Origins and Control of Polarization Effects in Single-Mode Fibers, J. of Lightw. Techn. 1(2) (1983). 12. R. McCool, S. Garrington and R. Spencer, Signal transport and networks for the SKA, Proc. USRI General Assembly and Scientific Symposium 1-2 (2011). 13. E. K. Rotich Kipnoo, H. Kourouma, R. R. G. Gamatham, A. Leitch, T. B. Gibbon, R. Siebrits, R. Julie and F. Kapp, Characterization of the polarization mode dispersion of single mode fibre in KAT-7 telescope array network, South Africa, Proc. IEEE AFRICON 1-4 (2013). 12

13 Tim Gibbon is an Associate Professor at the Nelson Mandela Metropolitan University (NMMU). In 2007 he received a PhD in Physics from NMMU in the field of optical communication. From he was Postdoctoral Researcher at DTU Fotonik in Demark, working closely with industry developing next-generation WDM-PON technologies. His research interests including long-haul impairments, metro-access technologies, passive optical networks, FTTH, photonic UWB, and optical fibre telescope networks. He currently manages the NMMU Optical Fibre Research Unit, working on the SKA Project, with further industry and academic collaborations across Africa, Europe, China and Australia. Biographies and photographs of the other authors are not available. 13