Software Spectrometer for an ASTE Multi-beam Receiver. Jongsoo Kim Korea Astronomy and Space Science Institute

Transcription

1 Software Spectrometer for an ASTE Multi-beam Receiver Jongsoo Kim Korea Astronomy and Space Science Institute

2 Design Consideration software spectrometer for a near future ASTE multi-beam receiver spectrometer for future TP Array multi-beam receivers extension to correlator for a future ACA array

3 Correlation Theorem, FX v(t) () V (f) F-step, Fourier Transform Z v( + t)v( )d () V (f)v (f) X-step, Multiplication

4 Technologies for Spectrometers or Correlators ASIC (Application-Specific Integrated Circuit) ACSIS (Auto-Correlation Spectral Imaging System) in JCMT MAC for BEARS in Nobeyama 45m ALMA 64-antenna Correlator FPGA (Field-Programmable Gate Arrays) WHSF (Wideband and High-dispersion Spectrometer system with FFX correlator) for ASTE ALMA 16-antenna ACA Correlator Software (high level-languages, e.g., C/C++, MPI, CUDA/OpenCL) DiFX (Distributed FX) software correlator for VLBA, LBA (Australian Long Baseline Array), KVN (Korean VLBI Network), GSB (GMRT software Backend): real-time correlator for the GMRT CEP (Central Processing Facility): correlator for the LOFAR

5 software vs. hardware Advantages of software correlator rapid and easy development flexibility (e.g., RFI) and expandability 32bit floating point operations (high-precision) Commodity Off-The-Shelf Technology (COTS) Disadvantages of software correlator low performance/watt

6 Front and Backends for Heterodyne Multi-beam Receivers Nobeyama, 45m BEARS (25 BEam Array Receiver System) GHz, 25 beams, DSB MAC or AC45 32 LSIs (Large Scale Integrations), MHz, 1024 channels IRAM 30m, Pico Veleta HERA (Heterodyne Receiver Array) ~ 230GHz, two 9 beam receivers, HEAR 1 (H) and 2(V) WILMA (Wideband Line Multiple Autocorrelator) Motorola microprocessor, 18 streams, ~ 1GHz, 512 channels

7 Front and Backends for Heterodyne Multi-beam Receivers (cont.) JCMT, 15m, Mauna Kea HARP (Heterodyne Array Receiver Program) GHz, 16 beams, SSB, single polarisation ACSIS (Auto-Correlation Spectral Imaging System) 32 Quint correlator chips, 32 beams, 0.25~2 GHz bandwidth, channels, 50 msec data dump time

8 Specification receiver: assume 16 beams with dual polarisation capability algorithm: FX number of inputs per beam: 4 basebands x 2 GHz x 2 pols number of auto- and cross-correlatioin per beam: 3 = 2 (XX,YY) + 1 (XY) spectral configuration: bandwidth: 62.5MHz ~ 2 GHz highest resolution: khz (=2GHz/2 19 ; 1M points FFT) temporal integration: 16 msec number of output spectral channel per baseband: 4096

9 functions data bit conversion/baseband: 3bit > 32bit (software) : ~2 ν operations ( ν:bandwidth) FFT/baseband: ~ ν log2 (K) complex operations (2K: number of FFT points) Multiplication for auto- and cross-polarisation correlation: ~ ν operations integration: ~ ν operations

10 performance requirement sample interval: t=1/(2 ν), ( ν bandwidth of baseband) time interval for one FFT: 2K t (2K FFT points) number of complex multiplies and adds per second per BB: K log 2 K/(2K t)= ν log 2 K number of floating point operations per second per beam: 2(pol) x 4 (baseband) x 4 (complex to floating) ν log 2 K = Tflops for 2K=2 20, ν=2ghz, Total number of floating point operations per second for 16 beam: Tflops

11 input and output data rates input: 16 (beam) x 2 (pol) x 4 (baseband) x 4 Gsamples/sec x 3bit = 1536 Gbit/s (need 16x100GE or 39x40GE) 16 (beams) x 3(correlation pairs, XX,YY,XY) x 4(baseband) x 4096 points x 4Byte / (16 msec) = MB/sec

12 NVIDIA GPU roadmap in 2013 single precision 4Tflops

13 cufft performance 13

14 Software Spectrometer for a 16 beam receiver 16 beams total: 64*40GE 4*40GE total: 16 nodes 4*40GE, 2 GPU 2 Tflops/node 4*40GE, 2 GPU 4*40GE, 2 GPU - A GPU cluster consists of 16 nodes. - Each node has four 40GE ports and two GPU cards.

15 conclusions goal specification for a multi-beam receiver (ALMA specification): 4 basebands with 2GHz bandwidth, dual polarisation, 16 beams spectrometer capability: 62.5MHz ~ 2 GHz bandwidth, khz highest resolution, 16msec minimum temporal integration performance requirement: ~ 20 Tflops input and output data rates: 1.6Tb/s, 200MB/sec A software spectrometer based on a 16-node cluster with 64 40GE ports and 32 GPU cards is proposed.

16 ACSIS for HARP in JCMT - based 32 Quint correlator chips - 32 beams, 1 GHz, 50 msec 16