ASKAP Phased Array Feed Digital Beamformer Design Overview and Performance Characteristics

Transcription

1 ASKAP Phased Array Feed Digital Beamformer Design Overview and Performance Characteristics John Tuthill, Tim Bateman, Grant Hampson, John Bunton, Andrew Brown, Daniel George, Mia Baquiran August 2016 CASS

2 Outline: ASKAP digital signal processing overview ASKAP digital beamformer design performance Current and future work 2 ASKAP Digital Beamformer John Tuthill

3 ASKAP: Architecture and signal chain Mk-II PAF: checkerboard array, LNAs, RF filters, laser drivers Custom digital system: Raw data ingest: 130Tbits/s Raw processing power: 2.3PMAC s/s O/E Central Site DSP: FPGA Hardware X 188 X 188 X 72 X 72 Correlator Ethernet X 36 x S O/E CMAC Raw Visibilities RF over Fibre Optical-to-electrical, Samplers, Coarse filterbanks Beamformers, Fine filterbanks Imaging pipeline : ivec Galaxy Cray Inc. XC30 series supercomputer Located in Perth, ~800km from MRO. ~200 TeraFLOPS 3 ASKAP Digital Beamformer John Tuthill

4 ASKAP DSP Hardware Passive optical circuits for data cross-connects X 188 X 188 X 72 X 72 O/E X 36 x S CMAC O/E Dragonfly-3 Digital Receiver Beamformer Redback-3 platform Correlator 4 ASKAP Digital Beamformer John Tuthill

5 Detailed Signal Path single PAF Digital Receiver CFB Integrator Raw ADC sample capture CFB 1MHz Capture ADC Histogram CFB Correlator PAF port analog input ADC 12-bit real Oversampled Coarse Filterbank Packetiser/ Serialiser All number representations are fixed-point two s complement. Data transport to Beamformer Beamformer Calibration correlator Data transport from digital receiver optical cross-connect ACM Peak Hold FFB Integrator Data transport to correlator Deserialiser/ Depacketiser Beamformer Corner-turn and coarse delay Fine Filterbank Fringe Stopping Packetiser/ Serialiser (13+13)-bit complex Deep buffer - external DDR memory Number representation is fixed-point signed two s complement throughout 5 ASKAP Digital Beamformer John Tuthill

6 ASKAP FPGA DSP Stat s digital reveiver Sampling ADC part - Nat. Semi. (now TI) ADC12D1600 Resolution 12bits (9.4 ENOB) Analog BW 2.8GHz Direct digital down-conversion (2 nd and 3 rd Nyquist zones) Sample rates: 1,280MSps and 1,536MSps 3 overlapping sampling bands between 700MHz and 1,800MHz Coarse frequency channelization Dragonfly-3 Digital Receiver Processed bandwidth: 384 MHz (arbitrarily selectable anywhere in the observing band) Coarse frequency channels: 1MHz oversampled by 32/27 Spectral flatness: <0.2dB Sub-band alias rejection: >60dB 6 ASKAP Digital Beamformer John Tuthill

7 ASKAP FPGA Hardware Stat s beamformer Digital beamforming Narrow-band beamformer structure (weight-and-sum) Full 192x192 element array covariance matrix for all 1MHz coarse channels (floating-point O/P) Up to 72 single-polarized beams (usually configured as 36 dual-pol beams) Fine frequency channelization 6 frequency zoom modes: 18.5kHz, 9.3kHz, 4.6kHz, 2.3kHz, 1.2kHz, 580Hz Critically sampled (but not for long) Delay tracking and fringe-stopping Coarse delay to 1us resolution Time-varying phase slope applied to fine channels Redback-3 DSP Platform 7 ASKAP Digital Beamformer John Tuthill

8 Digital Beamformer (single DSP FPGA) 1MHz narrow-band beamformer modules Peak-hold monitors 1 2 8x 1MHz channels, 192 complex-valued port voltages per channel Up to 36 dual-polarized beam voltages for each 1MHz channel 8 Array covariance matrix New beam weights Off-line weight calculation 8 ASKAP Digital Beamformer John Tuthill

9 Individual 1MHz digital beamformer module 1MHz channelized port voltage stream 1 Port voltage replay memory buffers 2 18x 1MHz narrow-band beamformer engines Beam weights: (14+14)-bit complex (16+16)-bit complex Dual-bank beam weight memory allows synchronous weight updates Beam scales: Scale RAM Peak-hold (40+40)-bit complex x Beam scaler x Round (16+16)-bit complex Up to 36 dualpolarized beams 16-bit round 18 (40+40)-bit complex (56+56)-bit complex 56-bits 56-bits 30-bits 16-bits Arbiter Inferred binary point 16-bits 14-bits Convergent rounding with wrap on overflow behaviour 9 ASKAP Digital Beamformer John Tuthill

10 Beamformer engine performance Two main performance bounds: Degrees of freedom in beam weight selection (architecture) Maximum beam weight update rate (interface) 1. Degrees of freedom (# active weights) Sample rate = 1MHz x 32/27 -> T = ns Beamformer logic clock = 312.5MHz All contributing PAF port voltages must be processed in 263 clocks 18 physical instances of the beamformer process ~1 port voltage per clock cycle 18 single-pol beams: all 192 ports active 36 single-pol beams (1 replay): 130 active ports 72 single-pol beams (2 replays): 60 active ports Implications for sidelobe control, null-steering, spill-over control 10 ASKAP Digital Beamformer John Tuthill

11 Beamformer engine performance 2. Maximum weight update rate, determined by: ACM dump time Beam weight upload time ACM data volume and dump time: 192 x 192 ports 2 (conjugate symmetry) x 2re/im x 32-bit float 172kBytes Measured ACM dump time 4ms/ACM (172kB/4ms = 43MB/s = 344Mb/s) (More efficient to stream raw port voltages than ACMs for dump times < 0.2ms) Beam weight upload time 180ms/chassis (36 dual-pol beams, measured) Max dynamic beamforming period 48 x 4ms + 180ms 400ms (does not include off-line weight calculation) Implications for real-time/adaptive beamforming and RFI tracking 11 ASKAP Digital Beamformer John Tuthill

12 Current/future work (SKA.CSP) Strong collaboration with ASTRON and AUT Physical aspects size/weight, power/cooling important Liquid cooled heatsinks are better for RFI shielding Choice of on-board optics (short range MMF vs long range SMF) Integrated control and monitoring functions on the DSP FPGA Higher processing bandwidth and more beams Not so much a limiting factor now as devices are much larger Larger devices Local control loops Capacity to calculate weights (dynamic beamforming) Need more memory (external and internal) Large/fast corner-turns and transient buffers Higher reliability -> lower component count Scalable/flexible systems Ability to trade of Beams, bandwidth and PAF ports 16nm process 12 ASKAP Digital Beamformer John Tuthill

13 DSP algorithms and other work Algorithms and firmware Fast ACM dumps/weight uploads Implementing real-time/adaptive beamforming matrix eigenvalue decomposition in FPGA s (Power iteration, Lanczos Alg.) iterative update of ACM inverse (Woodbury s identity) Tied-array processor for ASKAP (second-stage beamformer + GPU post-proc.) Efficient oversampled synthesis filterbanks RFI mitigation through time-gating (ADSB) High dynamic range RF-over-fiber High-power laser drivers Balanced/differential Mach-Zehnder modulators Balanced InGaAs photodiodes 13 ASKAP Digital Beamformer John Tuthill

14 Serial interfaces (JESD204B) Low frequency Optical-to-Digital Module (prototype) RF Analog and Digital 60mm, ~2 in. Differential RF Signals ADC Clock and JESD204B Synchronisation JESD204B Synchronisation ROSA ROSA Amp. Amp. 350MHz Filter 350MHz Filter Balun Balun ADC XFP Connector FPGA ROSA supply (+RF Choke) Amplifier Supply (+RF Choke) Power C&M Voltage Regulation (off module) C&M CS-I2AL TCBT-14+ MAR-1SM+ LFCN-320+ TC4-19X+ AD9234BCPZ ASKAP Digital Beamformer John Tuthill

15 Power-over-fibre Full galvanic isolation for electronics Typical 45% optical-to-electrical conversion efficiency at 1W optical input levels 2W output power from single device Accepts 1W to 4.5W laser diode input power Constant output power with multi-mode fiber sizes from 62.5µm to 200µm Currently addressing safety aspects before we proceed further 15 ASKAP Digital Beamformer John Tuthill

16 Star-on Machine Thank you Dr. Seuss - The Sneetches and Other Stories CSIRO Astronomy and Space Science Dr John Tuthill Research Group Leader Digital Systems t e john.tuthill@csiro.au w CSIRO ASTRONOMY AND SPACE SCIENCE

17 Beamforming Algorithm Maximum SNR with phase matching R R w s n Array covariance matrix n weights modified for smooth phase: ˆ 1 w R n v ˆ (where ˆ max ) wˆ e j wˆ arg ŵ H wr New beamformer weights W 1 W 2 W 188 S x 384 1MHz frequency channels other approaches sub-space projection (RFI mitigation, see next presentation!) Shape-constrained beamforming 17 ASKAP Digital Beamformer John Tuthill