ASKAP Industry technical briefing. Tim Cornwell, ASKAP Computing Project Lead Australian Square Kilometre Array Pathfinder

Transcription

1 ! ASKAP Industry technical briefing Tim Cornwell, ASKAP Computing Project Lead Australian Square Kilometre Array Pathfinder

2 The Square Kilometre Array 2020 era radio telescope Very large collecting area (km 2 ) Very large field of view Wide frequency range (70MHz - 25 GHz) Large physical extent (3000+ km)

3 SKA design Up to 1500 antennas (15m diameter) in the central 5 km Another 1500 from 5 km to km Aperture arrays for all sky monitor Connected to central processor by an optical fibre network Radio camera All-sky monitor

4 Australian SKA Pathfinder = 1% SKA Wide field of view telescope (30 square degrees) Sited at Boolardy, Western Australia 36 antennas compared to ~ 3600 for SKA!

5 ASKAP Beamforming

6 An illustration of the speed of the Pathfinder Ilana s image of Centaurus Required 1200 hours observing on the Australia Telescope Compact Array in Narrabri The Pathfinder will take about 10 minutes!

7 The ASKAP team and our timeline SITE INFRASTRUCTURE BETA: Boolardy Engineering Test Array

8 Computing team members IPT management Lead: Tim Cornwell Engineer: Ben Humphreys Manager Alan Ng Scientist: Ilana Feain Team members Marsfield: Juan Carlos Guzman, Malte Marquarding, Tony Maher, Max Voronkov, Matt Whiting Narrabri: Dave Brodrick, Euan Troup Parkes: Simon Hoyle Dwingeloo: Ger van Diepen Socorro: Urvashi Rau

9 CMPT internal activities Twelve different activities Each activity lead is responsible for planning Use TRAC tickets for schedule and track work Started to use integration milestones

10 Controlling complexity Conventional model for radio synthesis arrays: Observing modes and data reduction software are independent Possible to observe data that cannot be reduced Observers are exposed to full system complexity ASKAP model: Observing modes and data reduction software are coupled Can only observe data that can be reduced Only ASKAP staff are exposed to full system complexity Wrap up functionality as Software/Science Instruments (SI s) SI s are developed, tested, and deployed by ASKAP staff Allows incremental delivery Expert mode will be available Will truly be expert - probably via scripting Model has been extensively discussed with community Generally accepted for this survey telescope

11 Integration milestones Scheduled Science and System integration milestones First is on 30 October 2009 CDR is 1-2 December 2009 Science 1 Provide an environment in which testing of the prototype versions of the imaging and analysis software is possible. Both continuum and spectral-line cases should be catered for. Allow end-to-end testing of the pipelines, from simulation through imaging, source-finding and quality evaluation. System 1 Prototype several ASKAP software functionality to be presented in the upcoming CDR and to be used in early BETA. Test all entities mainly involved with observation execution Test administration of software components. Define Scheduling Block, components interfaces and initial implementation of Executive, TOS, CP, MoniCA and Logging are major deliverables of this milestone. Data Service is optional. Deploy in SAPKAP machines.

12 Simulations of radio sky observed with ASKAP Computation limited, extended source model Noise limited, point source model

13 Sky mount Three-axis telescope to keep antenna sidelobes (and PAF) fixed on sky A triumph of software over hardware!

14 Progress in key computing issues Developed parallelized imaging software Developed parallelized source finder Imaging algorithm advances Published theory (Urvashi, Bhatnagar, Voronkov, Cornwell) Post hoc reweighting Multi-frequency multi-scale (Urvashi PhD) Developed detailed cost model for processing Vital for what-if analysis Investigated various hardware options Excellent help from vendors IBM, Intel, AMD, CRAY, NEC GPGPU, Cell, FPGA PDR passed, March CDR planned for December 2009

15 The Square Kilometre Array 2020 era radio telescope 4 prime characteristics Very large collecting area (km2) Sensitivity to detect and image hydrogen in the early universe Sensitivity ~ 50 x largest existing array (Very Large Array in New Mexico) Very large field of view Fast surveying capability over the whole sky Survey speed up to one million times faster than EVLA Wide frequency range required for the key science projects low : MHz mid: 300 MHz-10 GHz high: GHz Large physical extent (3000+ km) Capability for detailed imaging of compact objects, and astrometry with milli arcsecond angular resolution

16 Building SKA SKA is a fully global project USA, Europe, Australia, South Africa, etc. Currently funded SKA R&D ( ) via national and regional projects: 150 M PrepSKA (EU FP7) funds the SPDO engineering team Precursors (ASKAP, MeerKAT) Design Studies (TDP, SKADS) Pathfinders (ATA, LOFAR, Apertif, MWA, LWA, EVLA, emerlin, eevn) Two countries have proposed to host SKA Australia (Western Australia) South Africa (Karoo) SPDO team Located in Manchester, UK

17 SKA timeline External Engineering Review of design SSEC Site Rec Phase 1 funding request Phase 2 funding request Prod. Readiness Review Phase 1 complete SKA-mid+low Complete Pathfinder implementation Pathfinder operations Early Science SKA mid SKA Ops +low Concept Design System Design EC-FP7: PrepSKA System design Funding Governance Site Characterisation Detailed Design, Prod. Eng. &Tool g Mobilisation Infra Plan g Phase 1 Construction Concept design & for SKA-high Full SKA mid + low construction and commissioning System design SKA-hi SKA-high Construct SKADS US TDP 26 February 2009

18 Possible telescope configuration Comms links + power Dishes in stations along spiral arms Central Processing Facility 40 stations km 40 remote stations 200 to >3000 km Station Dishes Max. Distance for Dense AAs 200 km Dense AA Sparse AA Adapted from A. Faulkner

19 Conceptual block diagram of SKA

20 Signal transport networks European VLBI Network (1 Gbit/s) LBA in Australia (1 Gbit/s) SKA data rates (8 GHz BW/pol max) 160 Gbit/s/beam/dish (<200km) 160 Gbit/s/station (>200km)

21 Scaling ASKAP computing to SKA Roughly 100 times more antennas Processing scales as square of antennas SKA processing > 10,000 ASKAP processing for same field of view > 1 Exaflop/s processing rate 1 Exabyte/day data rate input to imaging machine Each complex visibility takes about 10,000 flops to get to an image Suitable for Grid? Correlation, calibration, imaging require access to full data Not suitable for very distributed (i.e. Grid) processing Science analysis requires access to derived results (much smaller) Better suited to Grid processing Correlation, calibration, imaging all done at one location Specific data streams may be sent to specialized processing centers Science analysis distributed around the world at regional centers Multiple copies of archive globally Allows centers to specialize scientifically Vital to ensure that communities thrive in each participating country

22 SKA Operational model Multiple observations running simultaneously All data-centric processing done close to telescope Impossible to ship 1 Exabyte/day! Science products globally available Images, catalogs are small enough to ship globally Expect to use CERN multiple Tier model Important for stakeholders Recognizes scientific diversity Software costs SKA will take 150M per year to operate Operational costs ~ 5 per second! Can hire 600 software developers for that! Plan for continuous software development after commissioning Release capabilities incrementally

23 Climbing Mount Exaflop SKA phase 2 SKA phase 1 ASKAP NCI Altix tests ASKAP dev cluster Note that Flops numbers are not achieved - we actually get much lower efficiency because of memory bandwidth - so scaling is relative

24 Exa-problems for SKA computing Timescale % science observations ~ 2020 Management of complexity Many stake-holders and users Networking ~ 1Tb/s per antennas short haul (< 200km) 160Gb/s per station long haul (> 200km) Data flow Data rate ~ 1 EB/day Real time processing vital! Parallel processing 1 EFlop/s ~ 1 billion cores How do we program these? Computing power requirements 1Eflop/s = 350MW at Blue Gene efficiency Need one to two orders of magnitude improvement Plus green power Minimize use of radio-astronomy specific technology

25 Open questions at SKA level Computing Scale of processing? Processing model? Data storage and distribution? Software Models for development? Buy vs build vs reuse? Dealing with complexity? Data processing Scalable algorithms? Calibration of specific telescope design? Simulation capabilities?

26 Structure!

27 PrepSKA structure