Users Expectations from the ALMA Software

Transcription

1 Users Expectations from the ALMA Software Leonardo Testi INAF Osservatorio Astrofisico di Arcetri ALMA Software: the users view Current status and goals Observation Preparation Science Pipeline Data Reduction Package What the users are (or should be) expecting for Early Science

2 The Camera Goal ALMA delivers images The astronomer specifies the science requirements: angular resolution field of view Image fidelity required Spectral configuration and calibration precision The observing preparation tools derive the observation Stringency Stringency concept: (inverse of the) Fraction of time in which the observing conditions are better than required to perform the project. Full service observing with dynamic scheduling The ALMA imaging pipeline ensures optimal (within current knowledge) data processing to produce the image ESO Garching Leonardo / Nov 2002Testi, INAF - Osservatorio Astrofisico ALMA di Arcetri, Status Users Expectations from the ALMA Software 8

3 The Camera Goal Remaining difficulties for Astronomers Proper definition of the science requirements Compromise Sensitivity / Angular Resolution The spectral correlator configuration definition (the correlator is very flexible...) a minimum understanding interferometry may still be important. High fidelity or very wide field images may require post-processing. Full polarization imaging may require post-processing Special observations (e.g. moving targets,...)... ESO Garching Leonardo / Nov 2002Testi, INAF - Osservatorio Astrofisico ALMA di Arcetri, Status Users Expectations from the ALMA Software 9

4 The ALMA software system Developed by the Computing IPT (B. Glendenning & G. Raffi) Designed to meet a set of Science Requirements prepared by the Science Software Requirements Commettee (R. Lucas) Subsystems Scientists (SSR members) ) are responsible for supporting the developers teams and ensure that each subsystem will meet the scientific requirements. Observation Preparation (Leonardo Testi) Offline Data Reduction (Debra Shepherd) (Science)) Pipeline (Christine Wilson)

5 Observation Preparation Handles all user interaction with ALMA up to the execution of the observations Phase I (proposal( preparation and submission) Phase II (observing( programme preparation) Interactive observing Concept based on existing tools (Gemini, JCMT, ) Supports an Expert mode for observatory staff and a Novice to Experienced user mode for external users

6 Observation Preparation

7 ALMA-OT Current status: Phase II Expert mode developed for Integration/Testing and ATF support Point source sensitivity calculator developed Full ALMA simulator being developed within the Offline Data Reduction Package Test plan External user tests started (Jan 2004); next test Nov 2004; ATF testing 1-2Q2005 Observation Preparation

8 Offline Data Reduction Package The ALMA Offline DRP will be based on AIPS++ A dedicated set of science requirements have been developed for the DRP by the SSR At the time of the previous European Community Day (Nov( 2002) AIPS++ was undergoing a deep scrutiny by ALMA to verify its useability Requirements Audit (based on available AIPS++ documentation) A so called IRAMTEST Phase 1: Implement PdBI calibration routines in AIPS++ Phase 2: Use of AIPS++ to reduce PdBI data by experienced mm astronomers Phase 3: Accurate absolute and comparative benchmarking

9 Offline Data Reduction Package Audit and all phases of IRAMTEST were completed between the end of 2002 and spring 2003 IRAMTEST results: Phase 1: Partly succesful; it was possible to complete the implementation of routines but it took more time than expected Phase 2: Essentially failed The user interface was found totally inadequate and the software too difficult to use The software was not stable against user mistakes The software failed on requirements that were found to be acceptable in the documentation based a Phase 3: The package was found to be on average a factor of 4 slower than GILDAS (and AIPS/MIRIAD)

10 Offline Data Reduction Package The ALMA project (and NRAO) took an aggressive attitude to attack and solve the AIPS++ problems A series of Technical reviews have been performed: NRAO-AIPS++ AIPS++ Technical Review 1Q2003 ALMA PDR/CDR1&2 1Q2003, 2Q2003, 2Q2004 The software is now project driven (to fit ALMA/VLA needs) and it is no more offered as a general package An Internal and External User Test Plan has been implemented Periodic benchmarking on every subsystem of the package is being performed Transition from the old Glish to ACS/Python Python. Redesign of the (Graphical) User Interface

11 Offline Science Software Development & Test Schedule Sim/test data: Sgl Fld, Mosaic, TP (separate&combo) Sim Data: Large Multi-field Mosaic Sim Data: TP & Mosaic (separate) Complex Interferometric Mosaic Interferometer/TP combo Small Multi-fld Mosaic VLA & BIMA mosaics Single Field VLA & PdBI 13mm 1/3mm User Acceptance Tests Increased capability ALMA Early Science Begins 11

12 Requirements Audit: 106 requirements graded: 57% A: Adequate (all necessary functionality available) 27% A/E (some enhancement would be nice) 12% Inadequate (function not possible or severely limiting). None of the requirements graded I prevented processing of any dataset Comparison of the first ALMA Offline Subsystem Audit (based mostly on documentation, limited testing of VLA data only). Percentage of requirements with A and A/E grades increased from 72% to 84% during TST2. Grade A A/E I U First Audit Results 66% 6% 26% 2% TST1 Audit Results 57% 27% 12% 4% 12

13 Offline Subsystem Performance Requirements AIPS++ & Gildas Comparison PdBI GG Tau at 1 & 3mm spectral line, extended to 64 ants via simulation * Phase III AIPS++/Gildas comparison with same data: A2/G = 4 (Mar 2003) 13

14 Offline Subsystem Performance Requirements AIPS++, AIPS & Miriad Comparison VLA gravitational lens in polarized cm continuum, self-cal, time extended via simulation * AIPS++/AIPS comparison with same data: A2/A ~ 4 (Aug 2003) 14

15 (Science)) Pipeline Based on AIPS++ routines called by the pipeline infrastructure Heuristics being developed in Europe First User Tests (1Q2004) succesful: : the pipeline automatically calibrated and imaged the same datasets used for the AIPS++ tests Pipeline heuristics will require a learning learning period with real ALMA data hence data processed through the science pipeline will not be offered to the general users in the early days of ALMA

16 Pipeline User Test 1 Results ( GRB GG Tau, 3mm continuum Pipeline image rms=0.0025mjy/bm Pipeline image rms = 0.47mJy/bm Peak=0.49mJy/bm Peak = 7.2 mjy/bm GRB Offline (glish)-pipeline (ACS/python) image rms=7.8e-4mjy/bm Peak=7.5e-3mJy/bm (Less than 1% difference, slight divergence after CLEAN iteration ~800) Offline (glish)-pipeline (ACS/python) image rms = 0.0mJy/bm Identical images 16

17 Expectation for Early Science Observation Preparation All modes offered to the users will be supported from the Novice to the Expert levels. At the very beginning of Early Science,, the Novice mode behaviour will be mostly based on expectation from simulation and initial testing rather than on a solid experience with ALMA Science Pipeline Not available to the users for Early Science (it will be in the learnig phase) Offline Data Reduction Package All modes offered will have been fully tested and supported No Graphical User Interface will be available for Early science A solid well tested User Interface and scripting language based on Python will be available