WFC3 Training Session 3 WFC3 Post-Observation Systems Howard Bushouse 1
Overview WFC3 OPUS pipeline and calibration largely based on existing ACS and NICMOS procedures Our WFC3 mantra: just like ACS Very similar, if not identical, data file formats & contents Calibration task calwf3 uses applicable ACS/WFC processing for UVIS images and applicable NICMOS processing for IR Original calwf3 based on direct ports of calacs and calnica C code Current calwf3 customized to handle WFC3 unique features Different CCD overscan regions than ACS/WFC CCD binned modes IR reference pixels IR subarrays MultiDrizzle will be used to remove geometric distortion and to combine multiple exposures for a given target 2
File Format Overview ISR WFC3 2001-07 Detailed description of data formats File formats utilize STIS, NICMOS, and ACS heritage All science files are FITS with image and table extensions Images represented by several data arrays, known as an image set ( imset ) 2 UVIS chip images stored as 2 imsets in one file (same as ACS WFC) Each imset has SCI, ERR, DQ extensions IR MultiAccum readouts stored as multiple imsets in one file (same as NICMOS) Each imset has SCI, ERR, DQ, SAMP, TIME extensions Associated exposures including members of CR-SPLITs, REPEAT-OBS, and Patterns stored in separate files 3
File Names WFC3 retains ACS and some NICMOS name conventions Suffix _raw _asn _spt _trl _ima _flt _crj _drz Type Image Table Image Table Image Image Image Image Contents Raw science image Association table Support file (telemetry and engineering data) Trailer file (processing log) Calibrated IR Intermediate MultiAccum image Calibrated image (for UVIS CR-SPLIT & REPEAT-OBS=1 and all IR) Combined, CR-rejected image (for CR-SPLIT or REPEAT-OBS > 1) Calibrated, geometrically-corrected, drizzle-combined 4
FITS Keywords Overall header content is copy of ACS/WFC header IR-specific entries copied from NICMOS UVIS and IR images use separate sets of mode-specific keywords All internal Engineering Snapshot values dredged from WFC3 telemetry and stored as keyword values in UDL and SNAP extensions of SPT files 5
Sample UVIS Header 6
Sample IR Header 7
Associations Associations will be formed for: Multiple CR-SPLIT and REPEAT-OBS exposures Exposures using pre-defined Patterns Combinations of the above Associations will produce sub-products & main products: Sub-products: Combined CR-SPLIT and REPEAT-OBS exposures Main products: Combined dithered/mosaiced exposures Inputs to main product can be either individual exposures (single exposure per pattern point) or sub-products (multiple exposures per pattern point) Main product not always created by calwf3 CR-SPLIT or REPEAT-OBS with no dither pattern produces only sub-product Final (main) product will be produced by MultiDrizzle Product formed as long as 1 or more exposures present (ASN_STAT=INCOMPLETE) 8
Sample Associations CR-SPLIT=2; no dither CR-SPLIT=no; dither=2 Exposure EXP-CRJ EXP-CRJ EXP-DTH EXP-DTH Sub-Product PROD-CRJ Main Product (none)* PROD-DTH* * Main product drz file produced by MultiDrizzle, not calwf3 9
Another Sample Association CR-SPLIT=2; Dither=2 Position #1 Position #2 Exposure EXP-CR1 EXP-CR1 EXP-CR2 EXP-CR2 Sub-Product PROD-CR1 PROD-CR2 Main Product PROD-DTH* * Main product drz file produced by MultiDrizzle 10
Basic Pipeline Functions Apply instrumental calibrations to individual images includes handling of binned and subarray images combination of IR MultiAccum readouts into single image Perform various types of combining for associated images combine CR-SPLIT and REPEAT-OBS images (calwf3/wf3rej) combine dithered/mosaiced images (MultiDrizzle) Grism images receive 2-D processing only spectral extraction/calibration/analysis tools (axe) and support provided by ST-ECF (reuse of ST-ECF supplied ACS grism tools) 11
Calibration Flow Diagram Another set of CR-SPLITs or RPT-OBS images? UVIS Detector WF3CCD DQI Blev Bias Flash Another Image In CR-SPLIT or RPT-OBS set? CRCORR Yes WF3REJ CR Rejection WF3CCD Dark Flat Shad Phot DRZCORR IR WF3IR Zoff DQI Blev Dark Nlin Flat Phot Crrej Another Image in RPT-OBS set? RPTCORR Yes WF3REJ CR Rejection Yes MultiDrizzle Drizzling Another set of RPT-OBS images? 12
CALWF3 UVIS Cal Steps Inputs Processing Step Keyword Switch Output _asn or _raw CCDTAB Initialize ERR array (compute statistical errors) N/A BPIXTAB initialize DQ array DQICORR OSCNTAB Overscan bias level subtraction BLEVCORR BIASFILE Bias image subtraction BIASCORR FLSHFILE Post-flash subtraction FLSHCORR _blv_tmp _blv_tmp CRREJTAB Combine CR-SPLIT or REPEAT-OBS CRCORR _crj_tmp _blv_tmp or _crj_tmp Dark Current Subtraction DARKCORR DARKFILE PFLTFILE DFLTFILE LFLTFILE Flat Field Correction FLATCORR SHADFILE Shutter Shading Correction SHADCORR PHOTTAB Compute Photometric Parameters (populate PHOTxxxx keywords) PHOTCORR _flt or _crj 13
UVIS Overscan Layout WFC3 CCD s use THREE forms of overscan: Serial physical Serial virtual Parallel virtual Calwf3 blevcorr uses serial virtual and parallel virtual to fit tilted 2-d plane to each quad Serial physical used for subarrays, if it s present 14
CALWF3 IR Cal Steps Input Files Processing Step Keyword Switch Output File _asn or _raw BPIXTAB Initialize DQ arrays DQICORR OSCNTAB Residual bias subtraction (reference pix) BLEVCORR Zero-read signal correction ZSIGCORR Zero-read subtraction ZOFFCORR CCDTAB Initialize ERR arrays N/A DARKFILE Dark subtraction DARKCORR NLINFILE Non-Linearity Correction NLINCORR PHOTTAB Compute Photometric Parameters (populate PHOTxxxx keywords) PHOTCORR CRREJTAB PFLTFILE DFLTFILE LFLTFILE Cosmic-Ray Rejection ( up the ramp fitting of readouts) Flat Field Correction CRCORR FLATCORR _ima & _flt _flt Combine REPEAT-OBS Images RPTCORR _crj 15
Flat Field Strategy Uses same approach as WFPC2 and ACS: Flats obtained and maintained in raw space, i.e. pixels with larger projected area have more counts Division into raw science images produces constant counts per pixel for uniform extended sources, but leaves point sources incorrect Point source photometry will have to be corrected by other means (e.g. correction look-up tables or drizzle resampling) 16
Status Processing and ingest of SMGT data very successful in finding problems in OPUS/DADS systems and calwf3 All discovered issues have now been corrected First public release of calwf3 was in STSDAS v3.8 Upgrades already in the plans Adopt recent upgrades to NICMOS up the ramp fitting (done!) Optimize IR reference pixel subtraction Update IR non-linearity correction algorithm Implement UVIS non-linearity correction MultiDrizzle implementation and testing in progress Archive ingest/retrieval implemented; in testing 17
References AV-03: www.stsci.edu/hst/wfc3/documents/requiredreading/av-03_initial.pdf ICD-47: www.ess.stsci.edu/projects/distribution/icd47/ WFC3 ISR 2000-10 Associations for WFC3 and Preliminary Pipeline Requirements WFC3 ISR 2001-07 WFC3 Data Formats in OPUS & STSDAS WFC3 ISR 2003-14 WFC3 UVIS CCD Image Overscan Region Layouts 18
WFC3 Training Session 3 MultiDrizzle: Distortion Correction And Image Combination Linda Dressel August 12, 2008 August 12, 2008 WFC3 Training Session 3 19
Status of MultiDrizzle What is the status of MultiDrizzle? Python/PyRAF code that transforms imaging data from a grid of detector pixels to a linearized grid of possibly smaller pixels Version that supports WFC3 is available now under IRAFX, expected to be released in October Currently uses distortion coefficients based on optical models for WFC3, to be replaced by coefficients derived from on-orbit data taken in SMOV programs 11444 (WFC3/UVIS) and 11445 (WFC3/IR) To be implemented in pipeline for single and associated exposures when onorbit distortion coefficients have been delivered Filter-based increments and time-dependent corrections to distortion coefficients to be determined in subsequent WFC3 calibration programs MultiDrizzle Manual is being developed on a wiki page, to be converted to a FrameMaker document August 12, 2008 WFC3 Training Session 3 20
Purposes of MultiDrizzle What does MultiDrizzle accomplish? Correct the image(s) for geometric distortion Combine images even if offset in position (dithered or mosaiced) Improve the resolution if dithered input improves the sampling of the point spread function Reject cosmic rays Rotate the output image to a desired orientation (e.g., north up) August 12, 2008 WFC3 Training Session 3 21
Geometric Distortion: Projection of Imaging Detectors onto the Sky Shape of projection of detectors on the sky: STIS/CCD: square STIS/NUV: square STIS/FUV: square WFC3/UVIS: rhombus WFC3/IR: rectangle (WFPC2: square) NIC1: square NIC2: square NIC3: square ACS/WFC: rhombus ACS/HRC: parallelogram ACS/SBC: parallelogram August 12, 2008 WFC3 Training Session 3 22.
Geometric Distortion: Physical Cause UVIS: focal plane tilted along its diagonal => projected as rhombus IR: focal plane tilted along one axis => projected as rectangle Both: curvature of folding optics => additional non-linear distortion August 12, 2008 WFC3 Training Session 3 23.
Geometric Distortion: Projection of WFC3 Detectors onto the Sky X Scale ( /pix) Y Scale ( /pix) XY Angle (deg) UVIS 0.0397 0.0395 86.07 IR 0.1395 0.1211 90.00 August 12, 2008 WFC3 Training Session 3 24.
Geometric Distortion: Maps of Linear and Non-Linear Components Linear component (squares) and non-linear component (lines, magnified x10) of geometric distortion for the UVIS and IR detectors August 12, 2008 WFC3 Training Session 3 25
Geometric Distortion: Maps of Relative Pixel Effective Area Maps of effective area of pixels relative to central pixel due to nonlinear component of geometric distortion for the UVIS and IR detectors August 12, 2008 WFC3 Training Session 3 26
Combining Images Exposures made a with Pattern, CR-SPLIT, or REPEAT-OBS are associated (have an asn.fits file) and will be combined by MultiDrizzle in the pipeline WFC3 Patterns (http://www.stsci.edu/hst/programs/hst/proposing/docs/p2pi.html) WFC3-UVIS-DITHER-LINE WFC3-UVIS-DITHER-BOX WFC3-UVIS-MOS-DITHER-LINE WFC3-UVIS-MOS-BOX-LRG WFC3-UVIS-MOSAIC-LINE WFC3-IR-DITHER-LINE WFC3-IR-DITHER-BOX-MIN WFC3-IR-DITHER-BOX-UVIS Generic patterns August 12, 2008 WFC3 Training Session 3 27
Improving the Resolution PSF sampling by pixels indicates the improvement to be achieved by dithering Wavelength (nm) 200 600 800 1100 1500 1700 UVIS FWHM (pix) 2.29 1.62 1.72 2.32 IR FWHM (pix) 1.00 1.03 1.15 1.25 August 12, 2008 WFC3 Training Session 3 28
Cosmic Ray Rejection MultiDrizzle treats cosmic rays even in the presence of dithers and distortion UVIS crj images are made only for CR-SPLIT and REPEAT-OBS exposures (made at the same target position) IR flt images are made using up-the-ramp fitting to remove cosmic rays from the series of non-destructive reads in a timing sequence exposure MultiDrizzle drizzles input flt images to a common frame, derives a cosmicray rejected image in that frame, maps that image back to the input frames, and compares each input frame to the cleaned frame to identify cosmic rays MultiDrizzle relaxes the rejection criteria in the vicinity of large spatial derivatives in the flux (cores of point sources) so that it is more tolerant of drift and focus change than traditional cr-rejection routines August 12, 2008 WFC3 Training Session 3 29.
MultiDrizzle Output MultiDrizzled Output Image, *drz.fits ext [1] = SCI: single or combined distortion-corrected science image ext [2] = WHT: weight image (effective exposure time map) ext [3] = CTX: context image (bitmaps --> which inputs contributed) Post-Pipeline Processing Can choose different output options: pixfrac, scale, rotation, Can get intermediate products and drizzled images of each input file: single_sci, single_wt, (distortion) coeffs, (bad pixel) mask, blt (single cr-cleaned median blotted images) August 12, 2008 WFC3 Training Session 3 30
References MultiDrizzle Handbook (in progress) http://incubator.stsci.edu/mediawiki/index.php/telescopedia:multidrizzle Phase II Proposal Instructions http://www.stsci.edu/hst/programs/hst/proposing/docs/p2pi.html WFC3 Instrument Handbook Appendix B (distortion) & Appendix E (pipeline) http://www.stsci.edu/hst/wfc3/documents/handbooks/currentihb/wfc3_cover.html August 12, 2008 WFC3 Training Session 3 31.