Applying full polarization A-Projection to very-wide fields of view instruments: An imager for LOFAR Cyril Tasse ASTRON/Leiden: Joris van Zwieten, Bas van der Tol, Ger van Diepen NRAO: Sanjay Bhatnagar
Bas van der Tol Sanjay Me Ger van diepen missing :( Joris van Zwieten
Overview 1- LOFAR and the Measurement Equation 2- How do you get AW-Projection? Why? 3- Implementations / Optimisations for LOFAR 4- Simulations / Test / Perfomance 5- Conclusion Paper submitted: arxiv:1212.6178
4-pol visibility for one baseline One off axis polarized source (IQUV=100, 40, 20, 10)
When Direction Dependent Effects (DDE) become a problem : Beam y x 100-250 MHz 30-80 MHz y x LOFAR stations are phased arrays - Beam is variable in frequency and time - Projection of the dipoles in the sky is non trivial - Beam can be station-dependent --> Strong effects on polarisation
When Direction Dependent Effects (DDE) become a problem : Ionosphere Incoming wavefront Outcoming wavefront PSF Ionosp here Big field of view : station, direction, time and frequency dependent Other direction dependent effects : - Projection of the dipoles on the sky - Faraday rotation + Effect on the polarisation
The Measurement Equation Direction independent Direction dependent Source coherency F Hamaker 1996 Linear transf. [Voltage antenna p] x [ Voltage antenna q]* Beam Geometrical delay +Correlator Ionosphere Van der Tol thesis Electric field F'
An imager for LOFAR Mesurement (106-109 points) Calibration Numerical instrument ( A-Projection ): generalized adaptative optics - Primary beam - Dipole projection - Ionosphere - Faraday rotation Constrain: Operate on the data in postprocessing real time (< obs time) GOAL: Dynamiques range of 1:1.000.000 Unknown Sky Calibration
The Vec Operator If Columns of a Matrix And then
The Vec Operator applied to the ME Applying: To: DDE (4*4) Then: 3D FT With:
The direction-dependent Mueller matrix for a given baseline LOFAR LBA beam: Off-diagonal Mueller element as high as 10%
A-Projection Bhatnagar 08 DDE are in general smooth on the sky --> small support in the uv-domain Convolution theorem DDE (4*4) Convolution 2D FFT W term (scalar)
DeGridding for a given time, frequency, baseline FFT XXgrid XYgrid YXgrid YYgrid Sky domain IQUV XXgrid XYgrid YXgrid YYgrid
DeGridding for a given time, frequency, baseline FFT XXgrid XYgrid YXgrid YYgrid Sky domain IQUV Convolved XXgrid XYgrid YXgrid YYgrid
DeGridding for a given time, frequency, baseline FFT XXgrid XYgrid Sky domain IQUV YXgrid Convolved YYgrid XX XXgrid XYgrid YXgrid YYgrid
DeGridding for a given time, frequency, baseline FFT XXgrid XYgrid Sky domain IQUV YXgrid Convolved YYgrid XX, XY, YX, YY XXgrid XYgrid YXgrid YYgrid
Testing fullpol A-Projection One off axis polarized source (IQUV=100, 40, 20, 10) It works, but: - 16 operation instead of 4-16 convolution function instead of 1 - More memory access
Saparating the element beam from the station beam - Naive AW-Projection with beam: - DDE to convolution functions: Y BUT: X
Saparating the element beam from the station beam Baseline independent 16 terms slow Element beam (4*4) Baseline dependent 1 term, fast variation (ionosphere) Array factor*w term*ionosphere (scalar) Practical HowTo: 1- In a time/frequency interval: correct the 4-pol grids for the element beam 2- Degrid using a scalar time/freq/baseline dependent CF Advantage: - CF estimate per baseline: 1 CF instead of 16 - gridder: 1 memory acess instead of 16 - per visibility 4 gridding step instead of 16 Disadvantage: - Assumption may happen to be wrong for the long baselines and for Faraday rotation - Code is more complicated
Tests on simulated data 100 sources with flux density following NVSS 1.4 GHz source counts
Tests on simulated data Wterm + Scalar /diagonal beam 100 sources with flux density following NVSS 1.4 GHz source counts
Tests on simulated data Wterm + full beam Mueller matrix 100 sources with flux density following NVSS 1.4 GHz source counts
Tests on simulated data W-Proj only W-Proj + Array Factor Convergence of CLEAN W-Proj + Full Beam W-Proj + Full Beam + fuzzy corr
Tests on simulated data 100 sources with flux density following NVSS 1.4 GHz source counts
Computation ballance For CF estimate every 20 minutes With zero padding interpolation But: can be much worst Huge limitations for: - Wide FOV - Ionospheric corrections (until November 2012)
New implementation: hybrid w-stacking In the current (old) implementation w-term is annoying because: 1- For each baseline, we have to compute a big CF 2- We have to grid each individual baseline with that big CF BUT: The w-term is not in itself baseline dependent!!! We can rewrite the problem (degridding - predict) Old New 2- Conv with a element beam-term 3- Conv with a w-term (only step 4 is baseline dependent!) 4- Conv with a Aterm and dw-term 1- FFT of sky model
New implementation: w-stacking Gridding... Put selected visibilities of given baseliene on w-plane 0, with a kernel containing delta-w and scalar A-term W-plane 0
New implementation: w-stacking Gridding... loop over baselines, time and frequency bins W-plane 0
New implementation: w-stacking Gridding...... Convolve this plane with the w-term of the plane 0 W-plane 0 w-term
New implementation: w-stacking Gridding... Loop over w-planes... W-plane 1 W-plane 0 w-term
New implementation: w-stacking Gridding... Sum the grids of all wplanes + apply the element beam W-plane 1 W-plane 0 w-term
New implementation: w-stacking Advantage: - The delta-w/a-term CF is much smaller: the CF calculation time too - The W-term is applied to all baselines simultaneously --> we win a lot of computing time for the quickly varying / long baseline DDE - The CF always have the same support: easy scheduling for parallel gridding
Parallel gridding All convolution functions have the same support! - Before: one grid per thread and awimager limitted to small grids Thread 1 Thread 2 Thread 3 Thread 4
Parallel gridding All convolution functions have the same support! - Before: one grid per thread and awimager limitted to small grids - now: one grid only and all threads operate on it [movie] Thread 1 Thread 2 Thread 3 Thread 4
Performance Scaling: Typically, 2-3 hours for 11000x11000 pixel image, 12 hours, 12 subbands, 10 major cycles We can now apply quickly varying DDEs NB: At the moment, the parellelisation only uses multicores on single node.
Putting ionosphere in... Bas van der Tol Incoming wavefront Outcoming wavefront PSF The same idea: FT Convolution function
Putting ionosphere in the imager... A grid of sources with equal flux 2D phase screen on top Corrected for the center only
Putting ionosphere in the imager... A grid of sources with equal flux 2D phase screen on top The direction dependent ionospheric has been applied.
Putting ionosphere in the imager... Phased array beam + Ionosphere corrupted Simulated dataset with many point sources (LOFAR's beam+ionosphere)
Putting ionosphere in the imager... Phased array beam + Ionosphere corrected Simulated dataset with many point sources (LOFAR's beam+ionosphere)
Putting ionosphere in the imager... Simulated dataset with many point sources (LOFAR's beam+ionosphere)
Putting ionosphere in the imager... Simulated dataset with many point sources (LOFAR's beam+ionosphere)
Conclusion Several AW-Projection implementation for LOFAR DDE: Time, Frequency, Baseline dependent 1- Full naive implementation too slow 2- Separate the element beam much better (x16) - LOFAR's architecture related 3- Separate the w-term from the A-Term (x5-10) - Faster than real time - Many pixels: wide fields, high resolution - and for quickly varying DDE Next steps - Memory consumption still big in certain cases - Pythonisation of the imager (Bas, Joris, Alexey): - Easy platform to include compressed sensing - GPUs development - Parallelisation of the imager over the cluster - Multi-Term / Wide-Band cleaning - Open issues in the image plane (CLEAN doesn't use varying Direction Dependent PSFs) - Calibration of quickly varying DDE