CCP4 Workshop Chicago 2011 Data Integration with XDS

Transcription

1 CCP4 Workshop Chicago 2011 Data Integration with XDS Dept. of Structural Chemistry, University of Göttingen June CCP4 Chicago 2011: XDS 1/29

2 The XDS Distribution XDS is an integration program for monochromatic X-ray data originally written by W. Kabsch now co-authored by K. Diederichs availability: XDS main program suite xds-viewer viewer for control images (.cbf-files) Wiki and auxiliary programs maintained by Kay Diederichs XDSi (GUI) pkursula/xdsi.html CCP4 Chicago 2011: XDS 2/29

3 XDS Characteristics Some of the special features of XDS: 3-dimensional spot integration Correction for Radiation Damage Optimised for new Pilatus Detector Command-line program Parallelised: Fast! Simple to read documentation CCP4 Chicago 2011: XDS 3/29

4 Learning Curve Why newbies often do not like XDS: 1. No GUI, only command line 2. error messages may initially appear cryptic 3. users must read (rather than look at graphics) Once understood, the program flow is simple and very easy to set up Understanding XDS gets you better data quality (also with other integration programs) CCP4 Chicago 2011: XDS 4/29

5 Documentation XDS and accompanying programs come with very clean (html-) documentation. Every step of the program and every keyword is described, and reading is not interrupted by many cross-links. Documentation contains both technical information for those who want to understand how XDS works in details as well as user information that help setting up the integration process. CCP4 Chicago 2011: XDS 5/29

6 Other Sources of Information Wiki Tips, tricks, tutorials,... CCP4bb Sign up at CCP4 Chicago 2011: XDS 6/29

7 Templates Templates of input scripts for all supported detector formats Only very few adjustments necessary to get started Beamlines often generate appropriate input scripts generate XDS.INP from XDSwiki for MARCCD, ADSC, and TODO CCP4 Chicago 2011: XDS 7/29

8 XDS.INP XDS is controlled by one single input file: XDS.INP. Name cannot be changed Each data set must be run in separate directory to avoid overwriting of files. Contains about 100 Keywords a of the form KEYWORD=VALUE Only about 10 Keywords must be modified for most data sets (e.g. distance, number of images, etc.) Most important one: image names, detector JOB= XYCORR INIT COLSPOT IDXREF DEFPIX XPLAN INTEGRATE CORRECT Each name stands for one of the steps XDS carries out during data integration. (XPLAN is optional and corresponds to the BEST [2] or STRATEGY [1] output to report optimal data collection range(s)) a also called cards for historical reasons CCP4 Chicago 2011: XDS 8/29

9 The Steps XYINIT writes files for positional corrections of the detector plane. Most modern detectors provide already corrected images so that these to files are normally flat. INIT determines initial detector background IDXREF indexing: unit cell dimensions and crystal orientation DEFPIX set active dectector area (exclude resolution cut-off, beam stop shadow,... ) (XPLAN, optional) generate strategy table with data completeness depending on starting angle total scan width INTEGRATE determine reflection intensities CORRECT applies corrections (polarisation, Lorentz-correction,... ), scales reflections, reports data statistics CCP4 Chicago 2011: XDS 9/29

10 Program Flow Each step must be passed at least once - the subsequent steps depend on files produced by the previous steps. Each step creates a log-file (XYINIT.LP, INIT.LP,...). IDXREF is the main hurdle - once unit cell and crystal orientation are determined, integration usually runs smoothly. CORRECT summarises the quality of the data. Mostly IDXREF.LP and CORRECT.LP should be inspected. CCP4 Chicago 2011: XDS 10/29

11 Step Dependencies Step XYCORR (Important) output files INIT COLSPOT IDXREF DEFPIX SPOT.XDS XPARM.XDS FRAME.cbf SPOT.XDS BKGPIX.cbf ABS.cbf red: Text files with parameters or data cyan: control images (use xds-viewer) black: data files for further processing INTEGRATE INTEGRATE.HKL FRAME.cbf CORRECT XDS_ASCII.HKL GXPARM.XDS CCP4 Chicago 2011: XDS 11/29

12 SPOT.XDS written by COLSPOT: Detector coordinates and Intensity of strong spots to be used for indexing: X(pixel) Y(pixel) #image counts amended by Miller-Indices IDXREF after determination of unit cell and crystal orientation X(pixel) Y(pixel) #image counts H K L < : cannot index within tolerance (MAXIMUM_ERROR_OF_SPOT_POSITION) CCP4 Chicago 2011: XDS 12/29

13 !!! ERROR!!! SOLUTION IS INACCURATE Correct indexing is crucial for data integration. If XDS cannot index more than 70 % of all reflections listed in SPOT.XDS, it stops with the above error message. Most common reasons: 1. Wrong ORGX, ORGY 2. Poor data quality XDS has a very robust indexing method and depending on the situation, one can usually continue integration by one or a combination of the following means. CCP4 Chicago 2011: XDS 13/29

14 Wrong ORGX, ORGY The detector origin ORGX, ORGY in XDS.INP is provided in pixels. Typically it is in the centre of the detector, so a good start is ORGX=NX/2, ORGY=NY/2, i.e. half the detector dimensions. Illustration of the sensitivity to ORGX, ORGY Diffraction image from Thaumatin with a medium size cell 58 Å, 58 Å, 150 Å, 90, 90, 90. Coordinates of the four magnified spots: Coordinates Difference Difference(mm) An offset of the origin of about 0.5 mm means a shift of 1 in Miller-Indices, mis-indexing. CCP4 Chicago 2011: XDS 14/29

15 Complaints about ORGX, ORGY A good indicator of a wrongly set origin: 1. refined detector distance far from real value 2. ORGX, ORGY drift far away from initial value IDXREF.LP: ***** DIFFRACTION PARAMETERS USED AT START OF INTEGRATION ***** [...] DETECTOR ORIGIN (PIXELS) AT (XDS.INP: ) CRYSTAL TO DETECTOR DISTANCE (mm) (XDS.INP: 200) The detector distance reported during data collection (image header) should be correct within fractions of mm at a proper beamline and not drift by 11 mm. CCP4 Chicago 2011: XDS 15/29

16 1. Start with ORGX=NX/2, ORGY=NY/2 (use adxv [3] to read out header information). Often XDS refines these values reasonably well. Correcting ORGX, ORGY CCP4 Chicago 2011: XDS 16/29

17 2. Check the image for the beam centre. At θ = 0 this correpsonds well to ORGX, ORGY. Ice rings are particularly usefule, they are centred at about the beam centre (use circle fitting option in mosflm). Correcting ORGX, ORGY CCP4 Chicago 2011: XDS 17/29

18 Carry on regardless The XDS error message SOLUTION IS INACCURATE does not necessarily mean that something is seriously wrong. XDS is rather careful. The step IDXREF still refines the experimental parameters and writes them into XPARM.XDS. This is all we need to continue. Check IDXREF.LP for Number of indexed reflections ***** INDEXING OF OBSERVED SPOTS IN SPACE GROUP # 1 ***** 1909 OUT OF 2506 SPOTS INDEXED. 0 REJECTED REFLECTIONS (REASON: OVERLAP) 597 REJECTED REFLECTIONS (REASON: TOO FAR FROM IDEAL POSITION) with 30 % of indexed reflections the parameters in XPARM.XDS are probably good enough to converged to the correct values during INTEGRATE and CORRECT. refined detector distance and detector origin do not shift too much Set JOB = DEFPIX INTEGRATE CORRECT and integrate your data. If something did seriously go wrong, the file XPARM.XDS was not written. CCP4 Chicago 2011: XDS 18/29

19 DEFPIX: active detector mask DEFPIX sets the area of the detector which is taken into account during integration. It takes into account: 1. INCLUDE_RESOLUTION_RANGE default: 20 Å to detector edge. 2. VALUE_RANGE_FOR_TRUSTED_DETECTOR_PIXELS exclude shadowed regions, e.g. beamstop, cryo stream nozzle 3. UNTRUSTED_RECTANGLE exclude gaps of CCD chips e.g. Pilatus detector 4. EXCLUDE_RESOLUTION_RANGE exclude ice rings from data integration CCP4 Chicago 2011: XDS 19/29

20 VALUE RANGE FOR TRUSTED DETECTOR PIXELS ABS.cbf BKGPIX.cbf, all included VALUE RANGE FOR TRUSTED DETECTOR PIXELS= CCP4 Chicago 2011: XDS 20/29

21 VALUE RANGE FOR TRUSTED DETECTOR PIXELS ABS.cbf BKGPIX.cbf, shadows removed VALUE RANGE FOR TRUSTED DETECTOR PIXELS= CCP4 Chicago 2011: XDS 21/29

22 INTEGRATE and CORRECT: Intensity determination and fine tuning INTEGRATE determine spot intensities based on parameters (saved in XPARM.XDS) CORRECT experimental corrections (e.g. Lorentz- and Polarisation-correction), refine parameters and determine space group (saved to GXPARM.XDS) CORRECT write FRAME.cbf: predicted spot positions encircled check correctness of predictions CCP4 Chicago 2011: XDS 22/29

23 Recycling Parameters (in GXPARM.XDS) depend on measured intensities Intensities (including corrections) depend on Parameters rename GXPARM.XDS to XPARM.XDS and rerun XDS (JOB = DEFPIX INTEGRATE CORRECT) to improve results. This way one should also set the correct high- and low- resolution cut-offs CCP4 Chicago 2011: XDS 23/29

24 Resolution Cut-Off The default resolution range in XDS is 20 Å to the detector edge INCLUDE_RESOLUTION_RANGE= Medium to low resolution data: increase 20 Å to 30 Å or even 50 Å (check BKGPIX.cbf) After second round of integration: determine high-resolution cut-off. My favourite: xprep - higher number of resolution shells than e.g. listed in CORRECT.LP Why after second round? Correct space group rather than P 1 more symmetry related reflections more reliable data statistics, especially I/σ I CCP4 Chicago 2011: XDS 24/29

25 High Resolution Cut-Off Resolution #Data #Theory %Complete Redundancy Mean I Mean I/s Rmerge Rsigma Inf <----- I/sigma > <----- I/sigma < Inf CCP4 Chicago 2011: XDS 25/29

26 Low Resolution Cut-Off BKGPIX.cbf BKGPIX.cbf Default resolution range (20 Å - edge) - includes noise at edge - loose low resolution reflections Adjusted resolution range (40 Å Å) - I/σ I 2 in outer shell - low resolution reflection important for shape of molecule (MR, refinement) CCP4 Chicago 2011: XDS 26/29

27 Generating XDS.INP Download detector specific template and fill in manually. Advantage: Double-check of experimental settings do never solely rely on correctness of image headers! Many beamlines offer automatic generation of XDS.INP. Crucial: Beamline specific chip borders and list of dead pixels for Pilatus detectors. Script generate XDS.INP by Kay Diederichs XDS.INP Works with MAR-CCD, ADSC, and Pilatus 6M detectors Number of images Reads all information from image header BKGPIX.cbf from Pilatus detector: Chip borders and dead pixels excluded. CCP4 Chicago 2011: XDS 27/29

28 XDSSTAT The program XDSSTAT (Kay Diederichs, also from the XDS Wiki) creates more detailed statistics from XDS integration. The output can be viewed with a file browser - similar tables like scala output. General rule of thumb: developers like encouragment if you miss a feature point it out to the program author to increase the chance of the feature being incorporated (One of the most efficient examples : Paul Emsley & Bernd Lohkamp (Coot), Pavel Afonine (phenix)). CCP4 Chicago 2011: XDS 28/29

29 References 1. Leslie, A.G.W., Recent changes to the MOSFLM package for processing film and image plate data, Joint CCP4 + ESF-EAMCB Newsletter on Protein Crystallography (1992), No A.N. Popov and Bourenkov, Choice of data-collection parameters based on statistic modeling Acta Crystallogr. (2003). D59, Andrew Arvai, arvai/adxv.html CCP4 Chicago 2011: XDS 29/29