Organization of validation tests

Size: px

Start display at page:

Download "Organization of validation tests"

Erik Garrett
5 years ago
Views:

Dominique FASBENDER, Cozmin LUCAU www.jrc.ec.europa.

1 Organization of validation tests (Ortho-imagery & GNSS receivers) Berlin, November 5-7, 2013 Philippe LOUDJANI, Dominique FASBENDER, Cozmin LUCAU Serving society Stimulating innovation Supporting legislation

2 Validation/certification overview In the frame of CAP direct payments and associated On-The-Spot checks, agricultural parcel areas shall be determined as laid down in Art.34 of R.1122/2009. According to this article, MS shall use measurement tools that are proven to assure measurement of quality at least equivalent to that required by applicable technical standard as drawn up at Community level. The quality of a measurement tool (e.g. GNSS equipment, remote sensing ortho-images - cf. Art.20 of R.73/2009) can be characterized by a number of parameters such as its bias, precision and accuracy. In order to evaluate the reliability and accuracy of area measurement with GNSS receivers and orthoimagery (because the assessment of GNSS point accuracy is not sufficient), the JRC elaborated a validation protocol for the area measurements based on the ISO 5275 norm. WikiCAP + direct contact!! 8 November

4 Validation/certification overview Since 2009 (2007) main tools used for area measurements are: GNSS receivers - Validated (MS or RL*) Laser, etc * Reference laboratories 8 November olerance_validation_method 4

5 Validation/certification overview Since 2009 (2008) main tools used for area measurements are: GNSS receivers - Validated (MS or RL*) - Certified (Navcert) Laser, etc * Reference laboratories 8 November olerance_validation_method 5

6 Validation/certification overview Since 2009 (2008) main tools used for area measurements are: GNSS receivers - Validated (MS or RL*) - Certified (Navcert) Orthoimagery - Validated according to the proposed protocol Laser, etc * Reference laboratories 8 November olerance_validation_method 6

7 Validation/certification overview Since 2009 (2008) main tools used for area measurements are: GNSS receivers - Validated (MS or RL*) - Certified (Navcert) Orthoimagery - Validated according to the proposed protocol - Rule of thumb (R=1.5 * pixel size)!!! Laser, etc * Reference laboratories 8 November olerance_validation_method 7

8 Validation/certification overview Since 2009 (2008) main tools used for area measurements are: OK for pixel size > 1 m GNSS receivers - Validated (MS or RL*) - Certified (Navcert) Orthoimagery - Validated according to the proposed protocol - Rule of thumb (R=1.5 * Not good pixel size)!!! correlation for smaller pixel size (<1m)!!! Laser, etc * Reference laboratories 8 November ance_validation_method_-_recommendation_for_orthoimagery

9 Validation/certification overview Since 2009 (2008) main tools used for area measurements are: GNSS receivers - Validated (MS or RL*) - Certified (Navcert) Orthoimagery - Validated according to the proposed protocol - Rule of thumb (R=1.5 * pixel size)!!! Laser, etc * Reference laboratories Orthoimagery from 2013 survey. all bellow 1.0 m We strongly recommend to validate the orthoimagery according to the proposed validation protocol!!! 8 November ance_validation_method_-_recommendation_for_orthoimagery

10 Validation/certification overview Reproducibility limit calculated in validation process allows to classify the area measurement method to the one of the following classes: (1) "1.50 m" for RL* inside (1.25m, 1.5m]; (2) "1.25 m" for RL inside (1.0, 1.25m]; (3) "1.00 m" for RL inside (0.75m, 1.0m]; (4) "0.75 m" for RL inside (0.50m, 0.75m]; (5) "0.50 m" for RL below 0.50m. *RL = Reproducibility limit can be used with GNSS devices!!! Buffer tolerance = 0 must be used with reference parcels!!! 8 November

11 Validation method components Number of parcels More parcels, more reliable assessment (but more workload) If GNSS at least 6 parcels If ortho image at least 30 parcels

be of the same object If GNSS (borders marked with

12 Borders Validation method components Parcels should have unambiguous borders to ensure that all measurements will be of the same object If GNSS (borders marked with sticks, 1 peg / 25m) If ortho image (shape around the parcel)

of the same object For ortho images some parcels should be

13 Borders Validation method components Parcels should have unambiguous borders to ensure that all measurements will be of the same object For ortho images some parcels should be selected with easily identifiable borders and some with fuzzy borders

Borders Validation method components Parcels should have unambiguous borders to ensure that all measurements will be of the same object For ortho images However, parcels should be carefully selected

14 Borders Validation method components Parcels should have unambiguous borders to ensure that all measurements will be of the same object For ortho images However, parcels should be carefully selected so as to avoid interpretation problems and lead to some parcels rejections later on during the analysis Examples of parcels with limits difficult to delineate (photo interpret) and that should not be part of the selected parcels sample

15 Validation method components Shape and size Sizes spread along the typical size range of the country Shape of the fields should vary from very simple shapes (e.g. rectangular) to some irregular shapes with high perimeter to area ratio. For ortho images Parcel sizes should cover the range observed in the control area S : small M : medium L : large Parcel shapes should vary SF1 : compact SF2 : elongated SF3 : very elongated

16 Validation method components Number of operators More operators, more reliable assessment (but more workload) If GNSS at least 2 to 3 If ortho image at least 6 Sets of repetitions of measurements If GNSS at least 2 sets / operator measured at different times (to consider changes of satellite constellations) of 4 repetitions (for GNSS 2 clockwise, 2 anticlockwise) made in a row At least 32 measurements per parcel If ortho image 1 set / operator of 4 repetitions made in different time (avoid copying limits) At least 24 measurement per parcels

17 For GNSS devices Obstructions of the horizon In mountainous areas or on fields (partly) bordered by trees, the visibility of the satellites may be limited. If such parcels are common in the region or country, such parcels should be included in the test. Settings Validation method components Perform the test with exactly the same settings that are planned to be used in the future work of the device (max DOP, S/N ratio, logging interval). Method Perform the test with exactly the same method that is planned to be used in the future work of the device (continuous (1 rec./s better than 1 rec./5m) or vertex (every 25m)). We strongly recommend to validate the device for the both methods.

reference area can (should) be taken form the LPIS If no corresponding LPIS

18 Validation method components Last but not least Need to compare measurements with a reference area ( true area ) and perimeter The reference area can (should) be taken form the LPIS If no corresponding LPIS parcel, the reference area should be established with a surveying tools or GPS RTK measurements

19 Validation GNSS devices/modes/methods validates since 2007 by MS!!! Ongoing analyses Number of validated GNSS receivers and ortho-images devices per year 8 November

20 Validation 2013 very long fields PL Elongated shape for parcels L < 1000 m W (50) m Vertices every m time consuming 8 November

21 Validation ATV 8 November

22 Validation forest 8 November

23 Thank you for your attention! 8 November

Some elements on On-The-Spot Checks (OTSC) and area measurement

Some elements on On-The-Spot Checks (OTSC) and area measurement DSCG/2014/32 FINAL Philippe LOUDJANI MARS (Monitoring Agricultural Resources) Unit DG Joint Research Centre Evidence-based scientific and