Factors that affect the accuracy of UAS surveys

Transcription

1 Factors that affect the accuracy of UAS surveys Dr. Dimitrios Bolkas; Department of Surveying Engineering, Pennsylvania State University, Wilkes-Barre Campus PSLS Surveyor s Conference Hershey, PA

2 Your Input How many of you consider yourselves as beginners in drone photogrammetry? How many of you consider yourselves as intermediate level in drone photogrammetry? How many of you consider yourselves as advanced level in drone photogrammetry?

3 Outline (1/2) UAS Applications/ Motivation Objectives Study area and data acquisition (UAS and TLS) Checkpoint and DEM comparison Scale error, camera selfcalibration, rolling shutter, image resolution and geometry Questions and discussion Break

4 Outline (2/2) Image matching and software comparison, shadows, altitude effect, histogram comparison, geo-referencing (GCPs and GNSS+GCPs), accuracy assessment Practical guidelines Questions and discussion

5 Geomatics Sensors medium.com drl.de Surveyequipment.com Nex and Remondino (2014)

6 Applications Archeology Neptune temple in Paestum, Italy Nex and Remondino (2014)

7 Applications (Cont.) Geological and mining Raeva et al. (2016)

8 Monitoring Applications (Cont.) Turner et al. (2015)

9 Applications (Cont.) Surveying Mapping Nex and Remondino (2014)

10 Motivation Knowing the accuracy of suas derived elevations is important for several applications Accuracy of photogrammetric UAS surveys can vary based on (Eltner et al. 2016): o Type of digital camera and camera self-calibration o Flying height o Target scene, texture, shadows o Number of images and their overlap (geometry) o Image matching performance (software) o Number of ground control points (GCPs) for georeferencing

11 Your Input What about your applications? What affects your accuracy? What have you noticed? Any challenging accuracy requirements?

12 Objectives Understand how discussed factors affect elevation accuracy Understand the difference in accuracy between ground control point and direct georeferencing Insights on accuracy assessment Learn best practices for field data acquisition

13 Data acquisition for this workshop

14 UAV acquisition and processing pipeline Nex and Remondino (2014)

15 Aibot X6 Dead weight: 3.4 kg suas: Equipment Take-off weight kg Flight height up to 500 m (1,640 ft) over ground; 3,000 m (9,843 ft) above sea level Flight time 8-10 minutes with one battery set

16 suas: Equipment Nikon D Megapixels Sensor size 23.5 mm 15.6 mm Image area 6,000 4,000 pixels Shutter up to 1/4,000 ISO ,600 Focal length: mm Max aperture: f/ Weight: 395 grams grams = 660 grams

17 suas Study Area and Control Three Leica GS 15 GNSS Collected data simultaneously for at least ten minutes before moving a receiver to the next GCP Average postadjustment standard deviations of GCPs was 1 mm for easting and northing coordinates, and 2 mm for elevations Penn State Wilkes-Barre Area 2.25 ha

18 Flight Parameters Date: June 2017, and October Altitudes: 90 m (300 ft) and 50 m (160 ft) Nadir Images only (some oblique is available from 30 m altitude in the October flight) Camera settings o 18 mm focal length o Shutter speed: 1/1,000, 1/2,000 o Ground speed of 3/m sec

19 Flight Parameters (Cont.) Photograph overlap was set to 70% forward and 50% side 76 pictures June pictures 57 pictures October pictures

20 Flight Parameters (Cont.) Ground spacing o 1.9 cm for 90 m o 1.1 cm for 50 m However, sub-sampled images by factor of 2: o 3.8 cm for 90 m o 2.2 cm for 50 m 90 m flight 50 m flight

21 Data Processing Workflow Processing in Agisoft Photoscan Images Initial alignment of Photos and camera selfcalibration Automatic detection of tie points Manual GCP addition Refinement of alignment, camera self-calibration, and geo-referencing Point-cloud generation Dense point-cloud DEM generation DEM

22 Structure from Motion Correspondence search Feature detection on each image Keypoint matching Keypoint verification

23 Structure from Motion (Cont.) 90 m: ~30,000 points 50 m: ~85,000 points

24 GCP Manual Matching 90 m flight 50 m flight

25 PC Generation 90 m flight 50 m flight

26 suas Point-cloud Example

27 suas Point-cloud Example (Cont.)

28 suas Point-cloud Example

29 suas Digital Elevation Models (DEMs) 90 m (300 ft) 50 m (160 ft)

30 Assessment Data Leica Scanstation P40 Average point spacing of 0.5 cm to 1 cm Total station Leica TS 15: 93 Checkpoints

31 TLS data Point-clouds from 12 locations on June Resolution: 2 mm at 10 m Average point-spacing: cm

32 TLS data Example (Cont.)

33 TLS data Example (Cont.)

34 TLS data Segmentation (Cont.) Mean (StD) of Number of TLS Average point- slope ( ) points spacing (cm) Parking (1.7) 12,000, Parking (1.7) 18,000, Road sloped 8.4 (1.7) 700, Road flat 4.0 (6.4) 10,000, Pavement 4.2 (2.6) 1,000, Roof 39.5 (17.0) 200, Observatory 41.5 (24.5) 400, Field area 6.0 (2.9) 45, Field sloped (5.6) 3, Field sloped (6.1) 7,

35 ΔE i image = E i ΔN i image = N i Coordinate Comparison Comparison from checkpoints UAS (imageሻ UAS (imageሻ Ei TS Ni TS ΔH i image = H i UAS (imageሻ Hi TS where,e TS i, N TS i, H TS i are the easting and northing coordinates, and orthometric height of total station point i; E i UAS (imageሻ, Ni UAS (imageሻ, Hi UAS (imageሻ are the easting and northing coordinates, and orthometric height of the corresponding point i in the UAS images; and ΔE i image, ΔNi image, ΔHi image are the differences in easting and northing coordinates, and orthometric height. RMSE = mean 2 + StD 2

36 Height Comparison suas-derived DEM TLS point-cloud ΔΗ i DEM = H i UAS (DEMሻ Hi TLS RMSE = mean 2 + StD 2 H i TLS is the orthometric height of point i derived from TLS; H i UAS (DEMሻ is the UAS-derived orthometric height that is interpolated from the UAS DEM onto the location of point i; and ΔΗ i DEM is their height difference at point i.

37 Theoretical Error Theoretical error σ theo = D2 Bf σ i o σ theo is the theoretical coordinate error o f is the focal length o B is the distance between images o D is the mean distance between camera and surface o σ i is the error image measurement Theoretical error ratio e t = σ m σ theo o σ m is the measured (actual) error Eltner et al. (2016)

38 Relative error Relative error e r = σ m D o e r is the relative error o σ m is the measured (actual) error o D is the mean distance between camera and surface Eltner et al. (2016)

39 Let s Look at Some Results

40 Theoretical Error Ratio Theoretical error is influenced only by the focal length, camera-surface distance, and distance between images Measured error typically is much greater than predicted from theoretical error Theoretical error ratio e t = σ m σ theo Eltner et al. (2016)

41 Scale and Distance Distance from sensor Error However, there is no linear trend (circles) <10 cm accuracy can be achieved for <200 m distances Eltner et al. (2016)

42 Image Resolution The larger the pixel size the higher the amount of light that can be captured A more distinct signal is measured Keep in mind: Pixel number gives no information about the actual sensor size Example: two sensors having the same number of pixels, but different pixel sizes o The larger sensor will provide better image quality due to reduced noise than the smaller sensor

43 Image Resolution (Cont.) Down-sampling of images speeds up processing However, image information is reduced due to interpolation Agisoft has the following image quality levels: o Ultra high (original size) o High (down-sampling by factor of 4 2 in each side) o Medium (down-sampling by factor of 16) o Low (down-sampling by factor of 64) o Lowest (down-sampling by factor of 256)

44 Image Resolution (Cont.) Memory requirement in Agisoft Building Model (Height-field mode) What is your memory capability?

45 Image Resolution: Example 90 m altitude flight RMSE in cm Area Quality Level Ultra high High Medium Parking Parking Road sloped Road shadow Pavement Roof Observatory Field area Field sloped Field sloped

46 Image Resolution: Example 50 m altitude flight RMSE in cm Area Quality Level Ultra high High Medium Parking Parking Road sloped Road shadow Pavement Roof Observatory Field area Field sloped Field sloped

47 Camera setting: Rolling shutter ons/4/49/rolling_shutter_smil.svg Image from: Depending on the flying and shutter speed, this can lead to additional distortions.

48 ΔR, Δc Rolling shutter modeling Each image row requires 6 exterior orientation parameters (these will depend with time!) In practice, the incremental rotations and translations are considered constant with time Describing the rolling shutter using 6 parameters as follows: R t = R 0 λδr R 0, c 0 c t = c 0 + λδc Where R t is the time-dependent orientation and c t the time-dependent position of the camera, R 0 and c 0 are the orientation and position, respectively, of the camera for the first row of the sensor, ΔR and Δc are the linear motions of the camera from the first to the last image row in terms of rotation and position, respectively, and λ 1,1 models the time (image row).

49 Shutter Speed: Motion Blur High ground speed and slow shutters can cause motion blur 1/ 1000 shutter Excel: Courtesy Leica

50 Shutter Speed: Motion Blur High ground speed and slow shutters can cause motion blur 1/ 2000 shutter Excel: Courtesy Leica

51 Shutter Speed: Motion Blur High ground speed and slow shutters can cause motion blur 1/ 500 shutter; we start to see some limitations Excel: Courtesy Leica

52 Shutter Speed: Motion Blur Motion Blur cm = 100 shutter speed 1 s groundspeed (m/sሻ Nominal GSD cm Object Distance m pixel size μm = focallength mm 10 1 Motion Blur px = Motion Blur(cmሻ Nominal GSD cm

53 Shutter Speed Setting: Example 1/2000: June and 76 pictures, 16 GCPs Ground speed 2.5 m/sec Motion blur: 0.1 cm or 0.1 px 1/1000: October and 57 pictures, 23 GCPs Ground speed 3 m/sec Motion blur: 0.3 cm or 0.3 px

54 Shutter Speed Setting: Results In general similar results for most areas (considering that more parameters vary e.g., images and GCPs) Area 90 m 50 m 1/2000 1/1000 1/2000 1/1000 Parking Parking Road sloped Road shadow Pavement Roof Observatory Field area Field sloped Field sloped

55 Camera Calibration Camera calibration is a key element for high quality point-clouds If a pre-calibration is not available then users have to perform a self-calibration It is preferable to use one single camera model for one image block

56 Camera Calibration Optimal camera calibration requires images taken from: o Different distances and positions o at different orientation (0-180 ) o with varying rotation along the optical axis o including tilted images o and using an abundant number of conjugate points (tie points) in the images Cramer et al. (2017): test site at Bochum University, Germany

57 Camera Calibration However, camera calibration parameters, estimated in laboratory conditions, may change under in-flight conditions In addition, a well-established calibration site might not be available Cramer et al. (2017) showed the instability of principal distance (equal to focal length if focused at infinity) and principal point components

58 Collinearity condition Camera Calibration The perspective center of the lens, the object point and the image point must be collinear A distortion means there is a deviation from collinearity Luhmann et al. (2006)

59 Camera Calibration Calibration parameters o Focal length o Principal point offset in x- and y- direction (c x and c y ) o Radial distortion coefficients (K 1, K 2, K 3, and K 4 ) o Tangential distortion coefficients (P 1, P 2, P 3, and P 4 ) o Affinity and non-orthogonality coefficients (B 1 and B 2 ) o Rolling shutter correction Image from Pullivelli A. 2005

60 Camera Calibration Radial (symmetric) distortion: variations in refraction Function of lens design, focusing distance, and object distance at a constant focus Correlated with image scale (principal distance) Include linear part that creates a second-zero crossing Luhmann et al. (2006)

61 Camera Calibration Tangential distortion (radial-assymetric) or de-centring distortion: caused by de-centring and misalignment of individual lens elements Much smaller effect that radial distortion (in low cost lenses significant tangential distortion can be present) Luhmann et al. (2006)

62 Camera Calibration Affinity and shear: deviations of the image coordinate system with respect to orthogonality and uniform scale of the coordinate axes Luhmann et al. (2006)

63 Fixed- versus Self- Calibration James and Robson (2014) showed effect of radial distortion error and benefit of having convergent images (e.g., oblique) Notice the benefit of converge images on self-calibration Note difference in colorbar labels

64 Fixed- versus Self- Calibration Oniga et al. (2018) flight of 21 images (17 oblique and 3 nadir) Using 3 GCPs Images and tie points positions for 23 m nadiral flight after the orientation process using (a) SfM method and (b) pre-calibration parameters RMSE as a function of GCP number and pre- versus selfcalibration Good image geometry and an abundant number of GCPs is necessary!

65 Experiment Experiment included two flights from 90 m and 50 m altitudes at Penn State Wilkes-Barre 23 GCPs were used (accuracy assessment from TLS) Only nadir images were used (sub-optimal scenario for camera calibration) Camera calibration parameters are examined for their correlation and significance (derive a camera model that makes sense) Show the effect of not modeling the rolling shutter 5 scenarios were created

66 Experiment Parameter f Yes Yes c x and c y Yes Yes B 1 and B 2 Yes Yes K 1 and K 2 Yes Yes K 3 Yes Yes K 4 Yes Yes P 1 and P 2 Yes Yes P 3 Yes Yes P 4 Yes Yes Rolling Shutter Yes No Camera self-calibration scenarios Correlations between parameters K 2, K 3 and K 4, and P 2, P 3 and P 4. For the 90-m flight: ρ K2,K 3 = 0.98, ρ K2,K 4 = 0.94, ρ K3,K 4 = 0.99, ρ P2,P 3 = 0.71, ρ P2,P 4 = 0.76, and ρ P3,P 4 = This suggested that 3 rd and 4 th order terms could be omitted from the calibration model as they are over-redundant

67 Experiment Parameter Camera self-calibration scenarios f Yes Yes Yes c x and c y Yes Yes Yes B 1 and B 2 Yes Yes Yes K 1 and K 2 Yes Yes Yes K 3 Yes Yes No K 4 Yes Yes No P 1 and P 2 Yes Yes Yes P 3 Yes Yes No P 4 Yes Yes No Rolling Shutter Yes No Yes High correlation between the principal point offset in the x- direction and P 1 (ρ cx,p 1 = 0.97 and ρ cx,p 1 = 0.87 for the 90- m and 50-m flight, respectively) Created two more scenarios

68 Experiment Parameter Camera self-calibration scenarios f Yes Yes Yes Yes Yes c x and c y Yes Yes Yes Yes No B 1 and B 2 Yes Yes Yes Yes Yes K 1 and K 2 Yes Yes Yes Yes Yes K 3 Yes Yes No No No K 4 Yes Yes No No No P 1 and P 2 Yes Yes Yes No Yes P 3 Yes Yes No No No P 4 Yes Yes No No No Rolling Shutter Yes No Yes Yes Yes

69 Camera Self-calibration: Result 90-m flight: RMSE in cm Camera self-calibration scenario Area Parking Parking Road sloped Road shadow Pavement Roof Observatory Field area Field sloped Field sloped

70 Camera Self-calibration: Result 50-m flight: RMSE in cm Camera self-calibration scenario Area Parking Parking Road sloped Road shadow Pavement Roof Observatory Field area Field sloped Field sloped

71 Camera Self-calibration: Result Using total station checkpoints Flight 90-m 50-m Coordinate Camera self-calibration scenario Easting Northing Orth. height Easting Northing Orth. height

72 Camera Self-calibration: Remarks In both flights scenario 5 was found to provide better RMSE values Rolling shutter effect is important and should be considered (especially in buildings) Buildings and objects with 3D structure (nonplanar) will be affected more from weak camera self-calibrations

73 Image Network Geometry Number of images o Higher number of images can improve quality o However, increase in image does not linearly increase accuracy Image overlap o Higher overlap leads to better results o But increases number of flight lines Obliqueness o Oblique images will enhance 3D reconstruction and camera self-calibration

74 Piermattei et al. (2015) Image Network Geometry Italian Julian Alps No of images = 35 No of images = 347 Accuracy 0.27 m to 0.33 m Accuracy 0.08 m to 0.16 m

75 Image Network Geometry Systematic doming as surface shape error James et al. (2017) 60% forward 30% side 80 images, collected from two sets of parallel flight An additional 18 images, in two gently banked turns (20 to the vertical) Ground pixel 1.25 cm

76 Image Network Geometry James and Robson (2012) Coastal cliff dataset Used images with near-parallel orientation versus near-parallel + convergent images Systematic error is reduced when convergent images are added

77 Image Network Geometry Addition of oblique images Point of interest with 126 additional images taken from 30 m altitude

78 Image Network Geometry Improves 3D reconstruction, but did not improve accuracy much Area Without Oblique With Oblique Mean StD RMSE Mean StD RMSE Roof +0.1 ± ± Observatory -6.2 ± ±

79 Nadir Imagery + GNSS What if we process the same images with precise GNSS positioning and all GCPs GNSS+23 GCPs: o RMSE for the 90-m flight data: 6.8 cm o RMSE for the 50-m flight data: 5.4 cm RMSE value dropped, but it is still high Some of the parameters that can affect these RMSE values: o Weak camera self-calibration o 3D reconstruction o Apparent vertical error in DEM comparisons

80 Z-Dimension Apparent Vertical Error Elevation error = tan (a) Horizontal Displacement Mean slope of observatory is 41.5 Horizontal error from checkpoints about 2 cm Elevation error = tan (41.5) 2 cm = 1.8 cm Important, but not the only factor Apparent position of measured point Sensor measures this point Horizontal error Slope angle Vertical error X-Dimension Hodgson, M. E., & Bresnahan, P. (2004). Accuracy of airborne LiDAR-derived elevation. Photogrammetric Engineering & Remote Sensing, 70(3),

81 Image Matching Extract primitives from two or more images and determination of 3D coordinates of matched feature points Important for alignment of images and reconstruction Approach of image matching and reconstruction can create different results

82 Software Comparison The following slides show processing software comparisons Material has been retrieved from published peer-reviewed journal publications Note that these comparisons provide a snapshot of software performance o As software are frequently updated Finding objects/datasets for benchmarking is an active research topic o Software can perform well on some datasets/object but not on others o Which dataset/object is the best for comparison? The goal of the following slides in to help you understand that you can get a different solution based on the software used

83 Software Comparison GCP and CP location DJI with Canon EOS 6D 20 MP Vertical 284 images 20 m altitude Oblique 88 images Terrestrial 133 images Jeon et al. (2017)

84 Software Comparison Context Capture Photoscan Pix4D 3D point cloud Jeon et al. (2017)

85 Software Comparison Context Capture Photoscan Pix4D 3D Model Jeon et al. (2017)

86 Software Comparison DJI with 20 MP camera 300 photos with 70-80% overlap 1 cm ground spacing Alidoost and Arefi (2017)

87 Software Comparison Alidoost and Arefi (2017)

88 Alidoost and Arefi (2017) Software Comparison

89 Software Comparison Remondino et al. (2017) tested software using various scenes and using terrestrial and aerial photogrammetry

90 Remondino et al. (2017) Software Comparison

91 Software Comparison Surface texture can also affect accuracy of point cloud Remondino et al. (2017)

92 Shadow Effect Areas of shadow create low contrast in pictures Which makes it difficult for the software to identify correspondences between conjugate pixels observed from two images

93 Shadow effect on DEM 90 m flight

94 DEM Shadows Effect of shadows in the DEM 90 m flight 50 m flight

95 DEM shadows: Results Flight 90 m flight 50 m flight Mean (cm) StD (cm) RMSE (cm) Mean (cm) StD (cm) RMSE (cm) No shadow -0.6 ± ± With Shadow Flight at 11 am in June 2017 Flight at 1-2 pm in October 2017

96 Altitude Effect Spatial depiction of height differences 90 m (300 ft) 50 m (160 ft)

97 Histogram Comparison: Overall Using ~15,500 TLS points (1 m spacing) 90 m flight 50 m flight

98 Altitude Effect Overall Height Comparison 90 m flight 50 m flight Mean StD RMSE Mean StD RMSE -2.1 ± ± Units: cm Majority of study area is in a vegetated region The two surveys show similar RMSE values due to the high uncertainty in vegetated areas

99 Coordinate Comparison Comparison from checkpoints 90 m flight 50 m flight Checkpoints Coordinate Mean StD RMSE Mean StD RMSE Target-based (7 points) Total station based (93 points) Easting -0.2 ± ± Northing 0.7 ± ± Elevation 0.7 ± ± Easting 1.1 ± ± Northing 0.0 ± ± Elevation -1.0 ± ± Units: cm Note: total station checkpoints are in parking lot areas and buildings

100 Histogram Comparison 90 m flight Red curve shows a fitted normal distribution Observatory Roof Parking 2 Field Area

101 Histogram Comparison 50 m flight Red curve shows a fitted normal distribution Observatory Roof Parking 2 Field Area

102 Geo-referencing Indirect geo-referencing Direct geo-referencing Image from Kersting APB, 2011

103 Questions Do you have GNSS-RTK capability for your UAS? Yes No How many GCPs do you use for indirect geo-referencing? >12 How many GCPs do you use for GNSS-assisted georeferencing? >6

104 GCP-only Previous Studies High accuracy with 3-4 GCPs (~20 GCP/km²) Accuracy significantly degrades even when using 4, 5, or even 9 and 15 GCPs Direct Geo-referencing <10 cm (~4 in) with no GCPs GCP+GNSS 3-6 cm (1-2.5 in) with 1 GCP

105 GCP Scenarios 0 GCPs 1 GCPs 2 GCPs 3 GCPs 4 GCPs 5 GCPs (a), (b), (c), and (d) are tested for the case of GNSS-assisted geo-referencing

106 GCP Scenarios (Cont.) 6 GCPs 8 GCPs 12 GCPs 16 GCPs 20 GCPs 23 GCPs

107 90 m altitude Results: GCP-only

108 50 m altitude Results: GCP-only

109 90 m altitude Results: GCP+GNSS

110 50 m altitude Results: GCP+GNSS

111 GCP-only GCP+GNSS Results: Checkpoint Comparison 90 m 50 m 90 m 50 m

112 Results: Geo-referencing GCP-Only

113 Results: Geo-referencing GCP+GNSS

114 Results: Self-Calibration GCP-Only

115 Results: Self-Calibration GCP+GNSS

116 Some GCP Geometry Scenarios Geometry of GCPs is not so important for > 4 GCPs, as we all expect to derive good/logical geometries Investigate geometry for: o GNSS + 1 GCP o GNSS + 2 GCPs o GNSS + 3 GCPs 200 m 210 m 70 m 165 m 140 m 155 m

117 GCP Geometry Scenarios: 90-m flight Results

118 GCP Geometry Scenarios: 50-m flight Results

119 GCP Number and Scenarios: Remarks GCP-only: 8-12 GCPs are needed for 1-2 cm RMSE (for example in parking lot areas and road segments) GNSS-PPK positioning: using 1-2 GCPs can lead to RMSE values of <5 cm in parking lot areas and road segments o RMSE values drop to 1-2 cm when using 4-5 GCPs. A higher number of GCPs is needed in complex areas than noncomplex ones (improve camera self-calibration) GCP geometry (1, 2, and 3 GCPs) did not provide significant results GNSS-PPK reduces variability of geo-referencing parameters (image position and orientation), as well as camera selfcalibration parameters, compared to the GCP-only case

120 What about accuracy assessment? How many checkpoints do you need to estimate your accuracy reliably? How good should they be?

121 Reference data superiority e s = σ m σ ref o o o Reference Superiority e s is the reference superiority σ ref is the reference error σ m is the measured error Reference data should be at least an order of magnitude better than measured error to be considered error free This can be difficult to achieve in practice (only with total stations and static relative GPS)

122 Reference Superiority Median of studies shows that measured error is merely twice the reference error! This means that error from the reference contributes in the estimated RMSE values! This becomes more challenging when flying at low altitudes (< 50 m) where ground spacing can be ~ 1cm Eltner et al. (2016)

123 Number of Vertical Checkpoints 90 m flight RMSE value at 100% is 4.6 cm

124 Number of Vertical Checkpoints 50 m flight RMSE value at 100% is 3.3 cm

125 Practical guidelines Trade-off between accuracy and altitude (if surveying a field area RMSE of 90 m = 50 m) Camera self-calibration becomes important for buildings and 3D objects o Need to derive a proper calibration model Include oblique images: o Enhances camera self-calibration o Complete 3D reconstruction Avoid shadows: deteriorate accuracy

126 Practical guidelines Different software can produce different results GCP-only: establish at least 12 GCPs for 1-2 cm accuracy (5-8 GCPs <10 cm) GNSS-RTK (PPK): establish at least 6 GCPs for 1-2 cm accuracy (1-2 GCPs <5 cm) GNSS-RTK enhances camera self-calibration as it de-couples dependency between interior and exterior orientation Validate survey accuracy using many checkpoints in different locations and surfaces o Reference accuracy should be <1 cm but can use ~1 cm for verification

127 Acknowledgements Mr. Timothy Sichler for flying the UAS Wyatt McMarlin, Matthew Boyes, Nicholas Myers, Aaron Martinez, and Evan Decker are acknowledged for their aid in the TLS and UAS data-acquisition Frank Lenik from Leica Geosystems for providing the Leica Scansation P40 This project was supported and funded by: o The Research Development Grant of ETCE (Engineering Technology and Commonwealth Engineering) o Undergraduate Research Allocation (Penn State Wilkes-Barre o Former Assistant DAA, Dr. Albert Lozano

128 Thank you for your attention! Questions?

129 References Alidoost, F., & Arefi, H. (2017). COMPARISON OF UAS-BASED PHOTOGRAMMETRY SOFTWARE FOR 3D POINT CLOUD GENERATION: A SURVEY OVER A HISTORICAL SITE. ISPRS Annals of Photogrammetry, Remote Sensing & Spatial Information Sciences, 4. Cramer, M., Przybilla, H. J., & Zurhorst, A. (2017). UAV cameras: Overview and geometric calibration benchmark. The International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, 42, 85. James, M. R., & Robson, S. (2012). Straightforward reconstruction of 3D surfaces and topography with a camera: Accuracy and geoscience application. Journal of Geophysical Research: Earth Surface, 117(F3). James, M. R., & Robson, S. (2014). Mitigating systematic error in topographic models derived from UAV and ground based image networks. Earth Surface Processes and Landforms, 39(10), James, M. R., Robson, S., & Smith, M. W. (2017). 3 D uncertainty based topographic change detection with structure from motion photogrammetry: precision maps for ground control and directly georeferenced surveys. Earth Surface Processes and Landforms, 42(12), Jeon, E. I., Yu, S. J., Seok, H. W., Kang, S. J., Lee, K. Y., & Kwon, O. S. (2017). Comparative evaluation of commercial softwares in UAV imagery for cultural heritage recording: case study for traditional building in South Korea. Spatial Information Research, 25(5),

130 References Luhmann, T., Robson, S., Kyle, S. A., & Harley, I. A. (2006). Close range photogrammetry: principles, techniques and applications. Whittles. Nex, F., & Remondino, F. (2014). UAV for 3D mapping applications: a review. Applied geomatics, 6(1), Oniga, V. E., Pfeifer, N., & Loghin, A. M. (2018). 3D Calibration Test-Field for Digital Cameras Mounted on Unmanned Aerial Systems (UAS). Remote Sensing, 10(12), Piermattei, L., Carturan, L., & Guarnieri, A. (2015). Use of terrestrial photogrammetry based on structure from motion for mass balance estimation of a small glacier in the Italian alps. Earth Surface Processes and Landforms, 40(13), Raeva, P. L., Filipova, S. L., & Filipov, D. G. (2016). VOLUME COMPUTATION OF A STOCKPILE-A STUDY CASE COMPARING GPS AND UAV MEASUREMENTS IN AN OPEN PIT QUARRY. International Archives of the Photogrammetry, Remote Sensing & Spatial Information Sciences, 41. Turner, D., Lucieer, A., & De Jong, S. M. (2015). Time series analysis of landslide dynamics using an unmanned aerial vehicle (UAV). Remote Sensing, 7(2), Remondino, F., Spera, M. G., Nocerino, E., Menna, F., & Nex, F. (2014). State of the art in high density image matching. The Photogrammetric Record, 29(146), Remondino, F., Nocerino, E., Toschi, I., & Menna, F. (2017). A CRITICAL REVIEW OF AUTOMATED PHOTOGRAMMETRIC PROCESSING OF LARGE DATASETS. International Archives of the Photogrammetry, Remote Sensing & Spatial Information Sciences, 42.