Metric Accuracy Testing with Mobile Phone Cameras

Transcription

1 Metric Accuracy Testing with Mobile Phone Cameras Armin Gruen,, Devrim Akca Chair of Photogrammetry and Remote Sensing ETH Zurich Switzerland Devrim Akca, the 21. ISPRS Congress, Beijing, July 9, 28. 1

2 WHY MOBILE PHONE CAMERAS? *) availability of small hardware: *) availability of small hardware labtops, PDAs,, etc.. *) available know-how & software modules: sequential estimation, semi-automated and automated triangulation and surface model generation and texturing We anticipate the (future) possibilities of on-line processing of the acquired image data by mobile phone cameras. This is a concept from Mobile Mapping to Mobile 3D Modeling (Mobile Photogrammetry) Objective of the work: Objective of the work: Geometric Calibration and accuracy validation of mobile phone cameras Devrim Akca, the 21. ISPRS Congress, Beijing, July 9, 28. 2

3 CONTENT OF THE PRESENTATION: Some applications of mobile phone cameras Technical specifications of the used cameras Calibration test field Accuracy tests Sony Ericsson K75i Nokia N93 Sony DSC W Sony DSC F828 - camera embedded mobile phone - camera embedded mobile phone - off-the-shelf digital camera - off-the-shelf digital camera JPEG test with Sony DSC F828 Temporal stability test with Nokia N93 Analysis of results & conclusions Devrim Akca, the 21. ISPRS Congress, Beijing, July 9, 28. 3

4 Rapid progress in mobile phone camera technology: Sharp (24) 2 Mpixel CCD camera module Sharp (25) 3 Mpixel CCD camera modules Samsung (26) Mpixel CCD camera integrated handy SCH-B6 Devrim Akca, the 21. ISPRS Congress, Beijing, July 9, 28. 4

5 Some applications of mobile phone cameras: Character / text recognition facial animation face identification / recognition panoramic image capturing context awareness content provision to GIS (+ GPS chip) LBS applications etc. In spite of the availability of a broad diversity of applications, the metric capabilities and characteristics of mobile phone cameras have not been investigated so far. Devrim Akca, the 21. ISPRS Congress, Beijing, July 9, 28. 5

6 Sony Ericsson K75i Nokia N93 Sony DSC W Sony DSC F828 Camera Sensor Pixel size Image format Lens Focal length Optical zoom Auto focus Aperture Output format K75i CMOS 1/3.2 type 4.5 x 3.4 mm 2.8 micron 1632 x mega pixel 4.8 mm No Yes F2.8 (fixed) Only JPEG N93 CMOS 1/3.2 type 4.5 x 3.4 mm 2.2 micron 248 x mega pixel Carl Zeiss Vario-Tessar mm 3X Yes F3.3 (fixed) Only JPEG W CCD 1/1.8 type 7.2 x 5.3 mm 2.2 micron 3264 x mega pixel Carl Zeiss Vario-Tessar mm F Only JPEG CCD 2/3 type 8.8 x 6.6 mm 2.7 micron 3264 x mega pixel Carl Zeiss T* Vario-Sonnar mm 7X Yes F JPG and TIFF Devrim Akca, the 21. ISPRS Congress, Beijing, July 9, X Yes F828

7 3.4m 1.m Test field HIL C57.3, ETH Zurich Leica TC 3 2.m Z Leica TC 22 Base for the scale:.51 ±. mm by Interferometry X Y Leica Axyz system Hardware + software 87 GCPs, the average precisions X,Y,Z: ±.3, ±.5, ±.3 mm. Leica Axyz system. Devrim Akca, the 21. ISPRS Congress, Beijing, July 9, 28. 7

8 Imaging Quality: K75i N93 W F828 1:623 1:585 1:32 1:358 Image scale 1:863 1:829 1:46 1:513 1:977 1:961 1:55 1:583 Low-level image enhancement effects at K75i, N93 and W. F828 has the best overall image quality considering all images. JPEG artifacts on N93 images. Image Measurements: LS Template Matching All process & analysis software are in-house developed Devrim Akca, the 21. ISPRS Congress, Beijing, July 9, 28. 8

9 K75i: Accuracy test network configuration stations, convergent geometry, 18 pictures 9 pictures normal case 9 pictures rotated, 3/ 3/ 3: -9 / +9 / 18 Devrim Akca, the 21. ISPRS Congress, Beijing, July 9, 28. 9

10 K75i: Absolute accuracy test - #18 results Number of images: 18 (JPEG) Pixel size: 2.8 micron Ver GCP 87 CHK TIE 9 APs Rej Sigma (μm) (pixel) STD-X of CHK+TIE points.291 STD-Y.558 STD-Z.251 RMSE-X at CHK points N.A. RMSE-Y N.A. RMSE-Z N.A. Ver GCP CHK TIE APs Rej : Version number : Number of control points : Number of check points : Number of tie points : Number of additional parameters : Rejected rays by data-snooping, Sigma : Standard deviation of unit weight a posteriori STD : Average theoretical precision values of CHK/GCP coordinates RMSE : Empirical accuracies of CHK/GCP coordinates in object space Devrim Akca, the 21. ISPRS Congress, Beijing, July 9, 28.

11 K75i: Absolute accuracy test #18 Residuals in image space (1-4) Devrim Akca, the 21. ISPRS Congress, Beijing, July 9,

12 K75i: Absolute accuracy test #18 Residuals in image space (5-8) Devrim Akca, the 21. ISPRS Congress, Beijing, July 9,

13 K75i: Absolute accuracy test #18 Residuals in image space (9-12) Devrim Akca, the 21. ISPRS Congress, Beijing, July 9,

14 K75i: Absolute accuracy test #18 Residuals in image space (13-16) 16) Devrim Akca, the 21. ISPRS Congress, Beijing, July 9,

15 K75i: Absolute accuracy test #18 Residuals in image space (17-18) 18) Devrim Akca, the 21. ISPRS Congress, Beijing, July 9,

16 K75i: Accuracy test Image rejection REDs: Excluded images GREEN: Remained images 9 pictures normal case, 1/1/1: -9 / +9 /18 rotated NOW totally 12 pictures Devrim Akca, the 21. ISPRS Congress, Beijing, July 9,

17 K75i: Accuracy test #12 results Number of images: 12 (JPEG) Pixel size: 2.8 micron Ver GCP CHK TIE APs Rej Sigma STD-X STD-Y STD-Z RMSE-X RMSE-Y RMSE-Z (μm) (pixel) of CHK+TIE points at CHK points free.21 Devrim Akca, the 21. ISPRS Congress, Beijing, July 9,

18 K75i: #12 - Residuals in image space (Ver.11: 87gcp/chk, pictures 1-4) 1 Devrim Akca, the 21. ISPRS Congress, Beijing, July 9,

19 K75i: #12 - Residuals in image space (Ver.11: 87gcp/chk, pictures 5-8) 5 Devrim Akca, the 21. ISPRS Congress, Beijing, July 9,

20 K75i: #12 - Residuals in image space (Ver.11: 87gcp/chk, pictures 9-12) 9 Devrim Akca, the 21. ISPRS Congress, Beijing, July 9, 28. 2

21 N93: Accuracy test Network configuration (6th( February, 27 6th February, 27 ) stations, convergent geometry, 13 pictures 9 pictures normal case, 2/2 pictures +9 /-9 rotated Devrim Akca, the 21. ISPRS Congress, Beijing, July 9,

22 N93: Accuracy test Results (6th February, 27 ) Number of images: 13 (JPEG) Pixel size: 2.2 micron Ver GCP CHK TIE APs Rej Sigma STD-X STD-Y STD-Z RMSE-X RMSE-Y RMSE-Z (μm) (pixel) of CHK+TIE points at CHK points free Devrim Akca, the 21. ISPRS Congress, Beijing, July 9,

23 N93: #13 Residuals in image space (Ver.21: 87gcp/chk, pictures 1-4) 1 Devrim Akca, the 21. ISPRS Congress, Beijing, July 9,

24 N93: #13 Residuals in image space (Ver.21: 87gcp/chk, pictures 5-8) 5 Devrim Akca, the 21. ISPRS Congress, Beijing, July 9,

25 N93: #13 Residuals in image space (Ver.21: 87gcp/chk, pictures 9-12) 9 Devrim Akca, the 21. ISPRS Congress, Beijing, July 9,

26 N93: #13 Residuals in image space (Ver.21: 87gcp/chk, picture 13) Devrim Akca, the 21. ISPRS Congress, Beijing, July 9,

27 W: Accuracy test Network configuration stations, convergent geometry 13 pictures 9 pictures normal case, 4 pictures +9 rotated Devrim Akca, the 21. ISPRS Congress, Beijing, July 9,

28 W: Accuracy test Results Number of images: 13 (JPEG) Pixel size: 2.2 micron Ver GCP CHK TIE APs Rej Sigma STD-X STD-Y STD-Z RMSE-X RMSE-Y RMSE-Z (μm) (pixel) of CHK+TIE points at CHK points free Devrim Akca, the 21. ISPRS Congress, Beijing, July 9,

29 W: #13 Residuals in image space (Ver.31: 87gcp/chk, pictures 1-4) 1 Devrim Akca, the 21. ISPRS Congress, Beijing, July 9,

30 W: #13 Residuals in image space (Ver.31: 87gcp/chk, pictures 5-8) 5 Devrim Akca, the 21. ISPRS Congress, Beijing, July 9, 28. 3

31 W: #13 Residuals in image space (Ver.31: 87gcp/chk, pictures 9-12) 9 Devrim Akca, the 21. ISPRS Congress, Beijing, July 9,

32 W: #13 Residuals in image space (Ver.31: 87gcp/chk, picture 13) Devrim Akca, the 21. ISPRS Congress, Beijing, July 9,

33 F828: Accuracy test Network configuration stations, convergent geometry 13 pictures 9 pictures normal case, 2/2 : +9 / -9 rotated Devrim Akca, the 21. ISPRS Congress, Beijing, July 9,

34 F828: Accuracy test Results Number of images: 13 (JPEG) Pixel size: 2.7 micron Ver GCP CHK TIE APs Rej Sigma STD-X STD-Y STD-Z RMSE-X RMSE-Y RMSE-Z (μm) (pixel) of CHK+TIE points at CHK points free.9 Devrim Akca, the 21. ISPRS Congress, Beijing, July 9,

35 F828: #13 - Residuals in image space (Ver.41: 87gcp/chk, pictures 1-4) 1 Devrim Akca, the 21. ISPRS Congress, Beijing, July 9,

36 F828: #13 - Residuals in image space (Ver.41: 87gcp/chk, pictures 5-8) 5 Devrim Akca, the 21. ISPRS Congress, Beijing, July 9,

37 F828: #13 - Residuals in image space (Ver.41: 87gcp/chk, pictures 9-12) 9 Devrim Akca, the 21. ISPRS Congress, Beijing, July 9,

38 F828: #13 - Residuals in image space (Ver.41: 87gcp/chk, picture 13) Devrim Akca, the 21. ISPRS Congress, Beijing, July 9,

39 F828: #13 JPEG Test Number of images: 13 (TIFF and JPEG) Pixel size: 2.7 micron Ver CoR GCP CHK TIE APs Sigma STD-X STD-Y STD-Z RMSE-X RMSE-Y RMSE-Z (μm) (pixel) of CHK+TIE points at CHK points TIFF 23,4KB Q 4,265KB Q7 562KB. CoR : Compression ratio Devrim Akca, the 21. ISPRS Congress, Beijing, July 9,

40 N93: Temporal Stability of the Int. Orientation (3th September, 27 ) Number of images: 13 (JPEG) Pixel size: 2.2 micron Same image data acquisition configuration with the version of 6th February, 27!! Ver GCP free CHK TIE APs Rej Sigma (μm) (pixel) STD-X STD-Y of CHK+TIE points STD-Z RMSE-X at CHK points RMSE-Y RMSE-Z Devrim Akca, the 21. ISPRS Congress, Beijing, July 9, 28. 4

41 Ver th February, 27 3th September, 27 GCP free CHK TIE APs Rej Sigm a (μm) (pixel) Sigma (μm) (pixel) STD-X STD-Y of CHK+TIE points STD-Z RMSE-X at CHK points RMSE-X at CHK points Devrim Akca, the 21. ISPRS Congress, Beijing, July 9, RMSE-Y RMSE-Y RMSE-Z RMSE-Z

42 N93: Temporal Stability of the Int. Orientation (3th September, 27 ) *) We compared the tie point coordinates of the Ver.21 (February 27) and Ver.61 (September 27) results: The means of the differences (February 7 September 7) are +.43, +.2 and +.4 mm (for the X, Y and Z axes, respectively). The standard deviations of the coordinate differences are ±.89, ±.169 and ±.6 mm (for X, Y and Z axes, respectively). Devrim Akca, the 21. ISPRS Congress, Beijing, July 9,

43 N93: Temporal Stability of the Int. Orientation (3th September, 27 ) **) The change of the principal point locations (x & y) and the focal length (c) between Ver.21 (of February 27) and Ver.61 (of September 27) are only -1.2, +1. and -1.8 microns, respectively. The corresponding standard deviations of the differences (calculated according to the law of error propagation without considering the correlations) are ± 1.1, ±.8 and ±.6 microns, respectively. Devrim Akca, the 21. ISPRS Congress, Beijing, July 9,

44 N93: Temporal Stability of the Int. Orientation (3th September, 27 ) N93 image: (version 21) N93 image: 13 (version 21) N93 image: (version 61) N93 image: 13 (version 61) Devrim Akca, the 21. ISPRS Congress, Beijing, July 9,

45 Image Residual Analysis Project all residuals to one image plane Average at pre-defined grid locations Devrim Akca, the 21. ISPRS Congress, Beijing, July 9,

46 Image Residual Analysis K75i version N93 version 21 W version 31 K75i version 11 N93 version 61 F828 version 41 Devrim Akca, the 21. ISPRS Congress, Beijing, July 9,

47 Results & Conclusions We have found; + unwanted effects from image enhancement in K75i, N93 and W + JPEG compression artifacts in N93 In spite of giving the worst results, K75i still can offer sub-mm accuracy. Sigma of image observations + K75i, N93 and W 1/5 pixel + F828 1/ pixel Only in the case of F828, residuals in (almost) random distribution The other cameras (esp. K75i), strong image-variant systematic errors Block-invariant APs cannot compensate the systematic errors in the individual images. So far, we cannot explain the reasons. Maybe: image enhancement procedure or any other electronic shortcomings. Devrim Akca, the 21. ISPRS Congress, Beijing, July 9,

48 Results & Conclusions Ver GCP CHK TIE APs Rej Sigma STD-X STD-Y STD-Z RMSE-X RMSE-Y RMSE-Z (μm) (pixel) of CHK+TIE points at CHK points K75i N93-I N93-II W F828. Devrim Akca, the 21. ISPRS Congress, Beijing, July 9,

49 Results & Conclusions Nevertheless, relative accuracies ( GCP versions): + K75i in-plane: 1 : 8 depth: 1 : 3 + N93 in-plane: 1 : 9 depth: 1 : 4 + W in-plane: 1 : 8 depth: 1 : 7 + F828 in-plane: 1 : 34 depth: 1 : 21 JPEG compression does not have a significant effect on the metric system accuracy. For a factor of 42 compression rate, RMSE-Y degraded 9%. N93, interior orientation is stable over time, even the systematic error pattern!! Mobile phone cameras are offering an interesting option for doing mobile photogrammetry in terms of accuracy, costs and flexibility. Devrim Akca, the 21. ISPRS Congress, Beijing, July 9,

50 Future Work Radiometric analysis Image noise analysis MTF analysis by Siemens starts method Linearity analysis 3D modeling 谢谢您的关注 Thank you for your attention Devrim Akca, the 21. ISPRS Congress, Beijing, July 9, 28. 5