IMPROVING IMAGE MATCHING BY REDUCING SURFACE REFLECTIONS USING POLARISING FILTER TECHNIQUES

Transcription

1 IMPROVING IMAGE MATCHING BY REDUCING SURFACE REFLECTIONS USING POLARISING FILTER TECHNIQUES N. Conen 1*, H. Hstedt 1, O. Khmen 1, T. Luhmnn 1 1 Jde University of Applied Sciences, Institute for Applied Photogrmmetry nd Geoinformtics (IAPG), Ofener Str. 16/19, Oldenurg, Germny - (nikls.conen, heidi.hstedt, oliver.khmen, luhmnn)@jde-hs.de Commission II, WG II/7 KEY WORDS: rising filter, cmer clirtion, imge mtching, endoscopy, ccurcy, completeness ABSTRACT: In dense stereo mtching pplictions surfce reflections my led to incorrect mesurements nd lunders in the resulting point cloud. To overcome the prolem of disturing reflexions rising filters cn e mounted on the cmer lens nd light source. Reflections in the imges cn e suppressed y crossing the rising direction of the filters leding to homogeneous illuminted imges nd etter mtching results. However, the filter my influence the cmer s orienttion prmeters s well s the mesuring ccurcy. To quntify these effects, clirtion nd n ccurcy nlysis is conducted within sptil test rrngement ccording to the Germn guideline VDI/VDE (2002) using DSLR with nd without rising filter. In second test, the interior orienttion is nlysed in more detil. The results do not show significnt chnges of the mesuring ccurcy in oject spce nd only very smll chnges of the interior orienttion ( c 4 µm) with the rising filter in use. Since in medicl pplictions mny tiny reflections re present nd impede roust surfce mesurements, prototypic trinoculr endoscope is equipped with rising technique. The interior nd reltive orienttion is determined nd nlysed. The dvntge of the rising technique for medicl imge mtching is shown in n experiment with moistened pig kidney. The ccurcy nd completeness of the resulting point cloud cn e improved clerly when using rising filters. Furthermore, n ccurcy nlysis using lser tringultion system is performed nd the specil reflection properties of metllic surfces re presented. 1. INTRODUCTION Correltion-sed mesurement procedures using two or more imges usully require overlpping, well-structured nd unmiguous imge contents. One of the min chllenges of such procedures is the mtching of corresponding imge points especilly in scenrios where surfce reflections occur. The loction of the reflection in the imge depends on the sptil rrngement of the cmer nd light source. Reflections (right hot spots) led to uniform intensity distriution nd thus my cuse miguities for imge mtching. Furthermore, reflection might e visile in one cmer position, ut invisile or less intense in nother cmer position. This unpredictle ehviour cretes dverse conditions for imge mtching. Some pplictions tht hve to del with reflections either segment, skip or interte these res for dense imge mtching (Arnold et l., 2010). For some purposes it might e useful to mesure the surfce including reflecting res, for exmple in medicl pplictions like surfce reconstruction from endoscopic imges or in industril pplictions like three-dimensionl inspection of welding sems or other shiny surfces. Since surfces in endoscopy re usully moist, the endoscopic light source cuses mny speculr highlights. In ddition, the surfce is usully not stle. In comprison, welding sems re rigid. However, reflections re similr to endoscopic surfces. 1.1 Contriution In this contriution, n endoscopic mesurement system is presented tht cn relily del with speculr reflections on ritrry surfces. The system provides dense point clouds y semi-glol mtching using three imges. Reflections on the surfce re eliminted or suppressed using rising filters for the cmer lens nd light source. Since rising filters dd n opticl component to the sensor system, the stndrd photogrmmetric cmer model (Brown, 1971) hs to e vlidted. According to the Germn guideline VDI/VDE (2002) the mximum length mesurement error (LMEmx) of DSLR with nd without rising filter is determined nd compred. Additionlly, the interior orienttion of the DSLR is evluted in further test field clirtion under more stle conditions using tripod nd high qulity lens. The interior nd reltive orienttion of the trinoculr endoscope is determined nd nlysed s well. The effect of the rising filter technique is demonstrted y oserving moistened pig kidney with different ristion directions. The surfce is reconstructed for ech imge triple representing one direction of ristion. The crossed ristion rrngement leds to the est reconstruction result since reflections re mostly suppressed. Furthermore, the specil reflection ehviour of metllic surfces is shown. In n ppliction for surfce reconstruction from lser profiles, rising filters llow the concurrent segmenttion of metllic surfces, non-metllic surfces nd the projected lser line. * Corresponding uthor Authors CC BY 4.0 License. 267

2 1.2 Light nd Polrising Filters Light cn e understood s trnsversl electromgnetic wve tht propgtes in spce long propgtion direction z (Figure 1). The wvelength defines the colour nd the mplitude defines the intensity. The direction of oscilltion (Figure 1, red field vectors) descries the ristion. In contrst to the wvelength nd mplitude, the ristion is not visile to humns. x y z Figure 1. Light s trnsversl wve with liner () nd circulr () ristion. The field vectors re perpendiculr to the propgtion direction nd cn rotte ritrry. In cse of liner ristion, the field vectors form plne (Figure 1). In cse of elliptic or circulr ristion, the field vectors rotte continuously round the propgtion direction. The rottion cn e descried s phse shift etween the horizontl nd verticl projection of the wve (Figure 1, lck sinusoidl wves). Light is not rised when the rottion is ritrry nd multiple ristions re present t the sme time. Nturl light sources (e.g. sun) nd most rtificil light sources (lightuls or LEDs) emit unrised light. A light em ecomes prtly rised when it is reflected, refrcted, or scttered y non-metllic oject (dielectric mteril). If the ngle of incidence corresponds to the Brewster ngle, complete ristion rises perpendiculr to the plne of incidence Jähne (2004). When (rtificil) light source emits rised light, the ristion of reflected em only depends on the type of reflection, which my e directionl or diffuse, depending on the roughness of the oject. If the roughness is less thn the wvelength of the incident light em, the reflection is directionl nd diffuse vice vers. A diffuse reflection chnges its ristion, while it remins constnt for directionl, i.e. speculr, reflection. Hence, speculr reflections from rised light sources cn e suppressed using rising filters of perpendiculr oscilltion. If the directions of ristion re prllel insted, speculr reflections would ecome slightly more prominent wheres diffuse reflections re reduced. In generl, two ristion settings re known s cross nd prllel ristion. Both settings re illustrted in Figure 2. Further informtion on ristion cn e found in Jähne (2004). c Figure 2. Experiment using rised light source. Polrising filters re not in use (), prllel () or crossed (c). 1.3 Relted Work Polrising filters re used in common nd commercil photogrphy nd re prticulrly interesting for professionl film nd dvertising industry. The filter llow cpturing imges with higher sturtion nd fewer reflections on shiny ojects (Bumch, 2014; Beitinger, 2016). Krus et l. (1992) nd Menn et l. (2016) pulished photogrmmetric pplictions of rising filters to enhnce contrst, increse colour sturtion, chieve more homogeneous lightning conditions nd, in prticulr, suppress speculr reflections. Krus et l. (1992) focused on the improvement in visul ppernce of cquired imges of the Behim Gloe; Menn et l. (2016) minly used crossed rising filters for reflection suppression nd improved dense mtching results for culturl heritge oject, the Etruscn Srcophgus of the Spouses. A criticl investigtion of the influence on the interior prmeters using rising filters is not conducted. Clncy et l. (2014) present n ppliction for endoscopy. The uthors simultneously cpture two imges with crossed ristion directions nd segment the colour chnnels using triple ndpss filter. Comining the ristions nd chnnels llow for contrst-enhnced imge in which very densely rrnged nd smll lood vessels ecome visile. The proposed imge enhncement my offer new possiilities in endoscopic dignostics. Atkinson et l. (2006) proposed Shpe-from-Polristion (SfP) procedure to reconstruct surfce from imges with three defined ristion directions (0, 45, 90 ). Herrer et l. (2013) pplied this procedure to medicl imges cquired with rised endoscope. Experiments with different ntomicl specimens show etter results from SfP thn from Shpe-from-Shding. The ristion cmer POLKA, distriuted y the Frunhofer Institute for Integrted Circuits (Frunhofer, 2018), consists of n imge sensor with rised microlens rry. Ech microlens hs one of four defined ristion directions (0, 45, 90 nd 135 ) in order to generte four imges with different ristions from one oservtion. This llows n immedite nlysis of the fire direction of cron fire composites. Another well-estlished ppliction of rising filters is the qulity ssessment of trnsprent dielectric (non-metllic) mterils (like plstic utensils, e.g. plstic ottles) in photoelsticity. The stress distriution round discontinuities in those mterils cn e visulised when rised light psses through n oject nd is cptured with cross-rised cmer system. The oject trnsmission leds to irefringence tht chnges the ristion direction nd cuses colour spectrum in the cptured imge (Beyerer et l., 2012). 2. CAMERA MODEL AND POLARISING FILTER Cross ristion hs not een used extensively in photogrmmetry, lthough it is known for decdes nd wellestlished in some res. An extensive vlidtion of the photogrmmetric mesurement ccurcy nd n investigtion of the influence on cmer model prmeters using rising filters could not e identified within pulished reserch ctivities. This pper investigtes the influence of rising filters y experiments using professionl DSLR, high qulity lens, photogrmmetric clirtion nd test procedures within undle djustment. Authors CC BY 4.0 License. 268

3 2.1 Accurcy Evlution According to the Germn guideline VDI/VDE (2002) the ccurcy potentil is determined for five different DSLR nd flsh setups with sme NIKKOR 24mm-lens with nd without rising filter. The cmer setups re A B C D E Nikon D2X with n eccentric flsh, Nikon D4 with n eccentric flsh, Nikon D2X with ring flsh mounted on the lens, Nikon D4 with ring flsh mounted on the lens nd Nikon D2X with ring flsh mounted on the tripod socket. Setup E is dded due to stility issues in setup C nd D. In setup E, the grvity of the flsh does not ffect the lens geometry when rotting or tilting the cmer, since the flsh is seprted from the lens. Figure 3 shows the different flsh types. constrints to the undle djustment. The mesured distnces for the clirted length re determined from the resulting oject point coordintes. The lrgest solute devition etween the 58 clirted nd mesured lengths (LMEmx) for ech cmer setup re summrised in Tle 1 nd Figure 5. The LMEmx provides informtion out the mesuring ccurcy of ech setup. Setup/Pol.filter LMEmx [mm] Test 1 Test 2 Test 3 A/ A B/ B C/ C D/ D E/ E Tle 1. Resulting LMEmx for cmer setups. LME mx for different cmer setups c Figure 3. Exemplry cmer setups. ) Nikon D4 with eccentric flsh nd rising filter (B); ) Nikon D2X with ring flsh mounted to the lens (C); c) Nikon D4 with ring flsh mounted to the tripod socket (E). The ccurcy evlution is sed on the determintion of length mesurement errors ccording to VDI/VDE (2002). Seven mesuring lines, consisting of three retro-trgets ech, re rrnged within volume of out m³ (Figure 4). In totl, 58 clirted distnces re given s mesurement rtefcts etween the retro-trgets. The clirtion ccurcy of the distnces is certified to less thn 10 µm. As qulity prmeter, the length mesurement error (LME) is determined s performnce-trget-devition. The mximum solute vlue of the length mesurement errors is tken s LMEmx nd s qulity mesure. The volume is oserved three times (test 1-3), ech from 12 positions, in three height levels nd with four different tilt ngles. This results in undle of 144 imges (Figure 4). LME mx [mm] A A B B C Figure 5. Digrm of LMEmx for different cmer setups. The digrm revels three setup groups of similr LMEmx. The LMEmx of the setups A-B, C-D nd E re similr, however, the LMEmx etween the groups re quite different. These differences re cused y the flsh setup. The grvity of the ring flsh mounted to the lens (group C-D) influences the lens geometry when tilting the cmer. Thus, cmer prmeters re not stle nd LME re lrger. The groups A-B nd E re cptured with flsh seprted from the lens nd show etter results. However, group A-B shows mrginlly decrese in ccurcy compred to E. Tht my e cused y n eccentric imge mesurement of the retro-trget for A-B. The differences in the tests re cused y the NIKKOR 24mm-lens which mechnicl geometry chnges slightly during oservtion. This influences the clirtion stility nd thus the LME. The rising filter hs no significnt effect on the LME. The LME of sme setup with nd without filter re quite similr. For tht reson, rising filters re pplicle to stndrd photogrmmetric cmer model nd undle djustment with respect to the investigted setups. C D Cmer setup D E E Test 1 Test 2 Test Anlysis of Interior Cmer Prmeters Figure 4. VDI test volume () nd imging setup (). A undle djustment for ech dtset results in n RMS of oject point precision in ech xis of out 20 µm. This corresponds to the expected qulity level of the used cmer setup. Three clirted lengths long the coordinte xes re pplied s scle The interior orienttion is determined simultneously y the undle djustments from the VDI dtsets for ech cmer setup. In these experiments, no significnt chnges in the interior orienttion hve een detected. However, the comprility of the results is limited due to the instility of the cmer nd lens geometry nd remining mesurement uncertinties. In order to llow for relile comprison of the interior orienttion prmeters, whether using the rising filter or not, second ssessment is evluted. Authors CC BY 4.0 License. 269

4 Therefore, n dditionl test-field clirtion using the Nikon D4 with 35mm-ZEISS-lens (eing of higher qulity thn the 24 mm-nikkor-lens) nd tripod is conducted. This setup offers higher stility of the interior cmer geometry during the tests while moving the test-field insted. The eccentric flsh is used for imge cquisition in order to enle cross-ristion. The cross-ristion setup is chieved y rising film psted onto the flsh nd rising filter mounted to the lens in crossed ristion direction (Figure 6). Six tests re performed, four with nd two without rising filters. Therey, shiny cron test field with stndrd trgets is oserved from out 34 different positions y moving the test-field. An exemplry imge without () nd with (c) crossed rising filters is shown in Figure 6. ech cmer nd lens comintion ecuse of limited mnufcturing ccurcy. [mm] determined prmeters w/o, w.filter men devition std. dev. of men without with s c x' y' A1-6.80E E E E-07 A2 3.81E E E E-09 A3 1.50E E E E-12 B1 2.88E E E E-07 B2 5.34E E E E-07 C1 8.48E E E E-06 C2-3.78E E E E-05 Tle 2. Exemplry interior orienttion prmeters with nd without rising filters; men over ll devition nd its stndrd devition. The rdil-symmetric lens distortion curves of the exemplry prmeter sets re visulised in Figure 7. The devition in the distortion curves t the imge order is only 0.3 px. The influence to rdil-symmetric lens distortion s well s decentring distortion, ffinity nd sher is very low ut present. c Figure 6. Nikon D4 with 35mm-ZEISS-lens with rising filter nd eccentric flsh with rising film (); cron test field cptured without () nd with (c) crossed rising filters. The interior orienttion prmeters for ech dtset re computed y undle djustment. The otined prmeters re the principl distnce (c), principl point H(x'0,y'0), rdil-symmetric lens distortion (A1, A2, A3), decentring distortion (B1, B2), ffinity nd sher (C1, C2) (Luhmnn et l., 2014). The prmeters re determined significntly with significnce level of 95% for ech dtset, except for prmeter C2 in one test. The resulting prmeters for n exemplry test with (w.filter) nd without (w/o.filter) rising filters is shown in Tle 2. The influence of the rising filter is indicted y the devition ( ) of the prmeters determined with nd without rising filter. Its stndrd devitions (s ) re propgted in order to test its significnce. The men devition ( ) nd corresponding stndrd devitions of the men (s ) re listed in Tle 2. The men differences re determined with significnce level of 95%, except of prmeter C2. The comprison shows the influence on ech prmeter cused y the rising filter. The most prominent prmeters re the principl distnce nd the principl point. The principl distnce is 4 µm longer in verge when rising filters re pplied. Additionlly, the principl point is slightly shifted to the imge centre. The shift equls 0.7 px in x- nd 1.6 px in y-direction with pixel size of 7.3 µm. These chnges re relile due to refrction of the dditionl glss lyer in front of the lens. Furthermore, the ehviour of the principl distnce nd principl point seem to e correlted. As the principl distnce gets longer, the principl point is shifted to the imge centre. Another reson for the devitions might e due to tilt of the filter with respect to the imge sensor, which my e different for distortion [px] Figure 7. Rdil-symmetric lens distortion curve of Nikon D4 with 35mm-ZEISS-lens with nd without rising filter. This experiment shows tht clirtion prmeters, especilly the principl distnce nd principl point, vry slightly when rising filters re used. To enle high ccurcy mesurements, it is pproprite to clirte the cmer system for ech setup individully. Furthermore, the prmeters of test should lwys e considered s unique set since prmeters correlte with ech other. 2.3 Orienttion of Trinoculr Endoscope According to the results of the previous susections, it is ssumed tht other cmer systems with rising filters nd similr imging properties cn lso e descried y the stndrd photogrmmetric cmer model. Hence, specil trinoculr cmer system equipped with ring flsh nd rising filter (Figure 8) is evluted regrding the determintion of orienttion prmeters nd dense imge mtching processing. The prototypicl system consists of three equilterl rrnged cmers with dimeter of 6 mm ech (Figure 8). The outer dimeter of the three cmers is out 14 mm. The imge resolution is px² with pixel size of 3.6 µm. The system resemles trinoculr endoscope tht llows stereovision nd relile 3D mesurements in surgicl scenrios (Conen et l., 2017). Due to size restrictions the height-to-se rtio is pproximtely 10:1. Authors CC BY 4.0 License. 270

5 distortion [px] Figure 8. Trinoculr miniture cmer system () nd system setup with light source nd filter (); direction of ristion is indicted s lue nd red lines, ngle of ristion is 90 (exmple cross-ristion). The interior orienttion s well s the reltive pose etween the three cmers (reltive orienttion) re determined with nd without cross-ristion setup. The reltive orienttion is defined y trnsltion prmeters (X, Y, Z) nd rottion ngles (,, ). The left cmer defines the origin of the locl cmer coordinte system (X0 = Y0 = Z0 = 0 = 0 = 0 = 0). As in the previous section, the interior orienttion prmeters re determined y undle djustment using three dimensionl test-field. The reltive orienttion is estimted y multiple resections with fixed interior prmeters chieved from previous undle djustment. The results re shown in Tle 3 nd the rdil-symmetric lens distortion curves re visulised in Figure 9. Tle 3. determined prmeters [mm] devition w/o, w.filter [rd] without with c x' y' c x' y' c x' y' X Y Z E E E E E E E E E-03 X Y Z E E E E E E E E E-04 Interior nd reltive orienttion prmeters with nd without rising filters nd their devitions; the devitions re clculted with exct vlues nd rounded. Figure 9. Rdil-symmetric lens distortion curves of miniture cmers with nd without rising filter. The results for the interior orienttion show similr ut weker trend in prmeters s in the Nikon D4 test. In comprison to the cmer size, the devitions re still in similr rnge of vlues. The principl distnces re slightly longer (6 µm) or do not chnge significntly with rising filter. The principl points re slightly shifted ( px) to the imge centres. The rdilsymmetric lens distortion is reltively low. Hence, the influence of the rising filter is minly less thn 0.2 px (Figure 9). The results for the reltive orienttions of the right nd top cmer to the left show lrger chnges in Z direction thn in XY direction. The projection centres of the right nd top cmer re locted pproximtely 0.1 mm frther from the oject thn the left cmer. Due to correltions etween the Z direction nd the principl distnce, the shift in Z position my lso e cused y refrction of the filter. The shift in XY position my e cused y tilt etween the filter nd imge sensor. Due to limited stility of the interior cmer, the lens geometry nd minituristion issues, the determintion of orienttion prmeters for n endoscopic system is not s relile s for DSLR. For prcticl pplictions stndrdised nd fst clirtion procedure is recommended. The resulting orienttion prmeters for cross-ristion setup re used in the following sections for further imge nlysis. 3. EXPERIMENTS AND RESULTS As descried in the introduction, reflections cn mke stereoscopic imge mtching quite difficult. To improve mtching results for pplictions hving poor illumintion conditions, trinoculr system which suppresses disturing reflections is introduced. As descried in Conen et l. (2016), dense point cloud from the oserved surfce is computed y semi-glol mtching pproch for three imges. The presented trinoculr system nd the mtching pproch from Conen et l. (2016) is used to evlute the effect of rising filters on imge mtching. Therefore, two experiments on moistened pig kidney re conducted (Figure 10). The first experiment evlutes mtching results from imges with different degrees of reflexion suppression. The second experiment compres mtching result from rised endoscope with result from lser tringultion sensor of higher ccurcy. In n dditionl third experiment, prototypicl lser tringultion sensor is equipped with rising filters to demonstrte the properties of different mterils regrding ristion. The rising filter llows for segmenttion of metllic surfces, lser lines nd photogrmmetric trgets in prllel. Authors CC BY 4.0 License. 271

6 The Interntionl Archives of the Photogrmmetry, Remote Sensing nd Sptil Informtion Sciences, Volume XLII-2, (cross) (prllel) Figure 10. Specimen of moistened pig kidney with highlighted res of investigtion for experiments. 3.1 Reflexion Suppression for Imge Mtching In the first experiment, n re of 2 2 cm² of moistened pig kidney (Figure 10, re 1) is oserved from one fixed position with five different ngles of ristion: 0 (prllelristion), 22.5, 45, 67.5 nd 90 (cross-ristion). An dditionl mesurement with cross-ristion setup is conducted to check the precision of the 3D reconstruction process. An ex-vivo specimen of kidney is chosen to simulte the oject properties during lproscopic prtil nephrectomy (i.e. resection of tumour on kidney using endoscopes for dominl or pelvic cvities). A dense mtching with identicl prmeters is performed on ech imge triple. Since the dtset cquired using cross-ristion (90 ) consists of no speculr reflections nd led to the most complete nd homogeneous result, its mesh is used s reference for 3D comprison (cloud-to-mesh). The comprison to the second cross-ristion setup leds to stndrd devition of shortest distnces of 105 µm. Since stle solute orienttion is ssumed, the stndrd devition represents the precision. A Monte-Crlo simultion performed in Conen et l. (2017) leds to stndrd devition for the depth of 247 µm for similr imging distnce (75 mm) nd n ssumed mesurement ccurcy in imge spce of hlf pixel. However, Conen et l. (2017) mentioned tht due to semi-glol optimistion nd supixel intertion more optimistic mesurement ccurcy would e more relistic. A qurter pixel ccurcy results in stndrd devition for the depth of out 124 µm, which is comprle to the derived precision of the second crossristion mesurement. Therefore, one qurter of pixel is considered s mesurement ccurcy in imge spce for this experiment. Reference s0 = [mm] Figure 11. Comprison of different ngles of ristion. Top: originl left imge; middle: point cloud top view; ottom: point cloud side view; 90 : reference in greyscle; other ngles: 3D devitions w.r.t. 90 in colour-code. 3.2 Comprison with Lser Tringultion Sensor An ccurcy ssessment of the trinoculr system equipped with crossed rising filters is conducted prcticlly y comprison with mesurements from lser tringultion sensor (KEYENCE LJ-V7080). According to the mnufcturer, the repetility is 0.5 µm in depth nd 10 µm in direction of the lser line. An incision of pproximtely 5 mm depth of the kidney (Figure 10, re 2) is cptured with XY resolution in oject spce of out µm². The trinoculr system cptures the sme re with n imging distnce of out mm nd resulting resolution in oject spce of out µm². The different resolutions re compensted y meshing the point cloud of the lser tringultion sensor. The dtsets re initilly registered using sphericl eds of out 2.25 mm in dimeter. In cse of the lser tringultion sensor, the centres of the eds re determined y est fitting spheres in the point cloud (Figure 12) s it is common for lser scnning purposes. In cse of the trinoculr cmer system, the centres re determined y mesuring ellipses in imge spce (Figure 12) nd pplying forwrd intersection s it is common in photogrmmetry. The results of the 3D comprisons etween different ngles of ristion re shown in Figure 11. The fully sturted red nd lue spots in the 3D comprison show pproximtely five times higher devitions compred to the precision of the 3D reconstruction process. As expected, the mount nd intensity of speculr reflections grow with decresed ngle of ristion. The reflections led to lunders tht prtly devite more thn ±0.5 mm from the reference. The stndrd devitions increse with smller ngles of ristion. In cse of the prllel ristion, the fully sturted regions seem to e smoothed y semi-glol mtching, wherey the stndrd devition ecomes smll. In conclusion, the stndrd devitions indicte loss in ccurcy with higher reflections. Figure 12. Trget point detection; () point cloud nd selected sphericl points (red) of lser tringultion sensor; () mesured ellipses in one imge of the trinoculr system. The registrtion leds to 3D RMS of mm using eight trget points. In order to just compre the surfce points of the kidney the eds re deleted from the point clouds. The 3D comprison leds to men offset of mm nd stndrd devition of distnces of mm. In order to nlyse the Authors CC BY 4.0 License. 272

7 The Interntionl Archives of the Photogrmmetry, Remote Sensing nd Sptil Informtion Sciences, Volume XLII-2, 2018 remining offset fine registrtion using itertive-closest-point (ICP) lgorithm is pplied. The ICP minimises the men offset to mm with stndrd devition of mm. The stndrd devition indictes the precision of the experiment. To define the solute mesurement ccurcy of the trinoculr system, the registrtion hs to e considered. As shown in the results of the registrtion using eds n offset is detected. The offset is influenced y different 3D point mesurement techniques nd not perfectly sphericl eds. For this experiment the solute ccurcy results to mm. Referring to the Monte-Crlo simultion tht equls mesurement ccurcy in imge spce of out hlf pixel. (Figure 14). In ddition, the reflections re discontinuous due to the ritrry wve-like microstructure of the welding sem. Thus, reflections vry, even when cmer position chnge slightly. Crossed rising filters cn overcome this prolem. Since the rised light reflected y the welding sem does not pss the crossed rising filter, only the reflection of the lser line nd pper trgets re visile in the imge (Figure 14c). Although lser is lredy rised, its reflection psses the filter due to different direction of ristion. The filter slightly reduces its intensity, however, it is still enough for relile imge mesurement. The devitions fter ICP re visulised in Figure 13 s colourcoded point cloud. The lrgest devitions occur t the deepest point of the incision due to limited resolution nd smoothing effect cused y the mtching. The flnks of the incision s well s the surrounding surfce consists of lower devitions. c Figure 14. Acquired imges of welding sem with lser line nd photogrmmetric pper trgets; () short exposure time; () compromising exposure time; (c) crossed rising filter. The lst experiment demonstrtes the specil ristion properties of metllic surfces nd gives n exmple for useful ppliction. The rising technique might e solution for mny pplictions in which disturing reflections occur. Despite the technique is well known in some res there is still potentil for photogrmmetric purposes. [mm] Figure 13. 3D comprison with tringultion sensor. model derived from Without crossed rising filters shiny or reflective res would proly e inccurte or not mesured t ll. With respect to the Monte-Crlo simultion, previous experiments from Conen et l. (2017) nd the results of this pper, the resulting stndrd devitions re considered relile. Even though rising filters dd n opticl component to the cmer model, their impct in ccurcy is mrginl. However, if there re reflections present in the scene, rising filters offer gret dvntge in ccurcy, reliility nd completeness. 3.3 Segmenttion of Metllic Mterils The reflection nd ristion properties of metllic surfces re different to those of non-metllic surfces. Any reflection, whether diffuse or directed, does not chnge its ristion properties. Hence, rised light reflected on metllic surfce is completely filtered y crossed rising filter resulting in lmost lck pixels in the imge. This property llows the distinction etween metllic nd non-metllic ojects. A prcticl ppliction of this property might e mesurement of welding sems using hndheld lser tringultion sensor. The mesured profiles of the tringultion sensor re oriented y pper trgets ttched to the oject. These trgets define locl coordinte system in which the oriented profiles form surfce of the welding sem. However, mesuring the lser line nd the pper trgets simultneously in one imge is quite chllenging due to different lightning requirements; the lser line requires short exposure times (Figure 14) wheres the pper trgets rely on longer exposure times nd dditionl light sources. With compromising lightning setting mny miguities for profile mesurement occur due to reflections on the welding sem 4. DISCUSSION AND CONCLUSION In this pper it could e shown tht rising filters cn e used with stndrd photogrmmetric cmer model nd undle djustment. The effect of rising filters is negligile in the chieved rnge of ccurcy with respect to trget mesurements in VDI test environment. An experiment with more stle cmer setting shows the vritions of the interior cmer prmeters when rising filters re used. The filter minly influences the principl distnce nd the principl point. However, the common photogrmmetric cmer model cn still descrie these vritions. Consequently, it is ssumed tht rising filters re pplicle to other cmer systems with similr imging properties like the presented trinoculr endoscope. The influence in Z direction of the reltive orienttion is noticele lrge. However, it is lso highly correlted to the principl distnce. The effect of cross-ristion on moistened orgnic surfces is impressive nd enles rised system to mesure in more complicted scenrios. A significnt drwck is the loss of intensity, which requires more powerful illumintion. The dvntge on the other side is very homogeneous ppernce of the moistened surfce since only diffuse reflections re cptured. Only the texture of the oject is visile nd no disturing reflections occur. An ccurcy ssessment is conducted on pig kidney with n incision using lser tringultion sensor nd the rised endoscopic system. After ICP the stndrd devition of the shortest distnces etween the dtsets (cloud-to-mesh) is determined to e mm. Considering the height-to-se rtio of out 10:1, the Monte-Crlo simultion nd tests from Conen et l. (2017), the chieved ccurcy is plusile. For lproscopic surgeries, which re opertions in the dominl or pelvic Authors CC BY 4.0 License. 273

8 cvities, the chieved ccurcy is sufficient. However, rel-time computtion nd stitching of point clouds over time to ccess lrger res is not implemented yet. Not only mtching lgorithms profit from rising filters. Another enefit is etter nd clener vision to the surgeon, which proly mkes n opertion less demnding to the surgeon s eyes. A mesurement of welding sem using lser tringultion sensor demonstrtes useful ppliction of rising filters for non-medicl purposes. The specil reflection properties of metllic surfces enle simultneous mesurement of the lser line nd photogrmmetric pper trgets for orienttion of profiles in only one imge. This experiment shows gret potentil for other pplictions tht hve to del with disturing reflections. Complex reflecting surfces of vrious mterils cn e mesured y conventionl imge mtching or lser tringultion techniques when rising filters re pplied smrtly. The filters cn e used with stndrd photogrmmetric cmer model nd eliminte disturing reflections relile. ACKNOWLEDGEMENT This project is funded lis OrthoScn y the Europen Regionl Development Fund nd the federl stte of Lower Sxony. REFERENCES Atkinson, G. A. & Hncock, E. R., Recovery of surfce orienttion from diffuse riztion. In: IEEE Trnsctions on Imge Processing, Vol. 15, No. 6, pp doi: /TIP Frunhofer POLKA. Frunhofer IIS, mer.html, Ptent: html ( ). Herrer, S.E.M., Mlti, A., Morel, O. & Brtoli, A., Shpefrom-riztion in lproscopy. In: IEEE 10th Interntionl Symposium on Biomedicl Imging, pp Jähne, B., Prcticl Hndook on Imge Processing for Scientific nd Technicl Applictions. CRC Press, 2nd Ed. Krus, K., Tschnnerl, J., Hlmer, A. & Rottensteiner, F., Working on the Behim Gloe using Photogrmmetric Methods. In: Journl for the Study of Gloes nd Relted Instruments, 40/41, pp Luhmnn, T., Roson, S., Kyle, S. & Boehm, J., Close- Rnge Photogrmmetry nd 3D Imging. De Gruyter textook, 2nd edition, ISBN Menn, F., Nocerino, E., Remondino, F., Dellepine, M., Cllieri, M. & Scopigno, R., D Digitiztion of n Heritge Msterpiece - Criticl Anlysis on Qulity Assessment. In: The Interntionl Archives of the Photogrmmetry, Remote Sensing nd Sptil Informtion Sciences, 41, pp VDI/VDE (2002): Opticl 3-D mesuring systems Imging systems with point-y-point proing. VDI, Düsseldorf. Arnold, M., Ghosh, A., Ameling, S. & Lcey, G., Automtic Segmenttion nd Inpinting of Speculr Highlights for Endoscopic Imging. In: EURASIP Journl on Imge nd Video Processing, 2010: , doi: /2010/ Bumch, D., 2014: Die Kreuz Blitztechnik. fotogrfenwelt, TB Foto Tem, ( ). Beitinger, A., Polristionsfilter. fotovideotec, ( ). Brown, D.C., Close-Rnge Cmer Clirtion. In: Photogrmmetric Engineering, pp Clncy, N. T., Ary, S., Qi, J., Stoynov, D., Hnn, G. B., & Elson, D. S., Polrised stereo endoscope nd nrrownd detection for miniml ccess surgery. In: Biomedicl Optics Express, 5(12), pp doi: /BOE Conen, N., Jepping, C., Luhmnn, T. & Ms, H.-G., Rectifiction nd roust mtching using oriented imge triplets for minimlly invsive surgery. In: ISPRS interntionl nnls of the photogrmmetry, remote sensing nd sptil, Inf Sci 3(3), pp Conen, N., Luhmnn, T. & Ms, H.-G., Development nd Evlution of Miniture Trinoculr Cmer System for Surgicl Mesurement Applictions. In: Journl of Photogrmmetry, Remote Sensing nd Geoinformtion Science, Vol. 85, Issue 2, pp Authors CC BY 4.0 License. 274