Bounded Ieraive Thresholding for Lumen Region Deecion in Endoscopic Images Pon Nidhya Elango School of Compuer Science and Engineering Nanyang Technological Universiy Nanyang Avenue, Singapore Email: ponnihya88@gmail.com Siew-Kei Lam School of Compuer Science and Engineering Nanyang Technological Universiy Nanyang Avenue, Singapore Email: siewkei_lam@pmail.nu.edu.sg Absrac The developmen of a fully auomaed roboic endoscopic seering sysem has been an acive area of research for more han a decade. This paper aims a proposing a hardware-efficien ieraive hresholding sraegy o locae he lumen region in capured endoscopic images in order o enhance radiional endoscopes wih cerain degree of auonomy and inelligence. The proposed mehod is characerized by a definie requiremen on he number of ieraions of hresholding in order o deec he lumen region. The proposed algorihm has been demonsraed o be robus agains varying characerisics using real endoscopic sample images. The reducion in he number of operaions required by he proposed mehod can be up o 71% compared o a previously repored mehod. FPGA synhesis resuls of he proposed approach confirm is viabiliy for realime realizaion. Keywords-compuer vision; FPGA; hardware acceleraion; medical images I. INTRODUCTION Medical endoscopy is a minimally invasive procedure o invesigae various inernal caviies in he human body (e.g. lower gasroinesinal rac and respiraory rac for diagnosis and herapy. Micro-roboic endoscopes wih suppor for compuer-aided auomaed navigaion have emerged o alleviae he pain and discomfor risks in endoscopy. As illuminaion in endoscopes is provided by a poin ligh source, he region characerizing he farhes piece of issue appears o be he darkes in he capured endoscopic image and is referred o as he lumen region. The idenificaion of he lumen region and navigaion of he endoscope owards he lumen cener is he principal echnique used for auomaed roboic endoscopic seering sysem. Since he adven of endoscopes wih digial imaging chips, several endoscopic image processing echniques have been proposed for he idenificaion of he lumen conour and lumen cener in order o provide decision suppor for diagnosis and o provide suppor for navigaion. Exracion of he precise lumen conour is essenial when i comes o diagnosis. The conour, however, migh no be conspicuous in every image and i is an opional requiremen for navigaion [1]. Moreover, exracion of he precise conour could be oo slow for meeing real ime requiremens. Thus, for he purpose of auomaing endoscope navigaion, he robus idenificaion of he cener of he lumen is sufficien o precisely conrol he endoscope orienaion []-[6]. Several hardware-based image processing algorihms on segmenaion and region growing have been proposed as sofware approaches opimized for microprocessors may no be fas enough for real ime processing. FPGAs come across as a viable choice for he implemenaion of real ime image processing algorihms allowing for a dedicaed hardware soluion ha explois parallel and pipelined design echniques. The challenges in designing robus image processing algorihms sem from imaging condiions in he gasroinesinal rac ha end o be challenging as he background illuminaion and reflecive properies may no be uniform hroughou. Also, he shape, size and pixel inensiies of he lumen region in capured images migh vary significanly. This paper aims a proposing a hardware-efficien ieraive hresholding sraegy o locae he lumen region in capured endoscopic images. The deecion of he lumen region serves as a pre-requisie sep for he idenificaion of he lumen cener. The proposed bounded ieraive hresholding algorihm is characerized by a definie requiremen on he number of ieraions of hresholding in order o deec he lumen region. Simulaions were carried ou in Malab, for a class of 40 graylevel endoscopic images of size 56x56, o demonsrae he robusness of he proposed mehod for lumen region deecion. When compared o a previously repored approach, he proposed mehod leads o significan reducion in compuaional complexiy. In addiion, FPGA synhesis resuls show ha he proposed approach lends iself well for real-ime realizaion. In he nex secion, we review exising works in he lieraure for he exracion of lumen region. In Secion 3, we highligh limiaions of an exising work for deecing he lumen region, and Secion 4 describes he proposed bounded ieraive hresholding algorihm for lumen region deecion. This is followed by a discussion on experimenal and FPGA synhesis resuls in Secion 5. Secion 6 concludes he paper. II. RELATED WORK The work in [7] presened a echnique for auomaic exracion of lumen region and is boundary by using a combinaion of a progressive hresholding echnique and region growing. A quasi Region of Ineres (RoI is obained afer wo ieraions of Osu s hresholding [8]. In order o faciliae auomaic segmenaion, an adapive progressive hresholding (APT approach was suggesed in [9]. A Cumulaive Limiing
Facor (CLF is used o idenify he opimal hreshold in every ieraion. The APT mehod was combined wih a Differenial Region Growing (DRG approach in [10] for he exracion of he lumen region based on he similariy of pixels. An efficien pipelined archiecure for he implemenaion of APT on FPGA has been proposed in [9]. The archiecure was furher improved by replacing he complex muliplicaion and division operaions involved in Osu analysis wih an efficien logarihm conversion uni [11]. However, he APT mehod resuls in more han 100 sequenial ieraions for cerain endoscopic images which may no be appropriae for real ime implemenaion. Wang e. al [5] suggesed a echnique for lumen cener deecion which involves adapive hresholding followed by erosion and dilaion. Facors ha represen he upper and lower limis of he raio of dark region o he area of he whole endoscopic image need o be predeermined by experimening wih a good se of images. The adapive hreshold value needs o be deermined based on he comparison of he whole area of he image agains hese predeermined facors. Tian e. al. [1][13] suggesed he use of an Iris filer in order o obain a very accurae boundary of he lumen in endoscopic images. A preliminary RoI is exraced hrough APT and he remaining areas of he image which are beyond he hreshold value are eliminaed and considered as background. Having such a uniform background is a precondiion o be me for he use of Iris filer. The lumen boundary obained on applying he Iris filer involved large number of rigonomeric compuaions in he convergence index calculaions. Though efficien CORDIC archiecures have been suggesed in [13] for he hardware implemenaion of he rigonomeric compuaions, such a complex echnique o obain he accurae lumen boundary is no essenial for he case of endoscopic navigaion. A disinc algorihm using fuzzy region growing was suggesed by Chang e. al. o segmen he lumen region from endoscopic images [14]. The fuzzy rule mehod has been implemened on FPGA wih a pipelined archiecure. The fuzzy region growing is preceded by hresholding he image wih an opimal hreshold obained a he end of an ieraion of Osu s echnique. In spie of being a feasible implemenaion, his mehod is limied in accuracy as only a single ieraion of Osu is employed. Lim e. al. [4] proposed a echnique for locaing he cener of mass (CoM of he lumen in endoscopic images using a novel APT approach o idenify he lumen region ha is followed by a windowing operaion o deermine he lumen cener. The APT is erminaed when he maximum Beween Class Variance (BCV idenified by he Osu mehod in an ieraion is less han he maximum BCV of he previous ieraion. III. LIMITATION OF EXISTING APT METHOD The Osu s echnique segmens he hisogram of he endoscopic image ino wo disinc classes. Le C 1 and C represen he wo classes o be obained afer a segmenaion by hreshold, wih P 1 and P referring o he probabiliy funcion of a pixel belonging o class C 1 and C respecively. The probabiliy funcions are calculaed as shown below. P1 = ω ni = N i == 0 P = ω 0 = 1 ω (1 ( Le n i be he number of pixels which have an inensiy value of i and N he oal number of pixels in he image, and L he number of gray levels. L 1 i = n μ i T (3 i = 0 N i = n μ i i = 0 N (4 The Osu mehod suggess ha he opimal hreshold can be obained hrough he sequenial search for he maximum of beween class variance (BCV σ B(, for he values of where 0 < L : μt μ σ B ( = w0 (1 w0 ( 1 w0 max( σ B 0 = 0 max( σ > σ B B i 1 max( σ B i max( σ B μ w0 Fig. 1: APT mehod in [4] The APT mehod proposed in [4] involves discriminan analysis by Osu mehod and inensiy normalizaion for every ieraion unil a erminaion condiion is me (see Fig. 1. In paricular, he APT is erminaed when he maximum BCV idenified by he Osu mehod in he curren ieraion is less han he maximum BCV of he previous ieraion. Our experimens reveal ha for cerain images, he APT algorihm (5
proposed in [4] does no erminae a he righ poin. While for some images he final hresholded image obained afer erminaion of he algorihm does no have a well segmened lumen region, for cerain oher images he final hresholded image has all he pixels merged ino he background. In order o validae he effeciveness of he APT mehod, we perform he windowing operaion in [4] on he final segmened lumen region o locae he lumen cener. For he endoscopic image shown in Fig. (a, he APT mehod in [4] erminaes afer ieraions. However, i can be observed from he final hresholded image in Fig. (b ha significan background remains in he image. Deermining he lumen cener on his segmened image gives incorrec resuls as here is significan influence from he background ha is lef behind and he problem is furher complicaed as he lumen region is small. I can be observed ha he final lumen cener idenified, highlighed in red in Fig. (c, is well ouside he lumen region. On he oher hand, for he endoscopic image shown in Fig. 3(a, APT erminaes afer 6 ieraions, hough ieraions of Osu based hresholding would acually suffice for his case. Fig. 3(b shows he final hresholded image obained for he endoscopic image in Fig. 3(a afer 6 ieraions, wih all he pixels having been merged ino he background. Fig. 3(c shows he hresholded image afer ieraions, which would have been sufficien for accurae deecion of he lumen region. IV. PROPOSED METHOD The proposed APT approach sems from observaion ha a maximum of 3 ieraions were sufficien in order o obain a hresholded image wih a well segmened lumen region. In addiion, he decision o perform he hird ieraion depends on he number of pixels beween he wo hisogram peaks obained in an ieraion of he Osu mehod. Fig. 4 illusraes he over-hresholding problem in [4]. The hisograms of he original image and he hresholded image afer he 1 s, nd and 3 rd ieraions of hresholding are shown in Fig. 4 for he endoscopic image in Fig. 4(a. The APT algorihm in [4] erminaes auomaically afer 3 ieraions and resuls in over-hresholding as shown in Fig. 4(f. I can be observed ha he hisogram of he hresholded image obained afer ieraions in Fig. 4(d, has discernible peaks, Peak1 and Peak. Peak1 refers o he maximum pixel coun for he inensiy in he low inensiy region which would consiue he lumen region, while Peak refers o he number of background pixels. The difference in pixel coun beween Peak1 and Peak afer ieraions of hresholding is highlighed in Fig. 4(d. Though he APT mehod in [4] erminaes only afer 3 ieraions, ieraions of hresholding would suffice o obain a final hresholded image wih a well segmened lumen, as i can be seen from Fig. 4(d ha is a large enough o discern beween he wo peaks. (a (b (c Fig. (a: Original endoscopic image; (b final hresholded image of (a; (c windowing operaion o deermine lumen cener (a (b (c (d (e (f (a (b (c Fig. 3(a: Original endoscopic image; (b over hresholded image of (a afer 6 ieraions; (c hresholded image afer ieraions Hence here is a need o refine he APT mehod in [4], as i resuls in under-hresholding (Fig. (b or over-hresholding (Fig. 3(b for some images. Moreover, auomaic erminaion according o he maximum condiion does no happen for all cases. Ou of 40 sample endoscopic images ha were esed, he APT mehod in [4] was unable o provide a well segmened image for 10 of hem. Fig. 4(a: Original endoscopic image; (b original hisogram; (c hisogram afer 1 s ieraion; (d hisogram afer nd ieraion; (e hisogram afer 3 rd ieraion, (f over-hresholded image afer 3 ieraions using he mehod in [4] Fig. 5 illusraes he under-hresholding problem wih he mehod in [4]. For he endoscopic image in Fig. 5(a, he APT mehod in [4] erminaes auomaically afer he nd ieraion. This resuls in an under-hresholded image as shown in Fig. 5(e. When compared o Fig. 4(d, i can be observed ha he hisogram afer he nd ieraion in Fig. 5(d has very low Peak1 and he gray level inensiies beween he Peak1 and Peak have a conribuion comparable o ha of Peak1. This makes i necessary o have a 3 rd round of hresholding in order o obain a final hresholded image wih a well segmened lumen.
(a (b (c (d (e Fig. 5(a: Original endoscopic image; (b original hisogram; (c hisogram afer 1 s ieraion; (d hisogram afer nd ieraion; (e under-hresholded image afer ieraions using he mehod in [4] The analysis of he hisograms in Fig. 4 and Fig. 5 demonsraes ha in order o have he lumen as a predominan objec, he hresholding ieraions should coninue unil Peak1 is significanly higher han he conribuion from oher inermediae pixel inensiies (pixels beween Peak1 and Peak. In addiion, i was observed ha even in he case where Peak1 has over 000 pixels afer he nd ieraion, a 3 rd ieraion of hresholding was necessary for he images which had a comparable conribuion from inermediae inensiies beween Peak1 and Peak. Specifically, i was observed ha when 3 inermediae inensiies have a significan pixel coun of over 500 pixels, i is necessary o furher hreshold he image o ensure ha he final hresholded image has he lumen as he predominan objec. algorihm. I can be observed ha he Osu hresholding is performed wice for each image, and a hird ieraion is only underaken if Peak1 is less han 000 afer he second ieraion, or if Peak1 is more han 000 and Peak_500 is less han 3 afer he second ieraion. The values used for he comparisons of Peak1 and Peak_500 were obained from experimens based on he 56x56 endoscopic images. I is noeworhy ha he proposed echnique can also be exended o images of a differen resoluion using he following mehodology. The number of ieraions of hresholding, say n, can be deermined based on a reasonable number of raining images. The hisograms obained afer n ieraions of hresholding for all he raining images can be analyzed o obain a reliable esimae for he minimum value of Peak1. If Peak1 of he hisogram obained afer n ieraions happens o be less han he minimum value for cerain images, anoher final ieraion of hresholding is needed in order o beer disinguish he small lumen by reducing he conribuion of background pixels. Also, for he endoscopic images where Peak1 of he hisogram obained afer n ieraions of hresholding exceeds he minimum value, anoher final ieraion of hresholding is necessary only if he conribuion of background pixels is sill comparable o he value of Peak1. For insance, if here are a significan number of bins in he hisogram obained afer n ieraions of hresholding, whose pixel couns add up o a value comparable o Peak1, hen anoher final ieraion of hresholding can help o obain a hresholded image wih he lumen as he predominan objec. Hence he deerminaion of he value of n and Peak1 are he key seps in exending he proposed echnique for a new se of images wih differen resoluions. V. EXPERIMENTAL RESULTS In his secion, we compare he proposed bounded ieraive hresholding algorihm wih he baseline algorihm in [4], in erms of he qualiy of he deeced lumen regions, and compuaional complexiy. We will also discuss synhesis resuls o demonsrae he viabiliy of he proposed mehod for real-ime implemenaion. The experimens were performed on 40 gray-level endoscopic images of size 56x56 A. Effeciveness of lumen region deecion σ B max( σ B Fig. 7: Comparison of lumen ceners idenified by he proposed and baseline algorihms Fig. 6: Proposed bounded ieraive hresholding algorihm Le s denoe Peak_500 as he number of bins beween he peaks, whose pixel coun exceeds a value of 500. Fig. 6 illusraes he proposed bounded ieraive hresholding
Fig. 8: Endoscopic images (op and he hresholded images (boom using he proposed echnique, which enables accurae lumen ceners o be deeced In order o compare he qualiy of he deeced lumen regions beween he baseline algorihm and he proposed mehod, we perform he windowing operaion in [4] o locae he lumen ceners of he deeced lumen regions for boh echniques. Fig. 7 shows he absolue difference beween he lumen ceners idenified by he proposed mehod and he baseline mehod. I can be observed ha he resuls of boh he algorihms are in very close conformance excep for image 19. For his endoscopic image, he windowing algorihm is unable o idenify he lumen cener correcly for he baseline algorihm as he lumen region is small. However, he proposed mehod leads o he idenificaion of he correc lumen cener. Fig. 8 shows he lumen cener idenified by he proposed mehod for some of he endoscopic images used in he experimens. I can be observed ha he proposed echnique correcly idenifies he lumen cener for all he cases. I can be observed ha he compuaional complexiy for a single ieraion of hresholding for boh he baseline and proposed mehod is noably high, and hence reducing number of ieraions for lumen region deecion can significanly lower he compuaional complexiy. The number of Osu ieraions incurred by he proposed and baseline mehod for he sample endoscopic images are presened in Fig. 9. B. Compuaional Complexiy Analysis The seps and maximum number of operaions required for a single ieraion of Osu is shown in Table 1. TABLE 1 I. Hisogram and Inensiy Area (IA Compuaion Deermine ni i [0,55] by 1. 65536 addiions scanning all pixels. i*n i i [0,55] 56 Muliplicaions II. Cumulaive Hisogram and Cumulaive IA Compuaion 3. ni i= 0 [0,55] 55 Addiions 4. i * ni [0,55] 55 Addiions i= 0 III. Maximum BCV Compuaion 5. μ [0,55] 56 Divisions 6. ω [0,55] 56 Divisions Muliplicaions and 7. σ B( [0,55] Division for every 51 Muliplicaions and 51 Divisions Fig. 9: Number of Osu ieraions required by he baseline and proposed mehod for 40 endoscopic images The number of Osu ieraions performed by he baseline algorihm ranges beween and 7 (zero ieraions for he baseline indicaes ha i is unable o erminae. On he conrary, he proposed algorihm performs o 3 ieraions of Osu based hresholding. The need for a 3 rd ieraion is deermined based on he inensiy hisogram of he image obained afer ieraions as explained in he previous secion. The proposed algorihm requires lesser or he same number of ieraions compared o he baseline algorihm for all he images considered excep for image 4, where he proposed mehod requires one ieraion more han he baseline. In addiion, here are wo cases (image 5 and 11, where he baseline algorihm is unable o erminae, bu he proposed algorihm is able converge in -3 ieraions. Hence he proposed algorihm leads o lower number of Osu ieraions, which in urn resuls in significan reducion in he compuaional complexiy. On average, he proposed algorihm requires 0% lesser number of muliplicaions and
divisions when compared o he baseline mehod. The reducion in he number of muliplicaions and divisions can be as much as 71% (for image 10. C. FPGA implemenaion resuls The main compuaional blocks of he proposed bounded ieraive hresholding echnique consis of 3 sages. The firs sage comprises of he module for compuing he inensiy hisogram. A dual por BRAM wih 56 locaions, each of widh 16-bi is used o sore he inensiy hisogram. VI. CONCLUSION In his paper, we proposed a bounded ieraive hresholding algorihm for deecing he lumen region of endoscopic images ha limis he number of Osu ieraions o 3. A 3 rd ieraion of hresholding is performed only if deemed necessary by analyzing he hisogram peak corresponding o he lumen region and he inermediae pixel couns beween he wo discerning peaks afer he second ieraion. Simulaions on an exensive se of 40 endoscopic images of size 56x56 show ha he proposed mehod is more robus and lead o significan reducion in he compuaional complexiy when compared o he baseline algorihm. The FPGA synhesis resuls of he proposed mehod furher jusifies is real-ime capabiliy. Fig. 10: Archiecure for compuing CH and CIA When all he 65536 pixels of he endoscopic image have been accouned for in he hisogram, a signal indicaing he end of his ask iniiaes he subsequen sage which involves he compuaion of he Cumulaive Hisogram (CH and Cumulaive Inensiy Area (CIA. Fig. 10 shows he archiecure for generaing he CH and CIA. Two arrays consising of 56 regisers each were used for obaining he CH and CIA. The conens of he Hisogram BRAM are read successively and accumulaed in a emporary regiser CIValue. In parallel, inensiy area daa are accumulaed in anoher emporary regiser CAValue (hird sage. Thus, as he conens of he Hisogram array are read, he updaed values of CIValue and CAValue are pushed ino he regiser arrays of CH and CIA respecively. The archiecure for compuing he maximum BCV is no shown. The proposed design was synhesized for he Xilinx Sparan 6 (XC6SLX45-CSG34 device and he synhesis resuls are shown in Table. From he simulaion resuls i can be esimaed ha i akes approximaely 65800 clock cycles o generae CH and CIA for a 56x56 endoscopic image in a single ieraion of he proposed mehod. Since he proposed mehod is bounded by a maximum number of 3 ieraions, i only requires a mos.3ms o deec he lumen region of a single endoscopic image. TABLE AREA MAXIMUM Number of Number of FREQUENCY SLICE LUTS SLICE regisers 13 56 85.75MHz REFERENCES [1] V. K. Asari, S. Kumar, I. M. Kassim, A Fully Auonomous Microroboic Endoscopy Sysem, Journal of Inelligen and Roboic Sysems, 8: 35 341, 000. [] M. Liedlgruber, A. Uhl, A Summary of Research Targeed a Compuer-Aided Decision Suppor in Endoscopy of he Gasroinesinal Trac, Technical Repor 011-01, Universia Salzburg, April 011. [3] M. Liedlgruber, A. Uhl, Endoscopic Image Processing - An Overview, Proceedings of he 6h Inernaional Symposium on Image and Signal Processing and Analysis, December 009, pp. 707-71. [4] C.S. Lim, H. Tian, G. Sadana, T. Srikanhan, High Speed Technique for Auomaic Guidance of Microroboics in Endoscopic procedures, Proc. of he 4h Annual IEEE EMBS Conference on Informaion Technology Applicaions in Biomedicine (ITAB, Birmingham, UK, April 003, pp. 130-134. [5] P. Wang, S. Krishnan, Y. Huang, N. Srinivasan, An Adapive Segmenaion Technique for Clinical Endoscopic Image Processing, Proceedings of he Second Join Conference and he Annual Fall Meeing of he Biomedical Engineering Sociey EMBS/BMES Conference, Vol., 00, pp.1084-1085. [6] R. Reilink, S. Sramigioli, and S. Misra, Image based Flexible Endoscope Seering, IEEE/RSJ Inernaional Conference on Inelligen Robos and Sysems, Ocober 010, pp. 076-08 [7] S. Kumar, K. Asari, D. Radhakrishnan, Real-ime Auomaic Exracion of Lumen Region and Boundary from Endoscopic Images, Journal Med & Bio. Eng. & Compu., Vol.37, No.5, Sepember 1999, pp. 600-604 [8] N. Osu, A Threshold Selecion Mehod from Gray-Level Hisograms, IEEE Transacions on Sysems, Man and Cyberneics, vol. 9(1, 1979, pp. 6-66 [9] K. Asari, T. Srikanhan, S. Kumar, D. Radhakrishnan, A Pipelined Archiecure for Image Segmenaion by Adapive Progressive Thresholding, Microprocessors and Microsysems, 3(8-9, December 1999, pp. 493-499. [10] K. V. Asari, A Fas and Accurae Segmenaion Technique for he Exracion of Gasroinesinal Lumen from Endoscopic Images, Medical Engineering & Physics, Elsevier, 000, pp. 89 96 [11] H. Tian, S. K. Lam and T. Srikanhan, Area Time Efficien Beween- Class Variance Module for Adapive Segmenaion Process, IEE Proc.- Vis. Image Signal Process, Vol. 150, No. 4, Augus 003, pp. 63-69 [1] C. S. Lim, H. Tian, T. Srikanhan, An Auomaed Technique for High Speed Segmenaion of Endoscopic Images, Proceedings of 004 Inernaional Symposium on Inelligen Mulimedia, Video and Speech Processing, Hong Kong, Ocober 004 [13] C. S. Lim, S. K. Lam, H. Tian and T. Srikanhan, Efficien Archiecures for Segmenaion of Endoscopic Images in Micro-Roboic Auo Navigaion Sysems, Inernaional Journal of Humanoid Roboics, Vol. 3, No. 4, World Scienific, 006, pp. 53 545 [14] C. H. Chang, H. Tian, T. Srikanhan and C. S. Lim, Field Programmable Gae Array based Archiecure for Real Time Image Segmenaion by Region Growing Algorihm, Journal of Elecronic Imaging 11(4, SPIE, Ocober 00, pp. 469 478