Accelerating Image Processing Algorithms with Microblaze Softcore and Digilent S3 FPGA Demonstration Board

Transcription

1 Acceleratig Image Processig Algorithms with Microblaze Softcore ad Digilet S3 FPGA Demostratio Board Computer Electroics 1 st Semester, 2011/ Itroductio This project itroduces a example for image processig algorithms for MicroBlaze (MB) softcore processor [3] for Xilix Field Programmable Gate Array (FPGA) devices [4] utilizatio. The itroduced cocepts of the tutorial i [1] will be used to create a system for image processig that also icorporates iterfaces for a image capture camera ad a VGA display, allowig for the visualizatio of the implemeted real-time algorithms. The system desig will be accomplished employig the Xilix ISE ad Embedded Developmet Kit (EDK) tools [5], versio The implemetatio will be supported o the Digilet S3 starterkit board [2] which embed a Xilix Sparta 3 FPGA (part XC3S1000-4), ad two custom iterfaces that coect to the S3 starterkit board through its geeral purpose I/O pis that allow for image capture ad display. Before goig through this project the studets are suggested to: to have completed the tutorial i [1]; have a comprehesive readig o the MB processor architecture ad supported istructios; study some image processig basic routies; kow basic cocepts of C ad assembly laguage; kow basic cocepts of VHDL hardware descriptio laguage. After the completio of this project the studets are iteded to kow how to: efficietly characterize a algorithm i software ad hardware compoets for a FPGA system; efficietly accelerate image processig algorithms with a MB based system. This project guide is orgaized as follows. I sectio 2 the backgroud o the hardware modules, amely the image capture ad display modules, is described. I sectio 3 the flow to obtai a complete system for image capture, processig, ad display is described, based o the itroductory desig described i [1]. I sectio 4 the image processig algorithms are addressed, ad differet approaches for their implemetatio are discussed, amely differet partitioig betwee software ad hardware resources. 1

2 S3 board Sparta 3 FPGA Camera Iterface MicroBlaze Geeral I/O 8-bit DAC VSYNC HSYNC OV9650AA module VGA display VGA Iterface Figure 1: MB based image processig system cofiguratio D7 D6 D5 D4 D3 D2 D1 D0 R 2R 4R 8R 16R 32R 64R 128R R G VGA display 75Ω 75Ω B 75Ω Figure 2: 8-bit DAC layout. RGB Sigal H/V Syc T pw T bp T disp T fp T S Figure 3: VGA sigals properties. 2 Prelimiaries I this project guide we aim the real-time image processig. I order to evaluate this requiremet, we provide two hardwired peripherals that allows to real time video acquisitio ad display. These peripherals coect to exteral video acquisitio/display devices. The video acquisitio device is a CMOS SXGA digital camera with 1.3 MegaPixels with a maximum spatial resolutio of 1280 x 1024 with 24 bits for color. The video output display is a VGA computer moitor. The MB peripheral that coect to the exteral devices were developed i VHDL specifically for this applicatio. While the camera provides the output i digital format, for the VGA display a simple Digital to Aalog Coverter (DAC) is required to obtai the results i aalog format. I Figure 1 is depicted the system cofiguratio for the image processig applicatio. The layout of the DAC circuit is preseted i Figure 2. The VGA sigals properties are preseted i Figure 3 ad Table 1. The schematic of the sychroizatio sectio of peripherals developed i VHDL that 2

3 Exteral ports Slave Master Slave Master Exteral ports HREF D D cout RESET VSYNC Risig edge detectio Table 1: Temporal properties of the VGA sigals. PXL_CLK D D Symbol Parameter Vertical syc [µs] Horizotal syc[s] T S Period 16,700 32,000 T disp Display time 15,360 25,600 T pw Syc pulse width 64 3,840 T fp Syc frot porch T bp Syc back porch 928 1,920 RESET D Colum cout WE Fallig edge detectio FODD CLK HREF VSYNC D D RESET Lie cout CLK mod 800 Colum cout PXL_CLK Risig edge detectio D D RESET Colum cout 1 Colum addressig sectio mod 521 >=656 <752 <640 <480 Horizotal syc pulse Blak Lie cout FODD D WE Lie addressig sectio >=490 <492 Vertical syc pulse (a) Iput (b) Output Figure 4: Schematic of the exteral video I/O devices iterfaces. CLK mod 800 Colum cout 1 Colum addressig sectio mod 521 Lie addressig sectio Iteral ets >=656 <752 Camera peripheral FSL Horizotal syc pulse MicroBlaze VGA peripheral <640 Blak <480 Iteral ets Lie cout Figure>=490 5: MB based EDK Vertical project syc overview for image processig. pulse <492 FSL Iteral ets iterface the MB softcore with the exteral video acquisitio ad display devices is preseted i Figure 4. 3 Image processig with MicroBlaze I this sectio we guide the desig of a system, with the structure defied i Figure 1, i order to evaluate image processig algorithms. We will employ the kowledge acquired i the tutorial [1] regardig the EDK project creatio ad attachmet of peripherals. The project to be developed (or icremeted to the project i the tutorial [1] ) i the EDK eviromet is represeted i Figure 5. 3

4 Table 2: Iteral et coectio iformatio. Camera peripheral VGA peripheral Exteral ports Ports Dir. Net Rage Class Ports Dir. Net Rage Class Ports Dir. Net Rage Class clock I sclk CLK clock I sclk CLK clock I sclk2dcm CLK c15 I sclk CLK v07 I s io01 I si 7:0 c14 I s v06 I sc 7:0 io02 I sj c13 I sm v05 O sb 12:0 io03 I sk c12 I sl CLK v04 O sa io04 I sl CLK c11 I sk v03 O sq 7:0 io05 I sm c10 I sj v02 O sp io06 I s RST c09 I si 7:0 v01 O so io07 O so c08 O sh io08 O sp c07 O sg io09 O sq 7:0 c06 O sf io10 O sf c05 O se io11 O sd c04 O sd io12 O se c03 O sc 7:0 io13 O sg c02 I sb 12:0 io14 O sh c01 I sa Implemetig the image processig system a) Create a project similar to the oe you created i the tutorial [1], or use the same if you already have the files. b) Reame the reset ad clock exteral ports ad ets to the same ames used i the project i the tutorial [1]. Reame the clock i clock_geerator to sclk as well. c) The peripherals for the camera ( camera_iterface_v1_00_a ) ad VGA ( VGA_iterface_v1_00_a ) iterfaces were provided to you with this project guide. Take a look o the VHDL files (specially the top level VHDL file) of these projects ad try to idetify the similarities to the project implemeted i the tutorial [1], amely the FSL bus sigals. Copy the folders that cotai these peripherals to the project_path/pcores folder. d) Click Project->Resca User Repositories. Now you ca fid the camera ad VGA peripherals at the ed of the IP Catalog. Add a camera ad a VGA peripheral to your project. Also, add two FSL buses. Reame oe of the FSL buses to camera2mb ad the other oe to mb2vga. e) Cofigure the MB ( microblaze_0 ) istace to accept aother FSL coectio. Also, cofigure the MB Istructios tab to use a barrel shifter, 32-bit iteger multiplier, ad iteger divider. We do ot eed floatig poit support or patters comparator. We will optimize the desig for area, which meas that a 3-stage pipelie will be used. f) Coect the buses accordig to Figure 5. Coect the clock ( sclk ) ad reset ( s ) ets to the FSL buses i the Ports tab. Add ad reame exteral ports ad coect the iteral ets betwee peripherals ad exteral ports as suggested i Table 2. g) Check the file mb.ucf ad copy the pi assigmet ad costraits related to your project s exteral ports to the *.ucf of your project. This completes the hardware cofiguratio for your system. Hece you ca geerate the software libraries for your system by selectig Software->Geerate Libraires ad BSPs. 4

5 (a) Origial Image. (b) Histogram. (c) Embossig. Figure 6: Image processig output examples. h) Create 2 extra software projects with the ames camera_sw ad camera_sw_asm. Copy the file egative.c to the src folder of the camera_sw project, ad the file equalizatio.s to src folder of the camera_sw_asm. These files cotai a sample applicatio to compute the egative of a image ad equalize a image, respectively. The latter is writte i assembly laguage usig the MB istructios, hece it is a optimized implemetatio. Aalyze these samples carefully ad try to perform your ow applicatio. i) Compile the software ad obtai the bitstream to program the FPGA. Remember to select the software project you wat to use i a curret cofiguratio bitstream by selectig Mark to iitialize BRAMs. Also, i the Compiler Optios of your project set the optimizatio level to O2. j) Dowload the bitstream to the FPGA ad check the results. O the FPGA there are 4 buttos. The butto 2 ( BTN2 ) makes the camera to cofigure its registers, hece you must push this butto otherwise you will ot be able to see the captured images. 4 Software ad hardware image processig I this sectio two examples of image processig applicatios are thoroughly described i order to be implemeted by the studets usig the system set i sectio 3. For this task, the studets must be aware of basic cocepts of C laguage, assembly (see the supported MB istructios i [3]) ad VHDL. The two applicatios that are iteded to be implemeted are the histogram ad image embossig. The output of these applicatios are depicted i Figure 6 for a 256-graylevels image with 512x512 pixels size. I the image processig system a image size of 64x128 must be used istead, thus the required modificatio i the algorithms must be performed. 4.1 Histogram The represetatio of the histogram is a bar like graphic, where each bar correspods to a gray-level ad the size of the bar correspods to the umber of pixels with that gray-level. Summarizig, i order to obtai the histogram, followig procedure should be followed: 5

6 Defie ad iitialize to zero a memory rage (variable) for the histogram, which should be a iteger array with the size of the umber of possible gray levels. Defie a memory rage (variable) to store the image, which should be a byte array (assumig 256 gray levels) with the size of the umber of pixels i the image. Read the image; Browse each pixel of the image ad icremet the histogram etry that correspods to that pixel. Draw the histogram picture to the image array. Write the histogram image to the output device. Special attetio must be payed to the histogram drawig. Note that the image width may ot be exactly the umber of gray levels, thus the the etries of the histogram must be replicated or collapsed, depedig of the image width to be larger or smaller tha the umber of gray levels, respectively. Also, the image height may be smaller or larger tha the maximum umber of pixels with the same gray level, which may result the histogram to overflow the image boudaries or very difficult to visualize. This meas, that a scale factor should be established to dimesio the histogram data to the image. Regardig this issue, two optios ca be take: Set a fixed scale factor, which should be a compromise for the histogram bars to be easy to visualize i all the image height; Set a variable scale factor, such that the larger bar to correspod to the etire image height ad the other bars to be scaled appropriately. 4.2 Image embossig Whe embossig a image we are iterested i turig light/dark boudaries i highlights ad shadows, while settig low cotrast areas to gray (middle of the gray levels rage) backgroud. The idetificatio of light/dark boudaries is accomplished by computig the image first derivative. The first derivative magitude is more proouced i boudaries we are iterested to idetify. After computig the derivative, the obtaied values ca be added to the backgroud gray level i order to light or dark the boudary regios, while maitaiig the low cotrast areas almost uchaged. Fially, the computatio result must be scaled such that the boudary pixels are scaled to fit the rage [0 : N G 1], where N G is the umber of gray levels. Summarizig, i order to emboss a image, the followig procedure should be followed: Defie a memory rage (variable) to store the image I, which should be a byte array (assumig 256 gray levels) with the size of the umber of pixels i the image. Defie a memory rage (variable) to store the derivative D, which should be a iteger array with the size of the umber of pixels i the image. Defie a memory rage (variable) to store the squared covolutio kerel of width k W = 3, which should be a iteger array with the k W xk W size ad iitialize with: K = (1)

7 Read the image; Compute the derivative o oe side of the boudary by computig the covolutio D = I K. Add the derivative i the other side of the boudary accumulatig the covolutio D = D I K, where the kerel elemet ( K) i,j correspods to (K) kw 1 j,k w 1 i: K = (2) The values i D are i the rage [D mi : D max ]. A scale factor s must be applied (D = sd) to assure that the values are i the rage [ N G /2 : N G /2 1]. At this time the values of D are aroud zero. However, we wat the backgroud level to be the gray level N G /2. Hece, the resultig image is obtaied by addig the backgroud level N G /2 to each value i D. Note that the image embossig performace ca be greatly improved by takig ito accout that the two covolutio operatios ivolved ca be merged ito oly oe. Also, the covolutio kerels have several etries equal to powers of 2, thus multiplicatios ca be efficietly replaced by shift operatios. The derivative computatio ca be also performed o a differet directio which would result for the dark image areas to become light ad vice-versa. To chage the directio, the covolutio kerel K ca be chaged accordigly: K = 4.3 Implemetig the algorithms (3) a) Most of the effort to compute the algorithms ivolves software developmet. Hece, the studets are suggested to implemet the proposed algorithms i sectios 4.1 ad 4.2 i stadard C usig as experimetal setup a persoal computer ad their favorite developmet ad debuggig tools. For this task, a start poit implemetatio is provided with this tutorial i the file image_processig_sample.zip. The curret implemetatio i this file computes the egative of a image. The studets should update this project with the proposed algorithms. This procedure is preferable tha implemetig the project directly o EDK sice it ivolves hardware placemet which is time cosumig ad does ot ehace the debug. b) Oce the algorithms are correctly implemeted, the routies ca be adapted to the image size that will be hadled i the MB base system. With this, the software implemetatio becomes closer to the implemetatio to be set i the MB system. Thus, after completig the image processig algorithms the trasitio to our image processig system will be o more tha C code portig to the EDK eviromet, with small modificatios regardig the image readig/writig ad the removal of system calls. 7

8 c) After update the software project i the EDK eviromet, compile the program ad load the updated bitstream to the FPGA device ad check both histogram ad image embossig applicatios. 4.4 Optimizig the algorithms Despite the possible optimizatios i the algorithms, other approach to ehace the system performace rely i the descriptio of the algorithm as close to the hardware as possible. The first approach towards this goal, is the descriptio usig the MB assembly istructios. The secod approach is the implemetatio directly i hardware usig a hardware descriptio laguage such as VHDL. I this tutorial the studets are suggested to implemet the image embossig with assembly istructios ad accelerate the histogram computatio with dedicated hardware Image embossig with assembly a) Chage the applicatio i the equalizatio.s file ad implemet the image embossig algorithm with assembly istructios Histogram with VHDL a) To be completed. Refereces [1] Computer Electroics Course, DEEC, IST. Microblaze Softcore ad Digilet S3 FPGA Demostratio Board: Tutorial, 1st Semester, [2] Xilix Ic. Digilet S3 Starterkit board. Data/ Products/ S3BOARD/ S3BOARD_RM.pdf. [3] Xilix Ic. Microblaze Referece Maual, versio support/ documetatio/ sw_mauals/ mb_ref_guide.pdf. [4] Xilix Ic. Xilix FPGA Documetatio. support/ documetatio/ idex.htm. [5] Xilix Ic. Xilix ISE ad EDK tools. support/ dowload/ idex.htm. 8