Figures from Embedded System Design: A Unified Hardware/Software Introduction, Frank Vahid and Tony Givargis, New York, John Wiley, 2002
Data processing flow to implement basic JPEG coding in a simple B&W digital camera: Acquire CCD/CMOS Data and Correct for Zero-offset either in Hardware or through Loop-processing in Memory. SDRAM fast local memory 6 Meg. More 8 x 8 pix blocks? No Yes Perform Discrete Cosine Transform (DCT) on 8 x 8 block Further reduce bit/byte count by Huffman coding (variable length value representation short bit patterns for common values, longer for rarer values.) Compress 2.5: or more. Store in serial flash memory (onboard or memory stick.) Reduce bit count by coarsened quantization. Saves half bits? Download to PC or printer by USB or remove memory card and plug into USB.
The DCT algorithm: 7 7 uv, 4 uv, i, j i= 0 j= 0 ( π ( ) ) ( π ( ) ) F = C D cos 2i+ u/6 cos 2j+ v/6 Here, (u,v) is the set of indices of the 8x8 array in the frequency domain. The set of amplitude values of the individual pixels as indexed by row and column with i,j. The leading factor is a trivial function that appropriately normalizes the frequency C uv, is the /2 if uv, = 0,0 domain values; it is defined as Cuv, / 2 if u =, v= 0; or if u = 0, v=. if u > and v> The cosine values, of course, are a fixed table that can be stored in memory trivially. The calculation nominally requires 64 mulitply-add operations, but one reason for the choice of the size and type of transform is that there are methods to reduce the number of operations needed to do the function by exploiting the properties of the cosine function. The inverse operation, performed at the printer or PC end of processing - not in the camera, is: 7 7 i, j 4 uv, uv, i= 0 j= 0 ( π ( ) ) ( π ( ) ) D = C F cos 2u+ i/6 cos 2v+ j/6 Design alternatives and their metrics: The text from which I have borrowed this example (available on the class web-site) examined four possible ways to build a system for a 4096 pixel black and white camera. (On the face of it, this is absurdly but detailed examples are not readily available.) One of the methods was impossibly absurd because the processor could not even acquire data in a reasonable length of time. The methods were Use a small (8-bit Intel 805) processor to do all arithmetic with floating point emulation Use some custom hardware in a standard-cell ASIC to offload part of the arithmetic, particularly the acquire and zero-offset operations because of their simplicity. Use the same system but do all arithmetic in integer format Use a larger, fully custom standard-cell ASIC with an 805 embedded in the design (SOC-IP) to take more of processing into hardware, keeping primarily interface functions in the 805. The first method simply did not work well enough to consider. The other methods all involved some amount of custom hardware gotten by using VHDL or Verilog descriptions and synthesizing to standard cell ASICs. (I hope that you will look at the D i, j
possibility of something similar for your project but into FPGAs.) The metrics for the remaining methods are shown in the table below. Huffman coding is lossless compression done as the picture is stored into serial flash memory. It is based on using variable length bit patterns for values and exploits the fact that some values are more frequent than others. The saving come if those frequent codes can be represented by two or three bit-long short patterns for an 8-bit or 0-bit original code. Here is a tree structure showing how the Huffman code derives from the frequency of codes in the example 8x8 block.
Color pictures: Color sensors do not have 3 pixels per nominal spatial site. Instead the filters are arranged over the underlying array in what is called the Bayer pattern that looks like: The additional computational steps for color processing are: Interpolate between color pixels to get three full images in RGB Smooth the resulting pictures to remove spatial aliasing effects from the undersampling. (This two-step process is called demosaicing. It turns out to be the most complex of the computational operations and is generally proprietary to each camera vendor.) Convert data format to luminance (essentially a B&W version) and two colors by matrix multiplication of the three-color intensity vectors pixel site by pixel site JPEG compress all three channels but compress the two color signal more thoroughly than the luminance. (We are more attuned to spatial detail than color and the object of the game is to produce pleasing not necessarily accurate pictures.)