WITH THE ADVANCE of digital technologies, digital

Transcription

1 678 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 15, NO. 3, MARCH 2006 Video Halftoning Zhaohui Sun, Member, IEEE Abstract This paper studies video halftoning that renders a digital video sequence onto display devices, which have limited intensity resolutions and color palettes, by trading the spatiotemporal resolution for enhanced intensity/color resolution. This trade is needed when a continuous tone video is not necessary or not practical for video display, transmission, and storage. In particular, the quantization error of a pixel is diffused to its spatiotemporal neighbors by separable one-dimensional temporal and twodimensional spatial error diffusions. Motion-adaptive gain control is employed to enhance the temporal consistency of the visual patterns by minimizing the flickering artifacts. Experimental results of halftone and colortone videos are demonstrated and evaluated with various halftoning techniques. Index Terms Digital halftoning, human visual system, motion estimation, spatiotemporal error diffusion, temporal flicker, video rendering and display. I. INTRODUCTION WITH THE ADVANCE of digital technologies, digital video is getting easier and more efficient to use in a wide variety of applications, such as entertainment, education, medicine, security, and the military. Accordingly, there is an increasing demand for video processing techniques [1]. Video halftoning is a task that renders video sequences onto display devices that have limited intensity resolutions and color palettes. It provides an alternative for video representation, rendering, storage, and transmission when continuous tone video is not necessary or not practical. And it can be used in various applications, including the following. Display. It can be used to render continuous tone video onto display devices, when there is a mismatch between the image/video representation and the display capability because of the constraints of cost and system complexity, such as small electronic gadgets (e.g., cellular phone, personal digital assistant (PDA), game console, and vehicle dashboard), large screen display (e.g., cinema poster, commercial billboard, and stadium screen), and flexible display (e.g., packaging label). Data reduction. With a shorter bit depth, the size of a halftone or colortone video is much smaller than its counterpart with continuous tone and can be further reduced after exploring the temporal consistency of the static and slow-moving visual patterns. Manuscript received January 22, 2004; revised April 6, This work was done when the author was with the Research and Development Laboratories, Eastman Kodak Company, Rochester, NY. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Zhigang (Zeke) Fan. The author is with the Visualization and Computer Vision Lab, GE Global Research, Niskayuna, NY USA ( sunzh@research.ge.com). Digital Object Identifier /TIP Error-resilient communication. As stochastic noise patterns are used to conceal the quantization errors in the spatiotemporal domain less visible to the human eye, any random perturbation on the halftone video, such as channel noise, is less pronounced in terms of image quality degradation. Therefore, it is particularly suitable for wireless communication. Related prior art includes digital image halftoning and the extension to handling image sequences. Image halftoning reduces the image intensity/color resolution for the best possible reproduction and has wide applications in the printing industry. A number of techniques have been proposed [2] [7], such as error diffusion, ordered dither, dot diffusion, and stochastic screening. A comprehensive review can be found in [8] [10]. Studies have also been carried out applying the halftoning technology to image sequences [11] [16]. A three-dimensional (3-D) error diffusion algorithm is proposed in [11], including a scheme to minimize the flickering artifacts. In [12], an iterative image halftoning algorithm is applied to the image sequence where the halftone map on the previous frame is used as the starting point for iterative refinement on the current image frame, thus minimizing the temporal flicker. In [14], spatiotemporal error diffusion filters are designed for the luminance and chrominance channels at different frame rates. And the direct-binary-search algorithm is applied to 3-D error diffusion in [15]. Motivated by the widespread use of image halftoning in the printing industry, we extended the concept to video halftoning for display applications, by reducing the video tone scale with minimum visual degradation. The major contributions of this paper are as follows: scheme of video tone-scale reduction by separable temporal and spatial error diffusion; method of temporal flicker reduction by the use of motion information. After the problem formulation in Section II, separable temporal and spatial error diffusions are employed in Section III to diffuse the quantization error of a pixel to its causal temporal neighbor along the motion trajectory and its causal spatial neighbors. The diffused interframe error is large for fast-moving patterns at high frame rates and small for static patterns at low frame rates. To alleviate the temporal flickering artifacts, the quantization threshold of a pixel is adaptively adjusted based on the motion information in Section IV, which increases the inertia of the interframe error diffusion in static regions to enhance temporal consistency and encourage free error diffusion in fast-moving regions for the best image quality. A video halftoning algorithm is presented in Section V. It is applied to the generation /$ IEEE

2 SUN: VIDEO HALFTONING 679 of halftone and colortone video sequences, and evaluated with the other halftoning techniques in Section VI. The paper is concluded in Section VII. A preliminary version of this paper appeared in [17]. II. PROBLEM FORMULATION A digital video sequence is a temporally varying, two-dimensional (2-D) spatial signal on frame, sampled and quantized at spatial location. Signal contains a single luminance channel for grayscale video and two additional chrominance channels for color video. Each channel is quantized to bits, e.g., 8-bit grayscale video and 24-bit color video when. The task of video halftoning is to transform a full resolution video with a continuous tone scale (e.g., ) to a dithered video with a shorter bit depth (e.g., ), such that the perceived visual difference is made as small as possible. As shown in Fig. 1, the difference between the continuous tone video and the halftone video is displayed and perceived by human eyes. Under the assumption of linearity and ignoring the device-dependent display MTF, the perceived visual difference can be written as where denotes convolution, and is the impulse response of the visual system. If is separable in temporal and spatial dimensions,, with and as the temporal and spatial impulse responses, can be further written as Accordingly, video halftoning can be formulated as an optimization problem i.e., seeking the optimal halftone video, which minimizes the perceived visual difference (a summation taken across the spatial and time coordinates). Video halftoning can be taken as an extension to image halftoning by spreading the quantization error of a pixel to its 3-D spatiotemporal neighbors (instead of its 2-D spatial neighbors only) and making the noise less visible to the human visual system (HVS). Video halftoning adds an additional temporal dimension to conceal the quantization noise and has more flexibility to enhance video intensity resolution. It also needs more computational power to process a large amount of data. The spatiotemporal characteristics of the human visual system are more complicated, and the artifact of temporal flicker needs special attention. In the following, the intensity values are normalized to [0,1], with as black, as white, and as the middle point. (1) (2) (3) Fig. 1. Video halftoning finds the best halftone rendering V with the minimal perceived visual difference. III. SPATIOTEMPORAL ERROR DIFFUSION We start with a discussion of 3-D error diffusion in Section III-A. Because of its complexity and difficulty in practice, we resort to separable error diffusion in Section III-B, which is followed by separate discussions on the one-dimensional (1-D) temporal error diffusion in Section III-C and the 2-D spatial error diffusion in Section III-D. At the end, the idea is extended to multitone and colortone video in Section III-E. A. Three-Dimensional Error Diffusion To evaluate the cost function in (3), a spatiotemporal model of the HVS [18] based on the psychophysical tests can be used. It has a modulation transfer function (MTF) of where is the spatial frequency in cycles per degree, and is the temporal frequency in Hz. The model shows lowpass in spatial dimensions and bandpass in temporal dimension. The basic idea is to spread the quantization errors to the stop bands as high frequency noise ( blue noise ) such that they are less visible after the spatiotemporal filtering of. The optimization in (3) can be carried out by 3-D spatiotemporal error diffusion. An incoming video, along with the previously diffused error, i.e.,, are quantized as the halftone video. The quantization error is spatiotemporally filtered and fed back to the input. An alternative approach searches the spatiotemporal solution space for the optimal halftone video.itflips a pixel from 0 to 1 or from 1 to 0 and accepts the new state if the flip brings down the cost function in (3). The process repeats until no flip occurs in one sweep of all the pixels on all the frames, when a local optimum is reached. Both approaches involve 3-D spatiotemporal filtering and require large memory and intensive computation. The 3-D error diffusion has a few difficulties in practice. Because the human visual system is very complicated [19], a model is only valid for certain viewing conditions, and error diffusion filters need to adapt to the video content. Therefore, only local optimal solutions are practical. As the operations of filtering and cost function evaluation are carried out on 3-D video entities [e.g., group of frames (GOP)] with a large amount of data, they need intensive computation, introduce delay, and require high system complexity. Any compromise tends to introduce additional artifacts, such as temporal flicker. Therefore, we move (4)

3 680 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 15, NO. 3, MARCH 2006 Fig. 3. (a) Diffusion of error " (p) to its spatial neighbors and the temporal neighbor on the next frame and (b) collection of error " (p) from its spatial neighbors " and temporal neighbor ". Fig. 2. Separable temporal and spatial error diffusion with motion-adaptive gain control. Fig. 3(a), part of the error is diffused along the motion trajectory to the temporal neighbor as and the rest to the intraframe neighbors as in the spatial domain. The diffused error in (5) is collected from its spatiotemporal neighbors from the previous computation, as shown in Fig. 3(b) away from the 3-D model and resort to the separable temporal and spatial filters studied in [20]. B. Separable Error Diffusion The 3-D spatiotemporal MTF can be approximated by a concatenation of a 1-D temporal MTF and a 2-D spatial MTF,, as shown in [20]. Therefore, the 3-D spatiotemporal error diffusion can be carried out by a temporal error diffusion followed by a spatial error diffusion, which greatly simplifies the system complexity. The basic idea is to diffuse part of the quantization error of a pixel into its causal temporal neighbor along the motion trajectory and the remainder to its causal spatial neighbors to minimize the spatial visual distortion. The exact amount is controlled by a temporal diffusion map. More interframe error is diffused temporally in fast-moving regions, leaving less error to be diffused to the spatial neighbors. The separable error diffusion scheme is shown in Fig. 2. The video frames are processed sequentially. The pixels inside the frame are scanned in a serpentine order, from left to right on even lines and from right to left on odd lines. At a pixel location, the image intensity and the quantization errors diffused from its spatiotemporal neighbors are quantized to, by a comparison of with the threshold if otherwise. Pixel on halftone video is denoted as a black dot if the adjusted intensity value is less than the threshold, or a white dot otherwise, yielding a quantization error To improve the visual quality, the quantization error is diffused to the spatiotemporal neighbors. As shown in (5) (6) Part of is contributed by from the temporal neighbor on the previous frame with a weight of, and the rest from from the spatial neighbors on the current frame with a weight of. Motion vector specifies the horizontal and vertical displacements at location in frame to its correspondence in frame. Bilinear interpolation is carried out at the noninteger locations on the temporal error image. The are the spatial error diffusion filter coefficients with. In Fig. 3, the spatial neighbors and filter coefficients are chosen as those defined in the variable-coefficient error diffusion [6], with varying with intensity code value. C. Temporal Diffusion The temporal characteristics of the HVS is complicated and less well known than its spatial counterpart. Here, we employ a model proposed in [21]. Based on the psychophysical experiments, the temporal model consists of a lowpass filter and a bandpass filter. Specifically, it uses function and its high-order derivatives to model the temporal mechanism of the targets perceived at the center of the human eye. Function and its normalized second-order derivative, with s and s, are shown in Fig. 4(a), and the frequency responses are depicted in Fig. 4(b), showing one lowpass filter and one bandpass filter. Finite impulse response (FIR) filters with linear phases can be designed to approximate and [22]. At the frame rates of 30 and 60 Hz, a total of five and nine video frames fall into the time span of and. The five-tap lowpass FIR (7) (8)

4 SUN: VIDEO HALFTONING 681 Fig. 4. Temporal characteristics of the human visual system. (a) Impulse response and (b) frequency response. out by various image halftoning techniques with adaptive gain control. Here, only, part of, is diffused spatially, and the split of temporal error and spatial error is motion adaptive and content dependent. Spatial error diffusion involves the choices of the causal neighbors and the design of the error diffusion filter. Based on the psychophysical experiments, a model of the spatial frequency response of the HVS (10) Fig. 5. (a) Lowpass filter g (t) and the bandpass filter g (t) at 30 Hz. (b) The lowpass and bandpass filters of g (t) and g (t) at 60 Hz. filter and the five-tap bandpass filter for 30 Hz video are shown in Fig. 5(a), and the nine-tap FIR filters and for 60 Hz video are shown in Fig. 5(b). The filter coefficients vary with the choices of the scale parameter and the time-to-peak parameter. The temporal diffusion map,, on frame [also denoted as in (7)] is content dependent and can be determined by the temporal characteristics of the HVS and the video frame rate. Based on the temporal filters and, we choose as so that the major part of the noise energy falls into the stop bands. In (9), is the temporally smoothed version of. At low frame rates ( 10 Hz), as, there is no temporal error diffusion, and all of the quantization errors are exclusively diffused to the spatial neighbors. It is the same situation in the static regions at high frame rates. In the fast-moving regions at high frame rates, approaches 1, allowing more quantization error to be diffused across frames, and leaving less quantization errors in the spatial domain. The high frequency noises become less visible after temporal smoothing by the HVS. At frame rates higher than 60 Hz, the temporal masking effect of the human eye should be taken into consideration, and some frames can be dropped because the sensation of a high-contrast pattern lasts for a finite duration. (9) has been proposed in [23], where is the frequency in degrees per cycle. It has low pass characteristics, with a peak at 8 cycles/degree and dropping to 0 beyond 30 cycles/ degree. Thus, it is desirable to distribute the quantization error to the high frequency bands as the less visible blue noise. In the following, we use the 2-D variable-coefficient error diffusion filter [6] for spatial error diffusion. E. Colortone Video The idea of separable error diffusion can be extended to videos with multiple tone-scale levels or multiple color channels. To generate multitone video for multilevel displays, the binary thresholding in (5) is replaced by a multilevel quantizer, and the quantization error of a pixel is diffused to its causal spatiotemporal neighbors in a similar way. With additional chrominance channels in a color video, video halftoning has more flexibility and complexity to diffuse and conceal the quantization errors in color space as well as the spatiotemporal domain. When color dependency is ignored, the digital video halftoning scheme presented in Fig. 2 can be applied directly to colortone video generation, by replacing the scalar intensity variable with a vector color variable. Separable error diffusion is independently carried out in the YUV color channels. This approach is taken in the following experiments. It is also desirable to explore the color dependency and diffuse quantization errors across color channels. For example, the human eye is less sensitive to noise in the chrominance channels than that in the luminance channel. However, it involves a more sophisticated vision model and more complicated error diffusion filters. IV. MOTION-ADAPTIVE TEMPORAL CONSISTENCY Temporal flicker is a special artifact that, over time, alternates black and white patterns at the same spatial location. It can be caused by model approximation or independent intraframe halftoning. To alleviate temporal flicker, we use adaptive gain control to increase the temporal consistency in by adaptively changing the threshold D. Spatial Diffusion After temporal error diffusion, the rest of the quantization error is diffused in the spatial domain, and can be carried if otherwise (11)

5 682 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 15, NO. 3, MARCH 2006 used in the quantization decision (5). The threshold is moved away from the middle point, based on the previous visual pattern, video frame rate, and local motion. The adaptive gain control increases the inertia of the interframe halftoning, making similar to unless the spatiotemporally diffused error,, is large enough. The gain control map on frame (also denoted as in (11)) is content dependent and can be chosen as (12) where is the motion vector from point in frame to its corresponding point in frame. In static and slowmoving regions, is close to 1, and the halftoning of is strongly biased to for enhanced temporal consistency. In fast-moving regions with large motion vectors, is close to 0, and free error diffusion is encouraged to conceal the quantization error. Scale factor, (e.g., 0.75), guides the transition from slow to fast motion. Numerous motion estimation algorithms can be used to compute, such as gradient-based, region-based, energy-based, and transform-based approaches. Here, we use a robust algorithm [24] for the computational purpose. In the regions with outliers, caused by occasional model violation or occlusion, is set to 0. It is also helpful to run a median filtering on to smooth out any inconsistent outliers. For some compressed video, such as the MPEG, QUICKTIME, or streaming video, the block motion vectors are readily available in the data stream without further computation. An alternative model of is the use of temporal variances of adjacent frames instead of the motion vectors Input: Digital continuous tone video. Output: Halftone/colortone video. 1) Initialize temporal filter, temporal diffusion map, gain control map, motion field, and frame index. 2) Process the frames sequentially, and scan the pixel on frame in a serpentine order. 3) Collect the diffused quantization error from the spatiotemporal neighbors (7). 4) Quantize to based on the motion-adaptive quantization threshold (5). 5) Compute the quantization error (6). 6) Diffuse part of to the temporal neighbor along the motion trajectory. 7) Diffuse the rest of to the causal spatial neighbors on frame. 8) Go to step 2) for the rest of the pixels on frame, then increase frame index. 9) Retrieve or compute the motion field from frame to frame. 10) Determine the temporal diffusion map (9). 11) Determine the gain control map (12), (13), (14). 12) Go to step 2) until all the frames are processed. (13) where expectation is the windowed average of temporal intensity, with scale factor specifying the intensity deviation (e.g., 5). Another alternative is to use the temporal highpass filtering as a measure of the intensity changes (14) The algorithm has a few advantages. First, it has relatively low time and space complexities. At each pixel location, only the causal spatial neighbors and the temporal neighbor are involved in computation. Furthermore, separable error diffusion is carried out by 1-D temporal diffusion followed by spatial diffusion, which greatly reduces the system complexity. Given the motion information, the algorithm can generate high quality halftone video for real-time applications with minimal delay. Second, the spatial error diffusion is flexible and compatible with the widely available image halftoning techniques, which can be used as the plug-in modules. Third, temporal flickering artifacts are minimized by motion-adaptive gain control. where is a bandpass/highpass temporal filter. V. ALGORITHM The video halftoning algorithm is summarized as follows. VI. EXPERIMENTAL RESULTS A. Setup The proposed video halftoning scheme is tested on two video sequences. The grayscale Trevor sequence has 99 frames and a spatial resolution of It is shot by a static camera with a static textured background and a moving foreground (a

6 SUN: VIDEO HALFTONING 683 Fig. 6. (a) Frames 2, 34, 66, and 99 of the grayscale video sequence Trevor overlaid with motion vectors to the previous frames. (b) Frame 34 of the halftone video at the frame rates of (left) 30 Hz and (right) 60 Hz. person wearing a highly textured shirt and tie). The color Football sequence has 97 frames and a spatial resolution of It is shot by a slow-moving camera with fast-moving players and a highly textured field. A weighted signal-to-noise ratio (WSNR) is used as a metric for performance evaluation, which is defined as (15) is chosen as or where the temporal filter shown in Fig. 4. It is a measure of the filtered signal energy over the filtered noise energy. A large WSNR indicates high video quality with small visual degradation. Subjective visual quality is also judged by observers. cyan, magenta, and yellow. The results on frame 34 at frame rates of 30 and 60 Hz are printed in Fig. 7(b) at 120 dpi. The colortone video uses only a fraction of the colors to produce a realistic tone scale rendering., on frame 34 of Examples of the gain control maps, the sequences are shown in Fig. 8(a) and (c). The white regions denote static and slow-moving patterns, which are strongly biased to the halftone patterns on the previous frames. The dark regions denote fast-moving patterns that allow free error diffusion for the best possible image reproduction. The temporal, determines the weights for the temporal diffusion map, error diffusion. It tends to increase at high video frame rates. Examples on frame 34 of the sequences are shown in Fig. 8(b) and (d). The dark regions diffuse all errors inside a frame, and the white regions spread more errors across frames. B. Halftone/Colortone Video C. Evaluation Selected frames of the Trevor sequence, overlaid with the motion fields to the previous frames, are shown in Fig. 6(a). The 8-bit grayscale sequence is rendered as a monochrome video with only black and white dots. The results on frame 34 of the halftone videos rendered at frame rates of 30 and 60 Hz are printed in Fig. 6(b) at a spatial resolution of 120 dpi. The random patterns, coupled with the characteristics of the HVS, provide a sensation of enhanced tone scale. The algorithm is also applied to the colortone video. Selected frames of the 24-bit color Football sequence, overlaid with the motion fields to the previous frames, are shown in Fig. 7(a). The continuous tone color video is rendered as a colortone video with a palette of only eight colors, black, white, red, green, blue, The proposed video halftoning algorithm is compared to five different halftoning techniques, including the Floyd-Steinberg error diffusion method [2], the ordered dither method [3], the frame-dependent image halftoning method (Gotsman) [12], the Hild-Pins halftoning method [11], and the 3-D error diffusion method (AFHBA) [14]. The visual results for the 30 Hz Trevor sequence are shown in Fig. 9, and the numerical results in terms of WSNR are presented in Table I, where the is chosen as the lowpass filter or temporal filter at 30 Hz [shown in Fig. 5(a)]. the bandpass filter The halftoning results by the six methods on a portion of frame 34 in Fig. 6(b) are printed at 100 dpi in rows (a) and (c) of Fig. 9. The rendering results by the ordered dither method in

7 684 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 15, NO. 3, MARCH 2006 Fig. 7. (a) Frames 2, 34, 66, and 97 of the color video sequence Football overlaid with motion vectors to the previous frames. (b) Frame 34 of the colortone video at the frame rates of (left) 30 Hz and (right) 60 Hz. (Color version available online at Fig. 8. Gain control maps (i; j ) in (a) and (c) and temporal diffusion maps (i; j ) in (b) and (d) on frame 34 of the sequences at 30 Hz. TABLE I PERFORMANCE COMPARISON OF THE HALFTONING SCHEMES III (a) and the Hild-Pins method in II (c) are not as good as the others. The differences between the halftone frames 34 and 33 (the flickering artifacts) by the six methods are shown in rows (b) and (d) of Fig. 9. With special attention, the temporal flicker can be dramatically reduced by the video-halftoning method in I (b), the Gotsman method in I (d), and the Hild-Pins method in II (d). The AFHBA and Floyd-Steinberg methods give good spatial rendering in III (c) and II (a); however, the flickering artifacts dominate the frame differences in III (d) and II (b), in the static background as well as the moving foreground, yielding poor temporal quality. Conversely, the order-dither and Hild-Pins methods enforce temporal consistency aggressively, so the frame difference is very small, which means the temporally adjacent halftone frames are almost identical, and the spatial quality is poor. Only the video-halftoning method enforces content-dependent temporal consistency. The observations are supported by the numerical results in Table I, which provide the WSNR in the lowpass and the bandpass temporal channels. Overall, the video-halftoning technique provides the best spatiotemporal halftone rendering. For the algorithm complexity, the Gotsman method is the most computationally intensive approach. A direct binary search is carried out to flip the black and white dots until the visual difference is minimized. It is not suitable for real-time video processing tasks with a large number of frames. Next comes the video-halftoning method. It needs extra computation to dynamically update the temporal diffusion map and the gain control map. However, given the motion information, it still can be used in real-time applications. The rest of the methods can all be implemented very efficiently. It is worth notice that all of the methods guarantee only local optimum as a result of model approximation. This opens up future opportunities to explore the best possible rendering based on the spatiotemporal model, the viewing condition, and the video content. VII. CONCLUSION We have presented a video halftoning scheme to render continuous tone digital video as halftone and colortone video on display devices with limited tone scales and color palettes. Separable 1-D temporal and 2-D spatial error diffusions are carried out to spread the quantization errors in spatiotemporal domains less visible to the human visual system. Temporal FIR filters are

8 SUN: VIDEO HALFTONING 685 Fig. 9. Halftone frame 34 and the flickering artifact at 30 Hz by the video halftoning method in I (a) and (b), the Floyd-Steinberg method [2] in II (a) and (b), the ordered dither method [3] in III (a) and (b), the Gotsman method [12] in I (c) and (d), the Hild-Pins method [11] in II (c) and (d), and the AFHBA method [14] in III (c) and (d).

9 686 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 15, NO. 3, MARCH 2006 designed at various video frame rates to diffuse temporal errors along motion trajectory across frames. And a motion-adaptive gain control scheme is presented to enhance temporal consistency and alleviate flickering artifacts. REFERENCES [1] A. M. Tekalp, Digital Video Processing. Englewood Cliffs, NJ: Prentice-Hall, [2] R. Floyd and L. Steinberg, An adaptive algorithm for spatial grey scale, Proc. Soc. Inf. Display, vol. 17, no. 2, pp , Mar [3] B. E. Bayer, An optimum method for two-level rendition of continuoustone pictures, in Proc. IEEE Int. Conf. Communication, vol. 1, 1973, pp [4] J. Sullivan, L. Ray, and R. Miller, Design of minimum visual modulation halftone patterns, IEEE Trans. Syst., Man, Cybern., vol. 21, no. 1, pp , Jan./Feb [5] B. Kolpatzik and C. A. Bouman, Optimized error diffusion for image display, J. Electron. Imag., vol. 1, no. 3, pp , [6] V. Ostromoukhov, A simple and efficient error-diffusion algorithm, in Proc. ACM SIGGRAPH, 2001, pp [7] N. Damera-Venkata, B. L. Evans, and V. Monga, Color error-diffusion halftoning, IEEE Signal Process. Mag., vol. 20, no. 4, pp , Jul [8] R. A. Ulichney, Digital Halftoning. Cambridge, MA: MIT Press, [9] H. R. Kang, Digital Color Halftoning. New York: SPIE/IEEE Press, [10] J. C. Stoffel and J. F. Moreland, A survey of electronic techniques for pictorial image reproduction, IEEE Trans. Commun., vol. 29, no. 12, pp , Dec [11] H. Hild and M. Pins, A 3-D error diffusion dither algorithm for half-tone animation on bitmap screens, in State-of-the-Art in Computer Animation Proceedings of Computer Animation. Berlin, Germany: Springer-Verlag, 1989, pp [12] C. Gotsman, Halftoning of image sequence, Vis. Comput., vol. 9, no. 5, pp , [13] J. B. Mulligan, Methods for spatiotemporal dithering, in Proc. SID Int. Symp. Dig. Tech. Papers, Seattle, WA, 1993, pp [14] C. B. Atkins, T. J. Flohr, D. P. Hilgenberg, C. A. Bouman, and J. P. Allebach, Model-based color image sequence quantization, in Proc. SPIE/IS&T Conf. Human Vision, Visual Processing, and Digital Display V, vol. 2179, Feb. 1994, pp [15] D. P. Hilgenberg, T. J. Flohr, C. B. Atkins, J. P. Allebach, and C. A. Bouman, Least-squares model-based video halftoning, in Proc. SPIE/IS&T Conf. Human Vision, Visual Processing, and Digital Display V, vol. 2179, Feb. 1994, pp [16] D. P. Scholnik and J. O. Coleman, Joint spatial and temporal deltasigma modulation for wide-band antenna arrays and video halftoning, presented at the IEEE ICASSP, Salt Lake City, UT, May [17] Z. Sun, A method to generate halftone video, presented at the IEEE ICASSP, Philadelphia, PA, Mar [18] D. H. Kelly, Motion and vision: II. Stabilized spatio-temporal threshold surface, J. Opt. Soc. Amer., vol. 69, pp , [19] B. A. Wandell, Foundations of Vision. Sunderland, MA: Sinauer, [20] E. H. Adelson and J. R. Bergen, Spatiotemporal energy models for the perception of motion, J. Opt. Soc. Amer. A, vol. 2, no. 2, pp , Feb [21] R. E. Fredericksen and R. F. Hess, Estimating multiple temporal mechanisms in human vision, Vis. Res., vol. 38, pp , [22] S. Winkler, Issues in vision modeling for perceptual video quality assessment, Signal Process., vol. 78, no. 2, pp , Oct [23] J. L. Mannos and D. J. Sakrison, The effects of a visual fidelity criterion on the encoding of images, IEEE Trans. Inf. Theory, vol. 20, no. 4, pp , Jul [24] M. J. Black and P. Anandan, The robust estimation of multiple motions: parametric and piecewise-smooth flow fields, Comput. Vis. Image Understand., vol. 63, no. 1, pp , Jan Zhaohui Sun (M 00) received the B.E. and M.E. degrees in electronics engineering and information science from the University of Science and Technology of China, Hefei, in 1992 and 1995, respectively, and the M.S. and Ph.D. degrees in electrical and computer engineering from the University of Rochester, Rochester, NY, in 1998 and 2000, respectively. He is currently with the Visualization and Computer Vision Lab, GE Global Research, Niskayuna, NY. From 2000 to 2005, he was with the Research and Development Laboratories, Eastman Kodak Company, Rochester, as a Research Scientist, a Senior Research Scientist, and a Principal Research Scientist. In 1998, he was an Intern Researcher with the Imaging and Visualization Department, Siemens Corporate Research, Princeton, NJ. His research interests include vision technologies, digital video/image processing, medical imaging, and multimedia computing.