New Algorithms and FPGA Implementations for Fast Motion Estimation In H.264/AVC

Transcription

1 Slide 1 of 50 New Algorithms and FPGA Implementations for Fast Motion Estimation In H.264/AVC Prof. Tokunbo Ogunfunmi, Department of Electrical Engineering, Santa Clara University, CA 95053, USA Presented at University of Texas, Arlington Friday, February 11, 2011

2 Slide 2 of 50 Acknowledgement Ph.D Student Obianuju Ndili SCU School of Engineering for Teaching Assistantship and Packard fellowship award. Xilinx Inc. and Xilinx University Program.

3 Slide 3 of 50 Summary of Research Problem Support high-quality, real-time video encoding on low power mobile devices and low bit rate applications. Solution: Speed-up motion estimation via fast search algorithms and via hardware. Algorithms and Architectures: Co-designed fast algorithms and supporting hardware architecture: 1. Modified Simplified Unified Multi-Hexagon (SUMH) 2. Hardware-oriented Modified Diamond Search (HMDS) 3. Modified Fractional Motion Estimation 4. Combined integer and fractional motion estimator

4 Slide 4 of 50 Introduction Motivation Research Outline Integer motion estimation - HMDS Fractional Motion Estimation Motion Estimator Conclusions Publications

5 Slide 5 of 50 Introduction H.264/AVC H.264/MPEG-4 AVC latest block oriented, motion compensation-based video coding standard. By JVT: ITU-T VCEG (H.26L) & ISO/IEC MPEG. Good quality video at substantially lower bit rates. Broad range of applications - low bit rate internet streaming, to HDTV broadcast and digital cinema. Versions: First - May 2003, FRExt - Sept. 2004, SVC - Nov. 2007, MVC - Nov Reference Software provided.

6 Slide 6 of 50 Features of H.264/AVC Related to motion compensation: Up to 16 reference frames 7 Variable block sizes: 4 4, 4 8, 8 4, 8 8, 8 16, 16 8, Quarter-/Eighth-pel precision. 6-tap filtering for half-pel luma samples. Bilinear filter for quarter-pel luma samples. Others: Integer transform, Hadamard transform, Entropy coding (CAVLC, CABAC), Flexible macroblock ordering, In-loop deblocking filter, etc.

7 Slide 7 of 50 Profiles CBP / BP Videoconference and mobile. MP Standard definition digital TV broadcast. Hi* - HDTV, Blu-ray storage, professional.

8 Supported Slide 8 of 50 Levels

9 Slide 9 of 50 Performance of H.264/AVC H.264/AVC achieves 50% bitrate reduction compared to H.263, MPEG-2, MPEG-4 pt. 2 ASP. Penalty complex encoder. By R. Schafer, T. Wiegand & H. Schwarz (2003)

10 Slide 10 of 50 Encoder

11 Slide 11 of 50 Introduction Motion Estimation Find best matching reference block for current macroblock. Matching criteria: Sum of Absolute Differences (SAD) Sum of Absolute Transformed Differences (SATD) Sum of Squared Differences (SSD)

12 Slide 12 of 50 Block Distortion Measures Used x(i,j) and y(i,j) - pixels of the current, and candidate blocks, (di, dj) - displacement of candidate block in search window, I J - size of the current block, H -4 4 Hadamard transform matrix.

13 Slide 13 of 50 Motivation Motion estimation (ME) is the most powerful compression tool but consumes the most encoding time in H.264/AVC. Need to speed up ME to achieve savings in power for low power, typically low bit rate, portable devices. Speed up ME via: fast motion estimation algorithms hardware acceleration. In this talk we propose fast motion estimation algorithms as well as co-designed hardware.

14 Slide 14 of 50 Hardware-Oriented Modified Diamond Search (HMDS) Low complexity, fast motion estimation algorithm for integer motion estimation. Targets low power devices and low bit rate applications which typically use H.264/AVC baseline 1 2. Test sequences from QCIF to HD 720p. Results show HMDS: has better rate-distortion performance and speed-up. losses compared to Full Search, are insignificant. Architecture is more hardware-efficient for FPGAbased architectures. Performance comparable to architectures implemented on ASICs

15 Slide 15 of 50 Related Prior Work - Algorithms Fast algorithms using fixed set of search patterns: TSS, 4SS, BBGDS, DS, HEXS. Amenable to hardware implementation. Suffer the local minima problem. Adaptive and hybrid fast algorithms. EPZS, UMHexagonS, SUMH, ACQPPS. Typically not amenable to hardware implementation.

16 Slide 16 of 50 Related Prior Work - Architecture Most architectures proposed for Full Search (FS). Regular search pattern less complex hardware, more reuse. Some architectures for fast algorithms. Chao et. al DS. Miyakoshi et. al BBGDS. Lin et. al 4SS. Chen et. al Adaptive 4SS. Qiu et. al ACQPPS.

17 HMDS Search Patterns Why DS? (Cheung & Po, 2002, 2005). Most translation, zooming, pan and tilt, fall on diamond corners. Slide 17 of 50

18 HMDS Description Slide 18 of 50 1 st Center 3 rd Center 2 nd Center

19 Slide 19 of 50 Simulation Conditions Reference Software JM 13.2, 1 or 2 reference frames, RDO on, IPPP sequence I refresh rate 15Hz

20 Slide 20 of 50 Simulation Results Complexity Search points per macroblock Performance Average Y-PSNR

21 Slide 21 of 50 Simulation Results Bit rate increase at low bit rates RD Performance at high bit rates

22 Slide 22 of 50 Results RD Curves % Time savings: 71% 97.6%.

23 Slide 23 of 50 Architecture for HMDS 41 SADs =

24 Slide 24 of 50 HMDS Processing Unit Input current and candidate pels. Output is 16 4x4 blocks. Output takes 16 clock cycles.

25 Slide 25 of 50 SAD Combination 5 cycles.

26 Slide 26 of 50 Comparison Unit Input 41 SADs from 4 PUs. Output is 41 minimum SADs. Output takes 5 clock cycles.

27 Slide 27 of 50 HMDS Architecture Comparisons Comparison with other FPGA implementations

28 Slide 28 of 50 HMDS Architecture Comparisons Comparison with other ASIC implementations

29 Slide 29 of 50 Fractional Motion Estimation Further improves motion estimation by 2 3 db. Accounts for about 40% of motion estimation time. Only 17 search points for H.264/AVC. Computation cost is significant for our target application. Profile results show that half-pel interpolation is computational bottleneck. Half-pel interpolation is critical since it gives significant improvement over integer ME. Quarter-pel interpolation gives further modest improvement.

30 Slide 30 of 50 Related Prior Work Hardware acceleration of Fractional Motion Estimation (FME) via: Static interpolation costs much memory. Algorithm decomposition to increase parallelism. Estimating SAD at quarter-pel positions. Support of only block types above 8x8. Proposal of different half-pel interpolation filter.

31 Slide 31 of 50 Background H tap filter [1/32, -5/32, 20/32, 20/32, -5/32 and 1/32]. horizontal vertical quarter horizontal - vertical

32 Slide 32 of 50 Sample Computations Horizontal half-pel position g g ' ( E 5F 20 G 20 g ( g ' 16) 5 H 5I J ) Vertical half-pel position k k ' ( A 5C 20G 20 M k ( k ' 16) 5 5Q S ) bit shift Horizontal-vertical half-pel position l l' l Quarter pels ( a 5b 20 g 20 ( l' 512 ) 10 ( g k 1) ( g l 1) 1 1 p 5q r)

33 Slide 33 of 50 Proposals for Half-Pel Filter Hyun s 5-tap, 4-input half-pel interpolation filter has weights [-5/32, 21/32, 21/32, -5/32]. No multiplications. E.g. Our proposed 4-tap, 4-input half-pel interpolation filter has weights [-2/32, 18/32, 18/32, -2/32]. No multiplications. E.g.

34 Slide 34 of 50 Frequency Response Comparisons Our filter is a better lowpass (interpolation) filter. Our filter has sharper cut-off, linear phase, thus higher fidelity. Consistent with simulation results.

35 Slide 35 of 50 Simulation Results Simulation done with H.264 Reference Software JM Reference frame, IPPP@15Hz, RDO on, Full Search for IME.

36 Slide 36 of 50 Simulation Results

37 Slide 37 of 50 Proposed Architecture

38 Slide 38 of 50 Interpolation Filters F G H I g i Input integer-precision candidate pixels. No multipliers. Output takes 4 clock cycles. Input integer- and half-pelprecision candidate pixels. No multipliers. Output takes 2 clock cycles.

39 Slide 39 of 50 Processing Unit Input current and candidate fractional pixels. Output is 16 4x4 blocks. Output takes 20 clock cycles.

40 Slide 40 of 50 Processing Element Illustrative clock cycle of PE.

41 Slide 41 of 50 SATD Combination & Comparison 5 cycles. 4 cycles.

42 Implementation Results Slide 42 of 50

43 Slide 43 of 50 Integer/Fractional Motion Estimator Many more proposals for architectures that support either integer or fractional motion estimation, but not both. Some proposals that combine both: Zhang et. al DS + CS for integer search, then H.264 fractional search. Chen et. al Full search at 5-bit precision for integer search, sub-sampling at half-pel rate and ASRA, then H.264 fractional search. Our proposal Use HMDS for integer search, then our modified fractional search.

44 Slide 44 of 50 Proposed Motion Estimator FPSoC The user / IP cores are our earlier proposed architectures for HMDS and FME.

45 Slide 45 of 50 Proposed FPSoC contd. Processor sends stimuli to user / IP cores and reads results back for comparison. PLB controller provides interface to external SDRAM which stores frame data. PLB also connects to the IME IP core through an IP interface. The OPB connects to the FME IP core through an IP interface. We employ FIFOs for read / write operations between the processor and IP cores.

46 Slide 46 of 50 Dataflow - Algorithmic State Machine

47 Slide 47 of 50 Results Rate Distortion Curves Insignificant loss compared to Full Search in H.264/AVC.

48 Slide 48 of 50 Results Rate Distortion Performance Average Y-PSNR loss: db compared to Full Search in H.264/AVC. % bit rate increase: 5% compared to Full Search in H.264/AVC.

49 Slide 49 of 50 Comparison With Other Architectures Our FPSoC: Has higher maximum frequency than [1] [4]. Has comparable throughput in terms of cycles per MB. Can support HD 720p at 24Hz. Has lower equivalent gate count than [2] [4].

50 Slide 50 of 50 Conclusions We presented our algorithms and architectures for motion estimation speedup. Results showed that algorithms on average, had better speed-up and rate-distortion performance at low bitrates, than some previous algorithms. Losses compared to Full Search motion estimation, were insignificant. Our algorithm proposals are suitable for low power devices and low bitrate applications.

51 Slide 51 of 50 Publications Journal Papers 1. O. Ndili and T. Ogunfunmi, Efficient fast algorithm and architecture for integer and fractional motion estimation in H.264/AVC, Under Review at IEEE Transactions on Consumer Electronics, O. Ndili and T. Ogunfunmi, Algorithm and architecture co-design of hardware-oriented, modified diamond search for fast motion estimation in H.264/AVC, IEEE Transactions on Circuits and Systems for Video Technology, O. Ndili and T. Ogunfunmi, FPSoC-based architecture for a fast motion estimation algorithm in H.264/AVC, EURASIP Journal on Embedded Systems, vol. 2009, Article ID , 16 pages, Oct

52 Slide 52 of 50 Conference Papers Publications 1. O. Ndili and T. Ogunfunmi, Efficient fast algorithm and FPSoC for integer and fractional motion estimation in H.264/AVC, IEEE International Conference on Consumer Electronics (ICCE), pp , Jan O. Ndili and T. Ogunfunmi, Efficient sub-pixel interpolation and low power VLSI architecture for fractional motion estimation in H.264/AVC, IEEE International Conference on Signal Processing and Communication Systems (ICSPCS), Dec O. Ndili and T. Ogunfunmi, An efficient hardware-oriented algorithm and its low power VLSI design for fast integer-pel motion estimation in H.264/AVC, IEEE Conference on Design and Architectures for Signal and Image Processing (DASIP), Oct

53 Slide 53 of 50 Publications Conference Papers Contd. 4. O. Ndili and T. Ogunfunmi, Hardware-oriented modified diamond search for motion estimation in H.264/AVC, IEEE International Conference On Image Processing (ICIP), Sept O. Ndili and T. Ogunfunmi, A hardware oriented integer pel fast motion estimation algorithm in H.264/AVC, IEEE Conference on Design and Architectures for Signal and Image Processing (DASIP), Nov O. Ndili and T. Ogunfunmi, On the performance of a 3D flexible macroblock ordering for H.264/AVC, IEEE International Conference on Consumer Electronics (ICCE), pp , Jan

54 Slide 54 of 50 Questions?