Eccentricity Effect of Motion Silencing on Naturalistic Videos Lark Kwon Choi*, Lawrence K. Cormack, and Alan C. Bovik Dec. 6, 206
Outline Introduction Background Visual Masking and Motion Silencing Eccentricity Effect of Motion Silencing Human Subjective Studies Result Analysis Conclusion and Discussion 2
Perception of Visual Distortions Humans are generally the ultimate arbiter of digital videos.? Natural Scene Capture Video Processing Processing, Display Human Spatial distortions Blocking Ringing False contouring Blur Noise Temporal distortions Motion compensation mismatch Mosquito effects Ghosting Jerkiness Flickering 3
Visual Masking Visibility reduction of a stimulus (target) caused by the presence of another stimulus (mask) in space and/or time. Visibility of distortions is strongly reduced Annoying blocking artifacts Spatial masking is well known and widely used for visual processing. [Bovik, 200; Wang and Bovik, 20] 4
Motion Silencing Illusion Temporal Visual Masking is Please not well-modeled, look but temporal at masking a white is evident. dot in the center. Perceive luminance changes of dots. [Suchow and Alvarez, 20] Flicker visibility is strongly reduced with motions. 5
Flicker Visibility on Naturalistic Videos Q: Does motion silencing work on real naturalistic videos? Q2: What s motion effects on motion silencing? Q3: What s eccentricity effects on motion silencing? Execute a series of human subjective studies. Develop LIVE Flicker Video Database http://live.ece.utexas.edu/research/quality/live_flicker_video.html Phase I: Motion effects Phase II: Motion and eccentricity / eccentricity effects A: Works well. Would be useful for developing VQA algorithms. A2: Large, coherent motions strongly reduce flicker visibility. A3: Large motion and eccentricity much more strongly reduces flicker visibility. 6
LIVE Flicker Video Database Source videos using RED ONE cinematographic camera - 3K (3072 728), 42MB/s (Highest quality), and 30fps. - Diverse object motions. - 6 Source videos and 92 distorted (72 test, 20 training) videos. *Tractor content was obtained from Technical Univ. of Munich. 7
Quality levels LIVE Flicker Video Database Quantization Flicker simulations Source video Downsample (720p) H.264 codec compression 4 different quality levels Alternate video frames Excellent (QP26) Good (QP32) (QP44) Poor (QP38) (QP47) Bad (QP44) (QP50) 3 3 3 2 3 5 Frames Bad Bad Bad Different Excellent quality level changes Different flicker frequencies QP * : Quantization parameter; Excellent Excellent larger value means more compression. Bad 8
Phase I: Motion effects on flicker visibility (Experiment) Extremely Target: Baseball batter in white uniform Highly Medium Please fixate and follow your eyes on the target, and rate the visibility of flickering on the target by moving the mouse up or down the scale continuously. Little When you are ready, Please press the spacebar to play Hardly [Choi et al. QoMEX 203, SPIC 205] 9
Flicker visibility Object speed (pixel/frame) Phase I: Motion effects on flicker visibility (Result) P32-QP26 P38-QP26 P44-QP26 bject speed - 43 subjects - Eye tracker (FaceLAb5) - 24 monitor Flicker Visibility 00 80 60 40 20 Bb QP32-QP26 QP38-QP26 QP44-QP26 Object speed 3Hz 5Hz 7.5Hz Object speed Flicker Visibility Flicker Visibility 00 80 60 40 20 40 30 0 0 60 0 00 200 300 0 40 frame number 20 Bb Bb 00 80 60 0 0 00 200 300 20 0 frame number 40 Bb 00 80 20 As motion increases, flicker visibility is strongly suppressed. 40 30 20 0 0 60 0 20 0 [Choi et al. QoMEX 0 0 00 203, SPIC 200 205] 300 0 40 30 0 0 00 80 60 40 20 0 00 80 40 20 0
Gaze Perceive Eccentricity? Motion silencing is a peripheral effect that does not occur near the fixation point. We study eccentricity effect and the combined eccentricity-motion effect on flicker visibility.
Phase II: Eccentricity and motion effects on flicker visibility Task Gaze the fixation mark : INSTRUCTION. Please find the red fixation mark (+) and fixate your eyes always on the fixation mark. 2. Rate flicker visibility on the target by moving the mouse up or down continuously. Target: Circled regions on the batter. Extremely Highly + Medium Little When the video begins, the instruction and the rating bar disappear except for a white score gauge. Do immediately move the mouse after pressing the spacebar. When you are ready, please press the spacebar to start. Hardly [Choi et al. GlobalSIP 205] 2
Phase II: Eccentricity and motion effects on flicker visibility Task 2 Follow the moving object : INSTRUCTION. Please find the target and fixate your eyes always on the target by following it. 2. Rate flicker visibility on the target by moving the mouse up or down continuously. Target: Circled regions on the batter. Extremely Highly Medium Little When the video begins, the instruction and the rating bar disappear except for a white score gauge. Do immediately move the mouse after pressing the spacebar. When you are ready, please press the spacebar to start. Hardly [Choi et al. GlobalSIP 205] 3
Results: Eccentricity and motion effects on flicker visibility - 33 subjects. Please find the red fixation mark (+) and fixate your eyes always on the fixation mark. 2. Rate flicker visibility on the target by moving the mouse moving up or down object ) continuously. Target: Circled regions on the batter. Task ( Gaze the fixation mark ) Eccentricity and motion Task : INSTRUCTION Task 2 ( follow the Motion Extremely Highly Medium Little When the video begins, the instruction and the rating bar disappear except for a white score gauge. Do immediately move the mouse after pressing the spacebar. When you are ready, please press the spacebar to start. As eccentricity increases, flicker visibility is strongly suppressed. Hardly 4
Phase II: Eccentricity and motion effects on flicker visibility Ratio = Task ( Gaze the fixation mark ) Flicker visibiity in Task Flicker visibiity in Task 2 + Task 2 ( follow the moving object ) As eccentricity increases, flicker visibility is strongly suppressed. 5
Phase II: Eccentricity and motion effects on flicker visibility Ratio = Flicker visibiity in Task Flicker visibiity in Task 2 6
Phase II: Eccentricity and motion effects on flicker visibility Correlation Analysis QP44- QP26 QP47- QP26 QP50- QP26 Ratio Ratio Ratio.2 Bb BMX La Mr Rc Tr -0.755-988 -745-35 -0.986, 0.775-0.7845 0 4 8 2 Eccentricity -0.9376 0 4 8 2 Eccentricity -0.9679. 0.9 0.7 0 4 8 2 Eccentricity 0.4 0.2 0 5 0 5 0.4-392 0.2 0 5 0 5 0.4-778 0.2 0 5 0 5.2 0 2 4-78 0.4 0 2 4 0.9 0.7 0.5-0.9006 0 2 4. 0.9. 0.9 0 2 4 6-0.98 0 2 4 6. 0.9-0.980 0 2 4 6 0.4 0.2 0 6 2 8-0.9934, -0.903 0.4 0.4 0 6 2 8-0.9825, -66 0.2 0 6 2 8. 0.9. 0.9 0.95 0.9 5 0 2 4 6-658 0 2 4 6-899 0 2 4 6 7
Conclusion and Discussions We study eccentricity effects of motion silencing on flicker visibility in naturalistic videos. Eccentricity? Flicker visibility Results show that large eccentric, large motion strongly reduces the visibility of flicker distortions on real videos. LIVE Flicker Video Database (Publicly available). Applications accounting temporal flicker masking: Perceptual flicker visibility models and VQA algorithms. 8
Questions? 9