Recent Advances in Sampling-based Alpha Matting

Transcription

1 Recent Advances in Sampling-based Alpha Matting Presented By: Ahmad Al-Kabbany Under the Supervision of: Prof.Eric Dubois

2 Recent Advances in Sampling-based Alpha Matting Presented By: Ahmad Al-Kabbany Under the Supervision of: Prof.Eric Dubois

3 Outline Motivation and introduction to matting basics Alpha matting online benchmark Sampling-based vs. propagation-based matting State-of-the-art sampling-based matting Sequential sampling matting directions and recommended resources 3

4 What is alpha matting? Hard Segmentation vs. Soft Foreground/Background Segmentation Source: A. Levin, A. Rav-Acha, and D. Lischinski. Spectral matting.ieee TPAMI, 30: , October

5 What is alpha matting? Applications of matting Source: Gastal, E. S. L., Oliviera, M.M.: Shared sampling for real-time alpha matting. Comput. Graph. Forum,

6 What is alpha matting? Applications of matting Alpha matting vs. chroma keying Source: Google Images 6

7 What is alpha matting? Applications of matting Alpha matting vs. chroma keying Natural image matting vs. material matting 7

8 What is alpha matting? Applications of matting Alpha matting vs. chroma keying Natural image matting vs. material matting Motivation Source: O.Woodford, I. D. Reid, P. H. S. Torr, and A.W. Fitzgibbon. On new view synthesis using multiview stereo. In Proc. BMVC., volume 2, pages ,

9 The linear convex image model (compositing equation) The trimap 9

10 The Trimap Manually generated Heavily affects the output of all existing algorithms 10

11 Motivation for an online benchmark Training and testing datasets Training and testing trimaps Alpha maps quality metrics More details can be found in [8] 11

12 Propagation-based Matting Propagating alpha values to unknown regions Several methods were used for interpolation Optimizing Markov random fields Solving affinity matrices Propagation and image continuity Failure cases and drawbacks 12

13 Sampling-based Matting Sampling a search pool of FG/BG pairs. Usually searching nearby regions in the trimap Which pair is the best? Drawbacks differences 13

14 Geodesic Distance Matting [7] Collects nearby samples What is the geodesic distance? An alternative to Euclidean distance Useful for complex object topologies Locally distributed samples 14

15 Shared Sampling Realtime Matting [3] Collecting samples by shooting rays Taking the nearest FG/BG on every ray Why shared? Is it realtime? 15

16 Shared Sampling Realtime Matting [3] Collecting samples by shooting rays Taking the nearest FG/BG on every ray Why shared? Is it realtime? A comment on the sampling process 16

17 Global Sampling Matting [2] The distant-but-true problem 17

18 Global Sampling Matting [2] The distant-but-true problem 18

19 Global Sampling Matting [2] 19

20 Color/Texture Matting [6] Which problem does it address? How does it extract the texture information? 20

21 Color/Texture Matting [6] The pair-selection process 21

22 Sequential Pair-Selection Matting What is the problem with global sampling? 22

23 Sequential Pair-Selection Matting An intuitive observation There is something true nearby, otherwise it is an isolated region. All the possible start points Local FG, Local BG, Non-local FG, Non-local BG What is meant by a half-pair? What is meant by sequential pair-selection? 23

24 Sequential Pair-Selection Matting 24

25 Sequential Pair-Selection Matting

26 Sequential Pair-Selection Matting Color/Texture Checks Near FG 1. We have a pattern 2. But we lack an expression Near BG Distant FG Distant BG 26

27 Sequential Pair-Selection Matting Color/Texture Checks Near FG 1. We have a pattern 2. But we lack an expression LEARNING Near BG Distant FG Distant BG 27

28 Recommended Resources (Books) Application of decision forests to: Object recognition Pose estimation Semantic segmentation Among others... 28

29 Decision tree classifiers for CV applications What is the decision tree learning? Classification vs. Regression trees Decision tree classifiers for inpainting and matting The suggested DT framework for matting Tree building (training) The hybrid classification/regression step 29

30 Some Recommended Resources 1. Wang, J., Cohen, M.F.: J. Wang and M.F.: Cohen. Image and video matting: A survey. Foundations and Trends in Computer Graphics and Vision, 3(2):97-175, He, K., Rhemann, C., Rother, C., Tang, X., Sun, J.: A global sampling method for alpha matting. In CVPR, pages , Gastal, E. S. L., Oliviera, M.M.: Shared sampling for real-time alpha matting. Comput. Graph. Forum, 29(2): , Levin, A., Lischinski, D., Weiss, Y.: A closed-form solution to imagematting.ieee Trans. Pattern Anal. Mach. Intell., 30(2): , Rhemann, C., Rother, C.,Wang, J., Gelautz, M., Kohli, P., Rott, P.: A perceptually motivated online benchmark for image matting. In CVPR, pages , Shahrian, E., Rajan, D.: Weighted color and texture sample selection for image matting. In CVPR, pages , C. Rhemann, C. Rother, and M. Gelautz. Improving color modeling for alpha matting. BMVC, Rhemann, C., Rother, C.,Wang, J., Gelautz, M., Kohli, P., Rott, P.: A perceptually motivated online benchmark for image matting. In CVPR, pages , natural 30

31 Recap Introduced matting basics Highlighted the online benchmark Discussed matting algorithms families Explained a few SoA sampling-based techniques Shed the light on our contribution Mentioned future directions and recommended resources 31

32 Thank you 32