User-Steered Editing of Natural Images Based on the Image Foresting Transform

Size: px

Start display at page:

Download "User-Steered Editing of Natural Images Based on the Image Foresting Transform"

Cathleen Alexander
5 years ago
Views:

1 User-Steered Editing of Natural Images Based on the Image Foresting Transform Thiago Vallin Spina (M.Sc. student) Advisor: Prof. Dr. Alexandre Xavier Falcão Laboratory of Visual Informatics - Institute of Computing University of Campinas tvspina@liv.ic.unicamp.br Thiago Vallin Spina (M.Sc. student) (LIV-IC-UNICAMP) IFT-Based Editing of Natural Images tvspina@liv.ic.unicamp.br 1 / 10

2 Outline This presentation is organized as follows: 1 Motivation 2 Main goals of our research 3 Current works Thiago Vallin Spina (M.Sc. student) (LIV-IC-UNICAMP) IFT-Based Editing of Natural Images tvspina@liv.ic.unicamp.br 2 / 10

3 Motivation For the past two years we have been developing efficient and effective IFT-based methods for interactive hard segmentation of natural images [1, 2]. Thiago Vallin Spina (M.Sc. student) (LIV-IC-UNICAMP) IFT-Based Editing of Natural Images 3 / 10

hard segmentation of natural images [1, 2].

details and deal with occlusions. Thiago Vallin Spina (M.Sc.

4 Motivation For the past two years we have been developing efficient and effective IFT-based methods for interactive hard segmentation of natural images [1, 2]. However, our current method cannot always capture fine details and deal with occlusions. Thiago Vallin Spina (M.Sc. student) (LIV-IC-UNICAMP) IFT-Based Editing of Natural Images 3 / 10

5 Main goals of our research 1 Improve the framework developed for image segmentation (i.e., enhance the user interface and combine DIFT [3] with live-wire-on-the-fly [4]). Thiago Vallin Spina (M.Sc. student) (LIV-IC-UNICAMP) IFT-Based Editing of Natural Images tvspina@liv.ic.unicamp.br 4 / 10

Main goals of our research 1 2 Improve the framework developed

Use IFT-based pattern classifiers and other operators to deal

6 Main goals of our research 1 2 Improve the framework developed for image segmentation (i.e., enhance the user interface and combine DIFT [3] with live-wire-on-the-fly [4]). Use IFT-based pattern classifiers and other operators to deal with alpha matting and inpainting Thiago Vallin Spina (M.Sc. student) (LIV-IC-UNICAMP) IFT-Based Editing of Natural Images tvspina@liv.ic.unicamp.br 4 / 10

7 Main goals of our research 1 Improve the framework developed for image segmentation (i.e., enhance the user interface and combine DIFT [3] with live-wire-on-the-fly [4]). 2 Use IFT-based pattern classifiers and other operators to deal with alpha matting and inpainting 3 Extend the segmentation framework for video [5]. Thiago Vallin Spina (M.Sc. student) (LIV-IC-UNICAMP) IFT-Based Editing of Natural Images tvspina@liv.ic.unicamp.br 4 / 10

8 Current works We have recently developed a method which simplifies user interaction through the use of a single set of marked pixels for segmentation [6]. The segmentation process is still divided into recognition, arc-weight estimation, and object extraction, but the method selects automatically the best pixels to learn the object s properties. We further understand user interaction to improve the arc-weight estimation while preserving the user s control. Thiago Vallin Spina (M.Sc. student) (LIV-IC-UNICAMP) IFT-Based Editing of Natural Images tvspina@liv.ic.unicamp.br 5 / 10

9 Marker pixel selection First, the user selects the markers wherever is necessary. Thiago Vallin Spina (M.Sc. student) (LIV-IC-UNICAMP) IFT-Based Editing of Natural Images 6 / 10

Marker pixel selection All the image pixels are then clustered in the feature space according to their image attributes (multiscale smoothing

10 Marker pixel selection All the image pixels are then clustered in the feature space according to their image attributes (multiscale smoothing of the Lab values). Thiago Vallin Spina (M.Sc. student) (LIV-IC-UNICAMP) IFT-Based Editing of Natural Images 6 / 10

11 Marker pixel selection The clusters are used to select dissimilar marked pixels for arc-weight estimation. Thiago Vallin Spina (M.Sc. student) (LIV-IC-UNICAMP) IFT-Based Editing of Natural Images 6 / 10

12 Marker pixel selection The selected pixels are used to learn object properties by fuzzy classification. Thiago Vallin Spina (M.Sc. student) (LIV-IC-UNICAMP) IFT-Based Editing of Natural Images 6 / 10

13 Marker pixel selection Arc-weight estimation combines the image-based and object-based gradients to produce a weight-image. Thiago Vallin Spina (M.Sc. student) (LIV-IC-UNICAMP) IFT-Based Editing of Natural Images tvspina@liv.ic.unicamp.br 6 / 10

14 Marker pixel selection All marked pixels are used for object extraction through the Differential IFT [3]. The user may add/remove markers to improve the result. Thiago Vallin Spina (M.Sc. student) (LIV-IC-UNICAMP) IFT-Based Editing of Natural Images 6 / 10

15 Understanding user interaction An overview of how the user interacts with our framework and how it tries to understand the user s input. Arc-weight Estimation User Interaction Marker Selection Object-based Weight Features-based Weight Yes No Do Relearning? Object Extraction User Interaction Segmented Object Thiago Vallin Spina (M.Sc. student) (LIV-IC-UNICAMP) IFT-Based Editing of Natural Images tvspina@liv.ic.unicamp.br 7 / 10

16 Understanding user interaction An example of the arc-weight reestimation. Thiago Vallin Spina (M.Sc. student) (LIV-IC-UNICAMP) IFT-Based Editing of Natural Images 8 / 10

17 Some overall results Thiago Vallin Spina (M.Sc. student) (LIV-IC-UNICAMP) IFT-Based Editing of Natural Images 9 / 10

18 Acknowledgements Thank you all for your attention! I would also like to thank FAPESP (M.Sc. scholarship) and IC-UNICAMP for the support, and the organizers of SP-ASC (FAPESP, IME-USP, and IC-UNICAMP) for the opportunity. Thiago Vallin Spina (M.Sc. student) (LIV-IC-UNICAMP) IFT-Based Editing of Natural Images tvspina@liv.ic.unicamp.br 10 / 10

19 [1] T.V. Spina, J.A. Montoya-Zegarra, A.X. Falcão, and P.A.V. Miranda. Fast interactive segmentation of natural images using the image foresting transform. In Proc. of the 16th Intl. Conf. on Digital Signal Processing, Santorini, Greece, IEEE. [2] T.V. Spina, J.A. Montoya-Zegarra, P.A.V. Miranda, and A.X. Falcão. Improving user control with minimum involvement in user-guided segmentation by image foresting transform. In Proc. of the 13th Intl. Conf. on Computer Analysis of Images and Patterns (CAIP), volume LNCS 5702, pages , Münster (North Rhine-Westphalia), Germany, [3] A. X. Falcão and F. P. G. Bergo. Interactive volume segmentation with differential image foresting transforms. IEEE Trans. on Medical Imaging, 23(9): , [4] A.X. Falcão, J.K. Udupa, and F.K. Miyazawa. An ultra-fast user-steered image segmentation paradigm: Live-wire-on-the-fly. IEEE Trans. on Medical Imaging, 19(1):55 62, Jan Thiago Vallin Spina (M.Sc. student) (LIV-IC-UNICAMP) IFT-Based Editing of Natural Images 10 / 10

20 [5] R. Minetto, J.P. Papa, T.V. Spina, A.X. Falcão, N.J. Leite, and J. Stolfi. Fast and robust object tracking using image foresting transform. In Proc. of the 16th Intl. Conf. on Systems, Signals, and Image Processing, Chalkida, Greece, IEEE. [6] T. V. Spina and A. X. Falcão. Intelligent understanding of user input applied to arc-weight estimation for graph-based foreground segmentation. In SIBGRAPI 2010, Gramado, Brazil, Aug To appear. Thiago Vallin Spina (M.Sc. student) (LIV-IC-UNICAMP) IFT-Based Editing of Natural Images 10 / 10

Image binarization techniques for degraded document images: A review

Image binarization techniques for degraded document images: A review Binarization techniques 1 Amoli Panchal, 2 Chintan Panchal, 3 Bhargav Shah 1 Student, 2 Assistant Professor, 3 Assistant Professor 1