Short Course on Computational Illumination

Transcription

1 Short Course on Computational Illumination University of Tampere August 9/10, 2012 Matthew Turk Computer Science Department and Media Arts and Technology Program University of California, Santa Barbara

2 Day 1 Introduction My background and interests Why computational illumination?

3 My background pres. BS, Electrical Engineering Virginia Tech (Blacksburg, VA) MS, Electrical and Computer Engineering Carnegie Mellon University (Pittsburgh, PA) Signal processing, robotics Martin Marietta Denver Aerospace (Denver, CO) Mobile robots DARPA Autonomous Land Vehicle program PhD, Media Arts and Science (MIT Media Lab) Massachusetts Institute of Technology (Cambridge, MA) Automatic face recognition Postdoc, LIFIA/IMAG (Grenoble, France) Vision-based interaction Teleos Research (Palo Alto, CA) Vision-based interaction Microsoft Research (Redmond, WA) Perceptual user interfaces, gesture recognition, University of California, Santa Barbara

4 4

5 UCSB Four Eyes Lab 4 I s: Imaging, Interaction, and Innovative Interfaces Co-directors: Matthew Turk and Tobias Höllerer Research in computer vision and human-computer interaction Vision based and multimodal interfaces Augmented reality and virtual environments Human-computer interaction Wearable and mobile computing Novel 3D displays and interaction Activity recognition and surveillance.

6 Four Eyes Lab: General motivation Provide better, more compelling HCI technologies in a variety of important computing environments Desktop, immersive, VR Mobile, AR, ubiquitous and for a variety of application areas Science, information visualization, collaboration, entertainment Investigate fundamental issues in developing robust, realtime, working technologies for interactive systems Computer vision Computer graphics Human-computer interaction

7 Currently: 2 professors 5 visitors and postdocs PhD students 4-5 MS students 3-4 undergraduates

8 Four Eyes Lab Primary recent conference venues ICCV (Computer Vision) IEEE CVPR (Computer Vision and Pattern Recognition) IEEE ICCP (Computational Photography) IEEE ICIP (Image Processing) IEEE ICPR (Pattern Recognition) ACM SIGGRAPH (Computer Graphics) ISMAR (Mixed and Augmented Reality) ACM MobileCHI (Mobile HCI) ACM RecSys (Recommendation Systems) ACM IE (Intelligent Environments) IEEE VR (Virtual Reality)

9 Perceptual Interfaces Highly interactive, multimodal interfaces modeled after natural human-to-human interaction Goal: For people to be able to interact with computers in a similar fashion to how they interact with each other and with the physical world Not just passive Multiple modalities, not just mouse, keyboard, monitor

10 Early multimodal example Put That There (Bolt 1980)

11 How can achieve the goals of PUI? To develop powerful, adaptive, compelling multimodal interfaces that reach well beyond the GUI, researchers need to develop and integrate various relevant sensing, display, and interaction technologies, such as: Speech recognition Speech synthesis Natural language processing Vision (recognition and tracking) Graphics, animation, visualization Haptic I/O Affective computing Tangible interfaces Sound recognition Sound generation User modeling Conversational interfaces

12 Vision Based Interfaces (VBI) Visual cues are important in communication! Useful visual cues Presence Location Identity (and age, sex, nationality, etc.) Facial expression Body language Attention (gaze direction) Gestures for control and communication Lip movement Activity VBI using computer vision to perceive these cues

13 Elements of VBI Hand tracking Hand gestures Arm gestures Head tracking Gaze tracking Lip reading Face recognition Facial expression Body tracking Activity analysis

14 Some VBI application areas Accessibility, hands-free computing Gaming devices Control (e.g., living room, airport runway) Social interfaces Teleconferencing Improved speech recognition (speechreading) User-aware applications Intelligent environments Biometrics Interactive art Movement analysis (medicine, sports)

15 Microsoft, 1998

16 Microsoft, 1998

17 And years later Microsoft Kinect 2010

18 Background: The mobile computing revolution Me, Microsoft (1994): There will be a camera with every (desktop) computer. What opportunities! For when you re in your office, and maybe at home If you have a computer and Internet access Instead, there s now a computer with every camera (and phone). What opportunities! For practically anyone, anywhere In the past decade, there has been a rapid proliferation of mobile devices around the world The post-pc era

19 The mobile computing revolution More significant than the PC revolution! Convergence of functionality, mobility, and connectedness Key product milestones Nokia: Got the world online BlackBerry: Got the business world hooked iphone: Raised the bar for an integrated mobile computing platform delivering a consumer-friendly user experience Android: Opened the market for intense competition and innovation

20 The bottom line I have become a convert! In the mobile computing revolution the future of computing To the idea that computer vision + mobile computing has many interesting opportunities Mobile computing is changing the field of computer vision both research and practice Opportunities and challenges Next section.

21 So why computational illumination? Great opportunities in the convergence of mobile computing and digital imaging Rapidly improving technologies Cost, size, density, power(?) Increasing maturity of methods and algorithms Interest in computational photography Largely consumer-driven Can leverage best of human + computation Interactive, human-in-the-loop solutions possible Opportunities in exploring illumination Has not changed much over the years yet!

22 Outline for Thursday and Friday Introduction Overview of computer vision and imaging Overview of computational illumination Research in computational photography and computational illumination: Generalized autofocus Frankencamera Composition context photography Multiflash imaging and applications Miscellaneous and brainstorming Wrap-up