Well..How Did I Get Here? Steve LaValle University of Illinois February 25, 2015 U of I Research Park - Startup Cafe - Feb 2015 1 / 32
Oculus VR: Quick Overview Timeline: 07/12 Oculus VR founded by Palmer Luckey, 19 08/12 Kickstarter, $2.4 million 09/12 I joined as consultant for software: math, head tracking 03/13 Development Kit shipped; I joined as full-time head scientist 06/13 Series A, valuation approx. $16 million 12/13 Series B, valuation approx. $275 million 03/14 Facebook acquisition, approx. $2 billion 07/14 Development Kit 2 shipping; I returned to UIUC U of I Research Park - Startup Cafe - Feb 2015 2 / 32
Well...How Did I Get Here? And you may ask yourself Well...How did I get here? - David Byrne, Talking Heads, 1980 U of I Research Park - Startup Cafe - Feb 2015 3 / 32
Me In The Early 1980s In 1982, I was living in video arcades and learning how to program BASIC and assembly language U of I Research Park - Startup Cafe - Feb 2015 4 / 32
Career as a Robotics Professor 1987-1989: BS Cpr. Engineering, U. of Illinois 1990-1995: EE MS/PhD, Beckman Institute, U. of Illinois 1995-1997: Postdoc, Lecturer, Stanford 1997-2001: CS Professor, Iowa State 2001-2015: CS Professor, U. Illinois My lab in Siebel Center: U of I Research Park - Startup Cafe - Feb 2015 5 / 32
Education at UIUC BS in Computer Engineering; MS, PhD in Electrical Engineering PhD Adviser: Seth Hutchinson Assembly, microcode Digital logic design and lab, processor design Semiconductor device theory Electromagnetics Signals, systems, and control Algorithms Also, Head TA for Digital Systems Lab (under Ricardo Uribe) Decades later, at Oculus: Useful for quickly understanding MEMS sensors, digital hardware issues, calibration, error modeling. Easy to talk with real engineers. U of I Research Park - Startup Cafe - Feb 2015 6 / 32
Research: Motion Planning My long-time favorite: The Piano Movers Problem (Similar to making autonomous video game characters) U of I Research Park - Startup Cafe - Feb 2015 7 / 32
Research: Motion Planning Think in the configuration space... C obs C obs C free q G C obs q I Great familiarity with SO(3) and SE(3) geometry. Numerical and sampling issues: No problem! U of I Research Park - Startup Cafe - Feb 2015 8 / 32
Motion Strategy Library Developed a widely used library in 2000: Open source, C++, 25000+ lines of code U of I Research Park - Startup Cafe - Feb 2015 9 / 32
Book on Planning Available since 2004: PART I Introductory Material Chapters 1-2 PART II Motion Planning (Planning in Continuous Spaces) Chapters 3-8 PART III Decision-Theoretic Planning (Planning Under Uncertainty) Chapters 9-12 PART IV Planning Under Differential Constraints Chapters 13-15 Free download ( 1000 pages): http://planning.cs.uiuc.edu/ In 2012, Jack McCauley (Guitar Hero inventor, an Oculus founder) was searching for Euler angles, quaternions,... U of I Research Park - Startup Cafe - Feb 2015 10 / 32
Filtering and Information Spaces My main research efforts from 2003 to 2012. DARPA SToMP (minimalist foundations of sensing and filtering) Combinatorial filters Sensor models: Preimages, lattices Short book in 2012: Working on a longer book on sabbatical in Finland, 2012-2013. U of I Research Park - Startup Cafe - Feb 2015 11 / 32
From Finland to California Sabbatical 2012-2013 in Oulu Writing a book on sensing and filtering... U of I Research Park - Startup Cafe - Feb 2015 12 / 32
From Finland to California Sabbatical 2012-2013 in Oulu Writing a book on sensing and filtering... September 2012: Palmer disrupted my life U of I Research Park - Startup Cafe - Feb 2015 12 / 32
VR Historical Perspective 1968: Ivan Sutherland, Sword of Damocles 1980s: Jaron Lanier, VPL Research, goggles and gloves 1992: University of Illinois, CAVE 1992: Lawnmower Man 1995: Nintendo Virtual Boy Virtual reality became one of the most over-hyped technologies ever. Problem: The component technology was not ready yet. U of I Research Park - Startup Cafe - Feb 2015 13 / 32
A Personal Interest My grandma (right) and her sister in 1993. Separated after WWII: One in US, one in Poland I brought them together in 1993. I wished they could feel present in each others homes. U of I Research Park - Startup Cafe - Feb 2015 14 / 32
Making Oculus Rift DK1 Over 70,000 headsets sold. $300 each. Open-source software. U of I Research Park - Startup Cafe - Feb 2015 15 / 32
No Upper Age Limit 90-year-old grandma in virtual Tuscany U of I Research Park - Startup Cafe - Feb 2015 16 / 32
Architecture and Real Estate Do you wish your home were bigger? U of I Research Park - Startup Cafe - Feb 2015 17 / 32
Movies A private virtual movie theater U of I Research Park - Startup Cafe - Feb 2015 18 / 32
Panoramas Pick your favorite street views and have a look around. U of I Research Park - Startup Cafe - Feb 2015 19 / 32
First-Person Shooter Games Team Fortress 2, Valve Inc. U of I Research Park - Startup Cafe - Feb 2015 20 / 32
VR Game Jam What could you do with an elephant trunk? U of I Research Park - Startup Cafe - Feb 2015 21 / 32
Thrill Seekers Virtual amusement park rides! U of I Research Park - Startup Cafe - Feb 2015 22 / 32
Brilliantly Disturbing Ever wonder how Louis XVI must have felt? U of I Research Park - Startup Cafe - Feb 2015 23 / 32
Socializing in Virtual Spaces Open Sim, Second Life,... U of I Research Park - Startup Cafe - Feb 2015 24 / 32
Head Tracking Integrate and fuse sensor data to determine orientation Avoid raw, pitch, roll angle representation U of I Research Park - Startup Cafe - Feb 2015 25 / 32
Perceptual Psychology / Vision Science Important factors: Latency Tracking jitter Optical distortion Brightness Focus IPD Tearing Screen door Frame rate Textures Virtual egomotion Head translation Adverse effects: disbelief, fatigue, simulator sickness Complications: Adaptation and variance across people U of I Research Park - Startup Cafe - Feb 2015 26 / 32
Oculus Development Kit 2 Low-cost, stationary camera 40 IR LEDs around the headset surface Works between 0.5m and 1.5m Define error function based on LED projection. Use IMU data for prediction. Iterative gradient descent onse(3). Main tracking people: Anna Yershova, Max Katsev, Dov Katz, myself U of I Research Park - Startup Cafe - Feb 2015 27 / 32
New Hardware Demands VR pushes the components to the next level: Faster switching pixels in displays Higher resolution screens Higher frame rates Improved sensors New optical systems Faster rendering pipeline Big challenges for industry. U of I Research Park - Startup Cafe - Feb 2015 28 / 32
New Software Issues Building a new platform that includes: User interfaces, including controllers High-performance, perceptually satisfying graphics Tracking: heads, eyes, full bodies Model building / scene acquisition Middleware to hide complexity from developers Exciting challenges for our industry and universities. Collaborations across disciplines: psychology, signal processing, dynamical systems and control, algorithms, humanities U of I Research Park - Startup Cafe - Feb 2015 29 / 32
Future Research and Development Sensory stimulation beyond vision: Audio Touch / haptic Olfactory / taste Vestibular Each has a low-level perceptual psychology component Higher levels of human interaction: Interfaces / HCI Communications Education Sociology, foreign language, history, art Human-centered research all around! U of I Research Park - Startup Cafe - Feb 2015 30 / 32
A Growing Ecosystem Microsoft Hololens Sony Morpheus Google Cardboard Samsung Gear VR U of I Research Park - Startup Cafe - Feb 2015 31 / 32
VR + Robots =???? U of I Research Park - Startup Cafe - Feb 2015 32 / 32