Virtual Reality and simulation (1) -Overview / 3D rotation- Shoichi Hasegawa http://haselab.net/class/vr/
Report Write answers for questions and email to report@haselab.net The number of words for the whole report should not exceed 1500 words. 1. What should we care, when we design human interfaces for virtual reality? 2. What should we care, when we model a virtual world? 3. Explain difference of axis angle and quaternion 4. Explain difference of analytical solution and numerical simulation. 5. Explain difference of physics engine for rigid bodies and numerical integration of equation motion. 6. Explain two methods for motion generation of interactive character.
Principle of Virtual Reality Measure human s inputs and operations Simulate a virtual world Present the same stimulus as the real world to the five senses Interface Computer Senses Present Simulation Motions Input Measure Virtual World =Model of Real World
Recognition of Surroundings (Outer world) Human perceives and recognizes surroundings through the Five senses. Reality exist in individual minds. To give sense of reality, presenting to five senses is a good strategy. Direct electrical stimulation to nerves will also be able to give reality like in the Matrix cinema. Through sensations, one discovers objective descriptions of the world and oneself existing in the world. Visual perception/auditory perception/ Can you prove existence of the physical Tactile perception/olfaction/gustation world? Can you say that Law of physics (ex: ) Tautology in logic Computer (ex: A B ) are proved true in similar way and both absolutely true? If you interested in there issue, refer "Philosophy of science" 5
How to make reality (1) Reality of sensation Interface Computer Senses Present Motions Measure Input Virtual World Simulation An interface design strategy come from five senses. It is enough to make stimulus which can satisfy the senses of human beings. It is not necessary to reproduce the complete physical phenomenon. 6
Sensory Window Light is a physical phenomenon, how about vision? Human beings can only perceive the visible spectrum. It is not necessary to present ultraviolet and infrared rays about vision. 7 From wikipedia
Sensory Window RGB Three primary colors of light From wikipedia Why there are just 3 primary colors
Sensory Window People s Vision Four kinds of visual cells Cones : mostly concentrate in and near the fovea S Blue Cone M Green Cone L Red Cone Rods : density is greater in the peripheral retina than in the central retina Visual Pigments Cells Absorption Spectrum S,M,L : Cones R : Rods Wikipedia より 9 From Wikipedia
Sensory Window Wavelength λ=590nm, Intensity I=1.0, Light It is visible to yellow. (Wavelength λ=600nm Intensity I=0.9) + (Wavelength λ=546nm, Intensity I=0.9) It is visible to yellow. People cannot distinguish between this two kinds of yellow. It is not necessary to reproduce the spectrum of light. 10
Color and wave length Yxy color space Brightness : z axis Brightness is constant in the right figure Color x and y axis In side of the triangle can be presented by three lights with wave lengths correspond to three vertices. Yxy 表色系
Aim to the reality of sensation A variety of equipment is developed Visual/Auditory/Olfactory CAVE Olfactory Display Ken Nakamoto, TITech Sensics pisight HMD 2200x1200 145 x 60 Computer Haptic Somatic SPIDAR at Sato lab Titech Torus Treadmill at Iwata lab, Univ. Tsukuba, Japan
How to make reality (2) Reality of recognition Interface Computer Senses Present Motions Measure Input Virtual World Simulation Reality of the virtual world recognized by the user Shape, color, sound and tactile sense of world Modeling (Form of models / measurements of parameters) Change and Response of the World Simulation of model 13
About Modeling Object Oriented Modeling Represent what kinds of functions, elements, and interactions are. Dynamical system Represent with differential equation Discrete system Represent with Discrete mathematics Probability system Represent model with probabilistic mathematics Representation = Modeling We can not represent the whole real world. Abstraction (extract the essence) is required. Friction Air resistance Is floor flat Heating of spring Static Electricity Material of the weight and floor Lightning Find the essence from a real world where many phenomena continuously exist, and then represent it. Finding the initial value is also a big issue.
Example of modeling Polygonal mesh model for shape Represent shape by tiling triangles. Measurement of the shape of a real object 3D measurement depth data polygonal mesh From home page of 株式会社データ デザイン Artec 3D Scanners
Are perfect modeling and simulation of a world possible? A world to be used by user of a virtual reality system Reproduce of daily life Creation, Control, Communication, Education, Entertainment) Fairly tail world is almost same to the daily world. Visualization clarify Specific models are needed. How about modeling and simulation of a whole town? Physics law working in daily life is known Equation of motion : rigid body, continuum mechanism (soft body, fluid) For chemical reaction, molecular dynamics is known. Number of transistors in CPU tripe or quadruple in every three year. Both speed and amount of memory are expanding.
Are perfect modeling and simulation of a world possible? My idea is impossible Simulation Simulation of one atom requires some bytes of memory. The scale between daily world and quark is 10 16 Modeling Powers of 10 A 1968 American documentary short film written and directed by Ray Eames and her husband, Charles Eames, rereleased in 1977. There is no method to measure initial state (position and velocity) of all atoms. Even if there is, the number of atoms are too much to measure. I believe that we need to extract essence of the world for each application area. 17
Powers of ten 18
Modeling to create virtual world Modeling It is impossible to input every thing in the world to computers. Abstraction (extraction of essence) is needed. What is the essence of the real world for a virtual world. It depends on what users do in the virtual world. From what kind of viewpoint, the user realizes the real world, when they do the work being replicated in virtual world?
What is essence depends on what will be done Depends on purpose examples of purpose: CAD, CAE, Design evaluation Communication, tele presence/conference/operation Education, Training Interactive art, entertainment Examples Evaluation of aurban vista design Essences -Shape of buildings -Color of the sky Ignorable -People in the town Training for action to keep order Essences -People -Cars Ignorable -Color of the sky 20
What is essence depends on what will be done Simulation of illumination http://www2.panasonic.biz/es/lighting/control/lightcontrol/lightc ontrol/lightcontrol.html Digital Michelangelo project http://graphics.stanford.edu/projects/mich/ Assembly simulation
What is essence depends on what will be done Check whether it is possible to assemble http://www.ddd.co.jp/product/smartcollisionsdk/%e3%83%89%e3%82%ad%e3%83%a5%e3%83%a1%e3%83%b3%e3%83%88%e3%83%bb%e3%83%a0%e3%83%bc%e3%83%93%e3%83%bc/ Shape is important and color is not. So, color is used to show collisions.
Time in Virtual World and Simulation Changes of virtual world = Making the time flow Is necessary to make changes according to the user s input Must be Interactive If the user s input can be limited, several models can be prepared in advance and preset changes by switching models. If the input vary a lot, preparation in advance is impossible. Compute changes of models based on low of nature. Real time Simulation Simulation must be computed on fly to match the user s input Cases where high degrees of freedom increase the simulation input Aspen Movie Map No Computer Graphics Walkthrough@Sato lab. 3D Computer Graphics Two handed direct manipulation environment in 1994 No physical simulation Two handed direct manipulation environment in 2006 with physical simulation 23
Aspen Movie Map (1978 80 MIT) 24
Walk through with 3D computer graphics Makoto Sato Lab 2005 25
Two handed direct manipulation environment 1996 26
Two handed direct manipulation environment 2006 27
Role of Simulation Simulation Technology Conventional Method What can be changed freely 3D Computer Graphics 2DAnimation, Video Disc (Show 2D images prepared in advance) Viewpoint The user can move everywhere Physics Simulation Script Program Motion Data Object Movement The user can move any object The simulation realize the changes according to the input variation Increases things able to do in the virtual world 28
Summary Interface Computer Good VR requires both good interface good virtual world Sense Motion Display Measurement Interface Input Virtual World Display devices matching to the human sensations, input devices matching to the human motion. Simulation Virtual World Modeling the virtual world by extracting the essence. What is the essence depends on what you do The perspective of catching the world by the worker during the work to reproduce. Simulations enable variety of input Increases things able to do in the virtual world 29
Hierarchies of reality Reality VR Reality of Senses Resolution, field of view, color range Reality of image Static: Number of polygons, illumination model Movie: Mocap, Animation AI Reality of cinema Action of actors, lines Reality of Cartoon/Manga Reality of Novel
Hierarchies of reality Human mind Human sensors Interface Computer Recognition Display Behavior Input Virtual world Control Reality of Interface Layer of sensors and devices Reality of world Layer of cognition and model 31
1993 My first force feedback experience Large robot arm like UNC s Grope I wrote some part of haptic rendering code. This system uses FES(functional electric stimuli) to present feed back force for each finger. d f=knd n
1994: non contact tracking haptic interface Our haptic interfaces are heavy to drag. For free space: Tracking hand without contact. Displaying object: Stop the tracking. Hand hit to the device. Method to measure distance: Attach an antenna to the seat. Measure the impedance between hand (surface of hunan) and sensor. I think this is new and submit a patent but MIT media lab Fish sensor 1995
1995 jumping locomotion device Locomotion device for all direction. We named this device G.O.D Gravity On Demand
Locomotion device by walking Walking sensation presenting device
Join to Makoto Sato s lab I won lots to join to Sato lab. Sato&Koike Gp. P&I Lab, Tokyo Institute of Technology They research SPIDAR since 1989
Virtual Basket My first SIGGRAPH
Virtual Rubik's cube
Both hand direct manipulation
Virtual Canoe
Virtual Canoe
Hardware 1993 ARMSII Link and Gimbal mechanism 1994 ARMSIII pantograph mechanism and ball screw 1995 G.O.D. Electro magnetic break, rubber cords 1996 ARMSV AC servo motors, linear rail 1997 Virtual Basket SPIDAR with 8 motors 2000 Virtual Rubik s Cube SPDIAR with 24 motors 2004 Both Hand Direct Manipu SPIDAR with 8 motors 2005 Virtual Canoe SPIDAR with 8 motors
Software 1993 ARMSII Sense8 s World Tool Kit 1994 ARMSIII Sense8 s World Tool Kit 1995 G.O.D. Super Scape s VRT 1996 ARMSV Original software based on Rend386 1997 Virtual Basket Direct X with simple physics simulation 2000 Virtual Rubik s Cube Direct3D and simple physical simulation 2004 Both hand haptic interaction Original physics engine based on Volumetric Penalty Method 2005 Virtual Canoe OpenGL and Original Fluid simulation