CS415 Human Computer Interaction

Transcription

1 CS415 Human Computer Interaction Lecture 1 Introduction Part-2 August 29, 2017 Sam Siewert

2 Questions on Ex #1? Posted on Canvas - Ex #1 Turn in on Canvas Next Week Friday Focus is on 1D CLI - Not as Easy as it Sounds! Start with Mini-Shell Example code (written as graduate student for undergraduate OS classes) Make a friendly shell - add some features! Why Did Microsoft Release powershell in 2006 and Open Source it in 2016? Sam Siewert 2

3 NUC/Jetson Linux - Go To Lab, Verify Login Option #1 Use King 122 NUC / Jetson Lab Learn Host and Embedded Linux Getting Started Jetson - Used for Self-Driving Cars NUC - Way Cool! ssh -X ubuntu@ [from Host 121] Option #2 Virtual-Box Linux with Ubuntu LTS Use Windows or Macintosh PC Learn Linux, it s Easy Ryan Sutton has SE VM Image you can Use!! Sam Siewert 3

4 Questions on Lab Resources? B NUC1 [ ] => B72-TK1-112 [ ] B NUC9 [ ] => B72-TK1-192 [ ] #5 & #6 Left Side - #1 to #4 Left Side - #7 to #9 Sam Siewert 4

5 Downloading and Copying Files Getting Files to NUC Option #1 - Bring files to Lab on USB Memory Stick, Plug into NUC (or Jetson) Option #2 - Download files from NUC Web browser (right click, Save-Link-as) Option #3 - Put files on a Shared P: Windows drive and access from NUC Getting Files from NUC to Jetson (Testing) Use Connect to Server from Places and use SSH to Jetson IP, using ubuntu, ubuntu Drag and drop files from NUC folder to Jetson folder Sam Siewert 5

6 Now Drag and Drop Files on NUC Desktop Transfer files to/from NUC Take code with you on memory stick or copy to P: drive before you leave (backed up) Remove test files from Jetson Compile code native on Jetson (like PRClab) Possible to crosscompile, cross-debug if you want with NVIDIA tools Sam Siewert 6

7 The Basic Elements Human Models of Perception and Cognition Brain Models [Simulations] Brain Scans [fmri, EEG, fnirs] Brain Processing of Sensory Information [Perception] Computer Theory of Computation and Devices What Can Be Computed? What Can t? How Fast? Traditional ALU, Stored Program Computer: Von Neumann Architecture, Von Neumann Bottleneck Turing Machines [Wikipedia], Church-Turing Thesis [Wikipedia], Turing Test Hypercomputation (Myth, Reality?) Quantum Computing, D-Wave, ANN, Analog Real Computing, Neuromorphic, Deep Learning (GPU) Interaction Physical Devices and Interfaces Physics, Time and Space, Chemistry Acoustics, Haptics, Optics, Olfaction, Gustation 5 Senses Hear, Feel, See, Smell, Taste Fusion of Senses Proprioception [Body movement] Extended Chronoception [Time], Nocioception [Pain] 20+ Senses: Internal [interoceptive], External [exteroceptive] Superhuman Sensing [E.g. Thermal imaging, Magnetic field] Sam Siewert 7

8 Why is Human Vision > Computer? Cortex=10 Billion Neurons (High fan-out) Approximately 100+ Mega-Pixel (Rod/Cone Count) James Cutting & Peter Vishton Perceiving Layout > 1 Trillion Synapses Red Epic Mega-Pixel Total=100 Billion Neurons 1. Neuron > Transistor 2. Better Programming? ROM? 3. More Richly Interconnected 4. Storage + Processing Neuroscience. 2nd edition. Purves D, Augustine GJ, Fitzpatrick D, et al., editors. Sunderland (MA): Sinauer Associates; CPU Local Bus 5 To 10 billion transistors CPU Memory Controller I/O Bus (x16 5Gbps = 8GB/sec) Camera Link Interface Card Human Brain Models Project Status Visualization Sam Siewert 8

9 Biological Vision vs. Machine Vision (Why A Honey Bee is Better than HPC for CV) Humans million Photoreceptors 10 billion Neurons (Cerebral Cortex) Brain with 100 billion Neurons Millisecond Transfer Massively Parallel Analog + Digital Computation Synapse Match is a Challenge 7000 Connections from 10 Billion Neurons 3 Year Olds Have Synapses CPU to Digital Camera/HDD Connects 10 s of millions of pixels to Several Billion transistors Through Sequential Logic and I/O Bus 960K Neurons in flight: Learns locations, complex odors, colors, and shapes; with high efficiency (500 Watt/Kg), 0.218g Brain plasticity for learning, connectedness, concurrency, integrated sensing, power efficiency, and resiliency billion? NVIDIA GK110 28nm, (7.1 billion) Pascal 15 billion Intel MICA 22nm (5 billion) Moore s Law Graphic - Wikipedia Sam Siewert 9

10 3D Perception We Live in an Animated 3D World We still Compute with 2D Desktops (WIMP) Tablets, MS Surface, Notebooks, Laptops, Desktops Mobile SmartPhones, iwatch, Android Gear Enabled by Key Devices (Pointer Mouse, Stylus, Touch-screen, Raster Graphics CRT, LCD, OLED, VR CAVE) 3D Limited to CAD, VR Worlds, Movies, Visualization So Far s 3D Batch Processing Printers and Scanners What About Interactive 3D? Sam Siewert 10

11 Real World 3D Perception Many Visual Cues Provide 3D Perception Open Research (Perceiving Layout) HCI Inventors Want to Exploit What Matters Most Stereopsis and Disparity (Left Eye and Right Eye View) Accommodation (Near and Far Field) Correspondence (Common Features Noted with Each Eye) Sam Siewert Sam Siewert 11

12 Play with RGB Depth Mappers Scene Depth Mapping + RGB Planar Image Scene Segmentation (in Z as well as XY) Skeletonization Hand Tracking and Gesture Recognition Interaction with 3D Virtual Worlds Sam Siewert 12

13 Data Glove vs. RGB Depth Mapper Hand and Finger Tracking Based on Segmentation Depth (Skeletonization) Processing is Somewhat Intense Nothing to Wear Data Glove Rotational Rates, Accelerometer, Finger Flexure Sensors M.S. research, CU Boulder, A Hybrid Sign Language Recognition System, Van Culver, 2004 OpenCV and Monovision Camera with 5DT Data Glove CreativeCam With Intel RealSense 3D SDK Sam Siewert 13

14 Interactive 3D Devices I/O? 3D Input Active Depth Mappers, Passive Stereopsis ASUS Xtion Interactive Depth Mapper PrimeSense Chip [Apple now] Revitalized Many Competing Chips and Devices Now Passive Compute Intensive, Less Robust (Flat Wall Problem) Time-of-Flight, LIDAR (Distance, Outdoor) 3D Output [Left, Right Eye Disparity] 120 Hz, Left / Right Eye Parallax Filtering, Polarization (Left/Right Eye) Active Shutter Glasses Entertainment and Visualization Success, Interactive, Less So Fool the Eyes! Sam Siewert 14

15 3D Alternatives? Augmented Reality (Interactive Mix of Real/Virtual) 1. Project Virtual World onto Real World 2. View Real World Through Goggles with Annotation 3. Project Virtual Images into retina as Light-field Overlay C130 HUD Heads-Up-Display Aviation [Gunsight Generalization] Pranav Mistry, MIT Media Lab, Samsung Project onto Real World We ll Study Later Google Glass VRD Failed, Glassholes MS Hololens [View Real World with Annotation] Magic Leap [VRD] MIT Tech Review Article Oculus Rift VR Headset [Dual OLED, Head Tracking] Wearable AR/VR Doomed? Limited? Unhealthy? Sam Siewert 15

16 Start with Basics - 1D CLI Move to 2D WIMP Then 3D Animation, AR and VR Interaction Theory and Guidelines Cognition and Perception Models Then, Back to the Future! Before We Go Further Sam Siewert 16

17 Linux Skills Introduction Session Part-2 August 24, 2004 Sam Siewert

18 Linux How-To Ubuntu LTS on Virtual Box and Jetson Local How-To Resources Linux Linux-Development-Getting- Started.pdf Linux-Basic-Makefile-by- Example.pdf Linux-Setting-up-VBOXSF- Automount-from-Host.pdf Linux-Programming-Top- Errors.pdf 6R8Q Virtual Box VM Installation of a New OS Like Ubuntu Linux is a Safe Way to Learn on the Platform you Know Best Sam Siewert 18

19 Server/Desktop, VM, Embedded A Key Advantage of Ubuntu is that One Linux Distribution Can be Used for Server/Desktop, VM and Embedded Use Server: PRCLab (prclab.pr.erau.edu) VM: Virtual Box with Ubuntu LTS Embedded: Jetson TK1 Developer System Linux Kernel supports the NDK Layer in the AOS (Android Operating System) x.html k/ndk/index.html Sam Siewert 19

20 A few Comments on Options Native Linux (Jetson) Best Performance for Interactive Applications VM Linux Nice Option for Convenient Development and Test Server Linux Safe Haven that is Highly Available, Backed-Up Moving Code Can be Done with SFTP or SCP SFTP Requires ZIP or Archive to Move Whole Directories SCP Moves Structure Automatically Sam Siewert 20