Communication and Sensing Using Light

Transcription

1 Communication and Sensing Using Light Xia Zhou Department of Computer Science Dartmouth College dartnets

2 Increasingly Connected World 2

3 Two Key Challenges Emerge Radio spectrum crunch Ever-growing user demands meet limited radio spectrum Interaction with diverse smart devices 3

4 Looking into the Visible Light Spectrum 0 4x x x x x x10 19 Frequency (Hz) Radio Microwaves Infrared Ultraviolet X-rays Gamma Rays 390 nm 700 nm in wavelengths THz in frequency 4

5 Looking into the Visible Light Spectrum 0 4x x x x x x10 19 Frequency (Hz) Radio Microwaves Infrared Ultraviolet X-rays Gamma Rays x ~ 10,000 Key Benefits ~400THz free bandwidth Free of electromagnetic interference Ubiquitous Energy-efficient Secure 5

6 Light as a medium that integrates communication and sensing 6

7 Roadmap Visible Light Communication Visible Light Sensing Light sensors 7

8 Visible Light Communication Encode data into light intensity changes of Light Emitting Diodes (LEDs) Eyes cannot detect fast light switching, but semiconductor-based photodetector can! 8

9 Key Differences From RF #1: RF communication can modulate frequency or phase of the carrier Light uses Intensity modulation and direct detection (IM/DD) Frequency (Hz) 0 4x x x x x x1019 Radio Microwaves Infrared Ultraviolet X-rays Gamma Rays x ~ 10,000 10K wider bandwidth 10K higher data rates 9

10 Key Differences From RF #2: Tight coupling of illumination Cannot affect light illumination (avoid flickering, > 1KHz) 10

11 Discussion: What s your idea to enable light communication? 11

12 VLC Modulation Schemes On-off keying (OOK) Color shift keying (CSK) Frequency-shift keying (FSK) Spatial keying Pulse amplitude modulation (PAM) Pulse width modulation (PWM) Pulse position modulation (PPM) Polarization based modulation OFDM (ACO-OFDM, DCO- OFDM) Your design? J 12

13 Inherent Challenges Blockage Distance Uplink Lights not always on 13

14 Inherent Challenges Blockage Distance Uplink Lights not always on 14

15 RetroVLC LED LCD Shutter Retroreflector Photodiode Solar panel Figure 3: RetroVLC/PassiveVLC design illustration. Retro-VLC: Enabling Battery-free Duplex Visible Light Communication for Mobile and IoT Applications. HotMobile'15. PassiveVLC: Enabling Practical Visible Light Backscatter Communication for Battery-free IoT Applications. MobiCom'17. 15

16 Inherent Challenges Blockage Distance Uplink Lights not always on 16

17 17

18 How about Infrared? Need infrared emitters Eye-safety issues 18

19 0 0 1 darklight Video link: 19

20 DarkLight: Key Idea Encode data into ultra-short light pulses LED Eyes Low temporal resolution Accumulative Photodiodes High temporal resolution 20

21 Challenges t 0100 Ultra-short Light Pulses Off-the-shelf LEDs Low-cost photodiodes Data Encoding and Decoding Extremely low duty cycle Ambient light variation Multiple Transmitters Pulses interfere at the receiver 21

22 #1: Dealing with Ultra-Short Light Pulses Light Intensity Time Light Intensity ns Time (μs) Light Intensity High Gain Slow Low Gain Time (μs) 22

23 Efficient Circuit Design FPGA LED: CREE CXA 2520 ($7) V=36V Photodiode: Honeywell SD 5421 ($6) USRP Gate Driver MOSFET Transimpedance Amplifier Dedicated ADC Can be replaced by 23

24 #2: Data Encoding and Decoding OOK: 1 bit per pulse (~190 bps) symbol Time 24

25 #2: Data Encoding and Decoding OOK: 1 bit per pulse (~190 bps) FSK: multiple pulses encoding 1 bit (~160 bps) 1 0 symbol Time 25

26 Encode Data Efficiently OOK: 1 bit per pulse (~190 bps) FSK: multiple pulses encoding 1 bit Our design: Overlapping Pulse Position Modulation (OPPM) 10 bits/symbol (~160 bps) Start rising at Slot 3... symbol 1024 slots... Time 26

27 Detect Pulses Reliably Light intensity Ambient light Time First-order derivative Time 27

28 #3: Multiple Transmitters LED 1 LED 2 From LED 1 or LED 2? 28

29 Identifying Pulse Sources LED 1 slots LED 2 slots Time LED 1 Pulses LED 2 Pulses 29

30 Open Research Challenges Limit of existing LED luminaries Blue LED + Yellow phosphor Lower modulation bandwidth More efficient lighting RGB LED Higher modulation bandwidth Less efficient lighting 30

31 Open Research Challenges Co-existence of other medium Power consumption of RX design Innovative application scenarios, diverse communication forms (e.g., screen to camera, LED to camera) 31

32 Roadmap Visible Light Communication Visible Light Sensing Light sensors 32

33 Occupancy detection Gesture recognition Activity estimation Indoor localization Skeleton pose estimation 33

34 Indoor Localization RSS-based [LHP+14] Pattern-based [KPH+14] igure 1: Conceptual 0.4m accuracy overview of Pharo 0.1m accuracy dispersor Light Polarization [YWZ+15] VLC Transmitter 3.1 lamp or id1 id2 id3 id4 Video Preview VLC transmitter VLC receiver or Lamp Locations VLC Receiver 34

35 Indoor Localization Exploiting Light inherent feature hencewecalit characteristicfrequency (CF NormalizedIntensity LabTube AisleTube Light s Characteristic Frequency GECFL RSS(100dB/DIV) LabFL AisleFL GECFL Time(µs) Frequency(kHz) Camera-based under fluorescent lights (LiTell, MobiCom 16) Camera-based under LED + fluorescent lights (ilamp, MobiSys 17) Photodiode-based under LED + fluorescent lights (Pulsar, MobiCom 17) 35

36 Skeleton Pose Estimation LiSense StarLight Aili 36

37 Minimalist Sensing: Replacing cameras with low-end, distributed photodiodes 37

38 Video link: 38

39 Video link: 39

40 Shadows! 40

41 Not That Simple Challenge #1: Diluted and complex shadow under multiple light sources 41

42 Not That Simple Challenge #2: Reconstruct a 3D posture from 2D binary low-resolution (18 x 18) shadows? N E 42

43 LiSense Overview Challenge #1: Diminished shadow under multiple lights Separate light rays via light beacons 43

44 LiSense Overview Challenge #2: 2D shadows à 3D posture Seek a posture best fitting shadows cast in multiple directions 44

45 Light Beacon Rationale Light intensity Flash frequency = 1.8 khz LED1 LED2 Light intensity Flash frequency = 2.2 khz Time Time Light intensity Time Photodiode FFT Frequency power LED1 LED2 0k 2k 4k 5k 7k 9k Frequency 20

46 Recovering Shadow Maps Infer a binary shadow map cast by each single LED light N E L 5 L 3 L L 4 2 L 1 L 2 L 3 L 1 N E L 4 L 5 21

47 Shadow-Based Inference Track nine key body joints 22

48 Shadow-Based Inference Search for the skeleton best matching observed shadow maps LEDs Photodiodes 23

49 Shadow-Based Inference Search for the skeleton best matching observed shadow maps LEDs Blocked rays Non-blocked rays Photodiodes 24

50 Shadow-Based Inference 50

51 7 users 169 cm 190 cm 60 kg 80 kg SLR Cameras Ground truth Human labelling using 3 cameras 51

52 Key Results 18 upper-body gestures 10-degree mean angular error Real-time reconstruction at 60Hz 5 lower-body gestures 2 combo gestures 52

53 Skeleton Pose Estimation LiSense StarLight Aili 53

54 Too many sensors... Static user with known orientation Furniture can block light too 54

55 Exploit the large number of lights to reduce sensors! 55

56 Video link: 56

57 Main Challenges Dense LEDs Sparse Photodiodes User Mobility 57

58 StarLight Overview Dense LEDs Sparse Photodiodes User Mobility Time based Light Beacon 58

59 Impact of Dense LEDs PD Readings LED 16 LEDs 144 LEDs Time (ms) 59

60 Impact of Dense LEDs FFT power f 1 f 2 f 3 f 4 f 5 f 6 f 7 f 8 f 9 f 12 f 14 f 15 f 16 f 1 f 10 f 11 f 13 1 LED 16 LEDs f f 64 1 f LEDs LED Flashing Frequency (khz) 60

61 Why Do Dense LEDs Make it Hard? Flashing frequency range is limited ( khz) The more frequencies, the smaller the interval between adjacent rising and fall edges Light intensity Time Interval between rising and fall edges Rising and fall edges can be too close for photodiodes to respond L 61

62 Time-Based Light Beacon Reuse light beacon frequencies over time Combine beacon frequency and beacon time slot to identify an LED LED1 LED2 LED3 20 khz 30 khz 40 khz (base) LED4 Time 62

63 StarLight Overview Dense LEDs Sparse Photodiodes User Mobility Sensor Placement Algorithm 63

64 Reduced to the maximum set coverage problem Monotonic and Submodular Efficient greedy solution with (1 1/e) approximation ratio FoV 64

65 StarLight Overview Dense LEDs Sparse Photodiodes User Mobility Feature Extraction 65

66 Tracking a Mobile User Search for the best-fit skeleton based on the light blockage information Blocked rays Non-blocked rays LEDs PDs 66

67 LEDs Body section Potential Orientations PDs 1 m 67

68 Setup 3.6m x 4.8m office room 1 Linux Server x cm 190 cm Ground truth Kinect 2.0 Kinect 14-degree mean angular error Real-time reconstruction at 40 Hz 68

69 Application: User Interaction Designs 69

70 70

71 Application: Behavior Monitoring 71

72 Reconstructing Hand Poses 72

73 Video link: 73

74 Open Research Challenges Lower deployment overhead, low-power sensing Deployment in a reasonable scale Fusion with other sensing modality Innovative, interdisciplinary applications HCI, robotics, graphics/vision, security/privacy, health 74

75 Reusing VR Screen Light for Gaze Tracking Light-sensing unit Solar cell Micro-controller Energy-harvesting unit FOVE VR headset 75

76 Communication and Sensing Using Light dartnets 76