Intelligent Sensors for Cyber Physical Human Systems 人物联网系统的智能传感器

Transcription

1 Intelligent Sensors for Cyber Physical Human Systems 人物联网系统的智能传感器 Wen Jung LI 李文榮 Dept. of Mechanical and Biomedical Engg. City University of Hong Kong, Hong Kong SAR

2

3 Cyber Physical Systems Important characteristics of CPS: Image Credit: Nicolle Rager Fuller, NSF Direct connection between the physical and digital world New System functions through information, data, and function integration Sensors and Actuator Networks Deployment under difficult physical boundary conditions Automation, Adoptivity, and Autonomy 31/08/2016 3

4 Cyber Physical Systems Example: Google car 31/08/2016 4

5 Cyber Physical Cities Intelligent and Green City in Longgang ( 龙岗 ) 以联泰智居中心为代表的 N 模式起到了很好的带动和示范意义

6 The Industry 4.0 Ecosystem Future Factories: IT systems will be built around machines, storage systems and supplies that are linked up as cyber-physical systems (CPS). 31/08/

7 Cyber Physical Farms Connected Cattle: Wearables are changing the dairy industry 31/08/2016 7

8 Cyber Physical Farms Examples of Cattle Disease: Rumen Acidosis (liver disease): Changes in body temperature, ph level, reduced feed take, pulse rate, etc. Bovine Babesiosis (tick-born disease): Fever, incoordination, teeth grinding and mania, involuntary movements of the legs, etc

9 Cyber Physical Human Systems Methodology: Modeling, Optimization, Algorithms, Security, Decision, etc. Key Technologies Sensors, Actuators, Robotics Power Source Wireless Networks Control/Modeling Algorithms/Big Data Analysis Data Security HUMANS Systems: Manufacturing, Health Care, Power Grids, etc. Figure modified from 31/08/2016 9

10 Micro-Force Sensing and Feedback in Cross-Continent Microassembly Mirror plane Supporting frame Lockers MEMS structures are fragile and easily breakable Assembly operation need micro force information Micro force sensing and feedback is challenging but necessary

11 Human Controlling Machines Housing cover Sensor RF transmitter Battery cover X Y X Y PCB MCU Buttons Z i Z i Battery holder PCB housing Overview

12 Centre for Micro and Nano Systems, The Chinese University of Hong Kong A Ubiquitous Digital Writing System Using MEMS Motion Sensing Technology 3D Digital Pen Z e Roll ω X φ M X M Y M Z O A X Pitch ω Y θ ψ Yaw ω Z A Y A Z Y e X b X e Y b Z b

13 E-Education Devices 3D Digital Pen Sensing of Pitch, Roll, Yaw, and accelerations of 3D axes X e Z e O X b A Y A X Y b Z b Roll ω X Pitch ω Y φ θ A Z M X ψ M Y M Z Yaw ω Z Y e Sensing of velocity and position is possible Size: ~2cmx2cmx5cm, with Sensors, battery, MCU, DSP and wireless Tx chips.

14 Micro IMU for Human Motion Tracking Magnetometer (ω X,ω Y,ω Z ) B (A X,A Y,A Z ) B IMU (θ X, θ Y, θ Z ) E Time Update Attitude Kalman Filter Attitude E Transform Matrix Stroke Divider Attitude Measurement Weight Measurement Update (A X,A Y,A Z ) E Position Kalman Filter Invariant Set (P X,P Y,P Z ) E

15 z position (mm) z position (mm) z position (mm) z position (mm) z position z position (mm) (mm) z position (mm) z position (mm) z position (mm) z position (mm) z z position position (mm) (mm) z position (mm) z position (mm) z position (mm) z z position (mm) z position (mm) z(mm) z position z positio z position z z position z position z positio MEMS+Vision: Experimental Results x position x position (mm) (mm) x position x (mm) x position 200(mm) x position (mm) x position (mm) x position (mm) x position (mm) x position (mm) x position 0 (mm) x position (mm) x position (mm) x position (mm) x position 0 (mm) x position (mm) x position (mm) x position (mm) x position (mm) x position (mm) x position (mm) IMU 200 x position (mm) IMU+Camera Camera Reference x position (mm) x position (mm) x position (mm) x position (mm) x(mm) position (mm) x position (mm) x position (mm) (a) 中. (b) 大.

16 A Mobile Human Airbag System based on MEMS Motion Recognition Technology Source from falling.htm Source from / shtml [1] Department of Mechanical and Automation Engineering [2] Department of Orthopaedics and Traumatology Centre for Micro and Nano Systems, Dept of Mechanical and Automation Engineering, The Chinese University of Hong Kong

17 Human Airbag: Saving Lives WSN Falls and fall-induced fractures are very common among the elderly Hip fractures account for most of the deaths and costs In worldwide, there are 4,000,000 hip fracture cases per year the mortality rate is 30.8% in first year. In Hong Kong, there are 4,000 cases per year. This leads to HK$150 million medical and rehabilitation expenditure Femoral Neck Fracture Intertrochanteric Region Fracture

18 Mobile Human Airbag System Sensing and Airbag Deployment in ~0.75sec!! Centre for Micro and Nano Systems, Dept of Mechanical and Automation Engineering, The Chinese University of Hong Kong

19 Motion Classification by Support Vector Machine l * f ( x) ( i i )( ( xi) ( x)) b i 1 l ( i i 1 * i ) K( x i, x) b αi, αi* are multipliers, b is a threshold xi are support vectors 100 times of lateral falling-down one hundred times of other motions, including: 10 times running 20 times walking 20 times sitting down 20 times squat 20 times stepping stairs 10 times jumping SVM filter B: Fall R: Other Centre for Micro and Nano Systems, Dept of Mechanical and Automation Engineering, The Chinese University of Hong Kong

20 Airbags for Hip Protection Human Motion 人体动作 Sensors (μimu) 传感器 ( 惯性测量单元 ) Sensor Signal Filtering 传感器信号滤波 Fall Detection Algorithm 摔倒检测算法 Gas-releasing Mechanism 气体释放机制 Compressed Gas 压缩气体 Airbag(s) Sensing and Airbag Deployment must be done in ~0.75sec!!

21 Gesture Recognition for MMI Page 21

22 Gesture Recognition for MMI A B C D E F G H I J K L M N O P Q R S T U V W X Y Z A B C D E F G H I J K L M N O 1 49 P Q R S T U V W X 50 Y Z Letters (26) Mix up characters: e.g. F and I H and J P and Q O and U Correct rate 40%:

23 Gesture Recognition for MMI Characters recognition: M and N o Character recognition only based on MEMS-sensor reconstruction not feasible o 100% correct rate of these two characters can be obtained by classification

24 Real-time Human Gesture Recognition 24

25 Wearable Cyber Physical Human Devices Wearable Cyber Physical Human Devices to put users into cyber space to improve health care, social interaction, human-tocomputer interaction, etc.

26 Basic Device Specifications Battery/power/memory management Rechargeable/Wireless Charge Energy scavenger/generator converting human motions into electrical energy Human motion sensing/tracking/recognition (6 DOF) Location tracking (GPS) Heart beat/ecg Temp/Humidity Wireless transmission (10m Bluetooth transmission; periodic transmission is acceptable) 3G/4G communication module Volume of each device similar to Pandora charms

27 Wearable Cyber Physical Human Devices Wearable Device Nodes Sensor Nodes Smell Mist Generator Vibration-based Powergenerator Wireless Transmitter Power Source/Management Node Jewellery Chain (Covered with gold, platinum or other noble metals) Data Transceiver Universal Bus Vibration-based Powergenerator Wireless Transmitter Sensor Node Universal Bus Connector

28 Wearable Flexible Motion Sensors with Ultra-Lowpower Input A MEMS motion sensor using single layer graphene sensing element

29 Vibration based Power Generator Vibration Source Peak Acceleration (m/s 2 ) Frequency (Hz) Clothes dryer Door frame just after door closes Small microwave oven HVAC vents in office building External windows (size 2 ft X 3 ft) next to a busy street Page Washing Machine Second story floor of a wood frame office building Inner housing Magnet Spring Outer housing Circuit Coil

30 AA Size Power Cell MEMS Electroplated copper resonator to pickup vibrational energy 23.9g 10.5g Page 30

31 Lead-Free Micro Atomizer (Mist Generator) H b 8 tb g Db H z N z w 2

32 Mice Motion Tracking for Drug Efficacy Studies Motion Tracking of Mice for biomedical research and drug-discovery industries: The US & Europe alone invest >20B USD annually on animal testing; >400M in behavior equipment Current products based primarily on video capture/analysis suffer major limitations: Slow, labor intensive and inefficient since animals are monitored one at a time The data collected are relatively limited, of low quality, inconsistent and unreliable due to small sample size and tremendous stress putting on the animals Thus large-scale and/or long-term studies are impractical with current systems and consequently slow down the drug discovery processes and increase the costs. 31/08/

33 Mice Motion Tracking for Drug Efficacy Studies Cost reduction by >85% Time saving by >95% Reliable large-scale and/or longterm dataset new and/or better medicines Fast output of more reliable, consistent and analyzable data faster to market Enable new approaches in research and drug discovery - new and/or better medicines Worldwide animal researchers in academic research institutes, pharmaceutical and biotech companies, commercial animal suppliers, and contract research organizations. Wireless Distance Measurement (Bluetooth/Zigbee) Wireless Charger 3-D Accelerometers Micro Control Unit Interface 3-D Gyroscopes Inertial Measurement Unit 3-D Magnetometers Flash Memory 31/08/

34 Earthquake: Pearl River Delta Region Recent 6.4 magnitude earthquake in Tainan, Taiwan Seismic epicenters and magnitudes in Guangdong and northern South China Sea Infrastructures, including bridges, railways, tunnels and subways, have been designed to withstand earthquakes of up to Intensity 7.0 in Hong Kong.

35 Bridge Cyber Sensing System V Sensor Node Pa H high-precision, low-noise, low-consumption PCB Intelligent data processing Heterogeneous Network Z Base Station BT/Z Z BT/Z Z Z Z Z

36 Bridge Cyber Sensing System SZAR Wireless Sensor node IN9828 Wired Sensor node SZAR wireless sensor hardware 广州黄埔大桥 Operating range: -40 to 125 Dynamic range: 138db Orientation: triaxial Full-scale range: ± 2G Frequency Response: DC to 300Hz Software

37 Flutter Energy Converter for Wireless Sensors Pressure distribution around Belt at 15 degree (Source: ) 31/08/

38 Structural Health Monitoring for Mines A Sensor Node Pa H high-precision, low-noise, low-consumption PCB Intelligent data processing Sensor Network Z Base Station BT/Z Z BT/Z Z Z Z Z Local Server Remote Server

39 Palpation Robotic Hand Ultimate goal Reducing individual variance of TCM 28 Types of Sphygmopalpations classified into 4 categories: 脈位 : Level of finger pressing, reaction to pressing, etc. 脈次 ( 數 律 ): Rate and regularity, etc. 脈形 : Shape, breath of pulse, magnitude, etc. 脈勢 : Flow smoothness, wall tension, degree of fill, strength, kinetic, etc. Technology improvements required Various degree of repeatability issue Not fully mimicing Chinese Medicine Practitioner (CMP) fingers lack of machine tuning variables (e.g. imposed pressure, finger separation) Not fully extracting what CMP can feel (e.g. vessel elasticity & hardness, blood flow volume)

40 Palpation Sensing Robotic Hand Aortic valve opens Notch due to closing of the aortic valve

41 Pulse Data Recorded from the /Robotic Hand Data at Systolic peak Data at Dicrotic notch (b) The corresponding pulses measured in time domain. Data at Diastolic peak (a) 2D maps and 3D maps of the measured pulses by the three robotic fingers (from Left to right; Cun, Guan and Chi).

42 Summary Cyber Physical Human Systems (CPHS): connections between physical world, cyber world, and humans to create innovative systems and applications. The development of Cyber Physical Systems technology is a key driver for the 4 th Industrial Revolution new paradigms for industrial and manufacturing engineering will be created. MEMS and Nanotechnology are fundamentally important to the development of low power, high-selectivity, ubiquitous, and mobile sensing platforms. CPHS Research and Education: Highly Interdisciplinary robotics, sensors, modeling, virtual reality, wireless communication, system engineering, control, computer science, biomedical engineering, etc. 42

43 Funding Acknowledgement Hong Kong Research Grants Council, Hong Kong Innovation Technology Commission, Hong Kong Hong Kong Research Grants Council, Hong Kong Natural Science Foundation of China, China National 863 Plan, China CAS-Croucher Joint Lab Scheme, Hong Kong 31/08/