Tackling the Battery Problem for Continuous Mobile Vision Victor Bahl Robert LeKamWa (MSR/Rice), Bodhi Priyantha, Mathai Philipose, Lin Zhong (MSR/Rice) June 11, 2013 MIT Technology Review Mobile Summit 2013
Human Attention resource poverty hurts no Moore s Law for human attention being mobile consumes greater human attention already scarce resource is further taxed by resource poverty Adam & Eve 2000 AD technology should reduce the demand on human attention clever exploitation of {context awareness, computer vision, machine learning, augmented reality} needed to deliver vastly superior mobile user experience courtesy. M. Satya, CMU
continuous mobile vision reality vs. movies COBOT, CMU (2013) Steve Mann (early 90s) C-3PO (1977) Mission Impossible 4 (2011) irobot (2004) Victor Bahl, MSR
perennial challenges MSR s SenseCam for memory assistance Augmented Reality computation cloudlets connectivity & bandwidth battery white space networks, small cell networks, mm-wave networks Resource constraints prevent today s mobile apps from reaching their full potential Victor Bahl, MSR
Wh/Kg battery trends 250 200 150 100 50 0 Li-Ion Energy Density 91 92 93 94 95 96 97 98 99 00 01 02 03 04 05 Year Lagged behind o Higher voltage batteries (4.35 V vs. 4.2V) 8% improvement o Silicon anode adoption (vs. graphite) 30% improvement Trade-offs o Fast charging = lower capacity o Slow charging = higher capacity CPU performance improvement during same period: 246x A silver bullet seems unlikely Victor Bahl, MSR
so where is the energy going? assuming a typical SmartPhone battery of 1500 mah (~5.5 W) Sensors + Memory + Disk ~ 15 mw Display ~500 mw Single Core Processor CPU + GPU ~150 mw Network Stack (5 min. of usage / hour) ~100 mw battery lifetime ~7.25 hours
power consumption of a typical image sensor Reduce frame rate 5 MP, 5 fps 345 mw 1 MP, 5 fps 250 mw 1 MP, 15 fps 295 mw Reduce resolution 0.3 MP, 5 fps 232 mw 0.3 MP, 15 fps 245 mw 0.3 MP, 30 fps 268 mw low resolution, low frame rate image sensing for vision related tasks can reduce battery life by > 25%
state of art Energy / pixel is inversely proportional to the frame rate & image resolution Profiled 5 image sensors from 2 manufacturers power vs. resolution Video at 30 fps 200 power vs. frame rate Video at 0.1 MP Power (mw) 300 200 100 Power (mw) 150 100 50 0 0 5 10 Npixels x 10 6 0 0 10 20 30 FPS Regardless of image resolution & frame rate, image sensors consume about the same power Victor Bahl, MSR
digging deeper (1 MP, 5 fps) Active Period function of pixel count & clock speed Idle Period function of frame rate
reduce power by reducing pixel readout time one pixel is read out per clock period reduce this Number of Pixels divided by Clock Frequency Victor Bahl, MSR
Power Power reducing pixel count (N) Region-of-Interest (Windowing) Scaled Resolution (Pixel Skipping) Active Active Frame Readout Readout Active Readout Time Time
reduce power by aggressive use of standby Turn off sensor during idle period Idle mode necessary to allow exposure before readout Active Readout Active Readout Active Readout Active Readout Active Readout Active Readout Idle mode Standby mode Best when frame rate and resolution are sufficiently low
reduce power by adjusting clock frequency Adjust clock frequency to minimize power 5 fps Adjust this 1 fps Tradeoff frequency Power frequency At low frame rates, run the clock as slow as possible
summarizing power reduction techniques reduce Tactive & increase Tidle decrease frame rate reduce total pixel readout time (by reducing N) adapt clock frequency Instead of idle-ing put sensor in standby state reduce Pactive (not covered in this talk, see paper)
Frame rate (FPS) Frame rate (FPS) Power (mw) applying these techniques 30 Unoptimized 30 Aggressive Standby & Clock Optimization 350 25 25 300 20 20 250 15 15 200 10 10 150 100 5 5 50 0 1 2 3 4 5 0 1 2 3 4 5 0 Resolution (MP) Resolution (MP)
impact on vision algorithms Image registration Person Detection 480 x 270 Image Registration Success Person Detection Success Actual Power Reduction with software assist Estimated Power Reduction with hardware assist Full Resolution (129600 pixels) 99.9% 94.4% 51% 84% Frame Rate- 3 FPS 95.7% 83.3% 95% 98% 30% Window (63504 pixels) Subsampled by 2 (32400 pixels) 96.5% 77.8% 63% 91% 91.8% 72.2% 71% 94%
MSR s Glimpse project
collaborators & references Robert Bodhi Matthai Lin R. LeKamWa, B. Priyantha, M. Philipose, L. Zhong, P. Bahl, Energy Characterization and Optimization of Image Sensing Towards Continuous Mobile Vision, Proceedings of ACM MobiSys 2013, Taipei, Taiwan, June 26-29, 2013 P. Bahl, M. Philipose, L. Zhong, Cloud-Powered Sight for All: Showing the Cloud What You See, ACM Mobile Cloud Computing & Services Workshop, Lake District, U.K. June 25, 2012
Thanks! 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.