Future Mobile Terminals (A Broad View)

Transcription

1 Future Mobile Terminals (A Broad View) Mark T. Smith KTH: Swedish Royal Institute of Technology School of Information and Communication Technology

2 Module Goals Some trends that directly influence the evolution of mobile terminals. The idea of device aggregation and its impact on terminals. Other key enabling technologies for these future devices. Some of the personal, social, political and other aspects of the deployment and usage of these technologies. The knowledge gained from topics in this module should be deployable across a number of business areas.

3 Communication is extended Simple communication is being extended It is turning into broad socialization Socialization occurs by people sharing things And, there is a lot to share..

4 Expression and media drive the actions Ergonomics - sound - light/video - command - control - notification - status Physical objects - media - regulations - spaces - vehicles - devices - food - clothes - money Social context/objects - presence - community - opinion - choice - customs - fashion - recommendation - emotion Interaction - chat -share - touch - comment - create - pay Social networks make it possible to now share these. These are esp challenging!

5 Consider Google as a trend setter (recall the chart about expression and media) If I want to share these. Presence (in addition to Geo) Community interaction Opinion Choice Recommendation Fashion Emotion Does Google enable me to: Chat (Yes, Google chat) Share (Yes, media: Google books) Touch (not yet.. or can they?) Comment (Yes, Google Buzz, gmail, almost everything) Create (Android and applications) Pay (Yes, PayPal)

6 From the CEO of Spotify s blog: We think music data is social objects, and we focus on building tools around them. We don t necessarily want to be a social network ourselves. That s also a hint on the future. Daniel Ek What kind of a terminal will handle such social objects?

7 Social objects heavily rely on context Recall from before, context is the combination of people, places and things. Again, we are most interested in are the universal technologies. All appropriate applications require these. 1. Tagging: This makes things machine readable. 2. ID management: It makes things identifiable. Who or what it is. 3. Transaction: Enables explicit events, especially payment. 4. Context management: Allows inference to be made based on events. 5. It s the combination of people, places, things and events that make up what is shared. It isn t just the playlist. It s also who is listening to it!

8 Sensors are still centric We know that: Different applications may require different sensor technologies. These sensor technologies give specific meaning or context to objects, interaction, ergonomics and social context. The use of sensors in applications like this is often referred to as making the application context aware. It is very challenging because context itself is unbounded. It also means that the number of sensors we could potentially use is unbounded. New sensors are being invented all the time.

9 While kind of mobile platforms will use all this? Position Person s location Proximity to other objects Orientation In objects In people Sound Communication Entertainment Speech as I/O Environment Biometrics Tactile Communication and notification Locators and pointers Display of sensations Images Capture and display Environment Tracking Object recognition Biometrics Environment Meteorological Electromagnetic and chemical sensing Physiological Biometrics Noninvasive sensors Biochemical sensing Personal state monitoring And, there are a lot more.. How will they all be exploited?

10 Next generations of mobile terminals Example: The FriendSense Emotion System developed at SICS. This system integrates sensors to determine if the user: 1. Gets warm or cold 2. Tightens their grip, once or in different frequencies and with different strength 3. React to something with motion and touch The sensor events are fused to give an indication of emotion. Note the mobile for emotion display and visual effects

11 Swatch WebWatch The WebWatch created a shared space occupied by people. The social object was who was with who. You could be whoever you wanted to be.

12 Swatch, the mobile terminal company? Swatch is owned by the Swatch Group (18 different wristwatch companies). Swatch is one of the brands. They sell fashion watches and jewelry. The average Swatch watch owner has 7 Swatch watches. Never the less, this isn t how Swatch looks at things. Although he was wearing several watches, Nicolas Hyack Sr. rarely mentioned wristwatch as a Swatch product.

13 Simple sensors create rich social objects 1. Communication between users Real time, or time shifted 2. ID management for application personalization 3. Ability to form and keep an enhanced personal identity 4. Ability to form augmented social groups to project into Both physically co-located and internet located 5. Interaction in these groups can be real time or time shifted This device used the access point and other WebWatches as its sensors.

14 Managing the social space Significantly, the space was persistent, and could be augmented. A Web Presence Manager service was designed to create such spaces. User 1 Web Presence xml User n Web Presence xml Web Presence Manager Description Directory Autobiographer Observer Listkeeper Indexer The result was a complex system for social interaction. A way to enhance who are for the groups you want to be in.

15 These are special integrated terminals From a business perspective, this may be problematic. Not practical to put every kind of sensor into every mobile device Not cost effective if a big enough market doesn t exist The design process is not supple enough Can t really make a customized, special product for everyone Or, can you?

16 The idea of device Aggregation Start with what we know. We think of mobile terminals as embedded systems. Cell Phone MP3 Player All the Hardware, Software, including the application, and other pieces that make up the device and define it s functionality have been aggregated to make the system. In this case the system is for communication and for entertainment.

17 You can take the pieces apart In other words, DISaggregate the phone into more fundamental pieces. If you could architect it right, ANY speaker, Microphone and UI could form the basis of a phone. (The network interface is one more piece).

18 Fundamental pieces as everyday devices All of these devices talk on a network as well. User Interface Speakers Microphone These devices can be aggregated into new systems.

19 Similarly, we can aggregate to form new virtual devices Here, a home service forms a virtual telephone. Digital Home Network Digital Interface Audio Output Analog Speaker PSTN or Internet Connection (VoIP) Session Management Device Discovery Virtual Device API Telephony App (ie Skype) Digital Interface Audio Input Analog Microphone Application Server Digital Interface UI Processing Analog Television

20 Advantages of aggregation Flexible: This allows an optimal combination of different components to correct environmental factors that are obstacles. It can correct many different obstacles. Cost effective: As commodity components can be used to build the solutions, economy of scale can be exploited. Dynamic: It can be changed very quickly to accommodate a different situation. For example, different device components can be used based on media type, or location of the user. Broadly useful: We all might want it. Being broadly applicable is an advantage overall. Think of people with special needs.

21 Practical and real aggregation Authentication and secure applications are good examples Future mobile terminals will provide secure authentication A resource + mobile trusted element -> virtual secured device Remember NFC? Here is a broad example

22 Example: Virtual secure resources Customer was a large medical center Provide site security and accountability Automate all data and resource access Verify procedures and process Don t replace any equipment Nothing to forget or lose or have stolen No passwords. No PINS. No codes. Useful for everyone in the medical center Even the patients

23 The trusted element Secure Badge Prototype

24 1. Aggregate a blank badge with a biometric device Employee arrives at workplace, and takes any available S.B. They are not pre-assigned User attached S.B. to clothing. Sensors take over. An aggregate is formed with a biometric device and a data base The biometric is cached onto the S.B. and data about the work day is also stored. The caregiver starts their day. Note that nothing is pre-assigned! The user is not burdened with passwords or PINS.

25 2. Create aggregates with existing devices New, secure devices are formed Equipment, tools and instruments all aggregate with the S.B. Also rooms, door and cabinets The S.B. provides the sensors, actuators and other resources to form new devices Access can be based on anything: Who, where, what, when, who with. It can also change dynamically.

26 3. Aggregates can also be formed to data Caregivers and patient data S.B. forms an aggregate with the room. Caregiver is given access based on context. When physician is facing the data terminal, correct data is shown. When physician faces away or leaves, data is removed. Other conditions can be used.

27 4. Verification and accountability Patient reception 9:05 W/ Dr. Adams Lab Pharmacy 11:45 Patients know who saw and changed their records Every access to a secure resource is logged. Every caregiver who sees a patient is logged. Patients progress through the medical center is verified. Patient Jones leaves 12:00

28 Also an example of Dynamic Persona The front is just a plastic display A Secure Badge only belongs to the person currently wearing it. Anyone finding a lost S.B. simply finds a S.B. that is now their own. At the end of the day, the caregiver just takes the S.B. off. It s now ready for the next user. Any device that can be personalized can have dynamic persona. Especially virtual, aggregated devices

29 The Human Element These new terminals have a strong human element in their design It s new, and different from traditional Human Computer Interaction Technical artifacts, such as sensors and data now are social objects Human behavior is not defined by engineering. It is tacit knowledge. Technology that tries to combine codified and tacit knowledge is very hard to design and get right To be successful, future terminals, applications and services must include elements of classic design.

30 The idea of Social Weight (See Toney et al. Minimal Social Weight User Interactions for Wearable Computers in Business Suits ). It is an early attempt to quantify some kinds tacit knowledge so as to make it useful as an engineering metric. Social Weight = CL + PP + TA CL is cognitive load. How much you need to focus on the device PP is physical presence. How conspicuous or big it is. TA is technology apprehension. How technically intimidating it is. Note that TA can be negative! Think of the I-Phone.

31 Social Weight in other words Social Weight is a concise way to restate these ideas about design: Norman D., The Invisible Computer Restated: Try to get Technology Apprehension = 0 Norman D., The Design of Everyday Things Restated: Try to get Cognitive Load = 0

32 Example of device aggregation to create protection services for elderly individuals New housing for senior citizens that use home appliances in apartments to look at living patterns, and tries to determine the probability of normalcy. The building aggregates whatever happens to be in it. For example, light switches, TVs, kitchen devices, switches under carpets and in chairs, bed, etc. If the conclusion is that something is abnormal, the correct civil services are contacted. No cameras, no microphones. The person is not looked at. I think the Social Weight = 0 or very near it.

33 Smart Garments are another example Smart Garments combine IT with clothing We want to look at them because they may be a new class of mobile terminals They are also great subjects to think of with respect to S.W. They are social objects. Clothes have always been. They represent a combination of classic design and engineering Their roots are in wearable computing.

34 Newer directions have much lower S.W.

35 esuit: Lower Social Weight Business suit as a mobile terminal The idea was to match the Social Weight with more predicable behavior. Business etiquette is often thought to be learnable, and so not tacit knowledge. This makes the design specifiable. Its just engineering. On the surface, this device seems to have low social weight. See Toney et al., Minimal Social Weight User Interactions for Wearable Computers in Business Suits, ISWC 2002

36 Social Weight factors PP is relatively low as seen in figs 3 and 4. That watch is big. TA for an observer is also relatively low, as fig 3 seems very normal. On the other hand, LEDs in the cuff makes for a higher TA CL is definitely non-zero. The more displays, the worse it gets. There are buttons in the coat hem which further makes CL non-zero.

37 Reducing CL with alternative displays Active shoulder pads have a lower CL than displays. Seventh International Symposium on Wearable Computers, October 21-23, 2003 A Shoulder Pad Insert Vibrotactile Display Aaron Toney, Lucy Dunne, Bruce H. Thomas, Susan P. Ashdown

38 Discussion and exercise The idea of social weight can apply to most ICT devices and applications. Discuss what ICT products in common use you think have the highest and lowest social weight. What would you do to reduce the social weight of the highest level devices on your list? For example, people texting (SMS) while driving probably has high social weight. How would you reduce it.

39 Devices that change shape Who says a mobile communicator needs to look like a mobile phone? These are prototype devices that can change their shape in real time. Devices that communicate by twisting, shrinking or expanding. The technology is related to artificial muscles. Pettersson, Oscar, LEGA VERSION 2, MS Thesis Mälardalens Höogskola, 2010

40 Mobile terminals for community interaction Example of terminals for community based health management

41 Ideas on the design process 1. Technology and product breadth increases Design products are often not discrete entities in an ICT way. For example, how does a piece of jewelry function? Engineers need to consider how design will broaden the context or purpose in which the product is used or worn 2. User s interpretation of ICT and design From the user s viewpoint, ICT is not important Social objects allow a user to project their persona into both a physical and a network space. Designers and engineers need to work to determine what the device really means to the user, and so how it is valued.

42 Ideas on the design process 3. Understand market expectations of ICT and design Helps clarify advantages, risks and comfort zones Ie, Swatch wasn t sure about tech push models A market pull model brought in the Web Presence Manager 4. Customers and Demographics Can be hugely different between design and ICT companies Each group needs to broaden their own model of customer demographics Always challenge existing marketing models

43 Ideas on the design process 5. Extend ICT personalization to humans (like fashion does) Engineers personalize ICT for ease of use, & aesthetics Social objects are used to personalize the human Need methods to apply the ideas of personalization to real humans 6. Extensions to existing product realization methods. The expression of tacit social knowledge as a design metric Effective design/engineering co-invention environments

44 Summary and moving forward What s new are the devices and how they incorporate the ability to create and use social objects. The ways they do this is very diverse. The manifestations of them are equally diverse. They will need new methods deployed to successfully design them. These methods combine elements of classic design with ICT. They form the foundations of future communication.