How we Got Here and Where to Take it. Scott Hudson HCI Institute Carnegie Mellon

Size: px

Start display at page:

Download "How we Got Here and Where to Take it. Scott Hudson HCI Institute Carnegie Mellon"

Jerome Buddy Stephens
5 years ago
Views:

1 How we Got Here and Where to Take it Scott Hudson HCI Institute Carnegie Mellon

2 Instagram: Two guys, two years 30 million users (now 300 million) What made that possible?

3 Enablers Making it easy to make things easy Advances in UI Software Starting with tools

4 Starting in the early and mid-80 s, technical HCI community had a very clear goal: Make it easy enough to create these new graphical user interface thingees to allow people with minimal programming skills to do it (quickly & well) At the time it took days of work by programmers with fairly rare skills to build even simple interfaces

people with minimal programming skills to do it (quickly & well) At the time it

5 Starting in the early and mid-80 s, technical HCI community had a very clear goal: Make it easy enough to create these new graphical user interface thingees to allow people with minimal programming skills to do it (quickly & well) At the time it took days of work by programmers with fairly rare skills to build even simple interfaces

6 A Great Concept: User Interface Management System (UIMS) Analogy to DBMS Major line of work: Layered modular abstractions like other big systems like e.g., compilers

7 A Great Concept: User Interface Management System (UIMS) Analogy to DBMS Major line of work: Layered modular abstractions like other big systems like e.g., compilers for (i=0; i<9; i++) { if (i== 42) { Lexer Parser Optimizer (optional) AST Once you work out the task abstractions can refine each layer, etc Code Generator

8 Lots of Systems Were Produced With This Mindset TIGER [Kasik 82]

9 Lots of Systems Were Produced With This Mindset TIGER [Kasik 82] Syngraph [Olsen 83]

10 Lots of Systems Were Produced With This Mindset TIGER [Kasik 82] Syngraph [Olsen 83] IPDA [Olsen 84]

11 Lots of Systems Were Produced With This Mindset TIGER [Kasik 82] Syngraph [Olsen 83] IPDA [Olsen 84] ADM [Schulert 85]

12 Lots of Systems Were Produced With This Mindset TIGER [Kasik 82] Syngraph [Olsen 83] IPDA [Olsen 84] ADM [Schulert 85] GWUIMS [Sibert 86]

13 Lots of Systems Were Produced With This Mindset TIGER [Kasik 82] Syngraph [Olsen 83] AIDE [Hix 86] IPDA [Olsen 84] ADM [Schulert 85] GWUIMS [Sibert 86]

14 Lots of Systems Were Produced With This Mindset TIGER [Kasik 82] Syngraph [Olsen 83] AIDE [Hix 86] IPDA [Olsen 84] ADM [Schulert 85] GWUIMS [Sibert ALGAE 86] [Bergeron 87]

15 Modules and Layers Boil Down to Something Like This User Input Output UI Core Application Interface Appl

16 And this led directly to

17 Very little we see today at least not directly, it took a lot more progress first

18 (Resulting interfaces were typically mediocre) Because we had missed something really fundamental Our CS intuition/methodology told us that if we could understand and decompose the input/processing/output problem of UI we d have it But we missed that this little bit on the left ( the user ) actually changes everything User

19 User Centered Systems Becomes a much harder problem e.g., can t open the user s head and pour in the right mental model To do well you have to structure each system around the user s concepts (AND do all the rest) The user profoundly impacted what should have been purely technical issues

20 In the end the big advance wasn t a particular architecture, algorithm, or system It was really a change in mind set To be user centered even in what seemed like strictly technical issues Led to adopting interdisciplinary methodologies (and a whole new set of issues to be addressed)

21 At the Same Time Lots of other research was going on before, during, and after UIMS work The shift to a user centered mindset helped all of it We now see the fingerprints of the work all over modern interactive systems

22 SmallTalk UIMS Work in the Overall UI Research/Product Flow Industrial Toolkits [Sun, Apple, ] Java Swing Early Academic Toolkits [CMU Andrew, MIT X Windows, ] Adv. Academic Toolkits [Stanford, CMU, UofAZ, GATech, ] Many Current Toolkits Input Models Academic UIMS Work Interactive systems infrastructures today are converging and we can trace many things we see there back through research toolkits that followed UIMS work

23 Examples Three major research toolkit efforts through the 90s Stanford [Linton et al.]: Interviews and Fresco CMU [Myers et al. ]: Garnet and Amulet UofAz/GATech [Hudson et al.]: Artkit and subarctic Some traceable contributions : Concurrency model [subarctic] Swing many Layout abstractions [Interviews] Swing many Layout w/ constraints [Garnet, subarctic, Others*] Adobe, Apple Input dispatch model (heavily modified/adapted) [Garnet/Amulet, Artkit/subArctic] Swing many Resizable rich icons [subarctic] Apple, Android Animation abstractions [Artkit, Amulet] Android,

24 Examples Three major research toolkit efforts through the 90s Stanford [Linton et al.]: Interviews and Fresco CMU [Myers et al. ]: Garnet and Amulet UofAz/GATech [Hudson et al.]: Artkit and subarctic Some traceable contributions : Concurrency model [subarctic] Swing many Layout abstractions [Interviews] Swing many Layout w/ constraints [Garnet, subarctic, Others*] Adobe, Apple Input dispatch model (heavily modified/adapted) [Garnet/Amulet, Artkit/subArctic] Swing many Resizable (9-part) icons [subarctic] Animation abstractions [Artkit, Amulet] Apple, Android Android, *esp. Alan Borning s work at UWash

25 Other Tool Successes: UI Builders Tools to let you draw the (graphical parts of the) interface Trillium (PARC) [Henderson 81, 86] SOS (INRIA) [Hullot 81] NeXT Interface Builder

26 Principle: Visual things should be expressed visually (with minimal code) This turned out to be extremely important in Making it easy to make things easy Nearly every modern IDE now supports this Microsoft produced a whole series of products: Visual Basic, Visual C++, Visual Studio where the Visual part was a UI Builder

27 Principle: Visual things should be expressed visually (with minimal code) This turned out to be extremely important in Making it easy to make things easy Nearly every modern IDE now supports this Microsoft produced a whole series of products: Visual Basic, Visual C++, Visual Studio where the Visual part was a UI Builder

28 Principle: Visual things should be expressed visually (with minimal code) This turned out to be extremely important in Making it easy to make things easy Microsoft Nearly every modern IDE now supports this Microsoft produced a whole series of products: Visual Basic, Visual C++, Visual Studio where the Visual part was a UI Builder

29 Principle: Visual things should be expressed visually (with minimal code) This turned out to be extremely important in Making it easy to make things easy Nearly every modern IDE now supports this Microsoft Apple Microsoft produced a whole series of products: Visual Basic, Visual C++, Visual Studio where the Visual part was a UI Builder

30 Principle: Visual things should be expressed visually (with minimal code) This turned out to be extremely important in Making it easy to make things easy Microsoft Apple Nearly every modern IDE now supports this Microsoft produced a whole series of products: Google Visual Basic, Visual C++, Visual Studio where the Visual part was a UI Builder

31 Principle: Visual things should be expressed visually (with minimal code) This turned out to be extremely important in Making it easy to make things easy Microsoft Apple Nearly every modern IDE now supports this Microsoft produced a whole series of products: Google Visual Basic, Visual C++, Visual Studio where the Visual part was a UI Builder

32 Principle: Visual things should be expressed visually (with minimal code) This turned out to be extremely important in Making it easy to make things easy Microsoft Apple Nearly every modern IDE now supports this Google Microsoft produced a whole series of products: Visual Basic, Visual C++, Visual Studio where the Visual part was a UI Builder

33 More Successes: Interaction Techniques Dozens (if not 100s) of interaction techniques in the literature Pinch gesture (simultaneous scale and translate)

literature Pinch gesture (simultaneous scale and

34 More Successes: Interaction Techniques Dozens (if not 100s) of interaction techniques in the literature Pinch gesture (simultaneous scale and translate) Dates to Kruger s 1983 VideoPlace system

35 Speaking of Scale: Zoomable Interfaces (ZUIs) PA3D [Bederson 94] Pioneered using the dimension of scale for interaction Never run out of space

36 Last Example Touring Machine Project (Columbia) [Feiner et al. 97]

37 Last Example Touring Machine Project (Columbia) [Feiner et al. 97] Mobile computation

38 Last Example Touring Machine Project (Columbia) [Feiner et al. 97] Mobile computation Wireless communications

39 Last Example Touring Machine Project (Columbia) [Feiner et al. 97] Mobile computation Wireless communications GPS

40 Last Example Touring Machine Project (Columbia) [Feiner et al. 97] Mobile computation Wireless communications GPS Handheld Display & Input (pen rather than touch)

41 Last Example Touring Machine Project (Columbia) [Feiner et al. 97] Mobile computation Wireless communications GPS Handheld Display & Input (pen rather than touch)

42 Last Example Touring Machine Project (Columbia) [Feiner et al. 97] Mobile computation Wireless communications GPS Handheld Display & Input (pen rather than touch) + See Through Display

43 Last Example Touring Machine Project (Columbia) [Feiner et al. 97] Mobile computation Wireless communications GPS Handheld Display & Input (pen rather than touch) + See Through Display

44 Taking a user-centered approach leads to moving from How do I implement it?, What s the algorithm/architecture? to If I had <new tech>, what could I do for the user?

45 A Success Story That Led to Apps (and Many Other Things Interactive) Today we have the tools that were the goal of the early work you can create a highly usable small interface in a few hours (sometimes minutes) rather than a few days (or weeks or months) And the UI part is simple enough that many people we would think of as non-programmers can do it that is a critical game changer that leads to an app for everything

minutes) rather than a few days (or weeks or months) And the UI part is simple enough that many people

46 A Success Story That Led to Apps (and Many Other Things Interactive) Today we have the tools that were the goal of the early work you can create a highly usable small interface in a few hours (sometimes minutes) rather than a few days (or weeks or months) And the UI part is simple enough that many people we would think of as non-programmers can do it that is a critical game changer that leads to an app for everything

47 And There are Many Other Success Stories I Didn t Talk About

48 And There are Many Other Success Wearables Stories I Didn t Talk About Thad will talk about that next

49 And There are Many Other Success Stories I Didn t Talk About Wearables Context Aware Computing Google Now

50 And There are Many Other Success Stories I Didn t Talk About Wearables Context Aware Computing Internet of Things Personal Tracking (Quantified Self)

51 And There are Many Other Success Stories I Didn t Talk About Wearables Context Aware Computing Recognition Based UI Microsoft Kinect

52 And There are Many Other Success Stories I Didn t Talk About Wearables Context Aware Recognition Based UI Visualization

53 Overall the mindset change from a systems view to a user-centered view has been a game changer which has amplified the impact of many things Even more so than individual technologies, ideas that amplify other ideas are what we should be after

54 Where Does This Go Next? Lessons from UI tools point to possibilities for Democratization of Computational Power Can we make what used to take teams of programmers something that e.g. every mom and pop small business can bring to bear on their own specialized problems? We now know a lot about how to make tools usable I think we can do this.

55 Questions and Discussion? Scott Hudson, CMU Contact:

A Brief Survey of HCI Technology. Lecture #3

A Brief Survey of HCI Technology Lecture #3 Agenda Evolution of HCI Technology Computer side Human side Scope of HCI 2 HCI: Historical Perspective Primitive age Charles Babbage s computer Punch card Command