A deck from a recent Digital Catapult event at Wired Sussex talking around the future of User Interfaces beyond the touchscreen.

1. UI Beyond the Screen
17 Feb 2016
Jason Mesut
Founder, Resonant
Local Leader, IxDA London
@jasonmesut

2. We’re going to watch some videos this evening

3. Corning — A day made of glass 2

5. How would these future solutions
work as part of your life?

6. What does your home look like?

7. Original iPhone
??
2007
—Direct manipulation
—Dynamic display
—Rich graphics

8. Knob handles
??

9. Knob handles
??

10. Knob handles
??

11. Knob handles
??

12. Knob handles
??

13. Knob handles
??

14. Original iPhone
??
2007
—Direct manipulation
—Dynamic display
—Rich graphics

15. Knob handles
??

16. Knob handles
??

17. Knob handles
??

18. Knob handles
??

19. Knob handles
??

20. Knob handles
??

21. Knob handles
??

22. Knob handles
??

23. Knob handles
??

24. Knob handles
??

25. Knobs
Buttons
Switches
Levers and handles

26. I believe…
We should be designing products and services
that fit our worlds, and enhance our lives

27. What are the things you ?

28. People
Activities
Apps?

29. Technics 1200 SH-101
OP-1Tenori-On
I love
objects

30. Q-Bert
Technics
1200

31. Q-Bert
Technics
1200
Roland
SH-101

32. Q-Bert
Technics
1200
Milo Mesut
Yamaha
Tenori-On

33. Teenage Engineering
OP-1

34. Technics 1200 SH-101
OP-1Tenori-On
I love
objects

35. Why do I care about the
future of User Interfaces?

36. 5years
Industrial Design
years
Digital User Experience
16

37. 4years
Local leader
Interaction Design
Association
L o n d o n
Subjects over 4 years
Internet of Things
Future of publishing
Urban environment
Smart materials
Mobility / Automotive
Clinical applications
Smart home
Upcoming…
Music interfaces
Consumer healthcare

38. Working on a new transportation
service for past 2 years
Icons created by Vicons Design from the Noun Project
?
– Helps people unlock London
– Mobile apps and wider digital services
– Spaces, places, people, vehicles, interactions
– Various connected things

39. – Design trends and strategy
– Interaction design
– Visual inspiration
– Animations
– Video
Also conceiving future
UI for vehicles

40. 2015 2017 2019 2021 2023 2025 2027 20292016 2018 2020 2022 2024 2026 2028 2030
Targeting a long term future

41. 2015 2017 2019 2021 2023 2025 2027 20292016 2018 2020 2022 2024 2026 2028 2030
The earliest release

42. 2015 2017 2019 2021 2023 2025 2027 20292016 2018 2020 2022 2024 2026 2028 2030
The time horizons digital designs for

43. 2015 2017 2019 2021 2023 2025 2027 20292016 2018 2020 2022 2024 2026 2028 2030
How long the solution may exist for

44. 2015 2017 2019 2021 2023 2025 2027 20292016 2018 2020 2022 2024 2026 2028 2030
Different time horizons

45. 2015 2017 2019 2021 2023 2025 2027 20292016 2018 2020 2022 2024 2026 2028 2030
Future fuzziness

46. Safety vs. sales
– In-car UI
– Considering autonomous driving
– A long term future

47. UI/UX/SD/DSUser Interface User eXperience Service Design Design Strategy
I could talk about all of the above

48. User Interface
UI/UX/SD

49. What is a
User Interface?
UI/UX/SD

50. User Interface
UI/UX/SD

51. User Interface
A human that uses a product or service:
customer, employee, citizen, parent, child
UI/UX/SD

52. User Interface
A human that uses something
A medium for a user to understand
and control a product or service
UI/UX/SD

53. Classic diagram by Bill Verplank

54. Assessment
Thought
Comprehension
UI
Inputs
Outputs
System
Processes
Rules
Attention
Perception
Intention
Cognition
Sensing
Processes
Control actions

55. Assessment
Thought
Comprehension
UI
Inputs
Outputs
System
Processes
Rules
Attention
Perception
Intention
Cognition
Sensing
Processes
Control actions
Reaction
Action

56. See Hear Touch Smell Taste
Icons from Volodin Anton, hunotika, Deivid Sáenz, Jakob Vogel via the Noun Project

57. Speed of perception
Amount of information
Interpretation accuracy
Ability to ignore
++
++
+
+
++
+
-
-
+++
+
-
- -
-
+
-
- -
-
+
-
- -
See Hear Touch Smell Taste

58. See Hear Touch Smell Talk

59. Handles and buttons
Continuous (analogous) and
Discreet (digital)
Inputs

60. The best interfaces combine
both handles and buttons
eg, Touchscreen, mouse
Bill Verplank

62. Graphical
User Interface
Allows users to
interact with
electronic systems

63. Visual Language
Motion Graphics
Information Design
Information Architecture
Interaction Design

64. Visual Language
Motion Graphics
Information Design
Information Architecture
Interaction Design

65. Visual language
Motion graphics
Information Design
Information Architecture
Interaction Design
Still relevant

66. But there’s more to
UI than just GUI
Physical
Telephone
Voice
Embedded
Distributed
Text

67. I want to talk you through
The future present
The future possible
The future likely
What you can do

68. What’s going on now?
The future present

69. We’re still getting to
grips with touchscreens

70. Touchscreens enable flexibility

71. We are often blind to their
Weaknesses

73. We are still addressing the
Input problem of fingers on glass

75. ???
Steve lied
We do need
hard keys

76. ???
Steve lied
We do need
a pointer

77. Screens dominating

78. Screens on
everything

79. Screens replacing
tactile controls

80. Screens in
front of babies
http://www.digitaltrends.com/wp-content/uploads/2013/12/fisher-price-ipad-apptivity-seat.jpg

81. What is going on
Beyond the screen?

82. Free space gesture

83. Wii

84. Leap

85. Theremin

86. Tiring
Coarse control
Confusion with natural gestures
Action != intention
Gestural

87. Nest Protect

89. We’re
attracted to
magic like
magpies

90. Emerging hardware revival

91. Tech companies
buying hardware
companies + skills

92. Service
companies
selling hardware

93. Wearable
excitement
Something other than a
smartphone to design

94. Even if they make
us look ridiculous

95. A recipe for connected products?
+ +Physical
Product
Physical/ Digital
Interface
Digital
Services
Connected
product
=

96. Aesthetic
Visceral. Visual, sonic, feel.
Interactive
Behavioural. Distribution of inputs and controls.
Experiential
Reflective. Fit with person’s context and ecosystems.
Key levels
of harmony
Components of integrated experience
+ +Physical
Product
Physical/ Digital
Interface
Digital
Services
Connected
product
=

97. What are the possible futures?
The future possible

98. Artificial Intelligence

99. How far can this go?

100. How far can this go?
Her

101. How far can this go?
Ex Machina

102. Natural UI

103. No such thing
as a natural UI

104. Ban the word intuitive

105. No UI

106. No UI is the best UI

107. Zero UI

108. The invisible interface

109. Most of the time
It’s still UI
Just not GUI

110. Less visual clutter
Less learning (in theory)
Less conscious interaction
Greater chance of false intention

112. KISSKeep it Simple Stupid
An overly simplistic business cliche

113. YMCAYou Must Consider Alternatives
What I always think in response

114. Simplicity
Misunderstood and overrated

115. Displaced complexity
System rules
Mental models

117. Complexity
Inevitable and important

118. Designers role
Tame complexity for
human + business advantage

119. Where should we focus?
The future likely

120. The future is already
here — it's just not very
evenly distributed.
William Gibson, 1993
Cliche
slide

121. Touch and gesture
will evolve

123. Screen form factors
will evolve

124. Round screens

125. Any shape screens

126. Curved screens

127. We will see more
immersive interfaces
for niche applications

128. VR

129. You can now fly

130. Sony projector

131. Escapism or focus?
Gamers and experts?

132. Use sound more

133. Wayfindr

134. Physical controls will
augment screens

135. Integrated
automotive
interface
Texas Instruments
Immersive Automobile
Physical controls
designed in concert with
graphical user interface

138. We’ll see more
strokable surfaces

139. Roli
Seaboard
Roli
Innovative expressive
piano. Keys you can bend.

141. There will be more
distributed interfaces
Mainly knobs

142. Distributed controls and displays

143. Distributed controls

144. Distributed
knobs

145. Slick knobs

146. Knobs with
embedded
displays

147. One thing is for certain, although we
are entering a digital future,
the Knob is here to stay!
http://knobfeel.tumblr.com/

148. There will be buttons

149. Loads of buttons

150. And keyboards will
still be important

151. And older interfaces
will survive

152. Mice are precise

153. How can you better
focus for the future?

154. Get Physical and Digital
designers together

155. Connect on shared motivations and values
Calibrate language, practices and timelines
Create together

156. Make the future real

157. Beware concept videos
Tell real stories and demonstrate
with working prototypes

158. The Internet of
Ubiquitous
Computing
Information
Appliance Things is
growing
Prototype out loud

159. Tinkertronics

160. SAM Labs

161. SAM Labs

162. Shun the shiny
Don’t be a magpie
to the magical

163. The Future of User Interfaces?

164. Knobs
Buttons
Switches
Levers and handles

165. Knobs
Buttons
Switches
Levers and handles

166. Almost anything can be done physically or
digitally now. Now it’s about choice.
Understanding industrial Design

167. The Future of User Interfaces?
It’s like today, with
some small differences
— Politer interactions
— Evolved touch, gesture, voice, sound
— New screen form factors
— Physical controls augmenting displays

168. Thank you jasonmesut@gmail.com
@jasonmesut
slideshare.net/jasonmesut