How Deep Learning Transforms Design

H O W D E E P L E A R N I N G
C H A N G E S T H E
D E S I G N P R O C E S S
© Lawrence Lek

What is Deep Learning?
© Lawrence Lek

Artificial IntelligenceAI landscape
© Michael Copeland

C H A N G E S T H E
AI landscape
THE TERMS ARTIFICIAL INTELLIGENCE
AND MACHINE LEARNING
ARE OFTEN USED INTERCHANGEABLY –
YET THEY AREN’T THE SAME

C H A N G E S T H E
Machine Learning
Machine learning is the ability to learn without
being explicitly programmed.
Programs that can learn from their own mistakes
and improve their performance over time.

C H A N G E S T H E
Machine Learning
Data mining
Image recognition
Facebook newsfeed
Netflix suggestions
Big Data analysis
Amazon suggestions
Predictive analytics

C H A N G E S T H E
Machine Learning
Notifications
Predictive Keyboards
Spam/Quality Control Systems
Copywriting
Design Patterns
Gesture Recognition
Content Actions/Feedback
Magic Numbers
Search
Related Content
Character Recognition

C H A N G E S T H E
Machine Learning
Notifications
Predictive Keyboards
Ranked Feeds
Spam/Quality Control Systems
Copywriting
Design Patterns
Gesture Recognition
Content Actions/Feedback
Magic Numbers
Search
Related Content
Character Recognition
© Jan Korsanke

Artificial IntelligenceAI landscapeAI landscape
© Narrative Science

E P I C O R C H A P T E R
AI landscape
© Narrative Science

C H A N G E S T H E
AI vs Big Data
© Matthew Mayo
Machine Learning
Artifical Intelligence
Data Mining
Data Science
Deep
Learning
Big Data

C H A N G E S T H E
Deep Learning
Deep Learning is based on Artificial Neural
Networks.
These are systems that are designed to process
information in ways that are similar to the ways
biological brains work. It uses a certain set of
Machine Learning algorithms that run in multiple
layers.

Artificial IntelligenceMachine Learning vs Deep Learning
© Rasmus Rothe

„H O W D E E P
L E A R N I N G C H A N G E S
T H E D E S I G N
P R O C E S S
AI HAS BY NOW SUCCEEDED
IN DOING ESSENTIALLY EVERYTHING THAT
REQUIRES ‘THINKING’
BUT HAS FAILED TO DO MOST OF WHAT PEOPLE DO
‘WITHOUT THINKING.’
Donald Knuth, Stanford University

Why it is important?
© Lawrence Lek

C H A N G E S T H E
Possible
Desirable
Profitable
A user experience
powered by
a system that learns
© Fabien Girardin

C H A N G E S T H E
Google has decided to pivot its business model from that
simple mobile-first toward machine learning.

Mobile Frameworks & libraries
TensorFlow /
TensorFlow Lite
Caffe2Go ML Kit

What can be done today?
© Lawrence Lek

C H A N G E S T H E
Google Lens
Can an algorithm see what you see and help you take
action based on this information?

C H A N G E S T H E
Google Lens
Google Lens turns the camera from a passive tool
that’s capturing the world around you to one that’s
allowing you to interact with what’s in your camera’s
viewfinder.

C H A N G E S T H E
Photoeditor SDK
Can an algorithm extract foreground content
from its background?

C H A N G E S T H E
Photoeditor SDK
Professional image editing tools help you to select
distinctive objects, but, they aren’t available on your
mobile device, where you take and publish the images,
and they usually require some hands-on time, before
you can produce anything usable.

C H A N G E S T H E
Photoeditor SDK

C H A N G E S T H E
Adobe - Deep Image Matting
Can an algorithm extract foreground content
from its background – in movies too?

C H A N G E S T H E
Adobe - Deep Image Matting
Previous algorithms have poor performance when an
image has similar foreground and background colors
or complicated textures. With neural networks you
can extract foreground content from its background
intelligently and accurately - in movies. This might kill
the green screen.

C H A N G E S T H E
Entrupy
Can an algorithm help to spot a real
Chanel bag from a fake?

C H A N G E S T H E
Entrupy
Counterfeits are a painful thorn in the side of luxury
fashion brands but they can be even more of a
headache for digital re-sellers. Trying to spot a real
Chanel from a fake? Deep Learning tech can help.

C H A N G E S T H E
Entrupy
Entrupy is a portable scanning device that instantly detects
imitation designer bags by taking microscopic pictures that
take into account details of the material, processing,
workmanship, serial number, and wear/tear. It then employs
the technique of deep learning to compare the images
against a vast database that includes top luxury brands and
if the bag is deemed authentic, users immediately get a
Certificate of Authenticity.

C H A N G E S T H E
Google Research
Can an algorithm understand what makes
good photography?

C H A N G E S T H E
Google Research
360-degree images are fascinating to explore but they’re not
exactly the best looking photos, often lackluster in color and
exposure. Google engineers used Deep learning to teach the
machine to understand what makes good photography. They
even added automatic fine-tuning of the images: Instead of
filters, the AI understands what elements exist in the image
and, in turn, tweaks the lighting accordingly.

C H A N G E S T H E
EyeEm - Vision
Can an algorithm learn and suggest aesthetics
in a way an art director would?

C H A N G E S T H E
EyeEm - Vision
Can a machine learn aesthetics in a way a human
would? Could it then look at a set of photos, and draw
on those same aesthetics to reproduce a different
set?

C H A N G E S T H E
EyeEm - Vision
What are aesthetics anyway? Is it just what you “like”?
How does it all work? When you as a human find it
hard to express what you like do you think a machine
going to find it easy?

C H A N G E S T H E
EyeEm - VisionEyeEm - Vision

C H A N G E S T H E
Neural Storyteller
Can an algorithm make up stories from
any given image?

C H A N G E S T H E
Neural Storyteller
Neural Storyteller is an artificial intelligence
model that when given an image, can generate a
story about the image using features in the image.

Neural Storyteller
C H A N G E S T H E

C H A N G E S T H E
Rapto
Can an algorithm turn your normal day
into a rap song?

C H A N G E S T H E
Rapto
Rapto lets you create music using artificial
intelligence & your camera. Simply point the camera
towards any object around you and Rapto will use it's
inbuilt neural network to understand the object &
create rap music!

C H A N G E S T H E
Rapto

Subtopic or example
New versions of facial recognition technologies use
Deep Learning, as it is especially effective for image
recognition because it makes a computer zero in on
the facial features that will most reliably identify a
person.

C H A N G E S T H E
Google Allo
Can an algorithm create a Bitmoji for you on the fly?

C H A N G E S T H E
Google Allo
With Google Allo simply snap a selfie, and it’ll return
an automatically generated illustrated version of you,
on the fly.

C H A N G E S T H E
Nest Cam IQ
Can an algorithm detect strangers?

Nest Cam IQ
The Nest Cam IQ can differentiate between friends
and family members, or a stranger. Insights can range
from telling you the kids are home from school to
sending an alert if an unfamiliar person is in the living
room.

C H A N G E S T H E
China & Russia
Can an algorithm register me at events?

China & Russia
In China, facial recognition technology is already
finding consumer applications.
In Russia, FindFace is an controversial an app used to
identify members of a crowd to match them with
social network accounts.

C H A N G E S T H E
Apple Notes
Can an algorithm understand what you are writing?

C H A N G E S T H E
Google Quick Draw
Can an algorithm understand what you are drawing?

C H A N G E S T H E
Google Quick Draw
Can a neural network learn to recognize doodles?
Help teach it by adding your drawings to the world’s
largest doodle data set.

C H A N G E S T H E
Google Quick Draw

C H A N G E S T H E
Google AutoDraw
Can an algorithm help to teach you drawing?

C H A N G E S T H E
Google AutoDraw
AutoDraw’s suggestion tool uses the same technology
used in QuickDraw, to guess what you’re trying to
draw. Right now, it can guess hundreds of drawings
and we look forward to adding more over time.

C H A N G E S T H E
Google AutoDraw

C H A N G E S T H E
pix2pix
Can an algorithm take your drawings
to generate images?

C H A N G E S T H E
pix2pix
Image-to-image is a Tensorflow port of pix2pix.
The pix2pix model works by training on pairs of
images such as building facade labels to building
facades, and then attempts to generate the
corresponding output image from any input image
you give it.

C H A N G E S T H E
pix2pix

C H A N G E S T H E
Drawing music
Can an algorithm take your drawings
to generate music?

C H A N G E S T H E
Drawing music
Kene Cogan has built an experiment where a
classified musical object triggers an according
sound track in Ableton Live.

C H A N G E S T H E
Drawing music

C H A N G E S T H E
Lyrebird
Can an algorithm help to mimic the voice of
a given person?

C H A N G E S T H E
Lyrebird
Lyrebird is an API for speech synthesis. Record 1 minute from
someone's voice and Lyrebird can compress her/his voice's
DNA into a unique key.

C H A N G E S T H E
Lyrebird

C H A N G E S T H E
pix2code
Can an algorithm program a user interface
from a given layout?

C H A N G E S T H E
pix2code
Transforming a graphical user interface created by a designer
into code is a typical task conducted by a developer in order
to build software, websites and mobile applications.
Deep Learning techniques can be leveraged to automatically
generate code given a graphical user interface screenshot as
input.

C H A N G E S T H E
Style Transfer
Can an algorithm re-create complex aesthetics
and even fine art?

C H A N G E S T H E
Style Transfer
In fine art humans have mastered the skill to create unique
visual experiences through composing a complex interplay
between content and style.
Style Transfer is an artificial system that creates artistic
images of high perceptual quality.

C H A N G E S T H E
Creative Adversarial Networks
Can an algorithm invent new styles of art?

C H A N G E S T H E
Creative Adversarial Networks
Researchers modified a type of algorithm in which two neural
nets play off against each other to get better and better
results. One creates a solution, the other judges it.
AIs that can tweak photos to mimic the style of famous
painters are already widely available. But the new system is
designed to produce original works from scratch.

C H A N G E S T H E
Fontjoy
Can an algorithm generate font combinations?

C H A N G E S T H E
Logo Experiment
Can an algorithm generate logo artworks?

C H A N G E S T H E
Logo Experiment
Rob Peart tried to generate Death Metal
logos from Neural Networks ;)

Waking The Cadaver
Logo Experiment

From the sunset, forest and grief
Logo Experiment

Logo Experiment

C H A N G E S T H E
NVIDIA - Iray
Can an algorithm squash 3D render times?

C H A N G E S T H E
NVIDIA - Iray
NVIDIA brings AI to ray tracing to speed graphics
workloads. Upload your partially rendered image and
AI Renderer renders the rest for you.
By predicting final images from only partly finished
results, Iray AI produces accurate, photorealistic
models without having to wait for the final image to
be rendered.

C H A N G E S T H E
Autodesk - DreamCatcher
Can an algorithm generate physical products
that solve complex problems?

C H A N G E S T H E
Dreamcatcher is a generative CAD system that
enables designers to craft a definition of their design
problem through goals and constraints.

C H A N G E S T H E
The three structural elements shown are all designed to carry
the same structural loads and forces.

C H A N G E S T H E
A Lightweight bike stem generated by an algorithm

C H A N G E S T H E
This is a car frame that is designed by a generative algorithm

C H A N G E S T H E
A a lightweight load-bearing engine block

C H A N G E S T H E
Objectifier
Can an algorithm help objects to respond to
human gestures?

C H A N G E S T H E
Objectifier
Objectifier empowers people to train objects in
their daily environment to respond to their unique
behaviors. It gives an experience of training an
artificial intelligence; much like training a dog -
you teach it only what you want it to care about.
Just like a dog, it sees and understands its
environment.

Objectifier
C H A N G E S T H E

C H A N G E S T H E
What’s next?
Computers can now see, hear, and translate
languages with unprecedented accuracies.
They are also learning to generate images,
sound, and text.

C H A N G E S T H E
What’s next?
However the systems we’re building still fall into the
category of “Narrow AI” — they can achieve super-
human performance in a specific domain, but lack
the ability to do anything sensible outside of it.

C H A N G E S T H E
What’s next?
However the systems we’re building still fall into
the category of “Narrow AI” — they can achieve
super-human performance in a specific domain,
but lack the ability to do anything sensible
outside of it.
© Jan Korsanke

C H A N G E S T H E
Open AI - UNIVERSE

„H O W D E E P L E A R N I N G
C H A N G E S T H E
D E S I G N P R O C E S SMACHINE LEARNING IS GOING TO UPEND YOUR
INDUSTRY AND YOUR PRODUCT.
Ken Norton, Google Ventures

C H A N G E S T H E
MACHINE LEARNING WON’T REACH ITS POTENTIAL
– AND MAY ACTUALLY CAUSE HARM –
IF IT DOESN’T DEVELOP
IN TANDEM WITH USER EXPERIENCE DESIGN.
Caroline Sinders, Fast Company

C H A N G E S T H E
Challenges for UX
Develop relevant use cases

C H A N G E S T H E
Spotify - Discovery Weekly
Discovery Weekly is an automated music
recommendation digest for each Spotify user every
monday. It uses a feedback loop mechanism to
personalize, optimize or automate the existing
service.

C H A N G E S T H E
D E S I G N P R O C E S SMACHINE LEARNING WILL CHANGE CUSTOMER
PERSONAS FOREVER
Andre Smith, Digitalist Magazine / SAP

C H A N G E S T H E
Challenges for UX
Re-think customer personas

C H A N G E S T H E
Challenges for UX
Re-think customer personas
Thanks to machine learning, computers will soon know your
customers better than your customers know themselves.
• They’re much better at „guesswork“ than humans are
• More efficient targeting of new customers
• More cost-effective

C H A N G E S T H E
Challenges for UX
Create intuitive AI interfaces

C H A N G E S T H E
THE FUTURE OF MACHINE LEARNING
IS COMING UP WITH A HYBRID LANGUAGE THAT
BRIDGES DESIGN AND ENGINEERING.
Caroline Sinders, Fast Company

C H A N G E S T H E
Challenges for UX
Make tons of data manageable

C H A N G E S T H E
Challenges for UX
Use data to be super-relevant or be silent

C H A N G E S T H E
Challenges for UX
Let users tell about poor information

C H A N G E S T H E
Challenges for UX
Let users tell about poor information
For example in banking, one could consider the temporal evolution of
account balances to segment savings behaviors. This type of
algorithms that leads to decision-making needs to learn to be more
precise.
It’s the designer’s job to find ways to let users tell implicitly or
explicitly about poor information.

C H A N G E S T H E
Challenges for UX
Design for discovery
• Filter Bubble - Tweak algorithms to
be less accurate
• Profile Detox - Let an open door to
reshape profiles
• Human Computation - Enlist humans
to give more diversity
© Fabien Girardin

C H A N G E S T H E
D E S I G N P R O C E S SEVERY TIME I CHECK MY PHONE, I’M PLAYING
THE SLOT MACHINE
Tristan Harris, a former Google product manager

C H A N G E S T H E
Challenges for UX
Design for engagement responsibly

C H A N G E S T H E
Challenges for UX
Design for engagement responsibly
Today, algorithms typically score the relevance of social and news
content. Major online services are fighting to hook people, grab their
attention for as long as possible. Their business is to keep users
active as long and frequently as possible on their platforms. They use
techniques that promote addiction = hooking people endlessly
searching for the next reward.
That new power raises the need for new design principles in the age
of machine learning.

C H A N G E S T H E
Challenges for UX
Empathy is not (yet) available

C H A N G E S T H E
Challenges for UX
Empathy is not (yet) available
The ethical and practical considerations of machine learning
have to be shaped by how products using machine learning
affect users and how users can understand and see those
effects.

C H A N G E S T H E
Challenges for UX
Questions are the new answers
In future computers won’t deliver answers before asking
you back (a string of) questions. But what are rules and
etiquette for machines?
© Jan Korsanke

C H A N G E S T H E
Challenges for UX
Illustrate for transparency

C H A N G E S T H E
Challenges for UX
Illustrate for transparency
When users don’t understand how an algorithm gets its results,
it can be difficult to trust the system. Transparency
communicates trust.

C H A N G E S T H E
Challenges for UX
Seamful design

C H A N G E S T H E
Challenges for UX
Seamful design
Designers must know that a „Prediction Feature“ is not the
same as informing, and consider how well such a prediction
could support a user action.

C H A N G E S T H E
Challenges for UX
Machine bias: AI can lead to discrimination

C H A N G E S T H E
Challenges for UX
Machine bias: AI can lead to discrimination
Most of the current facial recognition techniques use the same
data set, which was trained on mainly white people. It would
not recognise people with other skintones.

C H A N G E S T H E
ULTIMATELY,
DESIGNERS MUST ACT AS A BULWARK
AGAINST IRRESPONSIBLE,
UNETHICAL USE OF AI.
Katharine Schwab, Fast Company

C H A N G E S T H E
Principles for designing AI responsibly
• AI must be designed to assist humanity
• AI must be transparent
• AI must maximize efficiencies without destroying the dignity of people
• AI must be designed for intelligent privacy
• AI must have algorithmic accountability
• AI must guard against bias
Satya Nadella, Microsoft CEO

@KUTTIN3DGE
KRUNCHTIME.ORG
D A N K E 🙏

How Deep Learning Transforms Design

Recomendados

Recomendados

Más contenido relacionado

Similar a How Deep Learning Transforms Design

Similar a How Deep Learning Transforms Design (20)

Más de Alexander Meinhardt

Más de Alexander Meinhardt (20)

Último

Último (20)

How Deep Learning Transforms Design