Chatbots are most commonly used as a feature of mobile or desktop websites, where text is used for the primary interaction method. In this session we’ll explore how you can use Amazon Sumerian to create a virtual host that can be interacted with through Virtual Reality, Augmented Reality or just as a digital character. This makes voice a far more natural interaction method. We’ll also look at how you can extend this bot to interact with other rich services in the AWS cloud. This webinar was presented by Brent Maxwell in September 2018 as part of the ANZ Webinar Series for Australia and New Zealand.

1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Using Virtual Reality to bring
your ChatBot to life
Brent Maxwell, AWS Solutions Architect
braxwell@amazon.com

2. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Agenda
• Market, Tech & Consumer Dynamics
• Developing AR/VR Applications
• How Amazon Sumerian Helps
• Using Sumerian
• Use Cases
• Resources

3. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Virtual Reality Augmented Reality

4. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Market & Technology Dynamics
Real World Virtual World
Digital & physical convergence
enabled by technology is driving
the emergence of immersive
applications & experiences
The Virtuality Continuum (1994) – spectrum of how we engage &
communicate with people, products, services & our environment

5. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Consumer Dynamics
What is driving demand for immersive experiences?
• Consumers: fun & engaging
• Enterprises: brand awareness, innovation & retention
First wave was primarily in gaming & entertainment
… we are seeing a second wave of applications & use
cases across all industries

6. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
App Development
- Process
- Challenges

7. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Developer
has idea
Download &
install 3D
engine
Install device
SDK
Register on
platform
Install
platform
SDK
Develop
application
Submit for
approval
Publish
application
Customer
finds app on
store
Customer
hits
purchase
button
Customer
waits for
download
Customer
uses
application
VR/AR Application Development Process

8. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

9. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

10. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
We Simplify …

11. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
… So You Can Focus

12. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How do we do it?
Why browsers?
• They are ubiquitous
… digital displays … smart TVs … smart phones … VR headsets
• Make it easy to build, iterate, refresh & publish cross platform
• Enables nearly anyone to develop AR, VR & 3D apps
Sumerian apps are browser-based: build once … deploy
widely

13. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Sumerian: Benefits
• No AR/VR/3D expertise needed
• Prebuilt templates for business scenarios
• Lifelike characters to guide/narrate your
applications
• Visual State Machine/JavaScript support
for customizations
• Tight integration with AWS services
• Deploy to any platform that supports
WebGL

14. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

15. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
WebGL IDE displayed in browser
Entities panel
State Graph
Components Panel
Canvas Viewport
WebVR Mode
Sumerian Menu
Welcome to Sumerian
Asset panel

16. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Integrate with AWS Services
CORE SERVICES
Integrated Networking
Rules Engine
Device Shadows
Device SDKs
Device Gateway
Registry
Local Compute
Custom Model
Training & Hosting
Conversational Chatbots
Virtual Desktops
App Streaming
Schema Conversion
Image & Scene
Recognition
Sharing &
Collaboration
Exabyte-Scale
Data Migration
Text to Speech
Corporate Email Application Migration
Database Migration
Regions
Availability Zones
Points of Presence
Data Warehousing
Business Intelligence
Elasticsearch
Hadoop/Spark
Data Pipelines
Streaming Data
Collection
ETL
Streaming Data
Analysis
Interactive SQL
Queries
Queuing & Notifications
Workflow
Email
Transcoding
Deep Learning
(Apache MXNet,
TensorFlow, & others)
Server MigrationCommunications
MARKETPLACE
Business Apps Business Intelligence DevOps Tools Security Networking StorageDatabases
API Gateway
Single Integrated
Console
Identity
Sync
Mobile Analytics
Mobile App Testing
Targeted Push
Notifications
One-clickApp
Deployment
DevOps Resource
Management
Application Lifecycle
Management
Containers
Triggers
Resource Templates
Build & Test
Analyze & Debug
Identity
Management
Key Management
& Storage
Monitoring &
Logs
Configuration
Compliance
Web Application
Firewall
Assessment
& Reporting
Resource & Usage
Auditing
Access Control
Account
Grouping
DDOS
Protection
TECHNICAL & BUSINESS SUPPORT
Support
Professional
Services
Optimization
Guidance
Partner
Ecosystem
Training &
Certification
Solutions Management Account Management
Security & Billing
Reports
Personalized
Dashboard
Monitoring
Manage
Resources
Data Integration
Integrated Identity &
Access
Integrated Resource &
Deployment Management
Integrated Devices
& Edge Systems
Resource
Templates
Configuration
Tracking
Server
Management
Service
Catalogue
Search
MIGRATIONHYBRID ARCHITECTUREENTERPRISE APPSMACHINE LEARNINGIoTMOBILE SERVICESDEV OPSANALYTICS
APP SERVICES
INFRASTRUCTURE SECURITY & COMPLIANCE MANAGEMENT TOOLS
Compute
VMs, Auto-scaling, Load
Balancing, Containers,
Virtual Private Servers,
Batch Computing, Cloud
Functions, Elastic GPUs,
Edge Computing
Storage
Object, Blocks, File, Archivals,
Import/Export, Exabyte-scale
data transfer
CDN
Databases
Relational, NoSQL,
Caching, Migration,
PostgreSQL compatible
Networking
VPC, DX, DNS
Facial Recognition &
Analysis
Facial Search
Patching
Contact Center

17. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Language & Speech Services
Amazon Polly is a service that
turns text into lifelike speech
• 52 lifelike voices spanning 24 languages
• Natural-sounding, accurate, and intelligible
• Low latency responses enable developers to build real-time systems
• Developers can store, replay and distribute generated speech

18. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
“Today in Seattle, WA, it’s 11°F”
‘"We live for the music" live from the Madison Square Garden.’
1. Automatic, Accurate Text Processing
Polly: A Focus On Voice Quality & Pronunciation

19. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
2. Intelligible and Easy to Understand
1. Automatic, Accurate Text Processing
Polly: A Focus On Voice Quality & Pronunciation

20. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
2. Intelligible and Easy to Understand
3. Add Semantic Meaning to Text
“Richard’s number is 2122341237“
“Richard’s number is 2122341237“
Telephone Number
1. Automatic, Accurate Text Processing
Polly: A Focus On Voice Quality & Pronunciation

21. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
2. Intelligible and Easy to Understand
3. Add Semantic Meaning to Text
4. Customized Pronunciation
“My daughter’s name is Kaja.”
“My daughter’s name is Kaja.”
Polly: A Focus On Voice Quality & Pronunciation
1. Automatic, Accurate Text Processing

22. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• Media and Entertainment
• Education
• Accessibility
• Content Creation
• Telephony/Contact Center
• Internet of Things
• Gaming
Polly: Use Cases

23. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Language & Speech Services
Amazon Lex is a service for building
conversational interfaces into any application
• Enables intent-driven interactions
• Supports both voice and text conversations
• Deployable across mobile and messaging platforms
• Integrated speech recognition and natural language understanding, constantly
improved by Alexa users

24. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Lex: Conversational Chatbots
I’d like to book a hotel.
Sure! What city would you like to book it in?

25. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Lex: Conversational Chatbots
I’d like to book a hotel.
Sure! What city would you like to book it in?
New York City
Destination:
New York City

26. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Lex: Conversational Chatbots
I’d like to book a hotel.
Sure! What city would you like to book it in?
New York City
When would you like to check in?
Next Friday
Destination:
New York City
Check In:
26/05/2017

27. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Lex: Conversational Chatbots
Lex Bots
Web
Devices
Apps
Facebook Messenger,
Slack,
Amazon
Connect
Mobile
Mobile Hub
integration

28. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Lex: Conversational Chatbots
Lex Bots
Salesforce
Microsoft Dynamics
Marketo
Zendesk
Web
Devices
Apps
Facebook Messenger,
Slack,
Amazon
Connect
Mobile
Mobile Hub
integration
Quickbooks

29. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Lex: Use Cases
Informational Bots
Chatbots for everyday consumer requests
Application Bots
Build powerful interfaces to mobile applications
• News updates
• Weather
information
• Game scores ….
• Book tickets
• Order food
• Manage bank accounts ….
Enterprise Productivity Bots
Streamline enterprise work activities and improve efficiencies
• Check sales numbers
• Marketing performance
• Inventory status ….
Internet of Things (IoT) Bots
Enable conversational interfaces for device interactions
• Wearables
• Appliances
• Auto ….

30. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• Guide your experiences
• 3 Hosts to choose from: Cristine,
Preston and Luke
• Hosts are integrated with automatic
speech recognition (ASR) services
like Amazon Lex and Amazon Polly
• Additive Animations for tracking,
speaking, and expressing emotion
(Gestures and Emotes)
Drive Digital Characters Dynamically
Sumerian Hosts

31. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Customise Sumerian Hosts
• Three additional outfits per
Host (hoodie, tee, polo)
• Customize the color
• Add your own logo

32. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Development Pipeline & Costs

33. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Development Pipeline & Costs
Concept Artist

34. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Development Pipeline & Costs
3D Modeler

35. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Development Pipeline & Costs
Technical Artist

36. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Development Pipeline & Costs
Animator

37. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Development Pipeline & Costs
Engineers: Animation Specialist and Web Services

38. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Development Pipeline & Costs
Development Team
concept artist modeler technical artist animator engineer engineer
Total cost for an interactive character ~$80,000

39. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Demo Time!

40. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Use Case: Training

41. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Use Case: Simulation

42. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Use Case: Digital Signage

43. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Resources & Community
https://slack.sumerian.world
Amazon-Sumerian-User-Group/
https://twitch.tv/aws/
https://aws.amazon.com/twitch/
https://docs.sumerian.amazonaws.com/
https://docs.aws.amazon.com/sumerian/latest/userguide/
https://aws.amazon.com/sumerian/faqs/

44. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Thank you!
Brent Maxwell, AWS Solutions Architect
braxwell@amazon.com