Solvion Trendwerkstatt - Microsoft Azure + Bots

SOLVION TRENDWERKSTATT
MICROSOFT AZURE+AI+BOTS
Stephan Bisser

STEPHAN BISSER
? https://bisser.io
t @stephanbisser
E stephan.bisser@solvion.net
# Graz, Austria
` http://bit.ly/LEARN_TO_BOT

TIMING
09:00-10:30 Workshop
10:30-10:45 Break
12:15-13:00 Lunch

AGENDA
1 Überblick Microsoft AI 2 Überblick
Conversational AI
3 Praxisbeispiele 4 Best Practices
5 Hands On

QUESTIONNAIRE
Please fill out
http://bit.ly/AITrendwerkstatt1

Azure AI
Machine learning
Azure Databricks
Azure Machine Learning
Azure AI Infrastructure
AI apps & agents
Azure Bot Service
Azure Cognitive Services
Knowledge mining
Azure Cognitive Search
For every person and every organization

DIGITAL TRANSFORMATION + AI
Artificial
Intelligence

2000s: Mobile
• Social
• User download apps
from App Stores
The future: Conversations
• Natural language between people
and technology
• Conversational canvas
• Bots and agents
• Conversational AI
1990s: Internet
• Search
• User “visits” websites
1980s: PC
• Desktop
The evolution of computers and IT

CONVERSATIONAL AI
A new user interface paradigm
Exposes software services through
conversational interface
In apps, browser and on IoT devices
Pro-active capabilities
(Virtual Assistant etc.)
Bots
Conversational AI is a subfield of artificial intelligence focused on producing natural and seamless
conversations between humans and computers

CONVERSATIONAL AI
Software as a service, SaaS solutions
To enable out-of-the-box conversational AI services for key solutions
Pre-trained AI, Cognitive Services
To add and customize language, speech, knowledge & vision
Managed bot service, Azure Bot Service
To accelerate building, management and deployment of conversational AI
Popular open source frameworks, Bot SDK on Github
To fully integrate and customize your solution with any AI technology
Solution accelerators and templates, PaaS
To accelerate conversational AI solutions
From the Intelligent Cloud to the Intelligent Edge
Customer Care
Bot Template
Virtual Assistant
Solution accelerator
Enterprise HR
Template
LanguageSpeech
…
DecisionVision Web search
Customer ServiceCortana, Teams Virtual Agent
BotKit SDK
Bot
Framework
Azure Bot Service

BUILDING CONVERSATIONAL APPS & AGENTS
Vision
Language
Speech

BOTS 101
• “a computer program designed to have a
conversation with a human being,
especially over the internet” *
• Solves the user’s needs in the
quickest/easiest way compared to any
other option...
• Introduce AI in a friendly way
* https://dictionary.cambridge.org/dictionary/english/chatbot

WHY DO WE NEED CHATBOTS?
24/7
They are always available
and respond at any time
QnA
They answer your users’
questions with
knowledge stored in your
data silos
Personal Assistant
They can be seen as personal
assistants who ease the daily
work for your users

TYPES OF CHATBOTS
INTERNAL EXTERNAL
Personal assistant
Team assistant
Company assistant
Public facing website bot
QnA endpoint for
customers on various
channels
Partner assistant

MODERNIZING THE COMPANY FAQ
Company info
Departmental knowledge
Troubleshooting/support
+ ++

BOT FRAMEWORK V4 SDK
• GA - C# and Java Script
• C# support .NET Core
• Python and Java - still in preview
• Incorporates learnings from V3
• Open, Modular and extensible architecture
• Agile and customizable dialog system
• Share the same implementation across C#, Java Script, Java and Python
• Rich visual controls (cards, buttons, images, videos, adaptive cards)
• Seamless integration with Azure and Cognitive Services
• Modern toolchain (streamline development and deployment)
• Open Source and rich ecosystem

MICROSOFT BOT FRAMEWORK
Users
Direct Line
Mail
Bing
kik
Facebook
Messenger
Web Chat
3rd Party
3rd
Slack GroupMe
CortanaMicrosoft Teams
Telegram
Twilio (SMS)
SkypeOffice 365
Channels
* still in preview
Conversational AI Platform
Bot Builder SDKs
C#
.NET SDK Java SDK*JavaScript SDK Python SDK*
Cognitive ServicesVision Speech Search Language Knowledge
Patterns & ConceptsRecognizers CardsMiddlewareState Dialogs
Web App Bot Bot Channels Registration Bot Service Types
HostingMicrosoft Azure On-premises Other Cloud Service Providers

 10+ Public Repos
• SDK V4: C#, JavaScript, Python, Java
• SDK V3 (C#, JavaScript)
• CLI Tools
• Emulator
• WebChat Control
• Documentation
• BotBuilder-Samples
https://github.com/BotBuilderCommunity
• Extending the core Bot Framework
SDKs
• Middleware, dialogs, helper
Bot Framework – Open Source Project

Adapter
Creates a turn context and calls the
middleware.
Middleware
Linear set of components that are each
executed in a pipeline. Can short circuit a
turn, in which case the next turn handler
isn’t called.
Services
Pre-built components to easily add
Cognitive Services to your bot.
State
Persist context between interactions.
Your Bot
Acts on inbound message and applies
your custom business logic.
SDK: How does it work?SDK: How does it work?

Virtual Assistant
Bot Framework + Cognitive Services
Cognitive Services
Your Assistant
Your Brand
Your Personality
Language
Understanding
Speech
VisionQnA
Dispatch
Office Graph
Knowledge
Sources
Manuals
(PDF)
Data Sources
General
Knowledge
Points of
Interest
Mail
Automotive
Custom
Other
Bots
Skills
(Customizable)
APIs
Directory
Tasks
Calendar
3P Assistants
Type Speak
Tap
User
Input
Devices
Channels
Azure Bot
Service
Adaptive
Cards

Office Graph
Knowledge
Sources
Manuals
(PDF)
Data Sources
General
Knowledge
Points of
Interest
Mail
Automotive
Custom
Other
Bots
Skills
(Customizable)
APIs
Directory
Tasks
Calendar
3P Assistants
Skills include language models, dialogs, and cards
that can be reusable across bots.
Skill == Bot
• Dispatcher within your core Assistant
supports Interruption
• Skills contains a self-describing Skill Manifest
Skills are open source, fully customizable
• Skill Template
• Skill CLI
• C# and Typescript support
BOT FRAMEWORK SKILLS

Complete control of the
user experience
Your Brand, Your Personality, Your
Voice, Your Assistant
Complete ownership
and control of data
Your authentication,
GDPR, Data Processor and
monetization opportunity
Your Assistant,
everywhere
In the Cloud, On The Edge
All Azure regions and Languages
In your app, webpage, & chat apps
Integrated and Context
Aware
Integrated with your ecosystem and
infused with intelligence about your
customers
External assistant
integration
Integrate with your customers
chosen assistant to hand off certain
questions
Enterprise Ready
Compliant, fully enterprise ready
ISO, SOC, PCI, HIPAA
Gov Cloud
SOLUTION ACCELERATORS AND TEMPLATES
VIRTUAL ASSISTANT

Parts of Conversation
Input → Semantic Language Understanding
{
“Intent” : “order food”,
“Entity” : “type = pizza”
}
Bot business logic
Output → Language Generation
{
“Entity” : “type = pizza”
}

MODERN BOT ARCHITECTURE
Modern chatbots within the Microsoft Bot Framework rely on SKILLS
Those SKILLS are aligned to the Amazon Alexa approach of a plug & play solution
where you can add and remove skills as needed
Email
Skill
Skill
Skill
Skill
Skill

CONVERSATIONAL MECHANISMS
• Text, with optional media
attachments
• Traditional chat, can contain
media attachments
(e.g., image, video, audio,
file)
• Input prompts
• Suggested actions: Buttons,
numbered items in a list, etc.
• Rich cards
• Images, buttons, audio,
animations, etc within one
frame – Adaptive Cards

MODELLING YOUR CONVERSATION
Flexibility
Effort to implement
Dialogs
FormFlow
QnA
Bespoke
FAQs,
command & control
Data capture,
“Web forms” scenarios
Multi layered conversations
Roll your own
state management etc.

SearchLanguageVision
Give your apps a human side
DecisionSpeech
Recognize, identify,
caption, index, and
moderate your pictures,
videos, and digital ink
content.
Convert spoken audio
into text, use voice for
verification, or add
speaker recognition to
your app.
Allow your apps to
process natural language
with pre-built scripts,
evaluate sentiment and
learn how to recognize
what users want.
Build apps that surface
recommendations for
informed and efficient
decision-making.
Add Bing Search APIs to
your apps and harness
the ability to comb
billions of webpages,
images, videos, and news
with a single API call.

SearchLanguageVision DecisionSpeech
Bing Autosuggest
Bing Web Search
Bing Entity Search
Bing Video Search
Bing Visual Search
Bing Custom Search
Bing News Search
Bing Image Search
Bing Local Business Search
Content Decision
Anomaly Detector
Personalizer
Language Understanding
(LUIS)
Text Analytics
Bing Spell Check
Translator Text
QnA Maker
Speaker Recognition
Speech ServicesComputer Vision
Face
Video Indexer
Custom Vision
Form Recognizer
Ink Recognizer

LANGUAGE UNDERSTANDING (LUIS)
“Cloud-based API that applies
custom machine-learning
intelligence to a user's
conversational, natural language
text to predict overall meaning,
and pull out relevant, detailed
information”
Prebuilt models
Custom entities
CLI tools and v4 SDK integration
CSA STAR, ISO 27001, ISO 27018,
HIPAA and PCI DSS Compliance Always learning &
improving
Enterprise Grade,
available worldwide

LANGUAGE UNDERSTANDING EXAMPLE
Book a hotel in Graz.

{
"query": "Book a hotel in Graz.",
"topScoringIntent": {
"intent": "BookHotel",
"score": 0.921233
},
"entities": [
{
"entity": "Graz",
"type": "Location",
"startIndex": 10,
"endIndex": 13,
"score": 0.7615982
}
]
}

Book a hotel in Graz.
Intent = BookHotel Entity = Graz

QNAMAKER
“QnA Maker is a cloud-based API
service that creates a conversational,
question and answer layer over your
data.”
Data - Aggregate and extract
knowledge from non structured data
Architecture - The data and runtime
hosted in your Azure subscription.
Data privacy and compliance. QnA
data is hosted within your Azure
compliance boundary
Seamless chit-chat integration with
personalities like: Professional,
Friendly, Witty, Caring & Enthusiastic
No-code experience to
create a bot from KB
Scale as needed

TEXT ANALYTICS
Supports basic NLP scenarios
Up to 1000 1K docs per request
Call directly from Power BI & Data Lake
U-SQL; SDK supports .NET Standard +
Android
Not PAYG: paid tier starts at $150 per
month with 100K txns
Example
▪ I had a wonderful experience! The rooms
were wonderful and the staff was helpful.
"name": "English"
"score": 0.99979335069656372
wonderful experience",
"staff",
"rooms"

TRANSLATOR (TEXT)
Cloud-based open API service
Supports 60+ languages
Powers Bing, Edge, mobile apps, Office,
Yammer, VS, Skype, eBay, Etsy, HP, …
XML-based API ; supports detect, bulk
operations
Pay-per-character (2M free)
Cet outil de traduction est
un moyen facile d'étendre
la portée de vos contenus
internes et externes.
Client app or webpage
此翻译工具是可以轻松
地扩展您的内部和外部
内容的覆盖范围。
WEB API

MULTILINGUAL QNA CHATBOT
User can talk to bot in more than 70 languages
Bot translates message into his native language
Bot requests QnA Maker with translated question
which may be stored on a website or in
SharePoint Online and re-translates it into user‘s
language
Channels
Azure Bot
Service
Translator Text QnA Maker
Website/
SharePoint Site

CONVERSATIONAL CHATBOT FOR HOTEL
RESERVATIONS ON AZURE
Viewing a restaurant take-out
menu and ordering food
Checking hotel availability
and reserving a room
Searching available photos
and ordering prints

Situation: Solution: Impact:
“Our digital innovation platform built on Microsoft Azure helps us deliver the best
possible fan experiences for the world’s best sports league.”
LaLiga has hundreds of millions of
global fans, and the league is
always seeking new ways to keep
them engaged and entertained. To
take advantage of the popularity of
voice services, LaLiga wanted to
create a conversational AI assistant.
LaLiga created a digital
innovation platform with
Microsoft Azure services, and it
built the LaLiga virtual assistant
using Azure Cognitive Services
to give fans a multichannel,
voice-based source of league
and player information.
Fans can now engage even more deeply
with LaLiga by just asking questions using
their mobile phones. The success of the
digital innovation platform is driving the
league to explore exciting new
technologies like AI and augmented reality
to make its well-loved brand an even
bigger global phenomenon.
Customer:
LaLiga
Industry:
Media and entertainment
Size:
581 employees
Country:
Spain
Products and services:
Microsoft Azure
Azure App Service
Azure Bot Service
Azure Cache for Redis
Azure SQL Database
Microsoft Bot Framework
Read full story here
—Jose Carlos Franco, Head of Data & Analytics, LaLiga

HOW TO IMPLEMENT A CHATBOT
BRAINSTORMING
MAKE IT YOUR BRAND
MOCKUP
CONVERSATIONS
TEST / ITERATION /
CONTINOUS
IMPROVEMENT
ITERATIONUSE CASE & PERSONA
DEFINITION
6,6
3,8 4,1
9,7
3,6
FEEDBACK FROM USERS
PROTOTYPING

TYPICAL PROJECT PLAN
LIFT-OFF
Kickoff
Mockup
Prototyping
Releaseof
prototype
Feedback-
iteration#1
Test phase
Implementation
feedback#11
Feedback-
iteration#2
Implementation
feedback#11
Releaseofbot

Awesome SDKs
Rich tool set
Great samples, templates,
solutions and docs
https://github.com/Microsoft/botbuilder-tools
https://github.com/Microsoft/botbuilder-samples
https://github.com/Microsoft/AI/Solutions
COMPREHENSIVE BOT DEVELOPMENT

THE LIFECYCLE OF YOUR PROJECT
Comprehensive framework for the lifecycle of your project
Plan Build Test Publish Connect Evaluate
Author Dialogs
Design Cards
Visualize
Bot Builder SDK
Cognitive Services
Getting Started
Templates
Emulator
Application
Insights
Conversation
Transcripts
DevOps Tools
Azure
BYO Web Host
WebChat
Cortana
Skype, FB
+
Custom
(DirectLine)
Application
Insights
LUIS
Suggestions

DEVELOPMENT LIFECYCLE: PLAN

user=vishwac
bot=cafebot
user:What are your locations
bot:We have Cafe locations in Redmond,
Seattle, Renton and Bellevue
user:who is your CEO
bot:Joshua Newmann is the CEO of Contoso Cafe
user:Are you hiring
bot:You bet, we always are. Please visit
http://contosocafe.com/careers
CHATDOWN
Mocking Conversation

DEVELOPMENT LIFECYCLE: BUILD

TEMPLATES AND SOLUTIONS
To help more businesses and customers succeed with
great conversational experiences, we are creating a
series of Templates to greatly simplify getting started
and Solutions to help enable complex new scenarios.
Bringing together all of the best practices and
supporting components we've identified through
building the first wave of Conversational Experiences.
Templates
• Basic
• Core
• Echo
Solutions
• Virtual Assistant

TEMPLATES AND SOLUTIONS
5 minutes to a working end-to-end conversational experience
• Introduction message with Adaptive Cards
• Automated typing indicators
• .bot file driven configuration
• Example Dialogs
• Basic conversational intents in multiple languages
• Example responses abstracted into separate View classes.
• Inappropriate / PII detection with Content Moderator
• Transcripts of all conversations stored in Azure Storage
• QnAMaker Integration
• Integrated Dispatcher for LUIS and QnA targets
• Application Insights and PowerBI for Conversation Analytics
• Authentication
• Complete automated deployment of all aspects
• Open Sourced and updated regularly.

Modeling: language understanding
Creating: language understanding
# Greeting
- Hi
- Hello
- Good morning
# BookFlight
- book a flight from {fromCity=new york} to
{toCity=seattle}
### ? What can you do?
```markdown
I can greet and provide help
```
# Import LUIS model
luis import version --in myapp.json
# Train our LUIS model from LuDown
luis train version
# Publish the luis model
luis publish version
# Ask for a prediction
luis query -q <querytext> --region <region>
LUIS QnALanguage UnderstandingLANGUAGE UNDERSTANDING

LANGUAGE GENERATION
Create sophisticated responses
• Dynamic and randomize bot responses (avoid uncomfortable repetition)
• Use bot memory (entities)
• Template driven
• Conditional responses
DevOps
• Separate content from code and business logic and development lifecycle
• Make changes to bot’s responses without the need to redeploy the entire bot
• Enable non-dev personas to author bot responses
• Localization
Solution
• Standard, human readable, declarative file format representing language generation
• Built in support for template (including template nesting), binding to bot memory,
Conditional responses using a common expression language
Customer’s asks for bot response generation

LANGUAGE GENERATION
Example of a template that includes
composition reference to another template
# GreetingReply
- [GreetingPrefix], [timeOfDayGreeting]
# GreetingPrefix
- Hi
- Hello
# timeOfDayGreeting
- IF: {timeOfDay == 'morning'}
- good morning
- ELSEIF: {timeOfDay == 'afternoon'}
- good afternoon
- ELSE:
- good evening
Generating Sophisticated Responses
Hi, good morning
Hi, good afternoon
Hi, good evening
Hello, good morning
Hello, good afternoon
Hello, good evening

DEVELOPMENT LIFECYCLE: TEST

BOT FRAMEWORK
EMULATOR
• Test and debug bots
locally and in cloud
• View and manage bot’s
services
• Built in inspectors for
LUIS and QnA
• Train and publish directly
from emulator
• Extensible plugins for
inspectors

DEVELOPMENT LIFECYCLE: PUBLISH

PUBLISH VIA CLI
You can use the Azure CLI
tool to create, download
and publish your bot to
Azure Bot Service.
> az bot –h
Group
az bot: Manage Bot Services.
Commands:
create : Create a new Bot Service.
delete : Delete an existing Bot
Service.
download : Download an existing Bot
Service.
publish : Publish to an existing Bot
Service.
show : Get an existing Bot Service.
update : Update an existing Bot
Service.

DEVELOPMENT LIFECYCLE: CONNECT

Now supporting 20+ Channels!
What is new
Amazon Alexa • Google Assistant
Direct Line Speech (preview) • WebChat Adapter
Direct Line for Vnets
SUPPORTED CHANNELS

CONNECT BOT TO CHANNELS VIA AZURE CLI
Create bots and manage
channels
> az bot -h
Group
az bot: Manage Bot Services.
Subgroups:
directline: Manage Directline Channel on a Bot.
email : Manage Email Channel on a Bot.
facebook : Manage Facebook Channel on a Bot.
kik : Manage Kik Channel on a Bot.
msteams : Manage Msteams Channel on a Bot.
skype : Manage Skype Channel on a Bot.
slack : Manage Slack Channel on a Bot.
sms : Manage Sms Channel on a Bot.
telegram : Manage Telegram Channel on a Bot.
webchat : Manage Webchat Channel on a Bot.
Commands:
create : Create a new Bot Service.
delete : Delete an existing Bot Service.
download : Download an existing Bot Service.
publish : Publish to an existing Bot Service.
show : Get an existing Bot Service.
update : Update an existing Bot Service.

DEVELOPMENT LIFECYCLE: EVALUATE

EVALUATE
Built-In Bot Analytics
Application Insights

CONTINUOUS IMPROVEMENT
Instrumentation provided by
App Insights; added by default
in Bot Service
Extend instrumentation through
AI SDK
If you are not building bots
that will actually be used,
then you are not building
bots

HANDS ON/CHALLENGE
Start from https://qnamaker.ai & create a QnA bot with the
following URL to get QnA pairs from:
▪ http://bit.ly/nacs2019
Connect the bot to Microsoft Teams and chat with the bot
in Teams

FEEDBACK
http://bit.ly/AITrendwerkstatt2
Please fill out

Solvion Trendwerkstatt - Microsoft Azure + Bots

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (14)

Similar a Solvion Trendwerkstatt - Microsoft Azure + Bots

Similar a Solvion Trendwerkstatt - Microsoft Azure + Bots (20)

Último

Último (20)

Solvion Trendwerkstatt - Microsoft Azure + Bots