This was presented at RIGA COMM 2020.
Talk about what is Machine Teaching and how it can be used in task automation scenarios.
Machine Teaching is a new paradigm and approach on how to use ML/AI to solve highly specialized tasks that would normally require human validation. Creating workflow with a human in the loop that continuously improves ML model while also carefully curating high quality data.
4. Emergn’s Machine
Learning Lab
Our team's core competencies:
• Business understanding of how to profit
from ML
• Machine learning development
• Deep learning and reinforcement learning
• Data visualization
• Algorithms and ML techniques
• Data processing, cleaning and preparation
OUR TECHNOLOGY STACK
We have established and run the largest ML community
in the Baltics with 1260 field experts as members.
All these activities currently makes our company the
No. 1 Choice for young and experience ML talents.
Machine learning models
Tools and programming languages:
• Python
• TensorFlow, TensorBoard
• R ,OpenCV, Caffe2
• KNIME
• Azure Machine Learning
Studio
• C, C++
Deployment techniques
Platforms and environments:
• Stand alone models
• SAP Hana2
• Microsoft Azure (Cortana
intelligences suite)
• SQL Server
We are partnering with GDEXA to help enable the young
generation with highly demanded skills like applied
AI/ML, Big Data Analytics and Cloud Applications.
6. Business automation
challenge
From automation with replacement of
humans to augmentation and
empowerment of subject matter experts.
We predict that companies who use
augmented automation technologies to
empower their environments and educate
their people on how to use them for better,
more predictable outcomes, will win by
providing the best service and building
better products.
• Wheels for the mind
• Find a comfortable level of automation
7. Why are we looking for
machine learning
(ML) alternatives?
ML for automation and
workflows should:
• Be transparent and interactive for
business users
• Include natural language-based
solutions where humans have better
comprehension
• Understand context and learning
from smaller data sets
• ML models should be verified and
monitored
AI director at
Facebook
YANN LECUN
ATARI GAME
Self-driving cars need
millions of hours of
training to reach
human level trained in
about 20 hours.
In 80 hours machine
will reach human level
aquired by 15 minutes.
8. Change of paradigm
Key differences
• Role of subject matter expert (SME) changes – using our tool, SME trains/provides the logic to improve the model.
• SME is integral to the success, needs to be empowered and have the tools to do their work better.
• The "one and done" approach is not flexible/doesn't allow for market changes nor incremental knowledge.
MACHINE TEACHING
MACHINE LEARNING
9. Iterative machine teaching process
For use cases such as:
• Email/text/document
classification
• Email/text/document
anonymization
• Entity extraction
SME
DATA SCIENTIST
Initial
model
training
Models could be
regularly monitored
by SME
AUTOMATION
WORKFLOW
CLASSIFICATION
MODEL
10. Tools for model quality inspection
Machine Teaching
administration tools
help business users
and ML Power Users
control classification
model quality.
12. Finding quick wins
ROUTINE OPERATIONS
• RPA Robots
• Machine Learning
• OCR + Data Extraction
REPETITIVE
COGNITIVE TASKS
COLLABORATION
WORKFLOWS
• Machine Learning
• Rule Engines • Interactive applications
• Workflows
• Data Enrichment
DECISION MAKING
AND SUPPORT
• AI
• Process Mining
• Document classification
• Automation translation
• Collaboration apps
• Approvals
• Case management
• Prioritization of work
• Analysis
• Data Extraction
• Copy data
• Enter data
• Sort documents
FREE UP PEOPLE TIME SUPPORT SHIFT TO DIGITAL OPERATING MODEL
TASK AUTOMATION WORKFLOW AUTOMATION DECISION AUTOMATION
• Machine Learning
Natural
Language
based
use-cases
13. Automation of the document flow
DOCUMENT LIFE-CYCLE
RECEIVE
DOCUMENT
PROCESS
DOCUMENT
ARCHIVING
• Highly manual
• Need decision making, involving knowledge worker to do manual tasks
• Text comprehension (SME)
• Fraught with errors
• Difficult to research/go back, to find things
• Time consuming
• Not possible when scale is large
14. Automation of the document flow
DOCUMENT LIFE-CYCLE
RECEIVE
DOCUMENT
SAVE
DOCUMENT
EXTRA
META DATA
Document and
form recognition
using OCR
• Be physical
document or
email or video
any format can
be input
DOCUMENT
ROUTING
DOCUMENT
PUBLISHING
ARCHIVING
Meta-data
extraction
• Key fields
• Key words
• In text
• e.g. PO #
• Subject
• Amount
Document
classification
and routing e.g.
• Classification
• Invoice routing
• Approval,
level, direct
forward to
finance
Document
anonymization
and masking e.g.
• Bank and
personal info
Document
archiving by
classification e.g.
• Archiving based
on classification: in
cloud or on-site
HOT WARM COLD
15. Information extraction from documents
CLIENT RECORD
CLIENT RELATED
DOCUMENTS AND KEYWORDS DATA
EXTRACTION
MODEL
*Client information is
automatically enriched.
Company: BNP Paribas*
Title:
Name: Tom Barnes*
Other information:
Meta data
Retrieving information from text and documents helps you to obtain data
that can and must supplement the information of certain accounting cards.
DATA
EXTRACTION
CONFIG
16. Document classification
Email
• By title
• By object
• By action
Mail
• A
• B
• C
Contract
Application
Invoice
CLASS CATALOG
The document class is
determined by the
organizational
documents/text
classification catalog
Training of document class catalogues and document
classification algorithm by business user – specialist
The classification of documents results in the identification of the document class and can use it to
further process the document, usually routing and storing it in an appropriate document process.
DOCUMENT
CLASSIFICATION
MODEL
DOCUMENT
CLASS
17. Document anonymization
The purpose of anonymizing documents/text is to cover all or part
of sensitive information prior to publication of the document/text.
Person 1, social security number,
living, Address 1, closed contract
Contact Number, with Company
ABC for Contract Title
DATA
MASKING
George Bennett,
Social security number
989384843*****, address,
***** street *
DOCUMENT
ANONYMIZATION
ANONYMIZATED
DOCUMENT
ANONYMIZATED
TEXT
ANONYMIZATION
CONFIGURATION
ANONYMIZATED
DOCUMENT
18. Machine
Teaching (MT)
Tool
Machine
Learning (ML)
Solution
Technical architecture
SET UP AS AN INTERNAL TOOL
ON-PREMISE SERVICE
MODULAR STRUCTURE
SME/IT:
• MT Tool with UI for training & analytics
• Custom integrations possible
• Endpoints for production scenario provided
Data Scientist:
• ML model exchangeable - has standardized endpoints
• Data and integration – as needed
PYTHON
CAN BE INTEGRATED INTO EXISTING WORKFLOW MODEL
MT DATABASE
MODEL DATA
ML SOLUTION
MT SERVICE
CUSTOM
INTEGRATION &
PRODUCTION
MT TOOL
21. How to get most value from machine
teaching for Natural Language
Processing (NLP) and Non-NLP models
IN NLP MODELS
• Objective – Create
Optimal data set for best
known optimal models
• Use for data labeling
and data preparation
IN NON-NLP MODELS
• Objective – Find
optimal/best model with
existing data
• Use for model monitoring
and verification
Machine Learning is the subfield of
computer science that, according to Arthur
Samuel in 1959, gives “computers the ability
to learn without being explicitly programmed.”
NLP
Artificial
Intelligence
Machine
Learning
Statistics
22. Let's stay connected
Contact us at info@emergn.com
for an individual demo.
Follow us: @emergn
MUNTIS RUDZITIS
Lead Data Scientist
at Emergn
https://www.linkedin.com/in/muntis-rudzitis/
ARIADNA KRAMKOVSKA
Machine Learning Developer
at Emergn
https://www.linkedin.com/in/ariadnakramkovska/
