Presented in RIGA COMM 2023
In this presentation I briefly cover how I see and try to systematize work, what parts of it can be helped by using modern ML/AI and algorithmic solutions.
Then I show some research work and publications we have done regarding Intelligent workplace concept - a knowledge work environment that tries to help to solve repeatable tasks.
Along the way I show whats possible and whats limited by current ML/AI capabilities, our lessons learned and some tips.
4. Intelligent Workplace
• R&D work done in field of text analytics and processing
- Notification analysis
- Next-best activity
- Meeting analysis
- Similar tasks
• https://www.emergn.com/thought-papers/insights-on-intelligent-
automation-of-knowledge-work/
• https://www.emergn.com/insights/smarter-approach-notifications-ml-ai/
• https://www.emergn.com/insights/why-process-mining-is-the-top-skill-
to-learn-for-business-analysts-in-2022/
4
5. The knowledge work spectrum
5
TASK
EXECUTION
WORKFLOW
MANAGEMENT
DECISION-
MAKING
Copy
Enter
Sort
Translate
Classify
Extract
Prioritize
Analyze
Decide
With typical activities
6. Work area
6
• Of the different classes of demand related to
developing and then running a product or service, the
Intelligent Workplace is aimed at Operations Demand.
• This kind of work is usually known ahead of time – or is
knowable.
• It could, however, be planned or unplanned.
• Examples of places where this kind of work happens:
• Help Desk
• Call Center
• Warehouse
• A shop
QUITE A BIT OF CERTAINTY
Work structure
7. Problem we are solving: Goal Alignment
What should the focus be?
Is the work easy or hard?
How are we using capacity?
Objectives and goals
(Top-Down)
Smart working environment
Contextual work items
(Bottom-up)
Managers
Workers
8. Intelligent Workplace
concept
I
I
I
I
Insights available (reusable knowledge)
15 min 245
2 days
ago
Average time
per task
Similar tasks
done this
week
Last time you
did similar
task
Description available
Colegues to
ask about it:
• George
• Fred
• Austris
Workarounds:
• 1
• 2
• 3
Write feedback
Best/worst case:
• ACME
• Ministry of Finance
• If it is unique – most problably no
knowledge there
• If it is repeating – there are at least
statistics + insights could be collected
• On-prem
+ When it is usualy done: 9:00 – 12:00
9. Intelligent Workplace – visualize work
Insights available (reusable knowledge)
15 min 245
2 days
ago
Average time
per task
Similar tasks
done this
week
Last time you
did similar
task
Description available
Colegues to
ask about it:
• George
• Fred
• Austris
Workarounds:
• 1
• 2
• 3
Write feedback
Best/worst case:
• ACME
• Ministry of Finance
Work done – New type of analytics
5000 245
Number of
taks done
today
Task 1
Task 2
Task 3
Tasks with feedback
Score A
10. • Task list view
• Prioritized
• Aggregated
• Intelligent context sensitive help
Source system task
metadata
Task-level
recommendations
(similar tasks, known
standards or solutions)
Topic level help with
SME/colleague contacts
Analytics, statistics and
estimates
Intelligent Workplace interface:
Worker’s View
11. • Statistics
• Task information
• Similar tasks
• Who can help
• Past work examples
• Reviews
• Workarounds and additional materials
Analytical insights for the Worker
12. Intelligent Workplace interface:
Manager’s View
• Visualizing work
• Structure knowledge
• Improve process
• Day-to-day progress and problems
• Plan ahead based on analytics
From "people" manager to work
manager/coordinator.
Role of an operator, observer, coordinator,
productivity enabler.
14. • Statistics
• Task information
• Similar tasks
• Who can help
• Past work examples
• Reviews
• Workarounds and additional materials
• Overall goals to improve productivity through knowledge reuse/accumulation:
• Reduce the number of unique tasks
• Apply knowledge to optimize repetitive tasks
• Reuse knowledge for repetitive tasks
Tasks to solve
15. Task similarity
• Task similarity is a tricky
• In defined processes:
• point of reference
• known steps and context
• potential solution overlap
• In undefined process:
• no clear point of reference in process
• very varied and significantly differ even when looking
similar at high level
• context often overshadows the tasks underneath;
goals can end up needing similar steps to complete
15
TASK
EXECUTION
WORKFLOW
MANAGEMENT
DECISION-
MAKING
16. Task similarity: Concept
• Knowledge/support work or its artefacts are
often semi-structured/unstructured.
• Key aspect here – find distance/similarity
metric for tasks (or people)
• Some of the metrics we experimented with:
- Average time per task
- Estimated task time
- Last time you did a similar task
- Colleagues to ask about the task
- Similar tasks
- Best / worst cases
16
IWS
Description
Time
Person
Documents
Forms
Insights,
recommendations,
help, productivity
...
17. Task similarity: Data Source
Dataset:
• Jira dataset
• Real data
• 66 Open-source Apache projects (used only select few)
• Each project has about 2000 tickets
• Feature set is closer to reality
• Calculated metrics are more believable
• Experimented with local Ops dataset
https://zenodo.org/record/3942332
18. Task similarity: Solution
18
IDSummary Near_summary Near_ID distance
12507134Create a LOGO for Airavata Project Airavata Logo Drafts 12901695 0.7933585
12507134Create a LOGO for Airavata Project
Improve Logo and Banner text
for Airavata 12653988 0.7009898
12507134Create a LOGO for Airavata Project Airavata Designer Guide 12675281 0.639134
12507134Create a LOGO for Airavata Project
Update Airavata Logo Website
Banners and Branding 12895578 0.626145
...... ... .. 0.6241394
21. Task similarity: results & Lessons
learned
21
• Limit the context & scope
• Repeatable – so that results overlap & help
• Data evolves over time
• Useful but can't 100% rely on
Writing style variance Lessons learned
Different
people
Different
teams
Different
projects
… …
Different
technologies
Writing
style
variance
22. Using:
• Neural Networks
• Support Vector Machines
• Random Forests
22
Major
Critical
Minor
63%
Achieved accuracy
Issue priority / story points
24. Lessons learned
• Uneven data scale
• Initially easy/fast on high level for humans,
when doing top-down
• Difficulty often comes form technical nuances,
requires data from different contexts to
compare
• Data is fairly private
24
Lessons learned Confusion matrix for test
25. Learning from mistakes
aka “what would I do differently Today?”
• Data availability: lots of Dev not Ops
- Problem framing
• Flip the problem:
- Codex, Llama 2 (CoPilot, ChatGPT, Code Llama, LangChain) etc.
- Enrich data while its fresh
- Enrich data when its finished
• Better local models availability
• Limit scope differently
• Worth revisit, proven success in similar tasks
25
Done research in area
If you follow us on social media
To paint the landscape..
How we see work - Knowledge work spectrum
Part of the work is processing, (in Black) - Reading documents, writing Emails, Book meetings / zoom
Work that does not need much of the education or skillsets.
And then there is the knowledge work (in Red) - requires, discussion, analysis, collaboration and decision. Like making this PPT. More time per task.
(RE)USE vs CREATE
Teacher reference
We consider both these types of work is the way to create better experiences and better products.
Leaves DATA TRAIL
More on left, more data. Better ML.
Regulatory areas have processes, easier to base yourself
Insurance, government etc
For manager to help
From manager to operator, observer, coordinator
Covey, Eisenhower popularized Urgent/Not Urgent and Important/Not important work seperation/prioritization
Not talking about big data
UNDERSTANDING WORK
UNDERSTANDING DATA
DATA LEAVES FOOTPRINT
Task similarity is tricky metric to calculate
OPS
Development
For the undefined process, the tasks to do can be very varied and significantly differ even when doing similar looking task at high level.
https://ai.googleblog.com/2018/05/advances-in-semantic-textual-similarity.html
https://tfhub.dev/google/universal-sentence-encoder/4
And what do you get as an output?
Cant really show the data..
Limit the context & scope (picture)
Repeatable – so that results overlap & help, otherwise very little help, so Ops better
Data evolves over time – tickets get more data, status, time completed etc
Useful but can't 100% rely on
successfully used in other use cases like survey analysis
ServiceDesk, Jira etc – smart plugins and features appearing
MS Outlook showing relevant docs for meetings
