Presented in RIGA COMM 2023 In this presentation I briefly cover how I see and try to systematize work, what parts of it can be helped by using modern ML/AI and algorithmic solutions. Then I show some research work and publications we have done regarding Intelligent workplace concept - a knowledge work environment that tries to help to solve repeatable tasks. Along the way I show whats possible and whats limited by current ML/AI capabilities, our lessons learned and some tips.

1. Building Intelligent
Workplace:
Limits and Challenges
Lessons learned from ML / AI enabled
solution for better workplace
RIGA COMM 2023

2. Speaker
2
Muntis Rudzitis
LEAD DATA SCIENTIST
EMERGN
Bio:
• Machine Learning Lab
• Technical architect for AI+ML,
data enabled solutions
• Experimentation, prototyping
• Python, Azure, Power BI
• Speaker

3. • Intro
• Intelligent Workplace
• ML tasks & Challenges
• Lessons learned
3

4. Intelligent Workplace
• R&D work done in field of text analytics and processing
- Notification analysis
- Next-best activity
- Meeting analysis
- Similar tasks
• https://www.emergn.com/thought-papers/insights-on-intelligent-
automation-of-knowledge-work/
• https://www.emergn.com/insights/smarter-approach-notifications-ml-ai/
• https://www.emergn.com/insights/why-process-mining-is-the-top-skill-
to-learn-for-business-analysts-in-2022/
4

5. The knowledge work spectrum
5
TASK
EXECUTION
WORKFLOW
MANAGEMENT
DECISION-
MAKING
Copy
Enter
Sort
Translate
Classify
Extract
Prioritize
Analyze
Decide
With typical activities

6. Work area
6
• Of the different classes of demand related to
developing and then running a product or service, the
Intelligent Workplace is aimed at Operations Demand.
• This kind of work is usually known ahead of time – or is
knowable.
• It could, however, be planned or unplanned.
• Examples of places where this kind of work happens:
• Help Desk
• Call Center
• Warehouse
• A shop
QUITE A BIT OF CERTAINTY
Work structure

7. Problem we are solving: Goal Alignment
What should the focus be?
Is the work easy or hard?
How are we using capacity?
Objectives and goals
(Top-Down)
Smart working environment
Contextual work items
(Bottom-up)
Managers
Workers

8. Intelligent Workplace
concept
I
I
I
I
Insights available (reusable knowledge)
15 min 245
2 days
ago
Average time
per task
Similar tasks
done this
week
Last time you
did similar
task
Description available
Colegues to
ask about it:
• George
• Fred
• Austris
Workarounds:
• 1
• 2
• 3
Write feedback
Best/worst case:
• ACME
• Ministry of Finance
• If it is unique – most problably no
knowledge there
• If it is repeating – there are at least
statistics + insights could be collected
• On-prem
+ When it is usualy done: 9:00 – 12:00

9. Intelligent Workplace – visualize work
Insights available (reusable knowledge)
15 min 245
2 days
ago
Average time
per task
Similar tasks
done this
week
Last time you
did similar
task
Description available
Colegues to
ask about it:
• George
• Fred
• Austris
Workarounds:
• 1
• 2
• 3
Write feedback
Best/worst case:
• ACME
• Ministry of Finance
Work done – New type of analytics
5000 245
Number of
taks done
today
Task 1
Task 2
Task 3
Tasks with feedback
Score A

10. • Task list view
• Prioritized
• Aggregated
• Intelligent context sensitive help
Source system task
metadata
Task-level
recommendations
(similar tasks, known
standards or solutions)
Topic level help with
SME/colleague contacts
Analytics, statistics and
estimates
Intelligent Workplace interface:
Worker’s View

11. • Statistics
• Task information
• Similar tasks
• Who can help
• Past work examples
• Reviews
• Workarounds and additional materials
Analytical insights for the Worker

12. Intelligent Workplace interface:
Manager’s View
• Visualizing work
• Structure knowledge
• Improve process
• Day-to-day progress and problems
• Plan ahead based on analytics
From "people" manager to work
manager/coordinator.
Role of an operator, observer, coordinator,
productivity enabler.

13. • Intro
• Intelligent Workplace
• ML tasks & Challenges
• Lessons learned
13

14. • Statistics
• Task information
• Similar tasks
• Who can help
• Past work examples
• Reviews
• Workarounds and additional materials
• Overall goals to improve productivity through knowledge reuse/accumulation:
• Reduce the number of unique tasks
• Apply knowledge to optimize repetitive tasks
• Reuse knowledge for repetitive tasks
Tasks to solve

15. Task similarity
• Task similarity is a tricky
• In defined processes:
• point of reference
• known steps and context
• potential solution overlap
• In undefined process:
• no clear point of reference in process
• very varied and significantly differ even when looking
similar at high level
• context often overshadows the tasks underneath;
goals can end up needing similar steps to complete
15
TASK
EXECUTION
WORKFLOW
MANAGEMENT
DECISION-
MAKING

16. Task similarity: Concept
• Knowledge/support work or its artefacts are
often semi-structured/unstructured.
• Key aspect here – find distance/similarity
metric for tasks (or people)
• Some of the metrics we experimented with:
- Average time per task
- Estimated task time
- Last time you did a similar task
- Colleagues to ask about the task
- Similar tasks
- Best / worst cases
16
IWS
Description
Time
Person
Documents
Forms
Insights,
recommendations,
help, productivity
...

17. Task similarity: Data Source
Dataset:
• Jira dataset
• Real data
• 66 Open-source Apache projects (used only select few)
• Each project has about 2000 tickets
• Feature set is closer to reality
• Calculated metrics are more believable
• Experimented with local Ops dataset
https://zenodo.org/record/3942332

18. Task similarity: Solution
18
IDSummary Near_summary Near_ID distance
12507134Create a LOGO for Airavata Project Airavata Logo Drafts 12901695 0.7933585
12507134Create a LOGO for Airavata Project
Improve Logo and Banner text
for Airavata 12653988 0.7009898
12507134Create a LOGO for Airavata Project Airavata Designer Guide 12675281 0.639134
12507134Create a LOGO for Airavata Project
Update Airavata Logo Website
Banners and Branding 12895578 0.626145
...... ... .. 0.6241394

19. 19

20. Task similarity: Clustermap
20

21. Task similarity: results & Lessons
learned
21
• Limit the context & scope
• Repeatable – so that results overlap & help
• Data evolves over time
• Useful but can't 100% rely on
Writing style variance Lessons learned
Different
people
Different
teams
Different
projects
… …
Different
technologies
Writing
style
variance

22. Using:
• Neural Networks
• Support Vector Machines
• Random Forests
22
Major
Critical
Minor
63%
Achieved accuracy
Issue priority / story points

23. 23
Error distribution Confusion matrix for test
https://arxiv.org/pdf/1609.00489.pdf
Issue priority / story points

24. Lessons learned
• Uneven data scale
• Initially easy/fast on high level for humans,
when doing top-down
• Difficulty often comes form technical nuances,
requires data from different contexts to
compare
• Data is fairly private
24
Lessons learned Confusion matrix for test

25. Learning from mistakes
aka “what would I do differently Today?”
• Data availability: lots of Dev not Ops
- Problem framing
• Flip the problem:
- Codex, Llama 2 (CoPilot, ChatGPT, Code Llama, LangChain) etc.
- Enrich data while its fresh
- Enrich data when its finished
• Better local models availability
• Limit scope differently
• Worth revisit, proven success in similar tasks
25

26. 26

27. Muntis Rudzitis
Muntis.Rudzitis@emergn.com
www.linkedin.com/in/muntis-rudzitis