SlideShare a Scribd company logo
1 of 50
Download to read offline
May 7, 2024
UiPath DEV meet-up
UiPath Document Understanding
2
Dion Morskieft
RPA Product Owner at
Partou
Mikal van Eijk
Automation Developer at
Tacstone Technology
Jack Klein Schiphorst
Automation Developer at
Tacstone Technology
Let’s connect!
3
• Introduction
• Document Understanding
• SPIE Building Solutions use case
• UWV use case
• Lessons learned
Program
Tacstone is a consulting and venture boutique founded in 2006 as a
spin-off from a big-4 consulting firm
Technology division established in 2018
In 2024 we operate with a team of 40+ automation experts serving 60+
clients in the Netherlands and abroad
About Tacstone Technology
Document
Understanding
TheUiPathBusinessAutomationPlatform
PROCESS MINING TASK MINING COMMUNICATIONS MINING IDEA CAPTURE & MANAGEMENT
LOW-CODE DEVELOPMENT UI & API AUTOMATION​ PROCESS ORCHESTRATION
INTELLIGENT
DOCUMENT PROCESSING
INTEGRATED NLP & AI/ML
Discover
Continuously uncover opportunities for process and task
improvements ​helping you identify the highest ROI areas
Automate
Get more done with a digital workforce that seamlessly collaborates with
your people ​and automates work via UI and API, powered with native
integrated AI​
Your Applications Your
People
Your
Processes Systems of record – ERP, CRM, HCM | Communications and Collaboration | Personal
Productivity​
ANALYTICS CONTINUOUS TESTING
UNIFIED MANAGEMENT
& GOVERNANCE​
FLEXIBLE DEPLOYMENT
Operate
An enterprise-grade foundation to run and optimize
a ​mission critical automation program at high scale
9
• Like forms, passports, licenses, time
sheets
• Fixed in format and can contain
handwriting, signatures, checkboxes​
• Like invoices, receipts, purchase
orders, medical bills, utility bills
• Containing fixed and variable parts
like tables
• Like contracts, agreements, emails,
scripts, drug prescriptions, news
• No fixed format, free-form
sentences/paragraphs
Which documents can be handled by Document
Understanding?
Structured documents Semi-structured documents Unstructured documents
10
Data extraction – from rules to AI
to a hybrid approach
RegEx-Based Extractor
Structured fields
Machine Learning Extractor
Mostly semi-structured documents
Form Extractor
Structured documents (forms)
A combination of both –
rule-based and AI-based approaches
Mostly documents combining both structured and less structured formats
Rule-based
Forms AI
Structured documents (forms)
AI-based
Hybrid approach
11
Pre-trained models
Machine learning (ML) models
as a template-less approach
Custom models
• No-code light-weight models in Forms AI
• Custom ML models in AI Center
• Third-party models
Model retraining
Learn about sharing data for model retraining here
• Invoices
• Receipts
• Purchase Orders
• Utility Bills
• Passports
• ID Cards*
• Legal Contracts
• W-2 Forms
• W-9 Forms
• Delivery Notes
• Remittance
Advices
• ACORD 125
• I9 Forms
• 990 Forms
• 4506T Forms
• FM1003 Forms
• Pay slips & personal
earnings statements
• Certificates of origin
• EU declarations of
conformity
• Children’s product
certificates
• Certificates of
incorporation
• Shipping invoices
• CMS1500
• Retraining via AI Center
• Continuous learning loop based on human validated data
SPIE Use Case
13
SPIE Building Solutions
Efficient
buildings
Smart city Energies
14
15
8 May 2024 15
16
8 May 2024 16
17
17
18
Every customer has its own template
18
19
Every customer has its own template
19
Prio
Description
Location
Subject
Priority
Address
Building
Description
SLA
Solution
21
• Our developers were needed to add
new templates
• Complex logic and code
• Not flexible to adjustments
Downsides
Initial rule-based regex solution
22
Final Forms AI Solution
Forms AI combines the ease of using
templates with the benefits of
continuously improving AI in a
no-code approach to structured
document processing:
1. Upload 2-5 forms
2. Define fields for one document
3. Test out the automatically
generated model
4. Publish and use the model
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
Categorization with DataService
• Use the correct Forms AI endpoint
• Only send relevant data to the AI model
• Fill in pre-defined values
40
Send data to SAP
• Convert extraction results to JSON
• Use SAP Odata API
• Crete SAP ticket based on Service request
41
Lessons Learned
43
Lessons Learned
Don't hesitate, just start.
Choose the correct AI model
Be consistent with labeling
Get enough data
Involve the Business
Generic is not always the best
vertrouwelijk
Dion Morskieft
07 mei 2024
Document
Understanding
at Partou
UWV
07 mei 2024
Document Understanding at Partou vertrouwelijk
Who am I?
• Dion Morskieft
• At Partou for 14 years
• 7 years RPA Developer
My team?
• Team consisting of 3 RPA developers and 5 BI specialists
• For each department robots/optimized processes are implemented
• Over 120 processes start 200+ robots daily
• Workforce reduction of 25-30 FTEs
Introduction
45
07 mei 2024
Document Understanding at Partou vertrouwelijk
46
https://www.kidsfoundation.nl/nl/wp-content/themes/kidsfoundation/assets/images/logo.png
https://www.kidsfoundation.nl/nl/wp-content/uploads/lable-smallsteps-2.png
07 mei 2024
Document Understanding at Partou vertrouwelijk
47
900+ LOCATIONS IN
THREE COUNTRIES
80.000+ CHILDREN 12.000+ COLLEAGUES
Document Understanding
48
07 mei 2024
Document Understanding at Partou vertrouwelijk
• 60 letters a day
• Lots of manual labour (over 5 minutes per letter)
• Good process for document understanding
What did we do?
• Document and optimize the process
• Two separate flows with different queues
• Regex
• Document Understanding
• One DU-model with multiple training and evaluations sets
• Communicating with business through Action Center and Teams
Why this proces?
Letters UWV
49
07 mei 2024
Document Understanding at Partou vertrouwelijk
50
1. Download enough letters for a reliable set
2. Anonymize the data
3. Create training and evaluation batches
4. Label the documents
5. Evaluate the model
6. Add more data and retrain if needed
I can show you in the AI-center
How did we train the model
07 mei 2024
Document Understanding at Partou vertrouwelijk
1. Let the robot download the letters in the UWV-portal
2. Based on subject decide which letters follow what flow
3. Fill queue for the DU- and regex model (encrypt data)
4. Cut the first two pages of the letter and let the model get the data
5. When the DU-model is not certain enough:
1. Create a task in the Action Center
2. Send a Teams-message to the HR-Team
6. When the DU-model is certain enough (or task is completed)
1. Fill queue for processing letter
2. Based on Social Security Number find colleague in Afas/Profit
3. Add letter to personal-file
4. Add data from the letter to Afas/Insite
5. Start a workflow in Afas for HR based on subject of the letter
I can show you in Studio
How did we build the robot?
51
Volg ons:
partou.nl
pm.op.insta
facebook.com/partou.kinderopvang
linkedin.com/company/partou-bv

More Related Content

Similar to DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam

SharePoint Inspired 'Get more from your data with Office 365'
SharePoint Inspired 'Get more from your data with Office 365'SharePoint Inspired 'Get more from your data with Office 365'
SharePoint Inspired 'Get more from your data with Office 365'Xylos
 
WebXpress Business Intelligence Capability
WebXpress Business Intelligence CapabilityWebXpress Business Intelligence Capability
WebXpress Business Intelligence CapabilityWebXpress.IN
 
Convert manual paper-based business processes into automated paperless
Convert manual paper-based business processes into automated paperlessConvert manual paper-based business processes into automated paperless
Convert manual paper-based business processes into automated paperlessSmart ERP Solutions, Inc.
 
Making the most of your SharePoint – Intranet & Apps
Making the most of your SharePoint – Intranet & AppsMaking the most of your SharePoint – Intranet & Apps
Making the most of your SharePoint – Intranet & AppsAmos Wachanga
 
Transitioning to HR Tech: Leveraging Technology to Automate HR
Transitioning to HR Tech: Leveraging Technology to Automate HR Transitioning to HR Tech: Leveraging Technology to Automate HR
Transitioning to HR Tech: Leveraging Technology to Automate HR Naba Ahmed
 
Transitioning to HR Tech: Leveraging Technology to Automate HR
Transitioning to HR Tech: Leveraging Technology to Automate HRTransitioning to HR Tech: Leveraging Technology to Automate HR
Transitioning to HR Tech: Leveraging Technology to Automate HRAggregage
 
Intelligent Automation in Accounting and Finance with IMA Queens College Stud...
Intelligent Automation in Accounting and Finance with IMA Queens College Stud...Intelligent Automation in Accounting and Finance with IMA Queens College Stud...
Intelligent Automation in Accounting and Finance with IMA Queens College Stud...Diana Gray, MBA
 
Unlocking the Power of UiPath: A Journey into Document Understanding
Unlocking the Power of UiPath: A Journey into Document UnderstandingUnlocking the Power of UiPath: A Journey into Document Understanding
Unlocking the Power of UiPath: A Journey into Document UnderstandingDianaGray10
 
AI Orange Belt - Session 3
AI Orange Belt - Session 3AI Orange Belt - Session 3
AI Orange Belt - Session 3AI Black Belt
 
SPSChicagoBurbs 2019 - What is CDM and CDS?
SPSChicagoBurbs 2019 - What is CDM and CDS?SPSChicagoBurbs 2019 - What is CDM and CDS?
SPSChicagoBurbs 2019 - What is CDM and CDS?Nicolas Georgeault
 
Building a 360 Degree View of Your Customers on BICS
Building a 360 Degree View of Your Customers on BICSBuilding a 360 Degree View of Your Customers on BICS
Building a 360 Degree View of Your Customers on BICSPerficient, Inc.
 
Raed Ashour Profile
Raed Ashour ProfileRaed Ashour Profile
Raed Ashour Profilerashour
 
DEV Meet-Up Q2 2022 Amsterdam Slides.pdf
DEV Meet-Up Q2 2022 Amsterdam Slides.pdfDEV Meet-Up Q2 2022 Amsterdam Slides.pdf
DEV Meet-Up Q2 2022 Amsterdam Slides.pdfCristina Vidu
 
Ephesoft & Linux Webinar: Smart Capture™ is Now Even Smarter!
Ephesoft & Linux Webinar: Smart Capture™ is Now Even Smarter!Ephesoft & Linux Webinar: Smart Capture™ is Now Even Smarter!
Ephesoft & Linux Webinar: Smart Capture™ is Now Even Smarter!Zia Consulting
 
Migration to Oracle ERP Cloud: A must read winning recipe for all
Migration to Oracle ERP Cloud: A must read winning recipe for allMigration to Oracle ERP Cloud: A must read winning recipe for all
Migration to Oracle ERP Cloud: A must read winning recipe for allJim Pang
 
A case for salesforce.com for future technology professionals
A case for salesforce.com for future technology professionalsA case for salesforce.com for future technology professionals
A case for salesforce.com for future technology professionalsPrashanth Krishnamurthy
 
Become an RPA Citizen Developer
Become an RPA Citizen Developer Become an RPA Citizen Developer
Become an RPA Citizen Developer Diana Gray, MBA
 
Become an RPA Citizen Developer
Become an RPA Citizen Developer Become an RPA Citizen Developer
Become an RPA Citizen Developer DianaGray10
 
AXI & LoQutus: Getting the most out of Office 365
AXI & LoQutus: Getting the most out of Office 365AXI & LoQutus: Getting the most out of Office 365
AXI & LoQutus: Getting the most out of Office 365LoQutus
 
Executive Briefing: Why managing machines is harder than you think
Executive Briefing: Why managing machines is harder than you thinkExecutive Briefing: Why managing machines is harder than you think
Executive Briefing: Why managing machines is harder than you thinkPeter Skomoroch
 

Similar to DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam (20)

SharePoint Inspired 'Get more from your data with Office 365'
SharePoint Inspired 'Get more from your data with Office 365'SharePoint Inspired 'Get more from your data with Office 365'
SharePoint Inspired 'Get more from your data with Office 365'
 
WebXpress Business Intelligence Capability
WebXpress Business Intelligence CapabilityWebXpress Business Intelligence Capability
WebXpress Business Intelligence Capability
 
Convert manual paper-based business processes into automated paperless
Convert manual paper-based business processes into automated paperlessConvert manual paper-based business processes into automated paperless
Convert manual paper-based business processes into automated paperless
 
Making the most of your SharePoint – Intranet & Apps
Making the most of your SharePoint – Intranet & AppsMaking the most of your SharePoint – Intranet & Apps
Making the most of your SharePoint – Intranet & Apps
 
Transitioning to HR Tech: Leveraging Technology to Automate HR
Transitioning to HR Tech: Leveraging Technology to Automate HR Transitioning to HR Tech: Leveraging Technology to Automate HR
Transitioning to HR Tech: Leveraging Technology to Automate HR
 
Transitioning to HR Tech: Leveraging Technology to Automate HR
Transitioning to HR Tech: Leveraging Technology to Automate HRTransitioning to HR Tech: Leveraging Technology to Automate HR
Transitioning to HR Tech: Leveraging Technology to Automate HR
 
Intelligent Automation in Accounting and Finance with IMA Queens College Stud...
Intelligent Automation in Accounting and Finance with IMA Queens College Stud...Intelligent Automation in Accounting and Finance with IMA Queens College Stud...
Intelligent Automation in Accounting and Finance with IMA Queens College Stud...
 
Unlocking the Power of UiPath: A Journey into Document Understanding
Unlocking the Power of UiPath: A Journey into Document UnderstandingUnlocking the Power of UiPath: A Journey into Document Understanding
Unlocking the Power of UiPath: A Journey into Document Understanding
 
AI Orange Belt - Session 3
AI Orange Belt - Session 3AI Orange Belt - Session 3
AI Orange Belt - Session 3
 
SPSChicagoBurbs 2019 - What is CDM and CDS?
SPSChicagoBurbs 2019 - What is CDM and CDS?SPSChicagoBurbs 2019 - What is CDM and CDS?
SPSChicagoBurbs 2019 - What is CDM and CDS?
 
Building a 360 Degree View of Your Customers on BICS
Building a 360 Degree View of Your Customers on BICSBuilding a 360 Degree View of Your Customers on BICS
Building a 360 Degree View of Your Customers on BICS
 
Raed Ashour Profile
Raed Ashour ProfileRaed Ashour Profile
Raed Ashour Profile
 
DEV Meet-Up Q2 2022 Amsterdam Slides.pdf
DEV Meet-Up Q2 2022 Amsterdam Slides.pdfDEV Meet-Up Q2 2022 Amsterdam Slides.pdf
DEV Meet-Up Q2 2022 Amsterdam Slides.pdf
 
Ephesoft & Linux Webinar: Smart Capture™ is Now Even Smarter!
Ephesoft & Linux Webinar: Smart Capture™ is Now Even Smarter!Ephesoft & Linux Webinar: Smart Capture™ is Now Even Smarter!
Ephesoft & Linux Webinar: Smart Capture™ is Now Even Smarter!
 
Migration to Oracle ERP Cloud: A must read winning recipe for all
Migration to Oracle ERP Cloud: A must read winning recipe for allMigration to Oracle ERP Cloud: A must read winning recipe for all
Migration to Oracle ERP Cloud: A must read winning recipe for all
 
A case for salesforce.com for future technology professionals
A case for salesforce.com for future technology professionalsA case for salesforce.com for future technology professionals
A case for salesforce.com for future technology professionals
 
Become an RPA Citizen Developer
Become an RPA Citizen Developer Become an RPA Citizen Developer
Become an RPA Citizen Developer
 
Become an RPA Citizen Developer
Become an RPA Citizen Developer Become an RPA Citizen Developer
Become an RPA Citizen Developer
 
AXI & LoQutus: Getting the most out of Office 365
AXI & LoQutus: Getting the most out of Office 365AXI & LoQutus: Getting the most out of Office 365
AXI & LoQutus: Getting the most out of Office 365
 
Executive Briefing: Why managing machines is harder than you think
Executive Briefing: Why managing machines is harder than you thinkExecutive Briefing: Why managing machines is harder than you think
Executive Briefing: Why managing machines is harder than you think
 

More from UiPathCommunity

Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebUiPathCommunity
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPathCommunity
 
WomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneWomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneUiPathCommunity
 
Dev Dives: Master advanced authentication and performance in Productivity Act...
Dev Dives: Master advanced authentication and performance in Productivity Act...Dev Dives: Master advanced authentication and performance in Productivity Act...
Dev Dives: Master advanced authentication and performance in Productivity Act...UiPathCommunity
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
Test Automation with Gen AI_Final_Presentation
Test Automation with Gen AI_Final_PresentationTest Automation with Gen AI_Final_Presentation
Test Automation with Gen AI_Final_PresentationUiPathCommunity
 
Dev Dives: Leverage APIs and Gen AI to power automations for RPA and software...
Dev Dives: Leverage APIs and Gen AI to power automations for RPA and software...Dev Dives: Leverage APIs and Gen AI to power automations for RPA and software...
Dev Dives: Leverage APIs and Gen AI to power automations for RPA and software...UiPathCommunity
 
Unleashing the force of AI-powered intelligent document processing
Unleashing the force of AI-powered intelligent document processingUnleashing the force of AI-powered intelligent document processing
Unleashing the force of AI-powered intelligent document processingUiPathCommunity
 
Dev Dives: Fast-track time to value with UiPath Solution Accelerators
Dev Dives: Fast-track time to value with UiPath Solution AcceleratorsDev Dives: Fast-track time to value with UiPath Solution Accelerators
Dev Dives: Fast-track time to value with UiPath Solution AcceleratorsUiPathCommunity
 
Communauté UiPath Suisse romande - Séance de janvier 2024
Communauté UiPath Suisse romande - Séance de janvier 2024Communauté UiPath Suisse romande - Séance de janvier 2024
Communauté UiPath Suisse romande - Séance de janvier 2024UiPathCommunity
 
Dev Dives: Take your developer productivity to new heights with UiPath Autopi...
Dev Dives: Take your developer productivity to new heights with UiPath Autopi...Dev Dives: Take your developer productivity to new heights with UiPath Autopi...
Dev Dives: Take your developer productivity to new heights with UiPath Autopi...UiPathCommunity
 
UiPath Document Understanding_Day 3.pptx
UiPath Document Understanding_Day 3.pptxUiPath Document Understanding_Day 3.pptx
UiPath Document Understanding_Day 3.pptxUiPathCommunity
 
Dev Dives: Automate in full power with the latest Generative AI features in U...
Dev Dives: Automate in full power with the latest Generative AI features in U...Dev Dives: Automate in full power with the latest Generative AI features in U...
Dev Dives: Automate in full power with the latest Generative AI features in U...UiPathCommunity
 
Testautomatisierung: Heatmap für SAP und Community-Austausch
Testautomatisierung: Heatmap für SAP und Community-AustauschTestautomatisierung: Heatmap für SAP und Community-Austausch
Testautomatisierung: Heatmap für SAP und Community-AustauschUiPathCommunity
 
Africa Series 2 Session 6 - UiPath ReFramework
Africa Series 2 Session 6 - UiPath ReFrameworkAfrica Series 2 Session 6 - UiPath ReFramework
Africa Series 2 Session 6 - UiPath ReFrameworkUiPathCommunity
 
Africa Series 2 Session 7 - Introduction to UiPath APIs.pdf
Africa Series 2 Session 7 - Introduction to UiPath APIs.pdfAfrica Series 2 Session 7 - Introduction to UiPath APIs.pdf
Africa Series 2 Session 7 - Introduction to UiPath APIs.pdfUiPathCommunity
 
Africa Series 2, Session 5: UiPath Document Understanding
Africa Series 2, Session 5: UiPath Document UnderstandingAfrica Series 2, Session 5: UiPath Document Understanding
Africa Series 2, Session 5: UiPath Document UnderstandingUiPathCommunity
 
Dev Dives: Supercharge testing and RPA with coded automations
Dev Dives: Supercharge testing and RPA with coded automationsDev Dives: Supercharge testing and RPA with coded automations
Dev Dives: Supercharge testing and RPA with coded automationsUiPathCommunity
 
Africa Series 2 Session 1 - UiPath Studio
Africa Series 2 Session 1 - UiPath Studio Africa Series 2 Session 1 - UiPath Studio
Africa Series 2 Session 1 - UiPath Studio UiPathCommunity
 

More from UiPathCommunity (20)

Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio Web
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to Hero
 
WomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneWomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyone
 
Dev Dives: Master advanced authentication and performance in Productivity Act...
Dev Dives: Master advanced authentication and performance in Productivity Act...Dev Dives: Master advanced authentication and performance in Productivity Act...
Dev Dives: Master advanced authentication and performance in Productivity Act...
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
Test Automation with Gen AI_Final_Presentation
Test Automation with Gen AI_Final_PresentationTest Automation with Gen AI_Final_Presentation
Test Automation with Gen AI_Final_Presentation
 
Dev Dives: Leverage APIs and Gen AI to power automations for RPA and software...
Dev Dives: Leverage APIs and Gen AI to power automations for RPA and software...Dev Dives: Leverage APIs and Gen AI to power automations for RPA and software...
Dev Dives: Leverage APIs and Gen AI to power automations for RPA and software...
 
Unleashing the force of AI-powered intelligent document processing
Unleashing the force of AI-powered intelligent document processingUnleashing the force of AI-powered intelligent document processing
Unleashing the force of AI-powered intelligent document processing
 
Dev Dives: Fast-track time to value with UiPath Solution Accelerators
Dev Dives: Fast-track time to value with UiPath Solution AcceleratorsDev Dives: Fast-track time to value with UiPath Solution Accelerators
Dev Dives: Fast-track time to value with UiPath Solution Accelerators
 
Communauté UiPath Suisse romande - Séance de janvier 2024
Communauté UiPath Suisse romande - Séance de janvier 2024Communauté UiPath Suisse romande - Séance de janvier 2024
Communauté UiPath Suisse romande - Séance de janvier 2024
 
Dev Dives: Take your developer productivity to new heights with UiPath Autopi...
Dev Dives: Take your developer productivity to new heights with UiPath Autopi...Dev Dives: Take your developer productivity to new heights with UiPath Autopi...
Dev Dives: Take your developer productivity to new heights with UiPath Autopi...
 
Azure CICD - Day1.pptx
Azure CICD - Day1.pptxAzure CICD - Day1.pptx
Azure CICD - Day1.pptx
 
UiPath Document Understanding_Day 3.pptx
UiPath Document Understanding_Day 3.pptxUiPath Document Understanding_Day 3.pptx
UiPath Document Understanding_Day 3.pptx
 
Dev Dives: Automate in full power with the latest Generative AI features in U...
Dev Dives: Automate in full power with the latest Generative AI features in U...Dev Dives: Automate in full power with the latest Generative AI features in U...
Dev Dives: Automate in full power with the latest Generative AI features in U...
 
Testautomatisierung: Heatmap für SAP und Community-Austausch
Testautomatisierung: Heatmap für SAP und Community-AustauschTestautomatisierung: Heatmap für SAP und Community-Austausch
Testautomatisierung: Heatmap für SAP und Community-Austausch
 
Africa Series 2 Session 6 - UiPath ReFramework
Africa Series 2 Session 6 - UiPath ReFrameworkAfrica Series 2 Session 6 - UiPath ReFramework
Africa Series 2 Session 6 - UiPath ReFramework
 
Africa Series 2 Session 7 - Introduction to UiPath APIs.pdf
Africa Series 2 Session 7 - Introduction to UiPath APIs.pdfAfrica Series 2 Session 7 - Introduction to UiPath APIs.pdf
Africa Series 2 Session 7 - Introduction to UiPath APIs.pdf
 
Africa Series 2, Session 5: UiPath Document Understanding
Africa Series 2, Session 5: UiPath Document UnderstandingAfrica Series 2, Session 5: UiPath Document Understanding
Africa Series 2, Session 5: UiPath Document Understanding
 
Dev Dives: Supercharge testing and RPA with coded automations
Dev Dives: Supercharge testing and RPA with coded automationsDev Dives: Supercharge testing and RPA with coded automations
Dev Dives: Supercharge testing and RPA with coded automations
 
Africa Series 2 Session 1 - UiPath Studio
Africa Series 2 Session 1 - UiPath Studio Africa Series 2 Session 1 - UiPath Studio
Africa Series 2 Session 1 - UiPath Studio
 

Recently uploaded

PLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsPLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsStefano
 
Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024Patrick Viafore
 
IESVE for Early Stage Design and Planning
IESVE for Early Stage Design and PlanningIESVE for Early Stage Design and Planning
IESVE for Early Stage Design and PlanningIES VE
 
TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024Stephen Perrenod
 
Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessUXDXConf
 
Enterprise Knowledge Graphs - Data Summit 2024
Enterprise Knowledge Graphs - Data Summit 2024Enterprise Knowledge Graphs - Data Summit 2024
Enterprise Knowledge Graphs - Data Summit 2024Enterprise Knowledge
 
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdfSimplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdfFIDO Alliance
 
Easier, Faster, and More Powerful – Notes Document Properties Reimagined
Easier, Faster, and More Powerful – Notes Document Properties ReimaginedEasier, Faster, and More Powerful – Notes Document Properties Reimagined
Easier, Faster, and More Powerful – Notes Document Properties Reimaginedpanagenda
 
Oauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoftOauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoftshyamraj55
 
Powerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara LaskowskaPowerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara LaskowskaCzechDreamin
 
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...CzechDreamin
 
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdfWhere to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdfFIDO Alliance
 
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdfThe Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdfFIDO Alliance
 
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...FIDO Alliance
 
Syngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdfSyngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdfSyngulon
 
Optimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through ObservabilityOptimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through ObservabilityScyllaDB
 
Designing for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at ComcastDesigning for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at ComcastUXDXConf
 
What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024Stephanie Beckett
 
How we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdfHow we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdfSrushith Repakula
 
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdfLinux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdfFIDO Alliance
 

Recently uploaded (20)

PLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsPLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. Startups
 
Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024
 
IESVE for Early Stage Design and Planning
IESVE for Early Stage Design and PlanningIESVE for Early Stage Design and Planning
IESVE for Early Stage Design and Planning
 
TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024
 
Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
 
Enterprise Knowledge Graphs - Data Summit 2024
Enterprise Knowledge Graphs - Data Summit 2024Enterprise Knowledge Graphs - Data Summit 2024
Enterprise Knowledge Graphs - Data Summit 2024
 
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdfSimplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
 
Easier, Faster, and More Powerful – Notes Document Properties Reimagined
Easier, Faster, and More Powerful – Notes Document Properties ReimaginedEasier, Faster, and More Powerful – Notes Document Properties Reimagined
Easier, Faster, and More Powerful – Notes Document Properties Reimagined
 
Oauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoftOauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoft
 
Powerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara LaskowskaPowerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara Laskowska
 
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
 
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdfWhere to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
 
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdfThe Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
 
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
 
Syngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdfSyngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdf
 
Optimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through ObservabilityOptimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through Observability
 
Designing for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at ComcastDesigning for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at Comcast
 
What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024
 
How we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdfHow we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdf
 
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdfLinux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
 

DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam

  • 1. May 7, 2024 UiPath DEV meet-up UiPath Document Understanding
  • 2. 2 Dion Morskieft RPA Product Owner at Partou Mikal van Eijk Automation Developer at Tacstone Technology Jack Klein Schiphorst Automation Developer at Tacstone Technology Let’s connect!
  • 3. 3 • Introduction • Document Understanding • SPIE Building Solutions use case • UWV use case • Lessons learned Program
  • 4. Tacstone is a consulting and venture boutique founded in 2006 as a spin-off from a big-4 consulting firm Technology division established in 2018 In 2024 we operate with a team of 40+ automation experts serving 60+ clients in the Netherlands and abroad About Tacstone Technology
  • 6. TheUiPathBusinessAutomationPlatform PROCESS MINING TASK MINING COMMUNICATIONS MINING IDEA CAPTURE & MANAGEMENT LOW-CODE DEVELOPMENT UI & API AUTOMATION​ PROCESS ORCHESTRATION INTELLIGENT DOCUMENT PROCESSING INTEGRATED NLP & AI/ML Discover Continuously uncover opportunities for process and task improvements ​helping you identify the highest ROI areas Automate Get more done with a digital workforce that seamlessly collaborates with your people ​and automates work via UI and API, powered with native integrated AI​ Your Applications Your People Your Processes Systems of record – ERP, CRM, HCM | Communications and Collaboration | Personal Productivity​ ANALYTICS CONTINUOUS TESTING UNIFIED MANAGEMENT & GOVERNANCE​ FLEXIBLE DEPLOYMENT Operate An enterprise-grade foundation to run and optimize a ​mission critical automation program at high scale
  • 7. 9 • Like forms, passports, licenses, time sheets • Fixed in format and can contain handwriting, signatures, checkboxes​ • Like invoices, receipts, purchase orders, medical bills, utility bills • Containing fixed and variable parts like tables • Like contracts, agreements, emails, scripts, drug prescriptions, news • No fixed format, free-form sentences/paragraphs Which documents can be handled by Document Understanding? Structured documents Semi-structured documents Unstructured documents
  • 8. 10 Data extraction – from rules to AI to a hybrid approach RegEx-Based Extractor Structured fields Machine Learning Extractor Mostly semi-structured documents Form Extractor Structured documents (forms) A combination of both – rule-based and AI-based approaches Mostly documents combining both structured and less structured formats Rule-based Forms AI Structured documents (forms) AI-based Hybrid approach
  • 9. 11 Pre-trained models Machine learning (ML) models as a template-less approach Custom models • No-code light-weight models in Forms AI • Custom ML models in AI Center • Third-party models Model retraining Learn about sharing data for model retraining here • Invoices • Receipts • Purchase Orders • Utility Bills • Passports • ID Cards* • Legal Contracts • W-2 Forms • W-9 Forms • Delivery Notes • Remittance Advices • ACORD 125 • I9 Forms • 990 Forms • 4506T Forms • FM1003 Forms • Pay slips & personal earnings statements • Certificates of origin • EU declarations of conformity • Children’s product certificates • Certificates of incorporation • Shipping invoices • CMS1500 • Retraining via AI Center • Continuous learning loop based on human validated data
  • 12. 14
  • 15. 17 17
  • 16. 18 Every customer has its own template 18
  • 17. 19 Every customer has its own template 19 Prio Description Location Subject Priority Address Building Description SLA
  • 19. 21 • Our developers were needed to add new templates • Complex logic and code • Not flexible to adjustments Downsides Initial rule-based regex solution
  • 20. 22 Final Forms AI Solution Forms AI combines the ease of using templates with the benefits of continuously improving AI in a no-code approach to structured document processing: 1. Upload 2-5 forms 2. Define fields for one document 3. Test out the automatically generated model 4. Publish and use the model
  • 21. 23
  • 22. 24
  • 23. 25
  • 24. 26
  • 25. 27
  • 26. 28
  • 27. 29
  • 28. 30
  • 29. 31
  • 30. 32
  • 31. 33
  • 32. 34
  • 33. 35
  • 34. 36
  • 35. 37
  • 36. 38
  • 37. 39 Categorization with DataService • Use the correct Forms AI endpoint • Only send relevant data to the AI model • Fill in pre-defined values
  • 38. 40 Send data to SAP • Convert extraction results to JSON • Use SAP Odata API • Crete SAP ticket based on Service request
  • 39. 41
  • 41. 43 Lessons Learned Don't hesitate, just start. Choose the correct AI model Be consistent with labeling Get enough data Involve the Business Generic is not always the best
  • 42. vertrouwelijk Dion Morskieft 07 mei 2024 Document Understanding at Partou UWV
  • 43. 07 mei 2024 Document Understanding at Partou vertrouwelijk Who am I? • Dion Morskieft • At Partou for 14 years • 7 years RPA Developer My team? • Team consisting of 3 RPA developers and 5 BI specialists • For each department robots/optimized processes are implemented • Over 120 processes start 200+ robots daily • Workforce reduction of 25-30 FTEs Introduction 45
  • 44. 07 mei 2024 Document Understanding at Partou vertrouwelijk 46 https://www.kidsfoundation.nl/nl/wp-content/themes/kidsfoundation/assets/images/logo.png https://www.kidsfoundation.nl/nl/wp-content/uploads/lable-smallsteps-2.png
  • 45. 07 mei 2024 Document Understanding at Partou vertrouwelijk 47 900+ LOCATIONS IN THREE COUNTRIES 80.000+ CHILDREN 12.000+ COLLEAGUES
  • 47. 07 mei 2024 Document Understanding at Partou vertrouwelijk • 60 letters a day • Lots of manual labour (over 5 minutes per letter) • Good process for document understanding What did we do? • Document and optimize the process • Two separate flows with different queues • Regex • Document Understanding • One DU-model with multiple training and evaluations sets • Communicating with business through Action Center and Teams Why this proces? Letters UWV 49
  • 48. 07 mei 2024 Document Understanding at Partou vertrouwelijk 50 1. Download enough letters for a reliable set 2. Anonymize the data 3. Create training and evaluation batches 4. Label the documents 5. Evaluate the model 6. Add more data and retrain if needed I can show you in the AI-center How did we train the model
  • 49. 07 mei 2024 Document Understanding at Partou vertrouwelijk 1. Let the robot download the letters in the UWV-portal 2. Based on subject decide which letters follow what flow 3. Fill queue for the DU- and regex model (encrypt data) 4. Cut the first two pages of the letter and let the model get the data 5. When the DU-model is not certain enough: 1. Create a task in the Action Center 2. Send a Teams-message to the HR-Team 6. When the DU-model is certain enough (or task is completed) 1. Fill queue for processing letter 2. Based on Social Security Number find colleague in Afas/Profit 3. Add letter to personal-file 4. Add data from the letter to Afas/Insite 5. Start a workflow in Afas for HR based on subject of the letter I can show you in Studio How did we build the robot? 51