Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Product deep-dive on Document Understanding capabilities
Similar to Product deep-dive on Document Understanding capabilities (20)
More from Rohit Radhakrishnan
More from Rohit Radhakrishnan (20)
Recently uploaded
Recently uploaded (20)
Product deep-dive on Document Understanding capabilities
- 1. Product deep-dive The UiPath ™ word mark, logos, and robots are registered trademarks owned by UiPath, Inc. and its affiliates. ©2023 UiPath. All rights reserved. Document Understanding capabilities
- 2. 2 Get your documents processed intelligently Teach your robots to understand documents using AI-enhanced skills for data extraction and interpretation. Drag and drop these capabilities directly into your automation workflows to embed AI
- 3. 3 • Like forms, passports, licenses, time sheets • Fixed in format and can contain handwriting, signatures, checkboxes • Like invoices, receipts, purchase orders, medical bills, utility bills • Containing fixed and variable parts like tables • Like contracts, agreements, emails, scripts, drug prescriptions, news • No fixed format, free-form sentences/paragraphs Which documents can be handled by Document Understanding? Structured documents Semi-structured documents Unstructured documents
- 4. 4 Load taxonomy for your documents How Document Understanding works Digitize images using multiple OCRs Classify documents Extract named entities in a taxonomy Validate and train supervised models Export extracted data 2 3 4 5 6 Digitize Classify Train & Validate Extract 2 3 5 4 Structured Semi-structured Unstructured Export RPA BPM API Other systems 6 1 Define taxonomy – document types and fields 1
- 5. 5 Taxonomy manager is used once at the start to define the collection of documents that you would want to process as well as business rules. Additionally, you can describe what data you would like to extract. Load taxonomy
- 6. 6 Move-For-You Co. We move so you don’t need to move PO: NP74006735 1 February 2020 PAYABLE WITHIN 15 DAYS OF RECEIPT 20800 ALMADEN AVE, SUITE 404 SAN JOSE, CA 95120-0520 T: +1 425 555 9876 F: +1 425 555 3456 E: billing@moveforyou-co.com www.moveforyou-co.com Bill To: Tony Tzeng 12345 Mango Lane Seattle, WA 98108 INVOICE DETAILS Packing services Storage fees (1 month) House move (white-glove service) Vehicle storage and transport Sales tax 10% Total Fee including Tax FEE $1,282.00 $1,884.00 $5,320.00 $5,186.00 $1,367.20 $15,039.20 Methods of payment Personal Check: Move-For-You LLC Wire Transfer: BigBank Co., Account 123456789-0987ABC Invoice No: 456200-TZE1 Digitize text in the documents using OCR
- 7. 7 Classify and split the documents Documents scanned into one file isn’t a problem – owing to classifiers, the robot can identify the document types and split the file to process the documents accordingly. Document Understanding offers different classification capabilities ranging from keyword-based to ML-based classification.
- 8. 8 Validate classification of the documents Classification Station is used to check, correct, and confirm the results of document classification and splitting.
- 9. 9 You can easily configure data extraction to choose most suitable extractor for each field. Use a combination of rule- based and model-based approaches to ensure smooth and accurate processing of different documents. Extract data from the documents
- 10. 10 Data extraction – from rules to AI to a hybrid approach RegEx-Based Extractor Structured fields Machine Learning Extractor Mostly semi-structured documents Form Extractor Structured documents (forms) A combination of both – rule-based and AI-based approaches Mostly documents combining both structured and less structured formats Rule-based Forms AI Structured documents (forms) AI-based Hybrid approach
- 11. 11 Make use of pre-trained ML models to process invoices, receipts, utility bills, ID cards, and many more document types. Retrain the models to optimize them for your custom documents and improve the model accuracy over time! Bring your own model or third party models and incorporate them in your automations. Pre-trained ML models
- 12. 12 Validate the extracted Information and handle exceptions using Validation Station. Now, ML models can also be retrained using the data confirmed or corrected in the Validation Station. Validate extraction of the documents
- 13. 13 Let the classifiers and extractors learn from the data corrected and validated in the Classification Station and Validation Station respectively. Learn about sharing data for model retraining here. Train classifiers and extractors
- 14. 14 Export the extracted data End-to-end intelligent document processing Start & continue the document processing workflows with other automation components. Export the data for further usage/automation – for example, to an Excel spreadsheet, send as email, SAP, and so on.
- 15. Best practices & performance metrics The UiPath ™ word mark, logos, and robots are registered trademarks owned by UiPath, Inc. and its affiliates. ©2023 UiPath. All rights reserved.
- 16. 16 Logic in the process to automatically determine if field(s) extracted correctly Art of the possible with business rules Character validation Basic math Date validation External source verification Invoice # = 7 digits Total = Line Items + Tax Valid date format Verify against PO Record PO Number Vendor ID Vendor Name 5928452 12345 PROTECH 3758292 98734 LORANA
- 17. How to get started Define which documents you want to process Explore how UiPath Document Understanding can automate this Give it a try – start Enterprise Trial Get trained on Document Understanding in UiPath Academy or via instructor-led training from UiPath Enterprise Trial, Academy Course, and more at uipath.com/document-understanding
- 18. What’s new in Document Understanding The UiPath ™ word mark, logos, and robots are registered trademarks owned by UiPath, Inc. and its affiliates. ©2023 UiPath. All rights reserved. 2023.4 release highlights and roadmap
- 19. 19 Document Understanding: AI & ML improvements More use cases and faster deployment • Automate more processes out of the box with pre-trained & retrainable ML models • Pay slips & personal earnings statements • Certificates of origin • EU declarations of conformity • Children’s product certificates • Certificates of incorporation • Shipping invoices • CMS1500 • Train ML models quickly with one-click ML for document classification and data extraction (in public preview)
- 20. 20 Document Understanding: business rules More accurate extraction with custom business rules • Set up your custom business rules, including mathematical rules, to reduce the need for human validation and ensure higher accuracy & straight- through processing rates
- 21. 21 Document Understanding as a service Seamless integration with third-party tools • Easily integrate core Document Understanding capabilities with any third-party tools via API (in public preview): • Discovery • Digitization • Classification • Extraction • Validation
- 22. 22 Roadmap: 2023.10 and beyond 2 Accelerate time to value • AI-assisted active learning- based training (tag the minimum, real-time retraining) • Pre-labeling documents with GPT • GPT-enabled document classification and data extraction 1 Expand unstructured document intelligence • GPT for querying documents with natural language • Enhanced Communications Mining + Document Understanding integration 4 Deployment insights & operations • Dashboards & document audit, containing metrics for STP rate, time saved, and more • Validation connected with Integration Services to enable data lookups in external applications 3 • Improved UX to help users build better models faster • Enhanced capability discovery, guided labeling experience and improved model insights User experience
- 23. 23 Example AI use cases Professional Svc Data Extraction from Charts RFP Opportunities Classification Deal Guidance Financial Svc Fraud Detection Personal Loan Approval KYC – Entity Identification Retail Packaging Quality Evaluation Inventory Management Merchandising Planning Healthcare Real Time Pregnancy Risk Evaluation Patient Receivables Management Propensity of Claim Denial Prediction General Help Desk Answers Customer Churn Prediction Resume Matching Auditing – Anomaly Detection AML Alert Classification Product Recommendation Fraudulent Medical Claim Prediction Customer Complaints - Email Classification Legal – Win/loss rate prediction ID Information Extraction Pricing Optimization Readmission Prediction Quality - Visual Inspection *Review appendix for details
- 24. UiPath AI Center Automate more by adding AI to your automations
- 25. 25 Use cases Real world use cases, and customer quotes Product deep dive AI Center overview, deployment types, Email AI, release highlights, and roadmap Next steps How to get started, and what to expect Intro What’s AI Center, ML models, demo, use cases, and key differentiators Agenda
- 26. 26 We’ve built AI into every part of the UiPath platform Classify emails Forms in VDI Semi-structured data Unstructured data Mine processes Mine tasks Speech to text Chatbots Understand documents AI Copy/paste Email event Fill forms Extract structured data Log in to apps Capture tasks Scrape data Read/write to databases Activities Moving file/folders RPA Process Mining Task Mining Document Understanding AI Computer Vision Custom/ Ecosystem Skills Action Center Chatbots AI Center Insights BUILD Teach your robots new AI skills DISCOVER Scientifically identify automation opportunities MANAGE Deploy, scale, and manage AI ENGAGE Bring humans in the AI loop
- 27. 27 AI Center enables you to insert AI into your workflow with drag-and-drop ease 1 2 3 ML Skill (Humans) Validate exceptions Choose a model from these options: Choose from over 25 pre-built models From UiPath Retrain the model Select from dozens of UiPath pre-build models Pick a model From UiPath marketplace Build an ML Skill with UiPath AI Center Drag and drop ML Skill into RPA workflows in UiPath Studio Bring your own
- 28. 28 Demo time
- 29. 29 AI Center key differentiators Bring your own models Models from UiPath and partners Multiple deployment options Continuously improve ML models with retraining Deep integration with other UiPath products Human validation
- 31. 31 Deployment Infrastructure Why? Why Not? Main Prerequisites Installation Experience Out-of-the-box Model Delivery UiPath Automation Cloud SaaS (Infrastructure managed by UiPath) • Fully managed by UiPath. No additional infrastructure needed • Scaling all handled by UiPath • Always get latest features first • Can work with Robots connected to on-prem Orchestrator • Data Sensitivity Simply an internet connection Sign-up and get started in seconds Automatically available and always up-to-date UiPath Automation Suite Service Fabric deployed onto Customer Infrastructure • AI Center comes bundled with entire UiPath suite • Products come pre-integrated • Want to maintain existing Orchestrator deployment on Windows • Don’t need most UiPath products • Very locked down Linux servers (e.g. STIG-hardened) • One or more (depending on scale) Linux VM • Data disk(s) • SQL Server EITHER: • Input values into inputs.json file and run setup.sh from Linux Shell OR • Input values via CLI for interactive install in Linux Shell CONNECTED MODE: • Daily CRON jobs ensure latest models are always available AIRGAPPED MODE: • One time image bundle must be run via Linux Shell for base images. • Then, each model package is uploaded via UI as Models packaged as OOTB model. AI Center Standalone Service Fabric deployed onto Customer Infrastructure PLEASE NOTE: This is different from the previous version of AI Center on Replicated, as we have standardized on the Rancher- based infrastructure provided by Automation Suite) • Want to maintain existing Orchestrator deployment on Windows • Can deploy to on-prem data center or your own cloud subscription • Don’t have existing Kubernetes footprint • Very locked down Linux servers (e.g. STIG-hardened) • One or more (depending on scale) Linux VM • Data disk(s) • SQL Server • Orchestrator 20.10 or higher Input values into inputs.json file and run setup.sh from Linux Shell CONNECTED MODE: • Daily CRON jobs ensure latest models are always available AIRGAPPED MODE: • One time image bundle must be run via Linux Shell for base images. • Then, each model package is uploaded via UI as Models packaged as OOTB model. AI Center Deployment Types • Please visit AI Center documentation pages for more details on this topic.
- 32. 32 AI Center Automate more by adding AI to your automations DEPLOY CONSUME MANAGE IMPROVE
- 33. 33 Use any ML models you choose—we are open Deploy Consume Manage Improve Bring your own ML model. Or choose a pre-built model from UiPath or UiPath partners AI Center offers dozens of ML models, enabling hundreds of use cases Jump start your first AI use case with a model without having to build them Easily deploy the model as an ML skill
- 34. 34 Drag and drop the ML model into your automation workflow Deploy Consume Manage Improve Drag and drop the ML activity to an RPA workflow Select an ML skill from the drop- down list Use multiple ML skills in one workflow if needed Easily test the ML skills before you run Scale the ML skill to as many robots as you want
- 35. 35 Monitor and manage your model Deploy Consume Manage Improve View all datasets, ML packages, and pipelines on the Dashboard page Get end-to-end visibility on ML model use Update model versions with a few clicks Keep track of your data, model performance, user actions, and ML pipelines
- 36. 36 Continuously improve your model Deploy Consume Manage Improve Label data, and configure custom fields Train a pre-built model with your own data Set confidence threshold Handle exceptions, validate predictions, route validated data back for retraining Choose model version, package version and dataset Choose when to run the training pipeline— now, time-based, recurring Get accuracy score, and training run report (data statistics, evaluation statistics, confusion matrix, per class statistics)
- 37. 37 Deploy ML Skills as REST API Use ML skill from UiPath Studio Use ML skill from a different tenant Use ML skill from other UiPath products or from third-party apps Use Orchestrator on-prem and AI Center in cloud Four different ways of using ML skills
- 38. 38 How to use UiPath ML models Model training & evaluation Model deployment Model maintain & retraining Data preparation & cleaning Prepare high-quality data Your training data should be as representative as of your production data Label your data correctly Mislabeled data result in poor model performance Distribute evenly Make sure your dataset is evenly balanced across your classes Separate your data Split your dataset into training and test data Train on base version When retraining, use a bigger dataset and retrain on the base version Work in iterations Train and evaluate the model on a small dataset. Reiterete until you are happy with the performance. Use GPU for speed GPU can be used for serving and training. GPU delivers 5- 10x improvement in speed Maintain model quality Monitor model performance, retrain the model periodically ML Package version After each training, the knowledge is stored in the next minor version Bring human in the loop Bring humans to validate model predictions, and save validated data for retraining
- 39. Use cases
- 40. 40 AI Enables Automation of Processes That Include Uncertainty You cannot determine an outcome with 100% certainty. High Variability There is too much variability for rules based. Unstructured Data Information like articles, documents, images, videos and emails. Property Valuation Loan Defaults Inventory Forecasts Resume Matching Purchase Decisions Language Translation Invoice Extraction Email Routing Speech to Text
- 41. 41 ML models and use case examples Image Classification Semantic Similarity Semantic Similarity Answer questions vis email based on FAQ docs Fetch articles based on emails or search queries Plagiarism detection on forms Detect intent Image Classification Vehicles Clothes Damaged products Logos This model shows which sentences or words correlate with each other. This model classified images into different classes. This model identifies key elements from text, then classifies them into pre-defined categories. Named Entity Recognition Emails Letters Webpages Call transcripts Named Entity Recognition
- 42. 42 Real-world Email AI use cases Top US Bank • Millions of emails, with different SLA requirements • With Email AI, many hours are removed. People can now focus on what they need to do instead of sorting, and pre-filling info from one system to another Financial Services company • Emails by business customers • Email AI extracts income data from those emails Printing, envelope and label provider • Orders to print labels • Email AI extracts info like label color, height, weight, quantity, materials, and other info in emails • Orders for different financial products, placed by customers via email • Email AI extracts information from emails, prioritizes based on business value, and auto- fill their OMS Large investment management org Metal solutions provider • Spot RFQs and offers to buy excess materials • Email AI helps determine what’s worth spending time on, separating real opportunities from the junk Financial services company in Japan • Notes details sent by issuers • With Email AI, they extract info from the emails, auto-fill Bloomberg terminal, create quotes for issuers Webinar link
- 43. Next steps
- 44. 44 Roadmap to automating more processes Take AI Center Academy course Sign up for trial Get UiPath support Prove ROI and scale up Enterprise Cloud Trial Enterprise On-premises Trial • Upskill your RPA developers • One of the most popular UiPath Academy courses in the past two years • All ML models included • Cloud and on-premises • Identify a use case – choose ‘low-hanging fruit’ projects rather than ‘moon shot’ projects • Watch AI use case demo videos • POC • AI Starter Pack • Immersion Lab • AI Center training course • AI playbook • AI 101 webinar • Instructor-led training • Professional Services • Example metrics: cost reduction, quicker customer response time, faster cash collection • Apply AI in more regions, more business units • Write a playbook and standardize the process
- 45. 45 AI Center Roadmap 2023 Enhance automations with ML Feature Enhancements • Data Labeling – Upload raw text data and annotate in the labelling tool for free text, and consume labeled data in training models • Tenant-level AI Units – Track and set limits of AI Unit consumption at the tenant-level ML Packages and Skills • Faster model training and deployment times • Test ML skills on AI Center before consuming in an automation workflow • AI Center API Availability – Enables programmatic management of ML Skills and Pipelines • Usability improvements including outage alerts, search / filter options, tooltips and accessibility Infrastructure, Security & Recovery • At-rest Data Encryption – Encrypt data with your own key • Tenant migration – enable AI Center projects to be migrated within and across geos • Resource queues for on-prem hardware • One-click access to logs and audit information Get more done with a digital workforce that seamlessly collaborates with your people and automates work via UI and API, powered with native integrated AI Automate
- 46. 46 Improved experience for pipeline parameter settings Auto-generated parameters for ML pipelines provides an easier user experience without coding. With this feature, users no longer have to remember or search for the parameters for a model anymore.
Editor's Notes
- Taxonomy Manager
- OCR engines
- Keyword-based Classifier Intelligent Keyword Classifier ML Classifier
- Classification Station
- RegEx-based Extractor Form Extractor Intelligent Form Extractor ML Extractor
- Invoices, Receipts – in GA Purchase Orders, Utility Bills, Invoices for Australia, India, Japan – in preview ML Extractor AI Center
- Validation Station
- Let’s build momentum here highlighting new capabilities for classification & splitting Two classification methods available already: a Keyword Based Classifier, focused on learning TITLES and searching for them, with no capability to split files into multiple docs; and the newly launched Intelligent Keyword Classifier, focused on document similarity from a word content perspective, also with learning capabilities, and also capable of identifying and suggesting splits between different documents within the same file.
- 2min 13 sec Vendor Confirmation Name Address Two and Three-Way Matching Invoice Item Quantity Invoice Item Price Cost Center Validation Date Format Sanity Check ID #’s
- Screenshot – new model for Shipping Invoices
- 2min 30 sec LLM – large language models
- We are seeing use cases across all industries. Refer to the Appendix to get deep dives on several of these.
- Start with traditional RPA – The core of RPA is to mimic human actions in interacting with digital system. Traditional RPA is a process-driven technology to automate repetitive, rule-based and non-value added task in business processes. Typical tasks would include auto-keying, screen/form integration, application or data integration, automated decisions acting on structured data. Focus is on ‘execution’ of tasks. When you add AI – By adding RPA to AI, you can automate more tasks that are knowledge-intensive. You can address scenarios that are predictive scenarios like whether a customer will default on their loans and ones that involve high-variability like matching resume job skills to open roles in the company. AI can: Understand unstructured and semi-structured data Detect fraud Classify and summarize text Read sentiment Detect objects in images and do many more Discover – one of the reasons automation programs don't scale to where they need to is due to the challenge of finding the next automation. This is where AI can be a natural way of discovering the next automation opportunity Gartner in ‘Emerging Technologies: RPA software advancements’ RPA scripts with integrated ML models allow buyers to automate more complex use cases and improve the scalability of their automation programs. 80%: https://www.i-search.com.cn/CompetitiveLandscape-RPA-Software.pdf 74%: https://www2.deloitte.com/us/en/insights/focus/cognitive-technologies/state-of-ai-and-intelligent-automation-in-business-survey.html 65%: https://www2.deloitte.com/us/en/insights/focus/cognitive-technologies/ai-adoption-in-the-workforce.html
- AI Center is UiPath's machine learning platform, which enables customers to deploy, consume, manage and improve machine learning models developed inhouse, by UiPath, or by UiPath partners.