- Discussed automated Contract Compliance solution built using Generative AI technologies. Objective was to explain students how multiple technologies are applied to solve real business problems in industry - Let us know what you think about this solution? Do you think RAG can be used here? Can we create a customised vector database for contracts?

1. Solving Business
Problems using
Date: 13th Sep 2023
Solving Business Problem
with Large Language
Models
Date: 25th Sep 2023

2. Table of Contents
2
Session’s Objective 03
Project Objective, Scope &
Benefits
04
Data Understanding & Contract
Rules
05-06
Infrastructure 07
Approach 08
Prompt usage for problem
solving
09-12
Prompting Techniques 13
Accuracy and Findings 14
Next Steps 15

3. Session’s Objective
Session’s Objective: How to solve a real business
problem with LLMs using Prompts

4. Project Objective, Scope and Benefits
Machine driven identification of non-complaint contracts of Oil and Gas
exploration and extraction in Europe
Objective
Benefits
• Efficiency Gains
• Reduced Human Errors
• Humanly impossible to check compliance of few thousand contracts
• Change in regulations require reassessment of existing contracts
Challenge
s

5. Data Understanding
~2000 Oil and Gas scanned contracts in PDF format
~500 Oil and Gas Scanned Contracts in PDF
format

6. Session’s Objective
Rules and Regulations of Oil and Gas Contracts
for Exploration and Extraction
Oil and Gas
Regulations

7. High Level Architecture
API Call
AI Engine
Input Layer
OneDrive
Contracts
Tesseract
Anaconda
Python ChatGPT 3.5
Output layer
4096 token
limit
Compliance
Report
Regulations

8. Approach
Oil & Gas
Contracts
Python
Contract in Text
Contract Split in parts
Compliance
Report
Read by Python Image to Text
Contract terms in
Image
Compliance
Rules
ChatGPT
Split in Parts
Does Contract
follow Rule?
ChatGPT used for code generation and to search rules in Contract; Python used for Data Processing

9. Licensed Area in Contract
Licensed Area comprises of one or more Blocks and its co-ordinates
The geographical area for which the firm, Tangram Energy Ltd and Summit
Exploration and Production Limited, has been given a license for oil and
gas extraction and production is referred to as "Block 14/26b.“
The coordinates of this licensed area are defined by a series of latitude and
longitude points, forming a polygon. The coordinates for Block 14/26b are
as follows:
58°10'00.000"N 1°00'00.000"W
58°10'00.000"N 0°48'00.000"W
58°00'00.000"N 0°48'00.000"W
58°00'00.000"N 0°50'00.000"W
58°05'00.000"N 0°50'00.000"W
58°05'00.000"N 0°56'00.000"W
58°06'00.000"N 0°56'00.000"W
58°06'00.000"N 1°00'00.000"W
58°10'00.000"N 1°00'00.000"W
These coordinates outline the boundaries of Block 14/26b, which is the
licensed area for oil and gas activities as specified in the provided text.
Co-ordinates - Gives
precise geographical
location
Block – A portion
within large area

10. Identify Licensed Area in Contract
Follow Prompt techniques to get desired output; ChatGPT output need to be monitored by humans
Contract Prompt Type ChatGPT Output
chrysaor-production-uk-limited-neo-energy-
zex-limited-p2521-exploitation
Zero shot prompt • No Block captured in Output
chrysaor-production-uk-limited-neo-energy-
zex-limited-p2521-exploitation
Multi shot prompt • 1 out of 4 Blocks captured correctly
chrysaor-production-uk-limited-neo-energy-
zex-limited-p2521-exploitation
Multi shot prompt with Business context • All 4 Blocks captured correctly
chrysaor-production-uk-limited-neo-energy-
zex-limited-p2521-exploitation
Business context from Domain expert • All 4 Blocks captured correctly
5258-apache-north-sea-limited-p2529-
exploitation-license-exploration-pdf
Multi shot prompt with Business context • 3 out of 4 Blocks captured correctly, 1
wrong Block captured
Step 1 : Identify prompt which captures License Areas & their coordinates
Step 2 : Indicates License Area present, partially present or not Present
Contract Prompt Type ChatGPT Output
chrysaor-production-uk-limited-neo-energy-
zex-limited-p2521-exploitation
1. Business context from Domain expert
2. Present fully, partially or not present
• Gives appropriate answer along with
additional information
Identify Licensed
Area

11. Data Confidentiality Clause in Contract
Data confidentiality is critical; data if leaked can result in economic and political consequences
All records, returns, plans, maps, samples, accounts and information (in
this clause referred to as "the specified data") which the Licensee is or may
from time to time be required to furnish under the provisions of this license
shall be supplied at the expense of the Licensee and shall not (except with
the consent in writing of the Licensee which shall not be unreasonably
withheld) be disclosed to any person not in the service or employment of
the OGA or the Crown

12. Identify Data Confidentiality Clause
Follow Prompt techniques to get desired output; ChatGPT output need to be monitored by humans
Contract Prompt Type ChatGPT Output
5263-ithaca-oil-and-gas-limited-p2534-
exploitation-license-explorationpdf
Multi-shot prompt with delimiters • Captures that data confidentiality clause
is present, no details
5263-ithaca-oil-and-gas-limited-p2534-
exploitation-license-explorationpdf
Multi-shot prompt, delimiters & only data
confidentiality details needed
• Captured most details of data
confidentiality, however few got missed
chrysaor-production-uk-limited-neo-energy-
zex-limited-p2521-exploitation
Delimiters & only data confidentiality details
needed
• Captured all details of data
confidentiality.
Step 1 : Identify prompt which captures Data Confidentiality Clause
Step 2 : Indicates Data Confidentiality Clause is present, partially present or not Present
Contract Prompt Type ChatGPT Output
chrysaor-production-uk-limited-neo-energy-
zex-limited-p2521-exploitation
1. Business context from Domain expert
2. Present fully, partially or not present
• Gives appropriate answer along with
additional information
Identify Data
Confidentiality Clause

13. Prompting Techniques
Prompting is an Art which improves with Experience
Prompting Techniques Short Description of Technique
Zero shot prompting Straight forward query with no example
Multi shot prompting Along with the requirement, examples are provided
Use of Delimiters Highlight portion of text using delimiters
Mention role ChatGPT should play Roles like Data Analyst, Business Analyst, Programmer etc.
Include 3 parts a) Role b) Requirement c) Output Format
Prompts with Business Context Business Context highly effective in getting accurate answers

14. Accuracy and Findings
Results
• 0.5% contracts with unclear
License area co-ordinates
• Data confidentiality
incomplete in 2% contracts
• 1.5% contracts without
damage clause
• No commitment to local
employment in 3% contracts
• No clause to protect natural
resources in 4% contracts
• 87% Accuracy
Accuracy

15. Next Steps
Fine tune ChatGPT 3.5 with 100
contracts
Evaluate performance improvement

16. Thank you!
Thank you!