Intelligent Automation 2019

WELCOME TO
INTELLIGENT
AUTOMATION
2019
#iascot

MARK STEPHEN
BBC SCOTLAND
@BBCSCOTLAND
#iascot

RAY BUGG
DIGIT
@DIGITFYI
#iascot

MARTIN SQUIRES
THE ANALYSIS
FOUNDRY
@MARTIN_SQUIRES
#iascot

Harnessing Data and
Machine Learning to
Improve Retail Decision
Making
Independent Consultant In Marketing &
customer analytics and data science
Founder/Owner of The Analysis
Foundry Ltd

“Harnessing Data and Machine Learning to Improve Retail
Decision Making”
•The winners in retail are driven by those with the best quality data
•You need an agile delivery method to succeed
•How to make good use of Business Intelligence
•There is still a lot of low hanging fruit that is ripe for automation
•Skills are a primary barrier for capitalising on opportunity

Why Are Retailers Interested?
“We are in the era of big data, and big
data need statisticians to make sense of it.
The democratization of data means that
those who can analyse it well will win.
Data is the sword of the twenty-first
century, those who wield it well, the
samurai.”
Eric Schmidt & Jonathon Rosenberg “How
Google Works”

And Retailers Have Actually Being Doing This Stuff

Where Are Retailers Trying To Go?
IBM “The Battle For Personalised
Loyalty”
What will set winners apart?
- Who has access to the most data?
- Who has the best ability to garner
insight & act on data?
- Who can execute the best at the
moment of truth?

Who Has Access To The Most Data?

Unstructured Data vs Untidy Data

Data Factory vs Data Laboratory
WhoHasAccessToTheBESTData?

Who Has Best Ability To Garner Insight?

Data Scientists The “Sexiest Job in the 21st Century”

Working The Factory And The Lab
“Your scientists were so pre occupied with
whether or not they could, they didn’t
stop to think if they should.”
Dr Ian Malcolm, Jurassic Park

The Right Skills For The Right Jobs

The “Day Job” Analytics Tool Kit

Not All Insights Are Automated

REGINA
BERENGOLTS
TVSQUARED
@TVSQUARED
#iascot

© 2019 TVSquared All Rights Reserved
Advertise.
Attribute.
Act.
Supporting Those Who Support Others;
Scaling Customer Support
Regina Berengolts
June 20, 2019

tvsquared.com 27
Agenda
• The “Abouts”
• The Relationship Between Automation and Business Scale
• Case Study: Customer Support

tvsquared.com 28
The Worldwide Leader in TV
Attribution
TVSquared is trusted by thousands of brands,
agencies and networks in more than 70 countries

tvsquared.com 29
Global Footprint

tvsquared.com 30
About Me
Head of Data
Science
New Product
Research
Internal Business
Improvements

tvsquared.com 31
To Grow a Business, You Need More Than Just Algorithms
Build and maintain the
platform
Market/sell to new clients
Onboarding of new clients
Support and customer
success
Do more with, at least,
the same resource
1
2
3
4
5

tvsquared.com 32
Automation and Scaling

tvsquared.com 33
Intelligent Automation as a Continuum
Automation and scaling
Automation
• With manual
intervention
Robotic Process
Automation
• With digital
triggers or self
service
Machine
Learning
• With analytics and
decision engines
Artificial
Intelligence
• With deductive
analytics
Process-Driven
Data-Driven
https://medium.com/@cfb_bots/the-difference-between-robotic-process-automation-and-artificial-intelligence-4a71b4834788

tvsquared.com 34
Case Study: Customer Support

tvsquared.com 35
Customer Support at TVSquared
Client
Onboarding
Data Checks &
QA
Model
Calibration
Integrations
Training
Sales and Pre-
Sales Support
Ad-hoc Change
and Service
Requests
Client Care

tvsquared.com 37
Where and How to Have the Greatest Impact
Automation
• Basic client
communications
Robotic
Process
Automation
• Self-service
onboarding
Machine
Learning
• Data Checking
• QA
• Model Calibration
Artificial
Intelligence
Process-Driven
Data-Driven

tvsquared.com 38
Get your hands dirty
Define what “good” looks like
Map out the process and iterate
“Training” the System

tvsquared.com 39
Overcoming Challenges in Adoption
Trust is hard to get and easy to lose
Collaboration Transparency Staged Rollout

tvsquared.com 40
Team Impact
Process Validation and Implementation
Client care (daily requests and tickets)
Training and demos
Supervision of junior employees
Customer success/insight for higher value clients
Provide expertise internally
Onboarding
70% time reduction
QA
50% time reduction +
Improved accuracy
Model Calibration
Immediate output +
Improved accuracy
Junior
Roles
Senior
Roles

tvsquared.com 41
Business Scale
More Customers
More Satisfied Customers
Maintained Costs

GUY KIRKWOOD
UIPATH
@GUYKIRKWOOD
#iascot

QUESTIONS &
DISCUSSION
#iascot

REFRESHMENTS,
NETWORKING &
EXHIBITION
#iascot

Presenters:
Intelligent
Automation (IA)
“The Journey to Scale”
Presenters:
Susan Weerts
EY Intelligent Automation
Delivery Leader
June 2019
Jonathan Angove
Blue Prism Solution Consultant

Intelligent Automation
Innovation enabled by an integrated suite of
digital tools, using existing applications and
interfaces, leading to cost reduction, improved
customer experience and staff satisfaction

Building a blended workforce
People Automation
Virtual
workforce
characteristics
•Empathy & sympathy
•Judgement
•Complex problem solving
•Scenario modeling
•Building relationships
•Delivering low-frequency and
exception tasks
•Managing change and
improvement
•Rules based execution of high
volume transactions
•Algorithm-driven insights
•Unstructured to structured data
translation
•Advanced data analysis
•Big Data focused
•Communicating via emails, text,
social media
•Optical Character Recognition
•Natural Language Processing
• Non-invasive
• Works 24/7 with consistency
and accuracy
• Benefits can include:
— Increased productivity for high
value employees
— Improved customer & staff
satisfaction
— Cost reduction or avoidance
— Enhanced revenue
— Unlocked capacity

14,000 hours
1 week to 2 days
For supplier query response times

Hear from EY’s Chairman
DRAFT - Confidential

What defines a successful IA adoption at scale?
Clearly defined
purpose
Well equipped
people
Enhanced
processes to
realised benefits
Technology fit for
purpose

What lessons can you learn from organisations at scale?
Strategy and planning
Discover and solution
Build and run

The Journey beyond process automation
Cognitive
automation
Intelligent
Chatbots
Basic process automation
Hybrid solutions

Work logically
with a clearly
defined
purpose and
aligned goals
Be structured but
flexible in your
approach
Communicate and be
transparent

Contact
Neil MacLean
EY Intelligent Automation Lead Partner
Office: +44(0)131 777 2035
Mobile: +44(0)7467 442037
Email: NMaclean@uk.ey.com
Jonathan Angove
Blue Prism Solution Consultant
Mobile: +44(0)7912673311
Email: jon.angove@blueprism.com
Susan Weerts
EY Intelligent Automation Delivery Lead
Mobile: +44 7552 271 211
Email: SWeerts@uk.ey.com
Ed Mitchell
EY Intelligent Automation Lead
Mobile: +44 (0)7799 620707
Email: emitchell2@uk.ey.com

EY | Assurance | Tax | Transactions | Advisory
Ernst & Young LLP
© 2019 Ernst & Young LLP. Published in the UK.
All Rights Reserved.
EYG00001-162Gbl
The UK firm Ernst & Young LLP is a limited liability partnership registered in England and
Wales with registered number OC300001 and is a member firm of Ernst & Young Global
Limited.
Ernst & Young LLP, 1 More London Place, London,SE1 2AF.
ey.com

EYAD ELYAN
RGU
@ELYANEYAD
BRIAN BAIN
DNV GL
@DNVGL
#iascot

Machine Learning & Vision Applications
Eyad Elyan, PhD
Case Study: Toward Fully Automated Framework for Analysing & Interpreting
Piping and instrumentation diagrams

1 Background
Machine Learning
Deep Learning
2 Challenges & Research at CSDM
Training Examples Authenticity
3 Case Study: Engineering Drawings
Heuristic-based Solution Advanced
Methods
4 Conclusion & Future Direction

Plan
1 Background
Machine Learning
Deep Learning
2 Challenges & Research at CSDM
Training Examples Authenticity
3 Case Study: Engineering Drawings
Heuristic-based Solution Advanced
Methods
4 Conclusion & Future Direction

Machine Learning
Machine Learning gives computers the ability to learn without being explicitly
programmed (Arthur Samuel, 1959)
Observations (past examples) are used to train computers to perform certain tasks
such as predicting future events:
Spam detection
Fraud detection
Give a customer a loan?
Shape and object detection and recognition, ...
Massive amount of data, computational power and advanced Deep Learning models

65 Years ago
Paul Meehl published his book ‘Disturbing little book’ and in one of his studies he com
different medicalcases
In each of these 20 cases, the simple algorithms outperformed the well-informed huPaul E. Meehl, Clinical Versus Statistical Prediction: A Theoretical Analysis and a Review of the Evidence Minneapolis, MN: University of
Minnesota Press,1954)

1
1
http://www.cs.nyu.edu/ yann/talks/lecun-ranzato-icml2013.pdf

2
2
http://www.cs.nyu.edu/ yann/talks/lecun-ranzato-icml2013.pdf

Object Recognition
Until 2012: Leading methods used hand-crafted features + encoding methods
(e.g, SIFT+Bag-of-Words+SVM)
ImageNet - 2012 Large Scale Visual Recognition Challenge 1.2 million high
resolution training images and each image belongs to one of the 1000 classes. The
task is to get "correct" class in the top 5 best.
Winning result - Kritzevsky et.al. - 16.4% - Convolutional Deep Neural Network
Source:http://image-net.org/challenges/LSVRC/2012/

Face Recognition
DeepFace3 , a face recognition system was first proposed by FaceBook in
2014 achieved an accuracy of 97.35%, beating the state-of-the-art then, by 27%.
3
Y. Taigman, M. Yang, M. Ranzato, and L. Wolf, “Deepface: Closing the gap to human-level
performance in face verification,” in 2014 IEEE Conference on Computer Vision and Pattern
Recognition, June 2014, pp. 1701–1708

Cancer Detection, 2017
Google AI Just Beat Human at Detecting Cancer (89% vs 73% humans accuracy) 4
4
https://www.fool.com/investing/2017/04/04/google-ai-just-beat-human-pathologists-at-
detectin.aspx

Computing Power
Deep Learning solutions can
be deployedon small devices
and normal pc’s. however,
training these models require
more computational power
(i.e. cloud, GPUs).

Challenge: Training Examples - Data Availability & Distribution?

Training Examples - Availability
Large number of annotated
training images
Transfer Learning has been
successfully applied to
different problems
However, for a specific
domains, manual annotation
is still the most common
approach
Results:Adamu Ali-Gombe, Eyad Elyan, Chrisina Jayne, "Fish Classification in Context of
Noisy Images". International Conference of Engineering Applications of Neural Networks
(EANN) 2017: 216-226,DOI: https://doi.org/10.1007/978-3-319-65172-9_19

Training Examples - Distribution
Learning algorithms including deepmodels tend to be biased towardthe dominant
class of examples (common problem in different domains: Security, health, banking,
energy. . . )

Challenge: Training Examples - Data Availability & Distribution?
Possible Solutions

CDSMOTE: Class decomposition
Class decomposition
improved classification
accuracy significantly
Can weapply it to
majority-class instances
to reduce its dominance?
Unlike other
undersampling methods,
no loss ofinformation

CDSMOTE
Apply (CD) to
majority-class instances
to reduce its dominance
Oversample
minority-class instances
N P
0200600
N_C1
N_C2
N_C3
N_C4
N_C5
P
0
150
100
50
200

Overlap-Based Method
Identify and remove potential overlapped majority-class instances Use
soft clustering technique i.e. Fuzzy C-means instead ofk-means
Results:P. Vuttipittayamongkol, E. Elyan, A. Petrovksi, C. Jayne, "Overlap-based undersampling
for improving imbalanced data classification. International Conference on Intelligent Data
Engineering and Automated Learning (IDEAL 2018). Springer; November 2018

Generate Training Examples
Use GANs5 to create
realistic training samples
and add it to the original
dataset
Challenge: Requires
large number of training
examples?
5
Goodfellow et al. Generative Adversarial Nets

Generate Training Examples
(a) Imbalanced dataset[1] (b) Generate data to improvelearning[1,2]
1 A. A Gombe, E. Elyan, and C. Jayne 2019 Multiple Fake Classes GAN for Data Augmentation in Face Image Dataset. To
appear at the International joint conference on neural networks 2019 ( I J C N N 2019), July 2019.
2 A. A Gombe, E. Elyan, Y Savoye, and C. Jayne 2018. Few-shot classifier GAN. Presented at the International joint conference on
neural networks 2018 ( I J C N N 2018), 8-13 July 2018, Rio de Janeiro, Brazil.

Learning from Imbalanced Data
Learning from few
examples
Results: A. A
Gombe, E. Elyan
MFC-GAN:
Class-Imbalanced
Dataset
Classification... To
appear
(GANS)

Challenge: Authenticity of Images, Videos and other Types ofData

[source https://www.theguardian.com/]

Video Tampering
"Deepfake videos could ’spark’ violent social unrest"6
Recent developments in deeplearning-based methods have made it possible not only to
also to fully synthesize video content
6
source https://www.bbc.co.uk/news/technology-48621452

Video Tampering - The Challenge
So many ways to edit/
forge a video or image
No training data
available!

Video Tampering Detection
P. Johnston, E. Elyan and C. Jayne, “Video tampering localisation using features learned
from authentic content”, Neural Computing and Applications, 2019, pp. 1-15, DOIL
https://doi.org/10.1007/s00521-019-04272-z
P. Johnston, E. Elyan and C. Jayne, “Spatial effects of video compression on classification in
convolutional neural networks”,2018 International Joint Conference on Neural Networks
(IJCNN), Rio deJaneiro, Brazil, 2018, pp. 1-8, DOIL
https://doi.org/10.1109/IJCNN.2018.8489370
P. Johnston, E. Elyan and C. Jayne, “Toward Video Tampering Exposure: Inferring
Compression Parameters from Pixels”, Engineering Applications of Neural Networks, EANN
2018. Communications in Computer and Information Science, vol 893. Springer, DOI:
https://doi.org/10.1007/978-3-319-98204-5_4,

1nd Phase (Jan-2017 to Jan 2018)
Digitising Engineering Drawings (P&ID Diagrams)
This Work was Supported by the Data Lab - Innovation Centre and DNV.GL

Engineering Drawings
Complex problem (different
shapes of symbols, text, lines,
lowquality documents. . .)
Little existing work, despite the
significant progress in core
image processing tasks such as
shape and object detection,
recognition(R-CNN, SSD,
R-FCN, YOLO)
C. F. Moreno-García, E. Elyan & C. Jayne, "New trends on digitisation of complex
engineering drawings", Neural Computing and Applications (2018) 31:1695.
https://doi.org/10.1007/s00521-018-3583-1

Symbols Detection
C. F. Moreno-García, E. Elyan, and C. Jayne, "Heuristics-Based Detection to Improve
Text/Graphics Segmentation in Complex Engineering Drawings", 2017 International on
Engineering Applications of Neural Networks. EANN 2017, Springer, DOI:
https://doi.org/10.1007/978-3-319-65172-9_8

Dataset
X =
x21
x11 x12
x22 2n
. .. . ..
.
. . .
. . . x
...
. ...
x1n
,Y = ..
..
xn1 xmn ym
A dataset A with m instances of symbols x1, x2, ..., xm, where each instance xi is
defined by an n features (pixels) as
xi = (xi1, xi2, ..., xin).
y1
..
Learn a function h(x) that maps a symbol xi ∈A to a class yj ∈Y .

Results & Limitations
87
E. Elyan,C. F. Moreno-García, and C. Jayne,
“Symbols classification in engineering
drawings”,to-appear in 2018 International Joint
Conference on Neural Networks
(IJCNN),DOI:https://doi.org/10.1109/IJCNN.2018.84890
Heuristics needto be adapted to meet other
types of drawings / standards
Requires extensive user interaction (two hours
to complete adrawing)

2nd Phase (Jul-2018 to July-2019)
Digitising Engineering Drawings (P&ID Diagrams)
This Work was Supported by the Oil and Gas Innovation Centre, The Data Lab
Innovation Centre andDNV.GL

Deep Learning Framework
Data collection
Data/ Image annotation
Train and Test Deep Learning
models (detection,
classification)
Front-end
Iterate & improve

Minimise user’s
intervention
Reduce processingtime
Overcome limitations
of the heuristics-based
solution
Account for different
types of diagrams
Create a pipeline of
work

Problem
Un-digitised, paper
work
Solution
Automated method

Problem
Un-digitised, paper
work
Solution
Automated method
(less than 5 minutes)

Results
172 Diagram 90%:10% for training(155) and
testing(16)
P&IDs annotation of 29 class of symbols
On average 180 symbol in each diagram
92% of symbols were recognised
80% of text elements wereaccuratelyidentified

Conclusion & Future Direction
Data is certainly the newoil
Progress in Deep Learning solved many problems and at the same time creating
new ones
Collaboration: Joint projects with academia/ industry (H2020, EPSRC, KTP, Data
Lab, OGIC, UKRI Innovate UK, Industry PhD programs...)

Thank You
e.elyan@rgu.ac.uk
@ElyanEyad
https://www3.rgu.ac.uk/dmstaff/elyan-eyad

DNV GL © 20 June 2019 SAFER, SMARTER, GREENERDNV GL ©
20 June 2019
OIL & GAS
Academic-Industrial Collaboration Case Study:
118
Automated Extraction of Information From Engineering Drawings
Brian Bain: DNV GL
Eyad Elyan: Robert Gordon University

DNV GL © 20 June 2019
Global reach – local competence
150+
years
100+
countries
100,000+
customers
12,000
employees
5% R&D
of annual revenue
MARITIME DIGITAL
SOLUTIONS
BUSINESS
ASSURANCE
ENERGYOIL & GAS
Technology & Research
Global Shared Services
119

Data is valuable – unlock it’s potential
120
Manage the quality, sharing and use of data for better
insights and analytics
Collate
data from
multiple
sources
Connect
the data to
provide a
more
complete
picture
Get
insights
and
analytics

Our innovation initiatives:
Near-term solutions and long-term foresight
121
EMPLOYEE-SOURCEDINNOVATIONSTRATEGY-LEDINNOVATION
Years to market2 3 4 5 60 1
Long-term foresight
into technology for
sustainable industry
growth
Developing in-house technology
expertise in collaboration with
universities
Intense projects to develop bold
concepts or deeply explore specific
fields of technology
Developing near-term solutions
to digitalizing the oil and gas
industry
‘Crowd-sourced’ independent projects in
collaboration with industry partners, solving
common challenges
Strategic Research
Technology Leadership
Extraordinary Innovation Projects
Digital Innovation
Joint Industry Projects

The Problem: Extraction of Data From
Piping and Instrumentation Diagrams (P&IDs)
122
Equipment Items
Instruments and Displays
Coded Text
Drawing
Continuation
Labels

The Problem: Extraction of Data From
Piping and Instrumentation Diagrams (P&IDs)
123
Event Name
Equipment
Name
Equipment
Size
Equipment
Count
ABC/JRP_WDP/JAS_SEP/G Vessel N/A 0.508
ABC/JRP_WDP/JAS_SEP/G Flange 6 3
ABC/JRP_WDP/JAS_SEP/G Flange 26 1.5
ABC/JRP_WDP/JAS_SEP/G Flange 0.75 2
ABC/JRP_WDP/JAS_SEP/G Man Valve 1 9.5
ABC/JRP_WDP/JAS_SEP/G Man Valve 0.75 9.5
ABC/JRP_WDP/JAS_SEP/G Man Valve 8 3
ABC/JRP_WDP/JAS_SEP/G Act. Valve 6 1
ABC/JRP_WDP/JAS_SEP/G Piping 12 6
ABC/JRP_WDP/JAS_SEP/G Piping 0.75 10
ABC/JRP_WDP/JAS_SEP/G Piping 16 2.5
ABC/JRP_WDP/JAS_SEP/O Vessel 1 1.5
ABC/JRP_WDP/JAS_SEP/O Flange 1 32
A lot of information
stored P&IDs is
required to obtain a
listing of process
equipment for input
to engineering
studies.
This is a labour intensive process which we would like to automate

Some More Detail
▪ The oil and gas industry (like most others) stores it’s data in multiple forms.
▪ More recent data will be in a digital form but a large mass of legacy information is in pdf files or
paper copies.
▪ Even information which was created in a “smart” format with imbedded information may not have
been passed on in this way. It may have to be reversed engineered to extract this information.
▪ This is a labour intensive process which is costly and prone to error.
▪ Can machine learning and deep learning techniques be used to
❑ automate this process?
❑ make it more accurate?
❑ make it faster?
❑ make it cheaper?
❑ add more value?
❑ establish an audit trail?
124

Establishing the Project and Obtaining Funding
▪ DNV GL and RGU discussed the problem and contacted the Data Lab to
seek funding.
▪ An initial one year project was funded for 2017 to prove the concept
▪ Phase 2 of the project was set up to advance the technology.
▪ Funded primarily by the Oil and Gas Innovation Centre (OGIC) with
further assistance from The Data Lab
▪ Current phase is due to be completed at end of July.
125

Making Use of the Extracted Data
▪ As shown, information on the location of equipment is saved in a file for further processing.
▪ For the initial application post processing is required to identify connection between components
and inherit data from one to the other.
▪ In this application the main components are;
• Vessels
• Vessel Regions (split between areas containing gas, oil and water)
• Vessel connection points to the pipework.
• Pipework sections
• Process equipment (valves, flanges, meters, heaters, coolers, pumps etc.)
• Instruments
• Pipe Connections
• Text
• Drawing connections
126

Constructing a Global Network
▪ Information obtained from the deep learning tool can be used to work out which
elements are connected.
▪ It can also determine which pieces of text are associated with some of the elements.
▪ Combining all the connection information enables a global network to be established.
▪ The drawing connection information allows the network to cover multiple drawings.
▪ Information may be know about some of the elements and through inference this
information can be transmitted throughout the network.
▪ The process allows us to contextualise the data automatically, making it richer while
reducing the man hours involved.
127

Global Network – Vertical View
Vessels
128
Vessel Regions
Connection Points
(Parent)
(Child/Parent)
(Child)

Global Network – 3D View
129
Vessel Nodes
Pipes Pipe Junctions
Equipment Items

Plane View – Plan View Seeding and Data Inference
6”/4”
8”
6” 6” 4”
4”
4”
4”
4”
4”
4”
3”
3”
3”
3”
3”
3”
3”
3”
3”
8”/10” 2”
2”
2”
8”
8”
8”
8”
8”
8”
8”
8”
8”
10”
10”
10”
10” 10” 10” 10”
8”
A pipe size can be used to seed
part of the network
Equipment which changes the
pipe size can also be used.
A similar process can be used for
other parameters/properties.

Extraction of Data From P&IDs – The Solution
132
Event Name
Equipment
Name
Equipment
Size
Equipment
Count
ABC/JRP_WDP/JAS_SEP/G Vessel N/A 0.508
ABC/JRP_WDP/JAS_SEP/G Flange 26 1.5
ABC/JRP_WDP/JAS_SEP/G Flange 0.75 2
ABC/JRP_WDP/JAS_SEP/G Man Valve 0.75 9.5
ABC/JRP_WDP/JAS_SEP/G Act. Valve 6 1
ABC/JRP_WDP/JAS_SEP/G Piping 0.75 10
ABC/JRP_WDP/JAS_SEP/G Piping 16 2.5
ABC/JRP_WDP/JAS_SEP/O Vessel 1 1.5
ABC/JRP_WDP/JAS_SEP/O Flange 1 32
Continuity Labels
Number Tag x y w h Direction Location
1 JA04-03-AP-00094 5581 3468 283 61 right G4
2 JA04-03-AP-00013 182 2437 284 61 right A3
8 JA04-03-AP-00045 5581 1080 283 60 right G2
9 JAO4-03-AP-00177 182 979 284 61 right A2
Sensors
Number Type Tag x y r Location
1 Local Sensor 103-PT-1029 3662 2230 70 D3
2 Local Sensor 103-LD-1018 3954 2264 70 E3
3 Local Sensor 103-PST-1028 1534 2264 70 B3
18 Panel Mounted Sensor 103-LI-1018 3954 2006 70 E3
19 Local Sensor 103-LI-1015 3784 1658 70 E2
20 Panel Mounted Sensor 103-TIC-1021 3572 2834 66 D4
21 Panel Mounted Sensor 103-LSI-1017 3338 1772 65 D3
22 Panel Mounted Sensor 103-UA-1028 2110 1828 65 C3
Tags
Number Assoc. Equip. No. Tag x y r Location
1 8 103-SDV-1024 3796 3520 70 E5
2 14 103-SDV-1026 5106 3362 66 F4
7 62 103-PSV-1027 1472 834 65 B1
8 61 103-PSV-10278 2538 834 66 C1
Equipment Symbols
Number Class x y w h Location
1 DB&BBV (Vertical-Left) 4865 3855 82 74 F5
2 DB&BBV (Vertical-Left) 5133 3854 83 76 F5
13 Valve Ball 4589 3430 72 71 E4
14 ESDV Valve Slab Gate 4960 3339 71 145 F4
15 DB&BPV (Vertical) 3963 3304 99 73 E4
16 Valve Plug (Vertical) 4226 3249 49 72 E4
17 Reducer (Vertical-Down) 3980 3216 37 39 E4
18 Valve Ball 4377 3059 72 71 E4
28 Vessel Connection 2764 2583 33 38 C3
29 DB&BPV (Horizontal) 2675 2566 71 99 C3
This involves breaking
the drawing down into
its component parts and
identifying them.
The data can be
exported to a file
Post processing of this
data can be used to
allocate the parts count
information to various
loss of containment
scenarios.
This processed data then
becomes input to further
analytical applications.

Equipment to
Sensor and
Instrumentation
Linking
Automated
Validation
Checks
133
Visualisation
and Search
Tools
HAZOP Node
Identification
Data InheritanceNetwork BuildingClassificationSegmentationLoading image
Parts
Count
Allocation
Process Stages and Alternative Information Sets From P&IDs

Extension to Other Drawing Types
134
Piping Isometrics
Structural Layouts
Tabulated Data
Electrical Diagrams

Business Impact
❑ Faster Processing
❑ Less manhours = less cost
❑ More accurate (possibly)
135
Existing Processes
New Opportunities
❑ Transform paper copies to a digital format
❑ Amalgamation of different data types
❑ Digital twins containing data on physical assets
❑ Improved visualisation and identification
❑ Automated Design Checks

SAFER, SMARTER, GREENER
www.dnvgl.com
The trademarks DNV GL®, DNV®, the Horizon Graphic and Det Norske Veritas®
are the properties of companies in the Det Norske Veritas group. All rights reserved.
Automated Extraction of Information From Engineering Drawings
136
brian.bain@dnvgl.com
eyad.elyan@rgu.ac.uk

MARTIN THORN
CYBG
@MARTIN_THORN
#iascot

Organisational
challenges of A.I
implementations
MartinThorn
CYBGPlc

Classification: Public
endto the possibilities
there is no we are only
limited by our
imaginations
Machine learning & A.I – it’s the future, right?

So what’s stopping us?
meet
Derek

Why isn’t Derek on side?
overselling
trust
unflattering
people don’t like
change
machine learning
= A.I
scaredabout job
terminator
the
those
in black mirror
robot dogs

Overselling
revolutionise
this will
your business
only going
we’re
to get
one shot
everyone
elsedoing it
is
voice
to chatleft behind
we can’tbe
the
savingswill be huge

Trust
trust me the
algorithm
works it all out
7th May
2018
IFttt
r a d a rcruise control
do the techies
trust the tech?
customerchurn model
human makes mistake
‘puter makes mistake

Change
can you have a
revolutionwithout changing things?
meaningful change
is never easy
people
liketheir way of doing
things
can I say
to my
goodbye
car?
rationally it might
make sense
giving the chat project
to the voice
person
people
rational
aint
@ home

Unflattering
this will make us
ten timesmore productive
you’re saying we
aren’t productive now?
first
time
resolution
i know a good
employee
when I see one
what if “it”
spots something
we missed?
if it was worth
we’d have
already
doing done it
we’ve already
optimised
this workflow

Hmmmmmm
well that’s exceptionally
depressing
i’m afraid
it is toughsorry
this is not
going to be
easy
sorry
sorry
if it was
everyone
easy
would do it

My top tips
start
reallysmall
convince a
non-techie
don’t
ask for £100k
ask for £10k lots and lots
of tiny experiments
overcommunicate
worst case:build it out of hours dumb in prod >
amazing on the shelf
hire data scientists
that can
talkto
humans
pattern
recognition
not machine learning algorithms
aren’t sexy
d.p.o
are your
friends

Thanks for
listening
martin.thorn@cybg.com
@martin_thorn

IVANA
BARTOLETTI
GEMSERV
@IVANABARTOLETTI
#iascot

Privacy and Security in AI
Ivana Bartoletti – Head of Privacy, Data Protection and Ethics

Contents
▪ Background to Privacy and Security in AI
▪ Approach to Privacy and Security in AI
▪ Key issues with Privacy and Security in AI
▪ Digital Persistence
▪ Repurposing
▪ Training and Testing Data
▪ Solutions
▪ Persistence
▪ Accuracy
▪ Transparency
▪ Security
▪ Regulation of AI systems
▪ Questions and Answers
Gemserv 152

Background to AI and Privacy
• Organisations are increasingly
looking towards data analytics to
make more informed and efficient
decisions.
• Data analytics allows companies to
make sense of data and develop
patterns and predictions.
• Artificial Intelligence (AI) can allow
for evolving analysis of data,
predictive functions and even
decisions.
Gemserv 153
However, AI can lead to:
• Persistence: data once created will persist
longer that human have created it.
• Lack of transparency: AI systems may
appear to be a “black box”.
• Synthetic and Inferred Data: Systems
could rely on presumptions in using or
creating data.
• Security issues: AI systems can use
extensive databases and wide data
collection, which can pose issues for
keeping data secure.

Approach to Privacy and Security in AI
Gemserv 154
Context and Setting
An individual’s ‘reasonable expectation’
of privacy will differ depending on the
context.
Security Elements
Different system setups used in AI
systems – including the use of
databases, APIs and cloud servers, can
involve different security vulnerabilities.
Transparency
Individuals may not be aware of
how their personal data will be used,
particularly when algorithms or “black
box” decisions are used.
Behavioural Economics
An individual’s desire for data
privacy will depend on how they
anticipate that data's effect on future
economic outcomes.

Background to the deployment of AI
• Online Advertising Systems characterise individuals into
social and demographic categories on the basis of tracking
their online behavioural interests.
• AI-based chatbots supplement the work of call-centre staff
in the pressure of rising demand and reduced resources.
• Smart Homes monitor residents’ and homeowners’ use of
appliances at home and behavioural habits, in order to
reduce water and energy use.
• HR systems powered by AI and used to shortlist and screen
candidates on the basis of their backgrounds or worthiness
assessments.
• Robotic Process Automation (RPA) for high-volume
repetitive activities which are difficult or costly to automate
with traditional system integration techniques.
Gemserv 155
• E-commerce and Retail websites using chatbots, cookies,
online advertising/profiling.
• Healthcare increasingly relies on diagnostic tools to monitor
patients and prescribe treatment.
• Energy sector is using AI in various applications, including
the context of analysing consumption trends, smart grids, and
oil and gas exploration.
• Manufacturing is using AI to improve customer interaction
with products, such as connected cars and smart devices.
• Local government are using AI to facilitate the delivery of
public services, including decisions around the allocation of
social benefits and ‘smart cities’.
SectorsProducts

Key Privacy Issues in Security and AI
Gemserv 156
PERSISTENCE ACCURACY TRANSPARENCY SECURITY

Persistence
The persistence of data creates problems for individuals to exercise their rights not be
subject to processing and for control or autonomy over their personal data.
Gemserv 157
Retention Periods
In the digital environment, information
can persist for long periods and in
different forms.
Organisations can continue to use
personal data for ongoing analysis.
Applicability
In particular, the currency of data may
change over time – rendering it
inaccurate or irrelevant.
Use of AI systems can exaggerate these
effects, where the algorithm continually
learns from and reuses the data.
Example
A woman uploads photos onto a
social media platform, aged 15.
Ten years later, the recruitment
department of a company where
she applies for a role examines
her Facebook profile to decide if
she will be a reputable candidate.

Accuracy
The inaccuracy of data may run into compliance issues when such data is used to make
decisions about individuals. Individuals generally have a right for it to be corrected.
Gemserv 158
Inferences
In the course of profiling, AI systems may
extrapolate or infer certain characteristics
about an individual.
Additionally, statistical data could be used to
profile people, based on demographic
characteristics and trends.
Discrimination
Discrimination can result from inaccurate data
that makes assumptions about trends and
behaviours to group individuals.
For example, defining ‘good employees’ and
‘productive workers’.
Example
A large retailer uses personal
data collected through cookies
about browsing history and
interests in order to create
profiles of individuals. This is
used to target specific
advertising at them.
Learning
If the system "learns" from biased or
inaccurate training data, AI systems can
exacerbate existing inaccuracies.

Transparency
Summary
Gemserv 159
Class labels
Issues can be unfairly applied to individuals
where companies can choose their own variables
and labels.
For example, defining ‘good employees’ and
‘productive workers’.
Efficiency trade-offs
There is a trade-off between limiting
algorithm functions to ensure transparency
and accountability and ensuring efficiency.
The case of the Black Box and the myth of
explainability.
The GDPR
• Art. 22
• Recital 71
• Article 15: the way
forward? Interplay with
information rights (13,
14)

Explanations
Gemserv 160
Explanations
• Model Centric Explanations (MCEs)
• Subject Centric Explanations (SCEs)
• Limits and barriers
Better explanations with the
GDPR?
• Right to erasure
• Data Portability
Privacy by Design:
• Bias & fairness
• External agencies to test
systems
• Audit trails
• DPIAs
• Certification Schemes
Scope of Article 22 GDPR

Security
Gemserv 161
Database security
Large databases used in AI systems
represent a significant target to identity
thieves, data breaches and other
incidents.
Organisations can protect such data by
using methods such as encryption at rest.
Authentication
Staff will typically have need access to AI
systems or database to support data
subjects with their right to human
intervention.
GDPR example
The GDPR requires organisations to
introduce appropriate technical and
organisational measures for data
security on a risk basis.
Organisations should consider relevant
measures to ensure the confidential,
integrity and availability of the data.
Summary
Cloud or server-based
If data is stored on cloud-based system,
including that:
• Communications for the transfer in and out
of the cloud is kept secure;
• Ensure you comply with legislation for
international data transfers.

AI and Privacy Solutions
Gemserv 162
The following principles should be followed in the delivery of AI solutions:
Data sets should be…
…Chosen and cleaned to avoid any issues with accuracy or content that
could have unfair impacts on individuals
Algorithmic functions need to be constrained…
…To avoid inferring or ascribing characteristics to individuals unfairly
Privacy by Design principles…
…Need to be embedded in the system to ensure data collected is the
minimum necessary
… The issue of the BLACK BOX and the question of explainability

Gemserv 163
Data Security
Organisations should ensure Data Security
around the use of AI analytics, such as by:
• Ensuring appropriate access controls are
introduced for uses on APIs and databases
used by AI systems.
• Carrying out an assessment of the
vulnerabilities on networks where data are
stored and accessed. Third Parties
With regard to third parties, organizations must:
• Carry out due diligence of third parties, including hosting
systems and bought-in data sets.
• Ensure that liability for decisions made as a result of AI
systems is apportioned with third parties.
Codes of Conduct
Organisations should ensure transparency around the use
of AI analytics, such as by:
• Publishing a Code of Conduct, outlining the
organisation’s commitment to fair and unbiased AI and
to protecting privacy.
• Use transparency notices to inform individuals of how
their personal data with be used and their rights.

Gemserv 164
Training and Testing
When training and testing AI systems, organisations
should:
• Introduce procedures for checking for accuracy and
data cleansing.
• Evaluate the impact of data analytics and profiling on
groups of individuals.
Individuals Rights
Policies and procedures should ensure that:
• Data subject rights, including access and correction to
personal data, can be processed by staff and systems.
• Staff are trained to recognized, respond and escalate such
requests.
Algorithmic Impact Assessments (AIAs)
Algorithmic Impact Assessments take the form of an
audit or gap analysis that allows an organisation to
determine whether they comply with legal, industry
and ethical requirements and norms.
They have been recommended to be conducted by
the IEEE.

Key Points
Gemserv 166
AI should service defined goals and the public interest
AI systems can exaggerate many existing data privacy issues
Transparency and data ethics should be at the core of systems
AI systems and networks present security challenges

Thank you, any questions?
Ivana Bartoletti
Head of Privacy, Data Protection and Ethics
dataprotection@gemserv.com

PANEL
MARTIN THORN - CYBG
IVANA BARTOLETTI - GEMSERV
SVEA MIESCH - SCOTLANDIS
ANDREW BONE - AIRTS
#iascot

DX3
Distributed - Decentralised - Disruptive
DX3os is a decentralised operating system for autonomous
industrial machines
simon@dx3os.com | www.dx3os.com
© DX3 2018-2019

The Machine Economy
"The Machine Economy is one in which machines are autonomous market
participants that have their own bank accounts. In the near future, it's
expected that M2M participants will be able to lease themselves out, hire
their own service engineers and pay for their own servicing and
replacement parts” - McKinsey
© DX3 2018-2019 | Simon Montford | www.dx3os.com | @dx3os

Machine Commerce

Machine Automation
Source: PWC

Real-World Examples

Open Platform

What is DX3?
DX3 is a decentralised ecosystem that will facilitate
autonomous machine-to-machine interactions and
transactions.
What is DX3os?
DX3os is an operating system [similar to middleware]
that connects autonomous industrial machines.

Project objectives
Objectives: increase operational
efficiency, trust, and safety. Reduce costs
of doing business and exposure to
cyberattack.
Energy | Mining | Construction

DX3 Consortium
Mission: open-source enterprise-
focused Blockchain Platform and
Ecosystem, that will give all participants
an equal opportunity to create and
obtain value.

Commercial Benefits
Value Creation: increase ROI by
automating tasks and enabling
autonomous industrial machines and
edge devices to operate autonomously.

DX3C - Contact us to find
out how to join The DX3
Consortium
Distributed - Decentralised - Disruptive
simon@dx3os.com | www.dx3os.com/dx3c.html

Geoff Ballinger / Head of Platform / @geoffballinger
The Internet Of (very big) Things:
Adventures in Using Event Based Vision Systems for
Localisation on Trains

Future transport will be highly automated and coordinated.
Knowing the position of each vehicle will be critical.

Sensors, data and algorithms are replacing infrastructure-
based localisation such as GPS
Image courtesy of Waymo

People didn’t need infrastructure to navigate in the past

The ground has measureable structure that is unique to a location

Image courtesy of Prophesee
Event-based vision solves many problems inherent in
traditional computer vision

Our on-vehicle camera-based mapping solution
determines accurate location using ground fingerprints

Currently no localization solution is accurate enough
for predictive maintenance

We improve localisation by reducing motion estimation error
AND continuously correcting for drift
State of
the art
GNSS
error
5% motion
est. error
1% motion
est. error

Initial trials gave 2cm accuracy at 100km/h
GPS

RailLoc
RailLoc Engine
RailLoc Sensor
RailLoc Map
Server
GPS
Track ID, Position
& Velocity
Offline GPS
Correction

Geoff Ballinger / Head of Platform / @geoffballinger
Thanks for listening!

Adopting
Automation
Andrew Bone
www.airts.co.u

“ To take the path of
automation, you must desire
progress and you must trust
technology”
Andrew Bone, a few seconds ago

What kind of Automation?
Projects
Optimised Plans
People

2. Controls a process that is:
• Strategically important
• High risk
• Emotive
Automation intertwined with people
1. Works in partnership with human planners

Automation
Time
0: None
1: Assisted
2: On
Demand
3: Always On
TrustAutomation Adoption Curve

Andrew Bone, 10 minutes ago
“To take the path of
technology”

Desire
Trust
Ostriches
Staunch
Traditionalists
Nervous
Flyers
Confident
Champions
Attitudes to automation

Case Study 1: Big 4 Delivery Centre
• >1,000 people based in Poland
• >50,000 tasks a month
• Tasks last minutes or hours
• Very dynamic
• Complex task assignment criteria
• Pressure to be more efficient

Automation
Time
0: None
1: Assisted
2: On
Demand
3: Always On
Automation Adoption Curve

Case Study 1: What was our journey?
Desire
Trust
12 3
Desire
Trust
1
2
3
4

Case Study 1: What did we learn?
Huge
Desir
e
Start simple and build trust!
Quest for
Perfectio
n
Too
Clever
Lack of
Trust

Case Study 2: UK Prof. Services Firm
• 4,500 auditors based in UK
• Approximately 36,000 tasks a month
• Tasks last days or weeks
• Somewhat dynamic
• Incumbent system 20 years old

Desire
Trust
2
1
3
Case Study 2: What was our journey?
Desire
Trust
1
2

Case Study 2: What did we learn?
Automation is not everything
Easy
Wins
Automatio
n
Deprioritise
d
Compellin
g Data
Future
Automation

“To take the path of
technology”
Andrew Bone, 25 minutes ago

Desire
Trust
Modest
Wins
ProveValue

Andrew Bone
andrew.bone@airts.co.uk

The Internet
The internet was invented in the 1970s by the US military, Sir Tim
Berners-Lee, and Al Gore to share photos of cats. It has grown a bit
since then. 2018 marked the point when most of the human race was
online. A huge step forward in information sharing. But not everything is
going as planned. Let’s talk about data, AI, and advertising.

GOOD-LOOP
We giveadvertisermoneyto
charity.
And everyone wins.

TalkStructure
1. Introduction
2. TheDataEconomy&Current Trends
3. Ethics
4. Howpersonaldataworksin ads
5. Questions&Comments

Howbig isthepersonal
dataeconomy?
● DMPs(minusthebig two):
$0.5 to $1.5 billion
(Forrester,MarketResearch Future)
● Overallmarket effect:
$18 billion to $1,000 billion
(OnAudience,EuropeanCommission)
DMP = Data Management Platform

EverySaleisaPersonalSale
EveryAdvert isTargeted…buthow
well?

CurrentTrendsin AI-for-Advertising
● Bigger,faster, stronger!
○ Morebuilt-in AI
○ Moreclosedloops
○ Moredata?
● Growthof chatand voice.
● Intentdrivenadvertising(i.e.search
keywords)
● AIassistantsfor campaigns
...includingsomecreative content

LiveRamp? Lotame? BlueKai?
Have you heard of them?
But they have heard of you. Or rather, they
provide a grey market for data on you to be
bought and sold. Welcome to data-laundering.

HowTrackingWorks
● Ads and webpages include tracking pixels
from various DMP companies(e.g. BlueKai,
AppNexus,butalsoGoogle, Facebook).
● A pixeldropsacookie onyourbrowser.
● Thebrowsercanthenberecognisedacross
sitesandadsthatcarrythatDMP’s pixels.
● Loginsorguessescantrackacrossdevices
(e.g.desktop→ phone).
● Cookie-syncing betweencompaniesjoins
activityacrossdatasets(maybeillegal).

Thank you
Questions &
Comments

Intelligent Automation 2019

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Intelligent Automation 2019

Similar to Intelligent Automation 2019 (20)

More from Ray Bugg

More from Ray Bugg (20)

Recently uploaded

Recently uploaded (20)

Intelligent Automation 2019