Data analytics in procurement: a brave new world of opportunity or a pandora's box of risk? webinar, 16 May 2019

Copyright Projecting Success 2019
Martin Paver
CEO / Founder
www.projectingsuccess.co.uk
martinpaver@projectingsuccess.co.uk
+44 777 570 4044
Data Analytics in Procurement: New
Opportunity or Pandora’s Box of Risk?
Thursday 16 May 2019

Experience
Chartered Engineer
Fellow
Chartered Project Professional
Professional Accreditation
Sectors
Project Manager $1bn
Programme Director $0.6bn
Portfolio lead $10bn
Roles
Icon credit: Icons8

Introduction to Project Data Analytics

The parent term
encompassing any technique
that allows a machine to act
like a human
AI, ML and Deep Learning
Artificial
Intelligence
(AI)
An AI technique that
focusses on learning from
experience
Machine
Learning
(ML)
A subset of ML that uses
neural networks based on
the brain
Deep
Learning

Why the Hype?
Data Cloud Algorithms
Icon made by Freepik from www.flaticon.com
In 2016, 90% of the world's data (that's 90% of all the data ever created)
had been created in the previous two years (IBM).

Potential Use Cases

Project Management
Risk lifecycleConnected risks Risks-Issues-Lessons
Informed risk
registers
Risk
trends
Risk mitigations
Risk
budget
Adaptive Scheduling

This presentation will stimulate debate
The topic has a Brexit / Marmite quality to it
Depends on perspective. Please think macro.
Its not a case of should we do it….
We can do it.
People will apply it
The question is how we shape it.

Use Cases Operating in Tension
Public Sector
Use cases:
• Encourage/maximise competition
• Improve quality of bids and solutions
• Scenario modelling
Hidden objectives
• Prevent precedent
• Prevent gaming
• Prevent reputational damage
• Limit number of appeals
Private Sector
Use cases:
Bid strategy
• Identify opportunities with greatest PWin
• Inform bid/no bid decisions
• Influence scoring mechanisms
• Identify areas for improvement
• Shape future strategy, which clients
• Shape improvement investments
• Which clients are loyal to which suppliers
Collaborate
• Identify ‘best’ collaborative partners
Challenge
• Identify bids which are outliers
• Insights into bias in bid decisions
Scenario modelling
• If we do this then…..
Exports

Construction
A focus on this use case because of the availability of data.
The approaches could apply to other sectors too.

A Backdrop in Construction
Credit: Building.co.uk
Credit: Newstatesman
Credit: FT.com
Some of the Government’s contractors developed
unsustainable business models over recent years,
underbidding for contracts, recklessly acquiring other
businesses and maintaining high bonuses and dividends.
…….. Government is in for the long term and cannot take
such risks with public money or with the security of
public services. The Government as the major customer
of these firms is responsible for the services they supply
and consequently needs to ensure that its contractors are
able to deliver those services sustainably.
Credit: Parliament; After Carillion: Public sector outsourcing and contracting

A Backdrop in Construction
30 April 2019

Gaining Access to Data
TED Data
Contract Finder Data
Subscription Services
FOI Requests
Plus
70,000+ Contracts
Unsustainable
Should this be in the public domain?

FOI Requests
Should the public know ….who won a competition?
….who competed?
….what the winner’s score was vs all other scores?
….composition of all scores?
….composition of the winning score?
….who achieved which scores?
Ruled in my favour Appealed

FOI Requests - Tribunal

Icons made by Freepik from www.flaticon.com
Machine Learning: Bid Data – a worked example

What Can be Derived from the Data?
• 1000s of features are derived from the data
• Patterns can then be identified in the features
• The data enables us to derive:
• Who is likely to bid
• Who will win – marginal or significant lead
• Top 3 and probability of each
• Success rates for each supplier
• Client loyalty
• Variance in performance of suppliers across different clients
• Anomalies

PWin
Aim – to use existing contract data to predict future bid
outcomes
Data - using ~10,000 contracts from construction industry
1. We can predict
bid outcomes to an
accuracy of 30%.
2. This increases where there is
a greater amount of unique
data, such as from Network
Rail (50%) and North Tyneside
(91%)
3. Will increase
significantly as
bid feedback is
added.

Examples of what can be achieved
Supplier PWin Dashboard
Who are the 5 most likely contractors for a particular opportunity?
Which contractor has the highest chance of winning?
What is your win rate for this opportunity?

Predicted Sector – in which sectors are
suppliers winning the most work?
Supplier Churn – do
certain clients keep using
the same suppliers?
Joint Ventures – which suppliers have
the greatest success of winning a JV?

2. Bid Scores
Aim – to take the scores awarded to bidders and identify variances between
suppliers and frameworks
Data - raised FOI requests for 42 frameworks and ~200 underlying contracts.
Outcomes
✓ 3500 bid scores obtained
✓ Who scored what
✓ Scores for each set of criteria

Bid analysis – against which criteria are
your competitors scoring better than you?
KPI Analysis – which
bidders have a greater
spread of scores
compared to the
average?
Box and Whisker Plot –
what was the distribution
of successful and
successful scores for a
particular framework or
set of frameworks?

What Insights Can be Derived from Data
Suppliers who bid for
more contracts (size of
bubble) tend to have a
higher average score and
lower spread of scores
*Bubble size = number of contracts bid for
Smaller
organisations
tend to have a
lower average
score and higher
spread of scores

Successful suppliers on average have a lower standard deviation (0.21) of
scores compared to unsuccessful ones (0.29)
*Bubble size = number of contracts bid for

*For top 10 suppliers by number of bids
There is a level of consistency across frameworks, but there are some outliers.
Bid feedback scores will enable us to understand this variance in greater detail.

Are criteria being scored consistently across the frameworks?
Health & Safety:
Min score – 40%
Max score – 75%
Environmental
Sustainability:
Min score – 49%
Max score – 92%

For a particular bidder, are criteria being scored consistently across the frameworks?
Environmental
Sustainability:
Min score – 70%
Max score – 100%
Business Resilience
& Continuity:
Min score – 50%
Max score – 92%
Health & Safety:
Min score – 70%
Max score – 100%

…. who achieved which scores?
• Identify variance in scoring within and between frameworks
• Encourage consistency
• Harmonise approach between frameworks
• Remove the lottery component
• Understand whether a supplier can ever close the gap
• Informs continuous development
• Informs market strategy
• Centralised scoring of common features (H&S, BIM)
• Refine market engagement strategies
We need to view this through a MACRO lens, not the lens of individuals or departments

The Challenges and Risks
Legal risks:
• Collusion
• Sharing open data vs sharing company only data
• Perception of companies withdrawing from competitions
• Can clients legally change scoring mechanisms?
• Using legacy data to shape procurement decisions
• Perception of blacklisting
Procurement risks:
• Competition collapses in areas with dominant supplier
• To what extent can legacy data shape procurement decisions
• Challenge procurement decisions
Data volumes:
• Limited performance with small data samples
• To what extent are past results an indicator of future results in:
• A rapidly changing sector, e.g. modular construction
• Collapse of key suppliers
• Changing company strategies

Scenarios: Collapse or Thrive?
Thrive
• Provide opportunities for suppliers to improve
• Harmonisation across frameworks – less of a lottery
• Have a view on PWin before committing to bid
• Reduction in bidders, but closer clustering
• Shape the scoring to encourage competition – legalities?
Collapse
• Dominant suppliers double down
• Weaker suppliers withdraw
• Suppliers are able to identify underbidding
• Saving in nugatory bid costs
• Client costs increase
Note: Suppliers are already being very cautious of how
they invest their business development funding.

Scenarios: Competition is Transformed
• Some elements scored centrally, periodically e.g. H&S
• Move towards more strategic partnerships
• Leverage existing designs, modular construction
• Greater sharing of project delivery data
• Pooling of data to drive productivity
• Improving confidence in out-turn, predicting issues
• Harmonisation across contract frameworks.
• Less of a lottery

Impact on Procurement Professionals
• Greater understanding of statistics and data science
• Transparency drives accountability – embrace it
• Identify variance
• Identify bias
• Scenario analysis
• Benchmarking; develop the ‘outside view’
• This is disruptive: Understanding of emerging case law
• How to adjust the requirement and scoring to:
• Secure the most cost effective solution
• Encourage competition
• Potential for increased automation:
• Tracking past performance across multiple
frameworks
• Automatic scoring of some elements of bids.
Civil service code
•‘integrity’ is putting the obligations of public
service above your own personal interests
•‘honesty’ is being truthful and open
•‘objectivity’ is basing your advice and decisions
on rigorous analysis of the evidence
•‘impartiality’ is acting solely according to the
merits of the case …
You must not:
•ignore inconvenient facts or relevant
considerations when providing advice or
making decisions
•act in a way that unjustifiably favours or
discriminates against particular individuals or
interests
Advanced data analytics underpins all of the
above

Data Trusts

Summary
• This is all possible today
• People will use it – how do we shape it?
• Insights will shape bid decisions for both parties
• Increase in number of challenges of public procurement decisions
• Legal issues are a grey area
• How will this transform public/all procurement?
The answers are emergent

• Free to join
• Free to attend
• Crosses professional boundaries
• Developing a community
• Developing into a force for good
Developing a Community
Like on Linked In or Retweet along with
#ProjectHack3.0 to be
Entered into a prize draw for a Google Home Mini

PROJECT DATA ANALYTICS COMMUNITY
The UK’s biggest Project Data Analytics Community
REGIONAL EVENTS
NATIONAL EVENTS
Founded Dec 2017
2,558 members
ConferencesProject:Hack Product Demos
Founded April 2019
167 members
Masterclasses
Founded April 2019
140 members
https://projectdataanalytics.uk/event-page

Contact
Please find me on Linkedin:
Martin Paver
Martin Paver
CEO / Founder
www.projectingsuccess.co.uk
martinpaver@projectingsuccess.co.uk
+44 777 570 4044
Also follow the
Project Data Analytics
Community

This presentation was delivered
at an APM webinar
To find out more about upcoming webinars
please visit our website
www.apm.org.uk/events

Data analytics in procurement: a brave new world of opportunity or a pandora's box of risk? webinar, 16 May 2019

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Data analytics in procurement: a brave new world of opportunity or a pandora's box of risk? webinar, 16 May 2019

Similar to Data analytics in procurement: a brave new world of opportunity or a pandora's box of risk? webinar, 16 May 2019 (20)

More from Association for Project Management

More from Association for Project Management (20)

Recently uploaded

Recently uploaded (20)

Data analytics in procurement: a brave new world of opportunity or a pandora's box of risk? webinar, 16 May 2019