This document presents a case study comparing the efficiency of different project delivery methods (DBB, DBF, DBFOM) using data envelopment analysis (DEA). The study analyzes the Presidio Parkway project in California. DEA is used to determine the relative efficiency scores of each delivery method. Dummy variables are then added to overcome issues with identical efficiency scores. Finally, a two-stage analysis is conducted, regressing efficiency scores on a contextual variable (level of service) to evaluate its impact, in line with the Banker-Natarajan model. The analysis finds private sector involvement improves factors like level of service and that a one unit change in this variable is associated with a 74.11% change in

1. Efficiency Based Comparison
of Project Delivery Methods
By
Deepak Sharma, Sudarshan Kurwadkar, Chandra
Putcha, Rajnish Lekhi
July 20, 2019
Calgary, Canada

2. Presentation Outline
• Introduction to Project Delivery Methods and PPPs
• Research Motivation
• Case Study
• DEA Model
• Results
• DEA with Dummy Variables
• Results
• DEA + OLS
• Results
• Conclusions
Revised as per comments
and suggestions

3. What is a Project Delivery
System?
• A project delivery system defines the
organizational framework for project development
and management.
• A project delivery system has tremendous influence
on contracts signed between parties, risk transfer,
payment mechanism, and many other important
aspects of construction project management.

4. What are Public Private
Partnerships?
• Public Private Partnerships (PPPs), are project delivery
systems that are used various infrastructure projects
such as highways, hospitals, schools, water treatment
systems, and many other infrastructure projects.
• PPPs are contractual agreements formed between a
public agency and a private sector entity that allow for
greater private sector participation in the delivery and
financing of projects.
• Many times, a highway PPP may result in a toll road.

5. Types of PPPs
• DB: Design-Build
• DBM: Design-Build-Maintain
• DBO: Design-Build-Operate
• DBOM: Design-Build-Operate-Maintain
• DBFOM: Design-Build-Finance-Operate-Maintain
• BOT: Build-Operate-Transfer
• BOO: Build-Own-Operate …… and several others

6. Types of PPPs
Image Source https://www.fhwa.dot.gov/policy/2006cpr/images/43h03.jpg

7. Types of PPPs
Image source: Roehrich J. K, Lewis M. A., George G. (2014)

8. The Procurement Process
• PPPs are NOT a “One Size Fits All Solution”
• Procurement can be roughly divided in two stages.
• First Stage
o Agencies aims to determine which PPP will be most
suitable.
o First stage purpose is to determine
a. If PPPs are suitable
b. If yes, which one?
• Second Stage
o Identify a suitable contractor for project delivery

9. Value for Money Analysis
A Quick Review
9

10. The VfM Analysis: Quantitative Part
The methodology for
carrying out a VfM analysis
involves preparing Public
Sector Comparator (PSC)
and comparing with all the
possible PPP options
Net Present Value (NPV) of
both options are obtained
and option that has higher
NPV is considered for
procurement
10
Image Source: FHWA (2013)

11. The VfM Analysis: Qualitative Part
• Besides quantitative evaluation, VfM also includes
qualitative evaluation
11
DECISION MAKING USING VfM ANALYSIS
Candidate
Project
Quantitative
Analysis
Qualitative
Analysis
NPV ($)
as Output
Non-dollar
Value Output
Decision
(Subjective
Integration)
VfM
Analysis

12. The VfM Analysis: Taking final
Decision
• It is difficult to combine qualitative assessment
with quantitative assessments (Prokopowicz 2014)
• “each party develops different perceptions
regarding these risks, potentially producing
unanticipated or undesired consequences”. (Chan et
al. 2009)

13. The VfM Analysis: After Final
Decision
• Her four is my three
Three

14. The VfM Analysis: After Final
Decision
• Her four is my three
Three
DBFOM
DBF
Time Saving, Less
Pollution, etc.,
etc., etc.
Freeways free of
congestion!
Three
Three
• In-efficient procurement practices have led to project cost
escalations (FHWA (2012); Kenny. C. (2010) & Flyvbjerg et al.
(2003))

15. Research Need
• PPPs have evolved but the VfM assessment has not
changed much (Tsukada 2015).
• VfM assessments results could be dubious because of
the assumptions and possible biases (Chan et al (2009) &
Garvin (2010)).
• There exists several issues that could lead to inaccurate
VfM assessments (Leigland and Shugart (2006), &
Grimsey and Lewis (2007) ).
• Governments need to strike the right balance between
qualitative & quantitative approaches (World Bank (2013))
• Methods that can integrate quantitative and qualitative
outcomes will increase transparency and public
confidence
15

16. Why this is important?
• As per World Bank, $69.9 B was invested on
transportation PPP projects throughout the world
(Kasper and Saha, 2015).
• This is a 53% increase from the past 5-year
average and 86% above the past 10 year
average.
• PPPs have enabled the public agencies to complete
several projects that would not have completed
without higher levels of private sector involvement

17. Why this is important?
• In the U.S., 33 states have passed PPP enabling
legislation and few more are planning for it or are
already in the process (Papajohn et al., 2011; Cui &
Lindly, 2010)
• We need PPPs!
• Current US infrastructure assessment resulted in a
D+ grade and the roads and highways got a further
lower grade of D (ASCE 2017).
• These issues prevail in every procurement,
irrespective of sector, where a decision is to be
made on the basis of combined understanding of
cost and quality.

18. Data Envelopment Analysis (DEA)

19. Data Envelopment Analysis
(DEA) – The Solution
• Data Envelopment Analysis (DEA) enables
efficiency-based comparison of various entities.
• DEA calculations require inputs and outputs to
determine relative efficiency scores.
• Banker and Morey (1986), for the first time
presented a DEA model that allows for seamlessly
combining qualitative data (categorical data) with
quantitative data. The model has a direct
application to the PPP procurement problem.

20. The Banker-Morey Model (1986)
𝑀𝑖𝑛𝑖𝑚𝑖𝑧𝑒 ℎ0 = 𝑍0 −𝜀
𝑖=1
𝑚′
𝑆𝑖
−
+
𝑟=1
𝑅
𝑆𝑟
+
𝑗=1
𝑁
𝜆𝑗 𝑋𝑖𝑗 + 𝑆𝑖
−
= 𝑍0 𝑋𝑖𝑗0
𝑗=1
𝑁
𝜆𝑗 𝑋𝑖𝑗 + 𝑆𝑖
−
= 𝑋𝑖𝑗0
i = (1, 2, ….., m’) (2)
i = (m’+ 1, m’+ 2, …, m) (3)
𝑗=1
𝑁
𝜆𝑗 𝑌𝑟𝑗 + 𝑆𝑟
+ = 𝑌𝑖𝑗0 r = (1, 2, …., R) (4)
𝑗=1
𝑁
𝜆𝑗 = 1 (5)
𝜆𝑗 ≥ 0 j = (1, 2, …., 𝑗0, … N) (6)
subject to:
𝑆𝑖
−
≥ 0, 𝑆𝑟
+
≥ 0 (𝑖 = 1, 2, … , 𝑚; 𝑟 = 1, 2, … , R)
(1)

21. Further Application: Evaluating
Contextual Variables
• Banker and Natarajan (2008) presented an approach to
evaluate contextual variables affecting productivity.
• This requires performing the following steps:
• First Stage: Determine technical efficiency ( 𝜃) by performing
DEA analysis on input-output data
• Second Stage: Estimate the following relationship using the
OLS method.
ln 𝜃 = 𝛽0 − 𝛽𝑧 + 𝑒1
• The two-stage method will enable determining the
effectiveness of the decisions made during the
procurement stage on the basis of socioeconomic
factors and performance-based parameters

22. Case Study
Presidio Parkway Project (Caltrans)
22

23. Case Study
Presidio Parkway Project (Caltrans)
Description Detail
Total Cost of project $1969 M
Number of lanes 6 Lanes
Length of each lane 1.5 mile
Southbound Additional Lane 1 Lane
Length of additional lane 0.5 mile
Length of each lane 1.6 miles
Pavement type Continuously Reinforced Concrete
Concession period 30 years
Beginning of Concession
Period
Year 2013
End of Concession Period Year 2043
PPP Type DBFOM
Payment Mechanism Availability Payment
Presidio Parkway also known
as Doyle Drive Replacement
Project is a $1969 M PPP
project currently in
construction phase in the
State of California. This
project will replace an existing
73 year old south access to
the Golden Gate Bridge.
23

24. Setting Up The DEA Problem
First Stage: DEA Analysis
24

25. • Let us say that each procurement route represents a
Decision Making Unit (DMU) which has one input and
several outputs. For the case study we have the following
inputs and outputs:
Case Study
Presidio Parkway Project (Caltrans)
DBB
Cost
(NPV)
Risk Transfer
Use of Public Funds
Cost & Schedule Certainty
Simplified representation of Presidio Parkway Project
25
Value for Money
Level of O&M
FIRST DMU

26. Case Study
Presidio Parkway Project (Caltrans)
DBF
Cost
(NPV)
Risk Transfer
Use of Public Funds
Cost & Schedule Certainty
Simplified representation of Presidio Parkway Project
26
Value for Money
Level of O&M
SECOND DMU

27. Case Study
Presidio Parkway Project (Caltrans)
DBFOM
Cost
(NPV)
Risk Transfer
Use of Public Funds
Cost & Schedule Certainty
Simplified representation of Presidio Parkway Project
27
Value for Money
Level of O&M
THIRD DMU

28. Case Study
Presidio Parkway Project (Caltrans)
Inputs
Cost of DBB = 0.635 (B$)
Cost of DBF = 0.642 (B$)
Cost of DBFOM = 0.488 (B$)
Outputs

29. Case Study
Presidio Parkway Project (Caltrans)

30. The Banker Morey Model
Applied to Case Study
𝑀𝑖𝑛𝑖𝑚𝑖𝑧𝑒 ℎ0 = 𝑍0 −𝜀
𝑖=1
𝑚′
𝑆𝑖
−
+
𝑟=1
𝑅
𝑆𝑟
+
𝑗=1
𝑁
𝜆𝑗 𝑋𝑖𝑗 + 𝑆𝑖
−
= 𝑍0 𝑋𝑖𝑗0 i = (1, 2, ….., m’) (2)
𝑗=1
𝑁
𝜆𝑗 = 1 (4)
𝜆𝑗 ≥ 0 j = (1, 2, …., 𝑗0, … N) (5)
subject to:
𝑆𝑖
−
≥ 0, 𝑆𝑟
+
≥ 0 (𝑖 = 1, 2, … , 𝑚; 𝑟 = 1, 2, … , R)
(1)
𝑗=1
𝑁
𝜆𝑗 𝑔 𝑟,𝑗
(1)
≤ 1,
𝑗=1
𝑁
𝜆𝑗 𝑔 𝑟,𝑗
2
≤ 1,
𝑗=1
𝑁
𝜆𝑗 𝑔 𝑟,𝑗
3
≤ 1,
𝑗=1
𝑁
𝜆𝑗 𝑔 𝑟,𝑗
(4)
≤ 1
(3)

31. Case Study
Presidio Parkway Project (Caltrans)

32. Case Study
Presidio Parkway Project (Caltrans)
• The results indicated that some of in two scenarios the
efficiencies were 100% for several cases.
• This would not be a very good scenario because
decision makers will need a crisp solution.
• To overcome this problem, dummy variables were
introduced in the mix. Total 18 dummy DMUs were
created and analyzed again for efficiency. This was
sufficient to overcome the requirement suggesting that
# of DMUs >= 3x(# of inputs + # of outputs)

33. Case Study
Presidio Parkway Project (Caltrans)

34. The Second Stage Analysis

35. Evaluating Contextual Variables
• Level of Service (LOS): Level of service (LOS) is a mechanism
used to determine how well a transportation facility is
operating from a traveler’s perspective.
• It is expected that with private sector’s involvement in
public projects the overall quality of construction, safety,
operations and maintenance will improve thus providing a
better LOS.
• Thus it can be said that for DBFOM the LOS will be best,
while in DBF and DBB the LOS will be average or will be even
low.
• With this assumption, LOS data was generated. The
efficiency score was then regressed as per Banker and
Natarajan (2008).

36. The Second Stage Analysis
Efficiencies LOS
1 0.769 1.010
2 0.760 1.005
3 1.000 0.604
4 0.891 0.895
5 0.831 0.936
6 0.902 0.813
7 0.871 0.816
8 0.853 0.946
9 0.760 1.025
10 0.870 0.802
11 0.830 0.984
12 0.821 0.910
13 0.861 0.871
14 0.835 0.949
15 0.793 1.097
16 0.804 1.005
17 0.764 1.055
18 0.989 0.607
19 0.773 1.060
20 0.886 0.873
21 0.816 0.974
-0.06
-0.04
-0.02
0
0.02
0.04
0.06
0 0.2 0.4 0.6 0.8 1 1.2
Residuals
Random LOS
Random LOS Residual Plot
-0.3
-0.25
-0.2
-0.15
-0.1
-0.05
0
0 20 40 60 80 100 120
ln(EffiScore)
Sample Percentile
Normal Probability Plot
Regression

37. The Second Stage Analysis
SUMMARY OUTPUT
Regression Statistics
Multiple R 0.938063774
R Square 0.879963644
Adjusted R Square0.873645941
Standard Error 0.027820218
Observations 21
ANOVA
df SS MS F Significance F
Regression 1 0.10780194 0.10780194 139.285378 3.43012E-10
Residual 19 0.014705326 0.000773965
Total 20 0.122507266
Coefficients Standard Error t Stat P-value Lower 95% Upper 95%
Intercept 0.332899542 0.043470345 7.658083692 3.18658E-07 0.241915064 0.42388402
Random LOS -0.554580459 0.046990688 -11.80192264 3.43012E-10 -0.652933099 -0.45622782
ln 𝜃 = 𝛽0 − 𝛽𝑧 + 𝑒1
ln 𝜃 = 0.3329 + 0.5545*LOS

38. Interpretations
• Exp(0.3329) = 1.395 is the geometric mean of PPP’s
Efficiency.
• A unit change in LOS by one unit is associated with
(Exp(0.5545) - 1)*100 % = 74.11% change in PPP’s
efficiency.
ln 𝜃 = 0.3329 + 0.5545*LOS

39. Future potential
• Other socioeconomic parameters that are linked to
PPPs can be incorporated in the above model.
• Factors such as (Chen et al 2017):
• Job creation
• Improvement in quality of life
• Contribution to Gross State Product (GSP)
• Reduction in congestion
can be added to this research
ln 𝜃 = 𝛽0 − 𝛽1 𝑍1 − 𝛽2 𝑍2 … − 𝛽 𝑛 𝑍 𝑛 + 𝑒1

40. Thank you for your time
Deepak Sharma, PhD (Corresponding Author)
Assistant Professor
Department of Civil and Environmental Engineering
California State University, Fullerton
Phone: 657-278-3450 (O)
Email: dsharma@fullerton.edu, dksharma77@gmail.com

41. References• ASCE (2017). 2017 Report Card for America's Infrastructure, The American Society of Civil Engineers,
www.infrastructurereportcard.org.
• Banker R. D., and Morey R. C. (1986). Use of Categorical Variables in Data Envelopment Analysis. Management Science.
32(12):1613-1627
• Banker R. D., and Natarajan R. (2008). Evaluating Contextual Variables Affecting Productivity Using Data Envelopment
Analysis. Operations Research.56(1):48-58
• Chan C., Forwood D., Roper H., and Sayers C. (2009). “Public Infrastructure Financing: An International Perspective”.
Productivity Commission, Commonwealth of Australia, ISBN 978-1-74037-262-6.
• Cui C., Liu Y., Hope A., and Wang J. (2018 ). Review of studies on the public–private partnerships (PPP) for infrastructure
projects. International Journal of Project Mgnagement. 36: 773–794
• Garvin, M. (2010). Enabling Development of the Transportation Public-Private Partnership Market in the United States. J.
Const. Engg & Mgmt., 136(4): 402-411.
• Grimsey D. and Lewis M. (2007). “Public Private Partnerships and Public Procurement”. Agenda, Vol. 14(2), Pp. 171-188
(Retrieved 4/4/2015, from http://press.anu.edu.au/wp-content/uploads/2011/06/14-2-A-7.pdf)
• Kasper, H. T., Saha, D., Chao, J. J. (2015). Global Private Participation in Infrastructure (PPI) update 2015 (English).
Washington, D.C. : World Bank Group. http://documents.worldbank.org/curated/en/167501521119675781/Global-Private-
Participation-in-Infrastructure-PPI-update-2015
• Leigland J., and Shugart C. (2006). “Is the Public Sector Comparator Right for Developing Countries? Appraising Public-
Private Projects in Infrastructure.” Note No. 4, PPIAF GridLines, World Bank
• Prokopowicz. A. (2014). The Value for Money for Transport Infrastructure Projects and Its Marketing Components –
Conceptual Thoughts and Reflections. Management and Business Administration. Central Europe Vol. 22, No. 4(127): p.
179–193, ISSN 2084–3356,
• World Bank (2013). “Value-for-Money Analysis – Practices and Challenges: How Governments Choose When to Use PPP to
Deliver Public Infrastructure and Service”. Report from World Bank Global Round-Table, World Bank Institute and PPIAF,
Washington DC
• Tsukada S. (2015). “Adoption of Shadow Bid Pricing for Enhanced Application of “Value for Money” Methodology to PPP
Programs”, Pub. Wrks. Mgmt. & Pol., Vol. 20(3), 248-263.