Sim Cheng Hwee

1. Secure Benchmarking
Sim Cheng Hwee
MD, IDSC
10 November 2013

2. Agenda
 Background
 Genesis of & motivation
 Secure benchmarking in transportation
procurement
 Other applications
 Conclusion & discussion

3. Genesis of Secure Benchmarking
 A major chemical MNC in China ran a tender but
refused to award as prices were 15-20% higher
than the previous tender (suspicion of corruption)
 Need for means to establish that the bids are in
line with market prices but all bids are confidential
& thus not accessible (especially that of
competitors) for benchmarking
 Developed secure benchmarking to resolve this
dilemma

4. Motivation for Secure Benchmarking
Raising Service Productivity
 Each company wants to know how
competitors perform but are reluctant to
share “sensitive” info about itself
 Secure benchmarking can overcome such a
problem preventing industries from
knowing key performance data crucial to
raising productivity of individual companies
& industries over time

5. SECURE TRANSPORTATION
BENCHMARKING
Safe & Insightful Market Economics Research

6. Freight Tenders
 Most companies tender
freight contracts on their
own
 Data format & analysis
method used in tenders
vary widely
 Bids are filtered using
rules which may not be
holistically optimal
 Lack of basis to know
whether which bids are
reasonable – problem of
“sell-in”
 Actual price invoiced may
deviate from awarded
rates - compliance may
not be audited & reasons
for deviation not studied
 No systematic
consideration for spot mkt

7. Freight Tender Considerations
 Historical & target
rates – trends & needs
 Shipping volume &
speed & their impact
on carriers’ bid rates
 Container size & type
 Desired no. of carriers
in freight contract
 Carriers’ lane &
network capacity /
frequency / speed &
performance reliability
 Shipper’s volume &
desired capacity /
frequency / speed &
performance reliability

8. Conventional Benchmarking
 Cost & performance data are not shared due to
confidentiality concerns as well as requirements
imposed by carriers
 Spread of prices analysed only as statistical
distribution or compared with single factors – no
insight into combined effect of factors shaping
price & performance in the market
 Yet accurate understanding of performance & cost
is crucial for both shippers & carriers to co-create
value & for industry to develop

9. Analytical Benchmarking
US Consortium
 1st year – 6 members only
 3rd year – >70 members with US$9.65b or 2.5% of
the US market
 Regression modeling of transportation cost & time
for dry van, reefer, intermodal, flatbed & shorthaul
– allows meaningful comparison of transportation
cost & time
 Participation by paid subscription & submission of
raw data

10. Transportation Performance / Cost
Truck Shipment Data by Lanes
 Cost – amount paid for
that shipment
 Delivery lead time
 Distance
 Weight / Volume
 Carrier
 Carrier code
 Vehicle type
 Fuel price & labour
cost
 Cargo
 5 digit Std
Classification of
Transported Goods
(SCTG) code
 Palletized or not
 Dangerous goods or
not

11. Confidentiality Concerns
Secure Multi-Party Computation
 Example - 2 millionaires who want to find out who
is richer, but both refuse to disclose their net worth
 Conventional way is to have an external trusted
party to perform the comparison & report the result
 Using special software allow us to receive sensitive
data from each member encrypted such that we are
still able to derive the desired aggregate result
without ever needing to know the raw data
 Nullifies danger of leaks (intended or otherwise)

12. Analytical Benchmarking
Submission of Data
 Shipment data sorted by lane are loaded by client
into our online spreadsheets which then produces
the encrypted data for that year / month / week / day
(depends on desired insight into seasonality)
 Client can inspect encrypted data to see what is
transmitted to us
 Benchmarks are produced for each time period for
lanes as well as collections of lanes in a region that
forms corridors

13. Secure Benchmarking Workflow
Client
Excel-spreadsheet
Server
Application
Data
Encryption
Data Validation
summary & a few samples
Client Data Regression
Global
Regression
Analysis Reports
Further Investigation
raw transportation data
Statistical Analysis

14. Front-End Work by Clients
• Microsoft Excel spreadsheet with specially prepared
macros are provided to clients to load their raw data so
that IDSC’s embedded algorithm can
• capture the essence of the raw data as summary
data
• collect a small randomised sample of data for
validation purpose (if authorised)
• encrypt the summary & sample data so that any
unauthorised person will not be able to make any
sense of the scrambled data (which is how the
submitted data is stored by IDSC)

15. Back-End Work by IDSC
• IDSC’s backend application to
 Confirm integrity of client’s summary data
 Conduct local & global regression using summary data
 Analyse regression results & generate reports
 Advantages of storing encrypted summaries
 Safe even if database was hacked
 Use of summary instead of raw data ensures database
remains compact & grow slowly

16. Key Features of IDSC’s Solution
 No hassle of dealing with huge
data sets as only the essence of
the data which reveals the
relationship between cost & its
explanatory factors as it pertains
to each client is kept & analysed
 No way for anyone to figure out
from the stored scrambled
summary data which carrier
bidded how much where or who
ships how much from where to
where
 Clients are in control & can
check the data provided to
ensure that no sensitive data are
sent – submissions can be
automated once clients trusts
the system

17. Bid Level Benchmarking
Illustrated using China-US Ocean
Freight Data

18. Company A Regression Result
 Reasonable explanatory
power esp. for seadistance
& container type even
with just 70 data pts but
transit/type & time are not
well explained
 APL seems higher priced
than Hanjin, OOCL &
ANL but reliability of
results for the latter two is
not high
Regression Statistics
Multiple R 0.916846
R Square 0.840607
Adjusted R Square 0.800032
Standard Error 357.4994
Observations 70
Coefficientsandard Err
Intercept 59.84632 338.4583
Container Type 643.1 105.6737
Sea Distance 0.202189 0.037112
Road Distance 0.365141 0.140651
Transit Type 64.7999 148.0413
Transit Time 26.53632 14.36221
Hanjin -426.022 165.6799
CMA-CGM 0 0
APL 55.59749 202.7216
ANL Container Line -138.827 397.7765
OOCL -89.9853 176.7303

19. Company B Regression Result
 Results for B is a lot less
reliable for carriers than A
even though it has 553
bids – largely because it
ignores potential
explanatory variables
 Reasonable results for sea
distance & road distance
 HMM clearly positions
itself higher than others
Multiple R 0.553339083
R Square 0.30618414
Adjusted R Square 0.291538091
Standard Error 617.2514241
Observations 553
Coefficients andard Err
Intercept 2202.676277 258.0929
SeaDistance(NauticalMiles) 0.179293246 0.011912
RoadDistance(Miles) 0.440283951 0.073352
EGV -130.399838 227.9095
HMM 68.94109664 227.5441
HPL -76.70961937 227.7625
HSL -141.3325011 227.8775
MSK -176.2701851 241.9074
NYK -218.1404706 240.8233
WSL -281.438057 259.817
YML -128.3968925 227.8957
ZIM 0 0

20. Observations from A & B
 B is paying less than A for
sea distance but more for
road distance
 B’s carriers are generally
discounting from the mkt
rates while A’s carriers are
not but overall signs point
to imperfect mkt
 Data could be made more
compatible
A B
Sea distance 0.202 0.179
Road distance 0.365 0.440
 Inclusion of carrier
frequency, capacity,
speed, etc on each lane
should give more insight
 Richer & more data
needed to get good picture
of cost structure in each
mkt

21. Bid Level Benchmarking
 Regression results representing
global & local mkt cost & value
structures & their interpretation
 Provides insights into factors
affecting the rates (cost & value
structure in each market) &
how they differ between
shippers & carriers – insights
that are useful for instituting
best practice & guiding value
co-creation & negotiation
 Insight into impact of spend,
contract period, etc to improve
procurement strategy
 Results show which lanes
should be clustered together as
corridors & regions as they
belong to the same market
 Benchmarking over time will
reveal patterns & trends which
can inform on how to forecast
& strategise for future cost &
performance
 Insights into forces such as fuel
price shaping the market

22. Secure Benchmarking
Freight Bids
Shipments by & Payments
to Contracted Carriers
Spot Market
Shipments
ReferenceCompliance
Cost & value structure
in each mkt
Ground realities as
expressed by spot mkt
& contract shipments

23. Shipment Level Benchmarking
Illustrated using Thai Data

24. Client Regression Generation
 Data source: Jan 2008 – June 2010
 Spot Market data from Thai Shippers (“SM-LTL”): Less than truck
load (LTL)
 Contract data from Thai Company X(“CX_FTL”): Full truck load
(FTL)
 Questions
 Q1: Is regression result generated from raw data similar to that
from encrypted data?
 Q2: Does regression result closely represent actual cost?
 Q3: What if Company X switches from Contract to Spot Market?

25. Regression Result - Spot Market LTL
Q2: Does regression result closely represent actual cost?

26. Regression Result - Company X FTL
Q2: Does regression result closely represent actual cost?

27. Regression Result
Q3: What if Company X moves more to Spot Market?

28. Regression Result
Q3: What if Company X moves more to Spot Market?
Truck Type Actual Cost (Baht)
(A)
Estimated Spot Mkt
Price(Baht)
(B)
Cost Saving(Baht)
(B-A)
% of Cost Saving
(B-A)/A*100%
4 Wheel 254,773 254,304 -469 -0.18%
6 Wheel 436,618 514,068 77,450 17.74%
Sum 691,391 768,372 76,981 11.13%
Conclusion:
1) Continue current contract transportation, specially for 6–wheel truck.
Total cost saving up to 11.13%
2) Although estimated Spot Market price is slightly less for 4-wheel truck,
admin costs may increase if switching to spot market.

29. Insight from Comparing Coefficients
1,535.86
1,000.00
1,600.00
1,378.62
0.00
200.00
400.00
600.00
800.00
1,000.00
1,200.00
1,400.00
1,600.00
1,800.00
A B C Global
Truck Type
A
B
C
Global
7.16
8.00
6.00
7.05
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
9.00
A B C Global
Distance (km)
A
B
C
Global
64.97
40.00
100.00
68.32
0.00
20.00
40.00
60.00
80.00
100.00
120.00
A B C Global
Weight (ton)
A
B
C
Global
Predictor
Coefficients
A B C Global
Truck Type 1,535.86 1,000.00 1,600.00 1,378.62
Distance (km) 7.16 8.00 6.00 7.05
Diesel Price (baht/liter) 40.74 40.74 40.74 40.74
Dangerous 256.66 256.66 256.66 256.66
Weight (ton) 64.97 40.00 100.00 68.32
Labor Cost(baht/day) 10.81 10.81 10.81 10.81
Constant -2,892.07 -2,900.07 -2,892.07 -2,894.73

30. Effect of Diesel Price in LTL
-100
-50
0
50
100
150
1 2 3 4 5 6 7 8 9 10
Coefficient-DiesePrice
Quarter
Seasonality of Coefficient - Diesel Price
0
10
20
30
40
50
Diesel price (Baht/liter)

31. Benchmarking Shipments
 Provides micro-level
factors & their impact on
the actual amount paid –
insights into why they
deviate from contracted
rates
 Serves to also audit
compliance with the
contracted rates
 Results show how
operational realities &
decisions can be better
managed to lower cost &
improve performance
 Benchmarking over time
will reveal seasonality
patterns & trends which
can inform on how to
forecast & strategise for
future cost & performance

32. OTHER APPLICATIONS

33. Benchmarking Forecasts
 Coefficients of forecast equations
are ideally suited for secure
benchmarking across companies
in various industries

34. Benchmarking Retail Productivity
 Inputs
 $ value of inventory held (display & warehouse)
 Display & w/h space used in terms of sq-m-days
 Shop staff & buyer experience in man-years
 $ value of capital & other fixed costs
 $ value of wages, rental & other recurrent costs
 Output
 Sales revenue

35. Benchmarking Services
 Inputs
 Service context data such as
• Space, budget, staff seniority, technology used, etc
for service provider
• Age, willingness to wait, language preference, etc of
service consumer
 Output
 Value in context

36. Benchmarking HR
 Salary (as an example)
 Inputs
• Specialisation
• Qualification
• Years of working experience in same industry
• Years of working experience in other industries
 Output
• Salary of employees with various qualifications

37. CONCLUSION & DISCUSSION

38. Conclusion
 Secure benchmarking allows companies to
 Objectively understand markets & their underlying factors –
establish “fair price”, productivity norms, patterns & trends, etc
 Learn from the best practice of other companies in the same
industries or of similar functions across industries
 Better explore opportunities to collaborate with other companies
 IDSC looks forward to working with service innovation
pioneers to transform services with the new benchmarking
technology

39. Discussion
 Comments?
 Questions?
 Next steps?