SlideShare a Scribd company logo
1 of 13
Responsible
Model
Development
Sell
Data Management
&

Sensitivity Analysis
When the client asks how wrong the model
can be, we should be able to look into his eyes
and say,
The recommendation based on the modeling
results has an error of $0.5Million.
Tell the client the model can be wrong is

And

the right thing to do.

to do the thing right, we need to tell how wrong it is.

And that’s something we believe can

sell.
Right and wrong are just the
flip sides of

RISKs.
Asset Management Guys tell us:

Business Risk Exposure =
Probability of Failure
X

Consequences of Failure
How Wrong the Model is?
Business Risk Exposure =
Probability of Model being Wrong
X
Consequences of Model being Wrong
An Example
Pipe 1

Pipe 1

Pipe 2
If GIS is right

Pipe 2
If GIS is wrong

Pipe 3

Pipe 3
Consequences of being wrong:

spend $30,000 on a good pipe

Probability of being wrong: GIS with no source attribution, 70% chance of a typo.

The error of the recommendation is
$30,000 * 70% =

$21,000
More generic formula

BRE = $30,000 * (0.2+0) = $6,000
It is the modeler’s responsibility to inform the client the limitation of the
model, and the extent.
It is a good way to sell Data Management and Sensitivity Analysis services.
It is a good way to add

true value and gain trust.
Model responsibly.

Reference: Rules for responsible modeling, James, William, CHI
http://www.chiwater.com/Publications/Books/r184.asp

More Related Content

Similar to Responsible model development

Growth Hacking with Cassie Lancellotti-Young
Growth Hacking with Cassie Lancellotti-YoungGrowth Hacking with Cassie Lancellotti-Young
Growth Hacking with Cassie Lancellotti-YoungMattan Griffel
 
Metrics that Matter: The 360-Degree Customer
Metrics that Matter: The 360-Degree CustomerMetrics that Matter: The 360-Degree Customer
Metrics that Matter: The 360-Degree CustomerCassie Lancellotti-Young
 
LTK - FC - Supply Chain - Startup Challenge v3.pdf
LTK - FC - Supply Chain - Startup Challenge v3.pdfLTK - FC - Supply Chain - Startup Challenge v3.pdf
LTK - FC - Supply Chain - Startup Challenge v3.pdfjeroen_tjepkema
 
Spreadsheet Errors
Spreadsheet ErrorsSpreadsheet Errors
Spreadsheet Errorskmannix
 
Storyfying your Data: How to go from Data to Insights to Stories
Storyfying your Data: How to go from Data to Insights to StoriesStoryfying your Data: How to go from Data to Insights to Stories
Storyfying your Data: How to go from Data to Insights to StoriesGramener
 
It's Social Business (Not Social Media)
It's Social Business (Not Social Media)It's Social Business (Not Social Media)
It's Social Business (Not Social Media)Jason Lauritzen
 
Combining Art & Science in Modern Marketing
Combining Art & Science in Modern MarketingCombining Art & Science in Modern Marketing
Combining Art & Science in Modern MarketingScott Brinker
 
How to Talk about AI to Non-analaysts - Stampedecon AI Summit 2017
How to Talk about AI to Non-analaysts - Stampedecon AI Summit 2017How to Talk about AI to Non-analaysts - Stampedecon AI Summit 2017
How to Talk about AI to Non-analaysts - Stampedecon AI Summit 2017StampedeCon
 
New Product Launch Plan Powerpoint Presentation Slides
New Product Launch Plan Powerpoint Presentation SlidesNew Product Launch Plan Powerpoint Presentation Slides
New Product Launch Plan Powerpoint Presentation SlidesSlideTeam
 
Multilevel Regression and Post-stratification (MRP) for Brand Tracking: PyCon...
Multilevel Regression and Post-stratification (MRP) for Brand Tracking: PyCon...Multilevel Regression and Post-stratification (MRP) for Brand Tracking: PyCon...
Multilevel Regression and Post-stratification (MRP) for Brand Tracking: PyCon...Latana
 
5 Conversion Mistakes You're Probably Making (And How to Fix Them)
5 Conversion Mistakes You're Probably Making (And How to Fix Them)5 Conversion Mistakes You're Probably Making (And How to Fix Them)
5 Conversion Mistakes You're Probably Making (And How to Fix Them)Louis Grenier
 
5 Conversion Mistakes You're Probably Making
5 Conversion Mistakes You're Probably Making5 Conversion Mistakes You're Probably Making
5 Conversion Mistakes You're Probably MakingSlices Consulting
 
Entering the Data Analytics industry
Entering the Data Analytics industryEntering the Data Analytics industry
Entering the Data Analytics industryGramener
 
"No Decision": The blind spot that prevents lawyers from doubling their income
"No Decision": The blind spot that prevents lawyers from doubling their income"No Decision": The blind spot that prevents lawyers from doubling their income
"No Decision": The blind spot that prevents lawyers from doubling their incomeMike O'Horo
 
Data mining - Machine Learning
Data mining - Machine LearningData mining - Machine Learning
Data mining - Machine LearningRupaDutta3
 

Similar to Responsible model development (20)

Growth Hacking with Cassie Lancellotti-Young
Growth Hacking with Cassie Lancellotti-YoungGrowth Hacking with Cassie Lancellotti-Young
Growth Hacking with Cassie Lancellotti-Young
 
Metrics that Matter: The 360-Degree Customer
Metrics that Matter: The 360-Degree CustomerMetrics that Matter: The 360-Degree Customer
Metrics that Matter: The 360-Degree Customer
 
LTK - FC - Supply Chain - Startup Challenge v3.pdf
LTK - FC - Supply Chain - Startup Challenge v3.pdfLTK - FC - Supply Chain - Startup Challenge v3.pdf
LTK - FC - Supply Chain - Startup Challenge v3.pdf
 
MidTerm memo
MidTerm memoMidTerm memo
MidTerm memo
 
Spreadsheet Errors
Spreadsheet ErrorsSpreadsheet Errors
Spreadsheet Errors
 
Data Visions Big Data Visual Analytics Tool
Data Visions Big Data Visual Analytics ToolData Visions Big Data Visual Analytics Tool
Data Visions Big Data Visual Analytics Tool
 
Storyfying your Data: How to go from Data to Insights to Stories
Storyfying your Data: How to go from Data to Insights to StoriesStoryfying your Data: How to go from Data to Insights to Stories
Storyfying your Data: How to go from Data to Insights to Stories
 
It's Social Business (Not Social Media)
It's Social Business (Not Social Media)It's Social Business (Not Social Media)
It's Social Business (Not Social Media)
 
Combining Art & Science in Modern Marketing
Combining Art & Science in Modern MarketingCombining Art & Science in Modern Marketing
Combining Art & Science in Modern Marketing
 
How to Talk about AI to Non-analaysts - Stampedecon AI Summit 2017
How to Talk about AI to Non-analaysts - Stampedecon AI Summit 2017How to Talk about AI to Non-analaysts - Stampedecon AI Summit 2017
How to Talk about AI to Non-analaysts - Stampedecon AI Summit 2017
 
New Product Launch Plan Powerpoint Presentation Slides
New Product Launch Plan Powerpoint Presentation SlidesNew Product Launch Plan Powerpoint Presentation Slides
New Product Launch Plan Powerpoint Presentation Slides
 
Multilevel Regression and Post-stratification (MRP) for Brand Tracking: PyCon...
Multilevel Regression and Post-stratification (MRP) for Brand Tracking: PyCon...Multilevel Regression and Post-stratification (MRP) for Brand Tracking: PyCon...
Multilevel Regression and Post-stratification (MRP) for Brand Tracking: PyCon...
 
Benchmarking Impacts
Benchmarking ImpactsBenchmarking Impacts
Benchmarking Impacts
 
5 Conversion Mistakes You're Probably Making (And How to Fix Them)
5 Conversion Mistakes You're Probably Making (And How to Fix Them)5 Conversion Mistakes You're Probably Making (And How to Fix Them)
5 Conversion Mistakes You're Probably Making (And How to Fix Them)
 
5 Conversion Mistakes You're Probably Making
5 Conversion Mistakes You're Probably Making5 Conversion Mistakes You're Probably Making
5 Conversion Mistakes You're Probably Making
 
Teaching students how to critically appraise organizational data
Teaching students how to critically appraise organizational dataTeaching students how to critically appraise organizational data
Teaching students how to critically appraise organizational data
 
Entering the Data Analytics industry
Entering the Data Analytics industryEntering the Data Analytics industry
Entering the Data Analytics industry
 
"No Decision": The blind spot that prevents lawyers from doubling their income
"No Decision": The blind spot that prevents lawyers from doubling their income"No Decision": The blind spot that prevents lawyers from doubling their income
"No Decision": The blind spot that prevents lawyers from doubling their income
 
Data mining - Machine Learning
Data mining - Machine LearningData mining - Machine Learning
Data mining - Machine Learning
 
Ultimate Guide to SaaS Pricing
Ultimate Guide to SaaS PricingUltimate Guide to SaaS Pricing
Ultimate Guide to SaaS Pricing
 

More from Mel Meng

USING GIS TO IDENTIFY CRITICAL PIPES IN THE COLLECTION SYSTEM- 2012 owea coll...
USING GIS TO IDENTIFY CRITICAL PIPES IN THE COLLECTION SYSTEM- 2012 owea coll...USING GIS TO IDENTIFY CRITICAL PIPES IN THE COLLECTION SYSTEM- 2012 owea coll...
USING GIS TO IDENTIFY CRITICAL PIPES IN THE COLLECTION SYSTEM- 2012 owea coll...Mel Meng
 
Smart Phones Move the Field Forces Forward - 2011 Ohio GIS Conference
Smart Phones Move the Field Forces Forward - 2011 Ohio GIS ConferenceSmart Phones Move the Field Forces Forward - 2011 Ohio GIS Conference
Smart Phones Move the Field Forces Forward - 2011 Ohio GIS ConferenceMel Meng
 
Envision version 2.0
Envision version 2.0Envision version 2.0
Envision version 2.0Mel Meng
 
Master planning app
Master planning appMaster planning app
Master planning appMel Meng
 
Why should we develop an app
Why should we develop an appWhy should we develop an app
Why should we develop an appMel Meng
 
Disruptive technologies
Disruptive technologiesDisruptive technologies
Disruptive technologiesMel Meng
 
The data is the model period.
The data is the model period.The data is the model period.
The data is the model period.Mel Meng
 
Hybrid planning
Hybrid planningHybrid planning
Hybrid planningMel Meng
 
Test driven model development
Test driven model developmentTest driven model development
Test driven model developmentMel Meng
 
Dont make me think
Dont make me thinkDont make me think
Dont make me thinkMel Meng
 
Smart infrastructure
Smart infrastructureSmart infrastructure
Smart infrastructureMel Meng
 

More from Mel Meng (11)

USING GIS TO IDENTIFY CRITICAL PIPES IN THE COLLECTION SYSTEM- 2012 owea coll...
USING GIS TO IDENTIFY CRITICAL PIPES IN THE COLLECTION SYSTEM- 2012 owea coll...USING GIS TO IDENTIFY CRITICAL PIPES IN THE COLLECTION SYSTEM- 2012 owea coll...
USING GIS TO IDENTIFY CRITICAL PIPES IN THE COLLECTION SYSTEM- 2012 owea coll...
 
Smart Phones Move the Field Forces Forward - 2011 Ohio GIS Conference
Smart Phones Move the Field Forces Forward - 2011 Ohio GIS ConferenceSmart Phones Move the Field Forces Forward - 2011 Ohio GIS Conference
Smart Phones Move the Field Forces Forward - 2011 Ohio GIS Conference
 
Envision version 2.0
Envision version 2.0Envision version 2.0
Envision version 2.0
 
Master planning app
Master planning appMaster planning app
Master planning app
 
Why should we develop an app
Why should we develop an appWhy should we develop an app
Why should we develop an app
 
Disruptive technologies
Disruptive technologiesDisruptive technologies
Disruptive technologies
 
The data is the model period.
The data is the model period.The data is the model period.
The data is the model period.
 
Hybrid planning
Hybrid planningHybrid planning
Hybrid planning
 
Test driven model development
Test driven model developmentTest driven model development
Test driven model development
 
Dont make me think
Dont make me thinkDont make me think
Dont make me think
 
Smart infrastructure
Smart infrastructureSmart infrastructure
Smart infrastructure
 

Recently uploaded

ERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage IntacctERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage IntacctBrainSell Technologies
 
Cyber Insurance - RalphGilot - Embry-Riddle Aeronautical University.pptx
Cyber Insurance - RalphGilot - Embry-Riddle Aeronautical University.pptxCyber Insurance - RalphGilot - Embry-Riddle Aeronautical University.pptx
Cyber Insurance - RalphGilot - Embry-Riddle Aeronautical University.pptxMasterG
 
2024 May Patch Tuesday
2024 May Patch Tuesday2024 May Patch Tuesday
2024 May Patch TuesdayIvanti
 
ADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptxADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptxFIDO Alliance
 
AI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAnitaRaj43
 
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...FIDO Alliance
 
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...ScyllaDB
 
WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024Lorenzo Miniero
 
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxFIDO Alliance
 
Vector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptxVector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptxjbellis
 
CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)Wonjun Hwang
 
AI mind or machine power point presentation
AI mind or machine power point presentationAI mind or machine power point presentation
AI mind or machine power point presentationyogeshlabana357357
 
How to Check GPS Location with a Live Tracker in Pakistan
How to Check GPS Location with a Live Tracker in PakistanHow to Check GPS Location with a Live Tracker in Pakistan
How to Check GPS Location with a Live Tracker in Pakistandanishmna97
 
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdfMuhammad Subhan
 
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...marcuskenyatta275
 
Event-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingEvent-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingScyllaDB
 
Design and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data ScienceDesign and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data SciencePaolo Missier
 
ChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityVictorSzoltysek
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe中 央社
 

Recently uploaded (20)

ERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage IntacctERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage Intacct
 
Overview of Hyperledger Foundation
Overview of Hyperledger FoundationOverview of Hyperledger Foundation
Overview of Hyperledger Foundation
 
Cyber Insurance - RalphGilot - Embry-Riddle Aeronautical University.pptx
Cyber Insurance - RalphGilot - Embry-Riddle Aeronautical University.pptxCyber Insurance - RalphGilot - Embry-Riddle Aeronautical University.pptx
Cyber Insurance - RalphGilot - Embry-Riddle Aeronautical University.pptx
 
2024 May Patch Tuesday
2024 May Patch Tuesday2024 May Patch Tuesday
2024 May Patch Tuesday
 
ADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptxADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptx
 
AI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by Anitaraj
 
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
 
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
 
WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024
 
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
 
Vector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptxVector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptx
 
CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)
 
AI mind or machine power point presentation
AI mind or machine power point presentationAI mind or machine power point presentation
AI mind or machine power point presentation
 
How to Check GPS Location with a Live Tracker in Pakistan
How to Check GPS Location with a Live Tracker in PakistanHow to Check GPS Location with a Live Tracker in Pakistan
How to Check GPS Location with a Live Tracker in Pakistan
 
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
 
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
 
Event-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingEvent-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream Processing
 
Design and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data ScienceDesign and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data Science
 
ChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps Productivity
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe
 

Responsible model development

  • 3. When the client asks how wrong the model can be, we should be able to look into his eyes and say, The recommendation based on the modeling results has an error of $0.5Million.
  • 4. Tell the client the model can be wrong is And the right thing to do. to do the thing right, we need to tell how wrong it is. And that’s something we believe can sell.
  • 5. Right and wrong are just the flip sides of RISKs.
  • 6. Asset Management Guys tell us: Business Risk Exposure = Probability of Failure X Consequences of Failure
  • 7. How Wrong the Model is? Business Risk Exposure = Probability of Model being Wrong X Consequences of Model being Wrong
  • 9. Pipe 1 Pipe 1 Pipe 2 If GIS is right Pipe 2 If GIS is wrong Pipe 3 Pipe 3
  • 10. Consequences of being wrong: spend $30,000 on a good pipe Probability of being wrong: GIS with no source attribution, 70% chance of a typo. The error of the recommendation is $30,000 * 70% = $21,000
  • 11. More generic formula BRE = $30,000 * (0.2+0) = $6,000
  • 12. It is the modeler’s responsibility to inform the client the limitation of the model, and the extent. It is a good way to sell Data Management and Sensitivity Analysis services. It is a good way to add true value and gain trust.
  • 13. Model responsibly. Reference: Rules for responsible modeling, James, William, CHI http://www.chiwater.com/Publications/Books/r184.asp