Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.

Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.

Successfully reported this slideshow.

Like this presentation? Why not share!

- Sergey A. Razin Ph.D., Chief Techno... by MLconf 1004 views
- Jeremy Stanley, EVP/Data Scientist,... by MLconf 2101 views
- Furong Huang, Ph.D. Candidate, UC I... by MLconf 684 views
- Ted Willke, Senior Principal Engine... by MLconf 1818 views
- Avi Pfeffer, Principal Scientist, C... by MLconf 1243 views
- Allison Gilmore, Data Scientist, Ay... by MLconf 1319 views

1,977 views

Published on

In this talk, I introduce Agency Theory, which is a mathematical framework for analyzing decision models to solve this problem. Agency theory borrows key ideas from machine learning, to solve a different purpose: rather than finding a set of parameters that best fits a data set, the objective is to find a set of decisions that leads to the most favorable set of outcomes, along with the data that is most valuable in supporting those decisions. Just as many foundational aspects of machine learning can be understood using information theory, I’ll describe how entropy and related concepts underlie Agency, and how to use this approach to prioritize data management and improve decision making.

Published in:
Technology

No Downloads

Total views

1,977

On SlideShare

0

From Embeds

0

Number of Embeds

6

Shares

0

Downloads

15

Comments

0

Likes

1

No embeds

No notes for slide

- 1. Agency Mark Zangari CEO Quantellia
- 2. Agency Agency’s aims: 1. Extend using machine learning into a new domain of problems. 2. The beginnings of a formal technique for linking data science to business levers. 3. Instead of answering “Given input data A, what output data B does my model predict?”, Agency answers “If I do A to complex dynamical system S, with the intention of achieving objective B, what is the probability that S’s outputs are closer to B?”
- 3. Agency The four pillars for effective machine learning: 1. Availability of training data 2. System stability over characterstic times (a) training data period (past) (b) prediction period (future) 3. Availability of input data at the time the prediction is to be made. 4. Sufficient signal-to-noise in data for pattern recognition to be possible.
- 4. Agency The four pillars for effective machine learning: What can we do if one or more of the four pillars are not present?
- 5. Agency The four pillars for effective machine learning: What can we do if one or more of the four pillars are not present? Example from finance: Commercial lending vs. consumer lending.
- 6. Agency Consumer Credit: 1. Ample, accessible data 2. Large, consistent, classifiable population 3. Relevant variables easily measured 4. Strong correlations between input variables and credit risk. Consumer Credit Transactions Data Warehouse Rating System Credit Score
- 7. Bill Fair & Earl Isaac Contacted 58 of the nation’s top lending institutions in 1958 offering to show them how using data would help them make better credit decisions... Only one responded. Image composed from stock images and portraits of Fair and http://www.fico.com/en/about-us#our_history
- 8. Agency Consumer Credit: 1. Ample, accessible data 2. Large, consistent, classifiable population 3. Relevant variables easily measured 4. Strong correlations between input variables and credit risk. Commercial Credit for Small/Medium Businesses: 1. Little data available, hard to obtain 2. Each business different, many are very new with little history. 3. Complex inter-entity relationships affect credit risk. 4. Many correlations , hard to isolate those that are good predictors of credit risk.
- 9. Agency Fully automated credit rating is rarely used for scoring Small/Medium Business Loans. The process commonly used is a good example of Agency in action…
- 10. Agency Type of Business Loan Amount Probability of Default Typical Model Decomposition for Commercial Loan Decision Grant this Loan? Profitable Loan Book Desired ObjectiveBusiness Lever External Inputs
- 11. Agency Agency Principle One: Learning and other analytics are designed to discriminate actions that will increase the probability of the objectives being met, from actions that do not. So… What is an “objective?”
- 12. Agency Objective: Map every element in the set of outcomes to a measure of the favorability of that outcome occurring, relative to the other outcomes. Probability of default DesiredLoanFrequency
- 13. Agency Objective: Map every element in the set of outcomes to a measure of the favorability of that outcome occurring, relative to the other outcomes. Probability of default DesiredLoanFrequency
- 14. Agency Type of Business Loan Amount Probability of Default Typical Model Decomposition for Commercial Loan Decision Grant this Loan? Profitable Loan Book Desired ObjectiveBusiness Levers External Inputs Measure Measure Learn Interest Rate
- 15. Agency Agency Principle Two: If a system does not ideally support machine learning because it does not satisfy the four pillars, decompose it using a dynamic system model until each link between each node either: (a) Satisfies the four pillars, or (b) Can be described using known domain-specific relationships (formulas, rules, approximations, etc.)
- 16. Agency Financial Spreading Values Market & Industry Data Management Team Bios & Experience Relationships to other Entities Key Ratios Aggregate Statistics Scoring Probability of Default Known mathematical relationships Machine Learning (intangibles) Rating Machine Learning Transform Tables Typical Model Decomposition for Business Credit Rating Feature Engineering
- 17. Agency Management Team Experience Agency gives us insight into decisions via the model levers. E.g. What interest rate do we charge and how does this affect our success? Interest Rate 1 Management Team Experience Interest Rate 2 Interest Rate 1 Interest Rate 2
- 18. Agency Summary of characteristics where Agency becomes highly relevant and useful: • Sparse data • Complex emergent dynamics, including feedback loops, phase changes, and other non-linear effects. • Provides a way of including intangibles with few measureables in the model • Can utilize relationships that the training data does not make apparent.
- 19. Agency A (brief) mathematical representation of Agency: We draw analogy from the information measure: The “Agency” A of a lever L, which maps some part of the output space to favorable outcomes, f and some other part to unfavorable outcomes, with measure M(f) and M(u) is given by:
- 20. Agency Agency: • Helps business users connect real-world decisions with support from data. • Provides a architecture for using machine learning in situations that are not typically well suited to ML solutions. • Allows ML results to be extrapolated beyond information contained in the data, by integrating knowledge of system dynamics.

No public clipboards found for this slide

Be the first to comment