This session was recorded in NYC on October 22nd, 2019 and can be viewed here: https://youtu.be/7h61dxKjhvg Machine learning in finance—the promise and the peril This talk will discuss how machine learning (ML) fits into the landscape of quantitative methods used in finance, and draw conclusions about application domains where ML is more promising versus domains where the perils are more acute. The talk will also discuss how to formulate a financial goal as an ML problem, and how to choose between solution approaches. Bio: Leading projects to apply machine learning and artificial intelligence across the firm. Evaluating opportunities to work with other organizations and consulting with clients.

1. ML in finance—the promise and the peril
Charles Elkan
Goldman Sachs
H2O World
October 22, 2019

2. Deep learning (DL)
 DL has enabled a Kuhnian wave of
scientific progress.
 Especially in vision and in combined
language/vision.
 DL yields higher accuracy inmany
applications, such as forecasting.
Image source: http://www.cs.toronto.edu/~mren/imageqa/
Why is the world excited about AI nowadays?
3

3. What is deep learning?
4
DL is the current generation of neuralnetworks
Neural networks were invented in the
1950s, inspired (loosely!) by
neuroscience.
Great research advances in the 1980s,
then again in the 2010s.
Current deep learning exploitsmodern
computational power.
Traditional statistical methods are too
simple for massive data, especiallyfor
text, image, and video data.
Image source: https://www.ais.uni-bonn.de/deep_learning/

4. Now and foreseeably, AI methods cannot achieve deep understanding.
6
What is understanding?
What is shallow understanding?
What is deep understanding?
Operationally, understanding is the ability to answer questions
of many different types correctly.
The ability to answer a limited range of questions thatare
similar to each other.
Understanding that is multilayered, and that involves broad
context, and that uses broad knowledge.
Etymology: The Latin verb intelligere means to understand, with the root meaning “to selectbetween.”
What can deep learning not do?

5. Is the answer really three? Maybe it isthree
and a half? Or three and twopieces?
Deep understanding combines language and
vision and knowledge and personal experience.
In this example, deep understanding requires
reasoning about occlusion, about gravity, about
social conventions, and more.
Deep understanding
7

6. Most applications remain brittle
Examples: Face recognition, street sign recognition, question answering:
9
Q: Alexa, how high is MountEverest?
Q: Alexa, where is it located?
A: “ … 29,029 feet …”
A: “The address … is 318 3rd Ave ...”
AI systems need excessive quantities of training examples
Consider the regional meaning of wicked as in “It’s wicked cold outside.”A human needs
only one example, and some thinking, to know when and when not to use thisidiom.
Consequences of shallow understanding

7. 11
How can we make healthcare visits more convenient for patients,
and more efficient for doctors?
1) The patient describes symptoms in freetext.
2) The app uses ML to select follow-ups from a setof standard
questions; the patient answers these.
3) The physician reads the answers, meets the patient, asks formore
information as needed.
4) The physician makes decisions: medicine, advice,referrals.
Only the physician needs deep understanding!
Example: Convenience and productivity in medicine
Source: https://www.geekwire.com/2018/98point6-launches-virtual-clinic-app-solve-americas-primary-care-crisis-ai/

8. In general, ML is empirical, while traditional quantitative methods aredeductive.
• Monte Carlo methods are randomized, but stilldeductive.
• Combined methods have an empirical part called“calibration.”
ML is useful for tasks with less opportunity fordeduction.
Note: Any method can be a self-fulfillingprophecy!
About ML specifically in finance
14

9. ML expands what is feasible:
• Taking advantage of new types of data such as text and video.
• Massive models, sometimes with millions of trainablecoefficients.
• Answering new types of questions, and/or maximizing customizedobjectives.
Example: Estimating a market price for a real estateasset:
• It is empirical how different locations, roads, lot sizes, room types, style, etc. influence prices.
• We can estimate the probability distribution of prices--which is not a standard distribution.
• We can use photographs and news articles asinput.
15
ML versus other quantitative methods

10. Someone has to be a product manager. This person must (ML-specific tasksin blue):
1. Quantify the size of the businessopportunity.
2. Understand exactly what will be useful for business stakeholders, especially customers.
3. Figure out exactly what metrics should be maximized.
4. Understand which data is useful, find this data, and evaluate itsquality.
5. Design how the new system fits into existingsystems.
6. Quantify latency, volume, and other system needs.
7. Help design the user interface.
8. Design guardrails and monitoring.
22
How to drive a project that uses ML?

11. Example: Consider probability of default (PD and loss given default (LGD). Loans that may default are
restructured, and not recorded as defaults. So are we missing data about true economic losses?
Understanding the true business process
23

12. 24
1. Lack of consensus on a relevant quantitativeobjective.
2. Too few decisions to be made, each one too important. Example: Venture capitalinvestments.
3. Outputs are not accurate enough to be useful. Example: Understanding legallanguage.
4. Real-time data and/or historical data are different, and/or notavailable.
5. The model is not explainable enough for stakeholders to feelcomfortable.
6. The ML method is held to a higher standardthan
humans, or than the current businessprocess.
Risks for an ML application

13. Do use the latest AI methods to discover patterns in data
Analyze multiple types of data simultaneously
Expect super-human performance in some ways, but not in deep understanding
Look for systems that use the complementary strengths of software and ofhumans
Do not expect genuine artificial intelligence in the foreseeablefuture
But don’t be confident that genuine AI isimpossible
We can’t predict when, but there will be more breakthroughs in research in the future
Speculating about super-intelligence is like speculating about life on otherplanets
32
The promise and the peril

14. Thank you
Questions welcome!
35