Business update Q1 2024 Lar España Real Estate SOCIMI
Algorithmic auditing 1.0
1. Pragmatic
Algorithmic Auditing 1.0
2020 Copyright QuantUniversity LLC.
Presented By:
Sri Krishnamurthy, CFA, CAP
sri@quantuniversity.com
www.quantuniversity.com
Data Club
Northeastern University
2. 2
Speaker bio
• Advisory and Consultancy for Financial
Analytics
• Prior Experience at MathWorks, Citigroup
and Endeca and 25+ financial services and
energy customers.
• Columnist for the Wilmott Magazine
• Author of forthcoming book
“Pragmatic AI and ML in Finance”
• Teaches AI/ML and Fintech Related topics in
the MS and MBA programs at Northeastern
University, Boston
• Reviewer: Journal of Asset Management
Sri Krishnamurthy
Founder and CEO
QuantUniversity
3. 3
QuantUniversity
• Boston-based Data Science, Quant
Finance and Machine Learning
training and consulting advisory
• Trained more than 1000 students in
Quantitative methods, Data Science
and Big Data Technologies using
MATLAB, Python and R
• Building a platform for AI
and Machine Learning
Experimentation
4. 1. Key trends in AI, Machine Learning & Fintech
2. Algorithmic Auditing – Introduction
3. 5 things to note when auditing an algorithm
1. Use case
2. Data
3. Model
4. Environment
5. Process
Agenda
6. 6
The 4th Industrial revolution is Here!
Source: Christoph Roser at AllAboutLean.com
As per Wikipedia*, “The 4th Industrial Revolution ….. marked by emerging technology breakthroughs in a
number of fields, including robotics, artificial intelligence, nanotechnology, quantum computing, biotechnology,
the Internet of Things, the Industrial Internet of Things (IIoT), decentralized consensus, fifth-generation wireless
technologies (5G), additive manufacturing/3D printing and fully autonomous vehicles.”
* https://en.wikipedia.org/wiki/Fourth_Industrial_Revolution
7. 7
Scientists are disrupting the way we live!
Source: https://www.ladn.eu/tech-a-suivre/mobilite-2030-vehicules-volants-open-data/
8. 8
Interest in Machine learning continues to grow
https://www.wipo.int/edocs/pubdocs/en/wipo_pub_1055.pdf
13. 13
• Machine learning is the scientific study of algorithms and statistical
models that computer systems use to effectively perform a specific task
without using explicit instructions, relying on patterns and inference
instead1
• Artificial intelligence is intelligence demonstrated by machines, in
contrast to the natural intelligence displayed by humans and animals1
Defining Machine Learning and AI
13
1. https://en.wikipedia.org/wiki/Machine_learning
2. Figure Source: http://www.fsb.org/wp-content/uploads/P011117.pdf
14. 14
Machine Learning & AI in finance: A paradigm shift
14
Stochastic
Models
Factor Models
Optimization
Risk Factors
P/Q Quants
Derivative pricing
Trading Strategies
Simulations
Distribution
fitting
Quant
Real-time analytics
Predictive analytics
Machine Learning
RPA
NLP
Deep Learning
Computer Vision
Graph Analytics
Chatbots
Sentiment Analysis
Alternative Data
Data Scientist
16. 16
The rise of Big Data and Data Science
16
Image Source: http://www.ibmbigdatahub.com/sites/default/files/infographic_file/4-Vs-of-big-data.jpg
17. 17
Smart Algorithms
17
Distributing Computing Frameworks Deep Learning Frameworks
1. Our labeled datasets were thousands of times too
small.
2. Our computers were millions of times too slow.
3. We initialized the weights in a stupid way.
4. We used the wrong type of non-linearity.
- Geoff Hinton
“Capital One was able to determine fraudulent credit
card applications in 100 milliseconds”*
* http://go.databricks.com/hubfs/pdfs/Databricks-for-FinTech-170306.pdf
21. 21
• Algorithmic auditing is a structured process conducted internally or
by a qualified independent third party that involves:
▫ Verifying and/or validating the working of the algorithm along with the
data, model, environment, process contextual to the use-cases in which
the algorithm is supposed to be used.
▫ Identification of issues that are clearly articulated and scoped for the
algorithm.
– Examples include : bias, discrimination, explainability, interpretability etc.
▫ Documentation of the understanding of the algorithm’s behavior, uses
as observed and evaluated by a qualified individual.
▫ Recommendations for mitigation, control and elimination of noted
risks.
Algorithmic Auditing
24. 24
Questions to ask:
• Do we really need this algorithm?
• How will this algorithm be used?
• Who/What will it affect?
1. Use cases are important
25. 25
Things to think about:
• How much data do we
have?
• How will this affect the
model?
• Do we have enough data?
• Are their privacy concerns?
2. Don’t forget the data
26. 26
All scenarios haven’t
played out
• Stress scenarios
• What-if scenarios
Challenges with real datasets
Figure ref: http://www.actuaries.org/CTTEES_SOLV/Documents/StressTestingPaper.pdf
28. 28
Questions to ask
• Blackbox/Whitebox
• Does the model work?
• How do we handle imbalanced
classes?
• Is it fair/biased?
• Can you explain the model?
3. Model Audit
32. Thank you!
Sri Krishnamurthy, CFA, CAP
Founder and CEO
QuantUniversity LLC.
srikrishnamurthy
www.qu.academy
Contact
Information, data and drawings embodied in this presentation are strictly a property of QuantUniversity LLC. and shall not be
distributed or used in any other publication without the prior written consent of QuantUniversity LLC.
32