SlideShare a Scribd company logo
1 of 22
Download to read offline
AI MODELS USAGE IN FINTECH PRODUCTS:
PM APPROACH & BEST PRACTICES
ISPMA | SPM Summit 2024 | February 2024
2
Profile
Kasthuri Rangan Bhaskar is a Product Leader and
banking/finance SME, with a proven track record of orchestrating
fintech products to success, from zero to one.
He currently leads products at BCT Digital.
He has 18+ years of diverse techno-functional exposure spread
across roles in commercial banking, digital transformation, risk
management, technology, pre-sales & B2B product management.
He has innovated products in the domains of Lending, Credit Risk,
Collateral Management, GRC and Model Risk Management. He
has extensively worked on AI models in Credit and Fraud Risk
Management space.
BCT Digital’s products have been recognized by leading analysts
such as Chartis. He has also worked with banking regulators as
part of sandbox testing.
He is a qualified Chartered Accountant, an Associate Member of
CPA (Australia), and holds an MBA from IIM Indore.
https://www.linkedin.com/in/kasrangan/
3
Agenda
 AI Models: Usage in Fintech Products
 Key Functional & Technical Challenges
 Key challenges of deploying models at scale: Indian context
 Key Regulatory Concerns: Bias and Explainablity of AI Models
 Product Management across AI Model Lifecycle: Development |Deployment |Usage
 3 Pillars for AI Models Product Management: Model Governance| Model Validation | Challenger
Models
 References for further study
4
AI Models: Usage in Fintech Products
Area Illustrative AI Use Cases
Lead Generation
• Personalized Marketing
• Tracking customer’s buying preference over a period
• Prioritizing leads for sales channels for maximizing conversion rates
Campaign Management • Identifying most effective campaign technique
Improving wallet share
• Personalized offers for cross-selling
• Optimal investment strategies for wealth management
Credit Scoring
• Credit Decisioning for BNPL/Instant loan companies
• Identifying repayment patterns
• Non-traditional sources such as Social Media
KYC
• Image recognition for ID cards
• OCR for digitalization of manually filled-up forms
Risk Management
• Predictive Analytics for identifying risk of default
• Suggesting risk management strategies such as hedging
Fraud Risk/AML • Suspicious Transactions, Funds Diversion
Customer Servicing
• AI based Chat Bots
• Contextual e-mail/voice-based responses
Back office (Operations)
• Automating repeated tasks such as form filling/data entry
• Automated workflows, MISMIS
AI Models are
increasingly
replacing human
judgement and
effort
5
AI Models Adoption: Key reasons for tremendous
increase
 High velocity/volume of transactions: AI helps in increasing the speed of decision making
 Overcoming human limitations in identifying patters through cognition
 Information overload: Beyond processing capacity of routine algorithms
 Need for improving models which can reduce frauds/AML incidents
 Need to automate very routine activities: Cost/Time Efficiency; boosting human productivity
 Increasing digital reach (UPI/Apps): No human factor involved in the value stream
 Decreasing cost of compute power (Memory & Processing)
 Easy Deployment: Ready-built platforms for AI Models deployment (E.g. AWS offers off-
the-shelf AI services for image/speech recognition, chatbots, personalization of websites,
Fraud Detection etc.)
6
..But the devil lies in the details
 An AI Model is not a standalone entity within
a product. Intended success of an embedded
AI model depends on factors such as:
 Infrastructure (E.g., Hardware, Processing
Power, Connectivity)
 Model Algorithm
 Modeler’s expertise
 Quality of code
 Quality of data
 Product Configurations, Parameters (E.g.,
Thresholds, logical operators)
 Regulatory Compliance
 Consequently, AI models need to be carefully
monitored at every stage of their lifecycle
Image Credit: https://home.paynearme.com/blog/beyond-innovation-what-it-takes-to-be-successful-with-ai/
7
Case Study
XYZ, a fintech company, disburses unsecured loans to customers for the lower/mid-income segment. Its
products embed AI Models in key areas such as origination, credit scoring, fraud risk and customer-servicing.
Ravi, the company’s Chief Data Scientist, is a worried man these days as there have been increasing
instances of :
- Stretched Payments/Defaults by customers flagged as ‘good’ during lead generation
- Low lead conversion rates despite extensive social media campaigns
- Frauds going undetected
- Customer issues such as improper responses by chatbots
- Auditors raising concerns on the transparency and efficacy of the models
Following this, he appointed a third-party consultant to investigate into the matter.
The consultant did a thorough analysis of all the AI Models and presented the senior management with the
following findings (explained in the next few slides):
8
AI Models: Key Functional Challenges for a PM/DS
Dev/Production
Data Mismatch
• Development of ML
models happens in
carefully cleaned
models while real-
life data may be
completely different
• Finding: XYZ’s
model was
developed on
anonymized data
obtained from a
third-party vendor
and was not
representative of
the company’s
primary customer
segments
• Implication:
Unstable/out-of-
sync models
Explainability
• We can observe the
input and the
output; however,
we may not be able
to explain the
relationship/connec
tion between them
• Finding: XYZ could
not explain how
changing income
levels impacted the
models, over a
period of time
• Implication: Why
and how the output
has come from the
input is not known
Changing Anything
Changes
Everything (CACE)
• Various components
of a ML model
(inputs, model &
parameters) are
entangled (i.e., they
do not have a
separate identify
• Finding: XYZ used
15 different
variables including
non-traditional data
points such as smart
phone usage
• Implication: Any
alteration/data
fluctuation in the
input may impact
the performance of
the model
Undeclared
Customers
• Output of a ML
model may be
silently used by
unintended/undecla
red customers (such
as other systems).
• Finding: Single
model was being
used across
business segments
• Implication: May
negatively influence
model’s behavior
(E.g. False Positives)
Unstable Data
Dependencies
• Finding: Input to a
ML model may be
consumed from
another source,
which is unstable
(such as Social
Media)
• Implication:
Unstable/out-of-
sync models
9
AI Models: Key Technical Challenges
 Data Management
 Managing data gaps, quality, governance and privacy
 Real Time/Near Real Time Computations
 Latency in infrastructure, penetration of internet/smartphone etc. may impact real time
decisioning
 Processing and queuing large data
 Frequent updation of scenarios
 Scenarios/use cases changes over a period (E.g.: New types of frauds emerge);
incorporating them existing models would be a challenge
 Managing multiple deployment approaches
 Models may have to be deployed on-cloud/locally/webservice, depending on the specific
need. Managing multiple deployment approaches is a huge challenge
 Version Control
 Difficulty in managing different versions of models, say across products, model lifecycle and
customer segment
10
AI Models: Key challenges from an Indian Context
 Lack of industry level data for model enrichment & validation
 Lack of shared data repositories such as industry consortium (excepting Credit Bureau);
access to certain databases are restricted to banks/NBFCsequate data standards
 Inadequate Model Governance Regulations
 Inadequate regulatory push to maintain model standards
 Lack of clean data
 Usage of assumptions, proxies to fill up data gaps
 Data Privacy
 Lack of effective control over usage of PII data
 Fast changing demographics
 Data not representative of the sample on which it was developed; model drift may happen
quickly (For example, income levels have undergone tremendous change in the last 5 years)
11
Key Regulatory Concerns: Explainability
Explainability is influenced by Type of decision & End-user of the explanation
Source Deloitte https://www2.deloitte.com/us/en/insights/industry/financial-services/explainable-ai-in-banking.html
12
Key Regulatory Concerns: Explainability
 Three components of Explainability
 Transparency: Ease of understanding
 Traceability of different model parameters/components
 Questionability: Questioning back ‘why it is so’ or ‘what if’
 Approaches to improving Explainability
 Not making it more complicated than required
 Trade off between explainability and predicted outcome
 Improving explainability of individual components and their relationship (E.g. Individual
Condition Expectations Plots)
 Improving explainability of significant components and their collective impact on the overall
model
13
Key Regulatory Concerns: Bias
 Bias unfairly restricts a segment from participation (For example, denial of credit)
 Regulators like RBI has highlighted bias as one of the key risks of AI Models
 There are various laws which explicitly prohibit such bias
 For example, the US Equal Credit Opportunity Act prohibits discrimination on the basis of
race, color, religion, national origin, sex, marital status, age or receipt of public assistance
 Sources of bias in an AI Model could be
 Training Data: For example, heavily skewed data
 Algorithm: For example, overweighing/double counting certain parameters
 Interpretation: Caused due to judgements based on past preferences/judgements
 Human oversight on the models at all stages is a very important factor to reduce bias
 Understanding and removing bias is most complex as it involves understanding of data, data
science and social science
“The manifestation of unfair or discriminatory behaviour in an AI/ML model’s predictions, influenced by biased
training data or the algorithm itself, leading to unequal treatment of different groups”
14
Explainablity/Bias: Illustrations
• Discrimination based on color: https://www.cnbc.com/2023/06/23/ai-has-a-discrimination-
problem-in-banking-that-can-be-devastating.html
• AI Models such as ChatGPT has ‘invented hypothetical case studies’
(https://apnews.com/article/artificial-intelligence-chatgpt-courts-
e15023d7e6fdf4f099aa122437dbb59b)
• Amazon was found using a hiring algorithm that favoured male candidates against female
candidates based on certain keywords like “executed”, usually found in resumes of male
candidates (https://www.reuters.com/article/us-amazon-com-jobs-automation-insight-
idUSKCN1MK08G/)
• Google’s ad algorithm was displaying more high paying jobs to males than to females
(https://www.washingtonpost.com/news/the-intersect/wp/2015/07/06/googles-algorithm-
shows-prestigious-job-ads-to-men-but-not-to-women-heres-why-that-should-worry-you/)
15
AI Models: Product Management across lifecycle
Model
Retirement
Model Ideation
Model Design &
Development
Independent
Validation
Testing
(E.g.: Sandbox,
Pilot, UAT)
Model
Deployment &
Usage
Ongoing
Validation
Calibration
Challenger
Models
Initial Model Development Stage
Continuous Model
Validation
Model Governance Model Documentation
AI Model Lifecycle management refers to the process of managing models across its lifecycle i.e., from
Development, Usage till the stage of Retirement. AI Models have a significantly larger domino effect.
16
AI Models: Best PM Practices - Development Stage
 Prior to new model development following factors to be considered:
 Choice of In-house vs Third Party Models
 Calibrate Existing vs Develop New Models
 Technology choice for Model Development platform: Carefully consider factors such as overall architecture,
functional capabilities, technology lock-in, any technical debt overhang
 Choice of model after considering various factors such as data availability, quality & appropriateness
 Trade-off between sophistication of model, data availability and model soundness
 Model development environment to be sanitized and demarcated to mix up data/models with
production environment
 Pre-Production Testing: Pilots, Limited Release & Stabilization; Followed by refinement/calibration
before final release
 Model release to authorized after rigorous testing phases (In-house & External Validation)
 Product design to factor be made flexible to factor in scenario change through parameterization,
configuration etc.
 Buy-in of Engineering teams is a must before model finalization, as it needs to be rolled-out
eventually
17
AI Models: Best PM Practices - Deployment Stage
 Strict product level controls, at levels, between models in production and those in other life
stages
 This can be achieved through fine-grained Role Based Access
 Strict control on code-base
 Version control through repositories such as GIT
 Linkage to source systems (real-time/off-line)
 Data pipelines to be clearly established and managed
 Do not over-engineer than required (Example: Too many transformations on-the-fly)
 Data validation to be undertaken periodically, depending on criticality (E.g. Models can have inbuilt
check to ensure data fields from source systems have not undergone a change by the provider)
 Must Have: Data Traceability (from where and when did the data originate)
 Real-Time/Near Real Time Computation
 Latency: Real world issues in network/connectivity
 Queuing/Bunching of data
 Off-line storage/computation
18
AI Models: Best PM Practices – Usage Stage
 Type I/Type II Errors: True flags not available immediately for retraining/recalibrating the
model
 Flagging (False Positives/True Negatives must be done at the earliest and models re-trained)
 Drift: Caused due to changing patterns of data due to factors such as changing
demographics, customer preferences etc.
 Drift may be gradual, sudden or intermittent
 Changing input data patterns; very different from the development sample
 Human oversight is a must, especially when quality of data is not up-to-the mark
 Backtest models periodically. If required, models need to be calibrated/modified
 Daily health check on various key parameters; any abnormal fluctuations can be immediately
noticed
19
AI Models Management: Three Pillars
Model Governance
Data Governance
Standards for model
design and development
Documentation for
internal/third party models
Change Management
Model Materiality
Model Inventory
Model Ownership
Automated process control
over development and
implementation
Feedback loop and
workflow remediation
Vendor Model
Management
Model Validation
Validation Calendar
depending on Model
Materiality
Standardised validation
process & Automation
Thresholds and Guardrails
for deviations
Documentation
Assessment of model
overrides
Remediation of issues
Model Issue Management
Creating feedback loops
Challenger Models
Parallel running of
Champion/Challenger
Models
Monitor major deviations
Benchmark performance
Measure against different
set of assumptions
Care to be taken to ensure
comparability of models
20
To Summarize
 These days, with ready made entablements, deploying ML models is easy but difficult to maintain
 ML models are not meant for solving problem once and forgetting about it; it needs constant attention
 When model is not performing, deciphering what went wrong is tedious
 There is no better substitute for rigorous Model Lifecycle Management and Model Risk Management,
embedded as part of Product Management
“Success in creating AI would be the biggest event in human history. Unfortunately, it might also be the last,
unless we learn how to avoid the risks”: Stephen Hawking (https://www.cam.ac.uk/research/news/the-best-
or-worst-thing-to-happen-to-humanity-stephen-hawking-launches-centre-for-the-future-of)
21
References/Further Reading
• What is an AI Model? https://www.ibm.com/topics/ai-model
• Hidden Technical Debt in Machine Learning Systems:
https://proceedings.neurips.cc/paper/2015/file/86df7dcfd896fcaf2674f757a2463eba-Paper.pdf
• Opening the black box of machine learning: https://www.fca.org.uk/insight/explaining-why-
computer-says-no
• The Equal Credit Opportunity Act [ECOA], 15 U.S.C. 1691 et seq. prohibits creditors from
discriminating against credit applicants on the basis of race, color, religion, national origin, sex,
marital status, age, because an applicant receives income from a public assistance program, or
because an applicant has in good faith exercised any right under the Consumer Credit Protection
Act
• Federal Reserve: Guidance on Model Risk Management
https://www.federalreserve.gov/supervisionreg/srletters/sr1107.htm
• European Central Bank: Guide to Internal Models
https://www.bankingsupervision.europa.eu/legalframework/publiccons/pdf/ssm.pubcon230622_g
uide.en.pdf
• EU AI Act: https://www.europarl.europa.eu/news/en/headlines/society/20230601STO93804/eu-
ai-act-first-regulation-on-artificial-intelligence
Thank You

More Related Content

Similar to AI MODELS USAGE IN FINTECH PRODUCTS: PM APPROACH & BEST PRACTICES by Kasthuri Rangan Bhaskar

EY + Neo4j: Why graph technology makes sense for fraud detection and customer...
EY + Neo4j: Why graph technology makes sense for fraud detection and customer...EY + Neo4j: Why graph technology makes sense for fraud detection and customer...
EY + Neo4j: Why graph technology makes sense for fraud detection and customer...Neo4j
 
Pixels.camp - Machine Learning: Building Successful Products at Scale
Pixels.camp - Machine Learning: Building Successful Products at ScalePixels.camp - Machine Learning: Building Successful Products at Scale
Pixels.camp - Machine Learning: Building Successful Products at ScaleAntónio Alegria
 
Preetam_Resume_Business Analyst
Preetam_Resume_Business AnalystPreetam_Resume_Business Analyst
Preetam_Resume_Business AnalystPreetam Sahu
 
Big Data, customer analytics and loyalty marketing
Big Data, customer analytics and loyalty marketingBig Data, customer analytics and loyalty marketing
Big Data, customer analytics and loyalty marketingKevin May
 
Impact of machine learning on banking sector
Impact of machine learning on banking sectorImpact of machine learning on banking sector
Impact of machine learning on banking sectorAbhishek Verma
 
How to get started in extracting business value from big data 1 of 2 oct 2013
How to get started in extracting business value from big data 1 of 2 oct 2013How to get started in extracting business value from big data 1 of 2 oct 2013
How to get started in extracting business value from big data 1 of 2 oct 2013Jaime Nistal
 
Valiance Corporate Deck
Valiance Corporate DeckValiance Corporate Deck
Valiance Corporate DeckRohit Pandey
 
DMA 2014: 6 Steps to Integrate Your Big Data
DMA 2014: 6 Steps to Integrate Your Big DataDMA 2014: 6 Steps to Integrate Your Big Data
DMA 2014: 6 Steps to Integrate Your Big DataSameer Khan
 
HWZ-Darden Konferenz: Building a Sustainable Analytics Orientation
HWZ-Darden Konferenz: Building a Sustainable Analytics OrientationHWZ-Darden Konferenz: Building a Sustainable Analytics Orientation
HWZ-Darden Konferenz: Building a Sustainable Analytics OrientationHWZ Hochschule für Wirtschaft
 
Bank offered rate based on Artificial Intelligence
Bank offered rate based on Artificial IntelligenceBank offered rate based on Artificial Intelligence
Bank offered rate based on Artificial IntelligenceIJAEMSJORNAL
 
Ai in insurance how to automate insurance claim processing with machine lear...
Ai in insurance  how to automate insurance claim processing with machine lear...Ai in insurance  how to automate insurance claim processing with machine lear...
Ai in insurance how to automate insurance claim processing with machine lear...Skyl.ai
 
Customer trust and viral growth presentation for partners
Customer trust and viral growth presentation for partnersCustomer trust and viral growth presentation for partners
Customer trust and viral growth presentation for partnersMicha Roon
 
Finlytica Solutions Brief​
Finlytica Solutions Brief​Finlytica Solutions Brief​
Finlytica Solutions Brief​PegasusKnowledge
 
Digital Engineering: Top Three Imperatives for Banks and Financial Services C...
Digital Engineering: Top Three Imperatives for Banks and Financial Services C...Digital Engineering: Top Three Imperatives for Banks and Financial Services C...
Digital Engineering: Top Three Imperatives for Banks and Financial Services C...Cognizant
 
Agile Mumbai 2022 - Balvinder Kaur & Sushant Joshi | Real-Time Insights and A...
Agile Mumbai 2022 - Balvinder Kaur & Sushant Joshi | Real-Time Insights and A...Agile Mumbai 2022 - Balvinder Kaur & Sushant Joshi | Real-Time Insights and A...
Agile Mumbai 2022 - Balvinder Kaur & Sushant Joshi | Real-Time Insights and A...AgileNetwork
 
Evolution of AI ML Solutions - A Review of Past and Future Impact.pdf
Evolution of AI ML Solutions - A Review of Past and Future Impact.pdfEvolution of AI ML Solutions - A Review of Past and Future Impact.pdf
Evolution of AI ML Solutions - A Review of Past and Future Impact.pdfChristine Shepherd
 
Responsible AI in Industry: Practical Challenges and Lessons Learned
Responsible AI in Industry: Practical Challenges and Lessons LearnedResponsible AI in Industry: Practical Challenges and Lessons Learned
Responsible AI in Industry: Practical Challenges and Lessons LearnedKrishnaram Kenthapadi
 
Machine Learning for Finance Master Class
Machine Learning for Finance Master Class Machine Learning for Finance Master Class
Machine Learning for Finance Master Class QuantUniversity
 
Enhancing E Business In The High Net Worth Client
Enhancing E Business In The High Net Worth ClientEnhancing E Business In The High Net Worth Client
Enhancing E Business In The High Net Worth ClientJohn Haley
 

Similar to AI MODELS USAGE IN FINTECH PRODUCTS: PM APPROACH & BEST PRACTICES by Kasthuri Rangan Bhaskar (20)

EY + Neo4j: Why graph technology makes sense for fraud detection and customer...
EY + Neo4j: Why graph technology makes sense for fraud detection and customer...EY + Neo4j: Why graph technology makes sense for fraud detection and customer...
EY + Neo4j: Why graph technology makes sense for fraud detection and customer...
 
Pixels.camp - Machine Learning: Building Successful Products at Scale
Pixels.camp - Machine Learning: Building Successful Products at ScalePixels.camp - Machine Learning: Building Successful Products at Scale
Pixels.camp - Machine Learning: Building Successful Products at Scale
 
Preetam_Resume_Business Analyst
Preetam_Resume_Business AnalystPreetam_Resume_Business Analyst
Preetam_Resume_Business Analyst
 
Big Data, customer analytics and loyalty marketing
Big Data, customer analytics and loyalty marketingBig Data, customer analytics and loyalty marketing
Big Data, customer analytics and loyalty marketing
 
Model Factory at ING Bank
Model Factory at ING BankModel Factory at ING Bank
Model Factory at ING Bank
 
Impact of machine learning on banking sector
Impact of machine learning on banking sectorImpact of machine learning on banking sector
Impact of machine learning on banking sector
 
How to get started in extracting business value from big data 1 of 2 oct 2013
How to get started in extracting business value from big data 1 of 2 oct 2013How to get started in extracting business value from big data 1 of 2 oct 2013
How to get started in extracting business value from big data 1 of 2 oct 2013
 
Valiance Corporate Deck
Valiance Corporate DeckValiance Corporate Deck
Valiance Corporate Deck
 
DMA 2014: 6 Steps to Integrate Your Big Data
DMA 2014: 6 Steps to Integrate Your Big DataDMA 2014: 6 Steps to Integrate Your Big Data
DMA 2014: 6 Steps to Integrate Your Big Data
 
HWZ-Darden Konferenz: Building a Sustainable Analytics Orientation
HWZ-Darden Konferenz: Building a Sustainable Analytics OrientationHWZ-Darden Konferenz: Building a Sustainable Analytics Orientation
HWZ-Darden Konferenz: Building a Sustainable Analytics Orientation
 
Bank offered rate based on Artificial Intelligence
Bank offered rate based on Artificial IntelligenceBank offered rate based on Artificial Intelligence
Bank offered rate based on Artificial Intelligence
 
Ai in insurance how to automate insurance claim processing with machine lear...
Ai in insurance  how to automate insurance claim processing with machine lear...Ai in insurance  how to automate insurance claim processing with machine lear...
Ai in insurance how to automate insurance claim processing with machine lear...
 
Customer trust and viral growth presentation for partners
Customer trust and viral growth presentation for partnersCustomer trust and viral growth presentation for partners
Customer trust and viral growth presentation for partners
 
Finlytica Solutions Brief​
Finlytica Solutions Brief​Finlytica Solutions Brief​
Finlytica Solutions Brief​
 
Digital Engineering: Top Three Imperatives for Banks and Financial Services C...
Digital Engineering: Top Three Imperatives for Banks and Financial Services C...Digital Engineering: Top Three Imperatives for Banks and Financial Services C...
Digital Engineering: Top Three Imperatives for Banks and Financial Services C...
 
Agile Mumbai 2022 - Balvinder Kaur & Sushant Joshi | Real-Time Insights and A...
Agile Mumbai 2022 - Balvinder Kaur & Sushant Joshi | Real-Time Insights and A...Agile Mumbai 2022 - Balvinder Kaur & Sushant Joshi | Real-Time Insights and A...
Agile Mumbai 2022 - Balvinder Kaur & Sushant Joshi | Real-Time Insights and A...
 
Evolution of AI ML Solutions - A Review of Past and Future Impact.pdf
Evolution of AI ML Solutions - A Review of Past and Future Impact.pdfEvolution of AI ML Solutions - A Review of Past and Future Impact.pdf
Evolution of AI ML Solutions - A Review of Past and Future Impact.pdf
 
Responsible AI in Industry: Practical Challenges and Lessons Learned
Responsible AI in Industry: Practical Challenges and Lessons LearnedResponsible AI in Industry: Practical Challenges and Lessons Learned
Responsible AI in Industry: Practical Challenges and Lessons Learned
 
Machine Learning for Finance Master Class
Machine Learning for Finance Master Class Machine Learning for Finance Master Class
Machine Learning for Finance Master Class
 
Enhancing E Business In The High Net Worth Client
Enhancing E Business In The High Net Worth ClientEnhancing E Business In The High Net Worth Client
Enhancing E Business In The High Net Worth Client
 

More from ISPMAIndia

"Scaling Product Leadership with AI" by Ravi Padaki
"Scaling Product Leadership with AI" by Ravi Padaki"Scaling Product Leadership with AI" by Ravi Padaki
"Scaling Product Leadership with AI" by Ravi PadakiISPMAIndia
 
"Elevate and Innovate: The Art of Product Leadership in the Age of AI" by Dee...
"Elevate and Innovate: The Art of Product Leadership in the Age of AI" by Dee..."Elevate and Innovate: The Art of Product Leadership in the Age of AI" by Dee...
"Elevate and Innovate: The Art of Product Leadership in the Age of AI" by Dee...ISPMAIndia
 
Building Products That Think- Bhaskaran Srinivasan & Ashish Gupta
Building Products That Think- Bhaskaran Srinivasan & Ashish GuptaBuilding Products That Think- Bhaskaran Srinivasan & Ashish Gupta
Building Products That Think- Bhaskaran Srinivasan & Ashish GuptaISPMAIndia
 
The Age of AI: Elevating Experiences & Delivering Customer Value!
The Age of AI: Elevating Experiences & Delivering Customer Value!The Age of AI: Elevating Experiences & Delivering Customer Value!
The Age of AI: Elevating Experiences & Delivering Customer Value!ISPMAIndia
 
AI Product Management by Abhijit Bendigiri
AI Product Management by Abhijit BendigiriAI Product Management by Abhijit Bendigiri
AI Product Management by Abhijit BendigiriISPMAIndia
 
SPM 2024 – Overview of and benefits of AI in Product Management
SPM 2024 – Overview of and benefits of AI in Product ManagementSPM 2024 – Overview of and benefits of AI in Product Management
SPM 2024 – Overview of and benefits of AI in Product ManagementISPMAIndia
 
"Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ...
"Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ..."Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ...
"Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ...ISPMAIndia
 
"Discovery and Delivery through Product IntelliGenAI framework" by Ramkumar A...
"Discovery and Delivery through Product IntelliGenAI framework" by Ramkumar A..."Discovery and Delivery through Product IntelliGenAI framework" by Ramkumar A...
"Discovery and Delivery through Product IntelliGenAI framework" by Ramkumar A...ISPMAIndia
 
"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google
"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google
"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, GoogleISPMAIndia
 
Product Manager vs Product Owner – Why Do Companies Still Struggle 23 Years A...
Product Manager vs Product Owner – Why Do Companies Still Struggle 23 Years A...Product Manager vs Product Owner – Why Do Companies Still Struggle 23 Years A...
Product Manager vs Product Owner – Why Do Companies Still Struggle 23 Years A...ISPMAIndia
 
"Customer- Centric AI: Personalising Product Experiences" by Apoorva Gudla, E...
"Customer- Centric AI: Personalising Product Experiences" by Apoorva Gudla, E..."Customer- Centric AI: Personalising Product Experiences" by Apoorva Gudla, E...
"Customer- Centric AI: Personalising Product Experiences" by Apoorva Gudla, E...ISPMAIndia
 
Journey of transforming legacy monolith to SaaS_Kartik_Suryanarayan.pdf
Journey of transforming legacy monolith to SaaS_Kartik_Suryanarayan.pdfJourney of transforming legacy monolith to SaaS_Kartik_Suryanarayan.pdf
Journey of transforming legacy monolith to SaaS_Kartik_Suryanarayan.pdfISPMAIndia
 
IIM BLR Smart Product & Deep Tech Strategy Healthcare 40 BANI world.pdf
IIM BLR Smart Product & Deep Tech Strategy Healthcare 40 BANI world.pdfIIM BLR Smart Product & Deep Tech Strategy Healthcare 40 BANI world.pdf
IIM BLR Smart Product & Deep Tech Strategy Healthcare 40 BANI world.pdfISPMAIndia
 
Careers360_Student Platform.pdf
Careers360_Student Platform.pdfCareers360_Student Platform.pdf
Careers360_Student Platform.pdfISPMAIndia
 
Billgren - Product Summit Speed Layers v12.pdf
Billgren - Product Summit Speed Layers v12.pdfBillgren - Product Summit Speed Layers v12.pdf
Billgren - Product Summit Speed Layers v12.pdfISPMAIndia
 
The Product Manager 2023.pdf
The Product Manager 2023.pdfThe Product Manager 2023.pdf
The Product Manager 2023.pdfISPMAIndia
 
SPM SUMMIT NESL.pdf
SPM SUMMIT NESL.pdfSPM SUMMIT NESL.pdf
SPM SUMMIT NESL.pdfISPMAIndia
 
ISPMA White Paper - Ensuring continuous PMF - Hans-Bernd Kittlaus - SPMS Indi...
ISPMA White Paper - Ensuring continuous PMF - Hans-Bernd Kittlaus - SPMS Indi...ISPMA White Paper - Ensuring continuous PMF - Hans-Bernd Kittlaus - SPMS Indi...
ISPMA White Paper - Ensuring continuous PMF - Hans-Bernd Kittlaus - SPMS Indi...ISPMAIndia
 
Billgren - Value Based Marketing Product Summit.pdf
Billgren - Value Based Marketing Product Summit.pdfBillgren - Value Based Marketing Product Summit.pdf
Billgren - Value Based Marketing Product Summit.pdfISPMAIndia
 
Creating a business case.pdf
Creating a business case.pdfCreating a business case.pdf
Creating a business case.pdfISPMAIndia
 

More from ISPMAIndia (20)

"Scaling Product Leadership with AI" by Ravi Padaki
"Scaling Product Leadership with AI" by Ravi Padaki"Scaling Product Leadership with AI" by Ravi Padaki
"Scaling Product Leadership with AI" by Ravi Padaki
 
"Elevate and Innovate: The Art of Product Leadership in the Age of AI" by Dee...
"Elevate and Innovate: The Art of Product Leadership in the Age of AI" by Dee..."Elevate and Innovate: The Art of Product Leadership in the Age of AI" by Dee...
"Elevate and Innovate: The Art of Product Leadership in the Age of AI" by Dee...
 
Building Products That Think- Bhaskaran Srinivasan & Ashish Gupta
Building Products That Think- Bhaskaran Srinivasan & Ashish GuptaBuilding Products That Think- Bhaskaran Srinivasan & Ashish Gupta
Building Products That Think- Bhaskaran Srinivasan & Ashish Gupta
 
The Age of AI: Elevating Experiences & Delivering Customer Value!
The Age of AI: Elevating Experiences & Delivering Customer Value!The Age of AI: Elevating Experiences & Delivering Customer Value!
The Age of AI: Elevating Experiences & Delivering Customer Value!
 
AI Product Management by Abhijit Bendigiri
AI Product Management by Abhijit BendigiriAI Product Management by Abhijit Bendigiri
AI Product Management by Abhijit Bendigiri
 
SPM 2024 – Overview of and benefits of AI in Product Management
SPM 2024 – Overview of and benefits of AI in Product ManagementSPM 2024 – Overview of and benefits of AI in Product Management
SPM 2024 – Overview of and benefits of AI in Product Management
 
"Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ...
"Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ..."Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ...
"Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ...
 
"Discovery and Delivery through Product IntelliGenAI framework" by Ramkumar A...
"Discovery and Delivery through Product IntelliGenAI framework" by Ramkumar A..."Discovery and Delivery through Product IntelliGenAI framework" by Ramkumar A...
"Discovery and Delivery through Product IntelliGenAI framework" by Ramkumar A...
 
"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google
"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google
"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google
 
Product Manager vs Product Owner – Why Do Companies Still Struggle 23 Years A...
Product Manager vs Product Owner – Why Do Companies Still Struggle 23 Years A...Product Manager vs Product Owner – Why Do Companies Still Struggle 23 Years A...
Product Manager vs Product Owner – Why Do Companies Still Struggle 23 Years A...
 
"Customer- Centric AI: Personalising Product Experiences" by Apoorva Gudla, E...
"Customer- Centric AI: Personalising Product Experiences" by Apoorva Gudla, E..."Customer- Centric AI: Personalising Product Experiences" by Apoorva Gudla, E...
"Customer- Centric AI: Personalising Product Experiences" by Apoorva Gudla, E...
 
Journey of transforming legacy monolith to SaaS_Kartik_Suryanarayan.pdf
Journey of transforming legacy monolith to SaaS_Kartik_Suryanarayan.pdfJourney of transforming legacy monolith to SaaS_Kartik_Suryanarayan.pdf
Journey of transforming legacy monolith to SaaS_Kartik_Suryanarayan.pdf
 
IIM BLR Smart Product & Deep Tech Strategy Healthcare 40 BANI world.pdf
IIM BLR Smart Product & Deep Tech Strategy Healthcare 40 BANI world.pdfIIM BLR Smart Product & Deep Tech Strategy Healthcare 40 BANI world.pdf
IIM BLR Smart Product & Deep Tech Strategy Healthcare 40 BANI world.pdf
 
Careers360_Student Platform.pdf
Careers360_Student Platform.pdfCareers360_Student Platform.pdf
Careers360_Student Platform.pdf
 
Billgren - Product Summit Speed Layers v12.pdf
Billgren - Product Summit Speed Layers v12.pdfBillgren - Product Summit Speed Layers v12.pdf
Billgren - Product Summit Speed Layers v12.pdf
 
The Product Manager 2023.pdf
The Product Manager 2023.pdfThe Product Manager 2023.pdf
The Product Manager 2023.pdf
 
SPM SUMMIT NESL.pdf
SPM SUMMIT NESL.pdfSPM SUMMIT NESL.pdf
SPM SUMMIT NESL.pdf
 
ISPMA White Paper - Ensuring continuous PMF - Hans-Bernd Kittlaus - SPMS Indi...
ISPMA White Paper - Ensuring continuous PMF - Hans-Bernd Kittlaus - SPMS Indi...ISPMA White Paper - Ensuring continuous PMF - Hans-Bernd Kittlaus - SPMS Indi...
ISPMA White Paper - Ensuring continuous PMF - Hans-Bernd Kittlaus - SPMS Indi...
 
Billgren - Value Based Marketing Product Summit.pdf
Billgren - Value Based Marketing Product Summit.pdfBillgren - Value Based Marketing Product Summit.pdf
Billgren - Value Based Marketing Product Summit.pdf
 
Creating a business case.pdf
Creating a business case.pdfCreating a business case.pdf
Creating a business case.pdf
 

Recently uploaded

All These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFAll These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFMichael Gough
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Mark Goldstein
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPathCommunity
 
Data governance with Unity Catalog Presentation
Data governance with Unity Catalog PresentationData governance with Unity Catalog Presentation
Data governance with Unity Catalog PresentationKnoldus Inc.
 
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security ObservabilityGlenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security Observabilityitnewsafrica
 
QCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesQCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesBernd Ruecker
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxLoriGlavin3
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxLoriGlavin3
 
Zeshan Sattar- Assessing the skill requirements and industry expectations for...
Zeshan Sattar- Assessing the skill requirements and industry expectations for...Zeshan Sattar- Assessing the skill requirements and industry expectations for...
Zeshan Sattar- Assessing the skill requirements and industry expectations for...itnewsafrica
 
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotesMuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotesManik S Magar
 
React JS; all concepts. Contains React Features, JSX, functional & Class comp...
React JS; all concepts. Contains React Features, JSX, functional & Class comp...React JS; all concepts. Contains React Features, JSX, functional & Class comp...
React JS; all concepts. Contains React Features, JSX, functional & Class comp...Karmanjay Verma
 
React Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkReact Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkPixlogix Infotech
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfNeo4j
 
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...Jeffrey Haguewood
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Hiroshi SHIBATA
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxLoriGlavin3
 
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality AssuranceInflectra
 
Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024TopCSSGallery
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityIES VE
 
A Framework for Development in the AI Age
A Framework for Development in the AI AgeA Framework for Development in the AI Age
A Framework for Development in the AI AgeCprime
 

Recently uploaded (20)

All These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFAll These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDF
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to Hero
 
Data governance with Unity Catalog Presentation
Data governance with Unity Catalog PresentationData governance with Unity Catalog Presentation
Data governance with Unity Catalog Presentation
 
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security ObservabilityGlenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
 
QCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesQCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architectures
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
 
Zeshan Sattar- Assessing the skill requirements and industry expectations for...
Zeshan Sattar- Assessing the skill requirements and industry expectations for...Zeshan Sattar- Assessing the skill requirements and industry expectations for...
Zeshan Sattar- Assessing the skill requirements and industry expectations for...
 
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotesMuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
 
React JS; all concepts. Contains React Features, JSX, functional & Class comp...
React JS; all concepts. Contains React Features, JSX, functional & Class comp...React JS; all concepts. Contains React Features, JSX, functional & Class comp...
React JS; all concepts. Contains React Features, JSX, functional & Class comp...
 
React Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkReact Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App Framework
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdf
 
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
 
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
 
Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a reality
 
A Framework for Development in the AI Age
A Framework for Development in the AI AgeA Framework for Development in the AI Age
A Framework for Development in the AI Age
 

AI MODELS USAGE IN FINTECH PRODUCTS: PM APPROACH & BEST PRACTICES by Kasthuri Rangan Bhaskar

  • 1. AI MODELS USAGE IN FINTECH PRODUCTS: PM APPROACH & BEST PRACTICES ISPMA | SPM Summit 2024 | February 2024
  • 2. 2 Profile Kasthuri Rangan Bhaskar is a Product Leader and banking/finance SME, with a proven track record of orchestrating fintech products to success, from zero to one. He currently leads products at BCT Digital. He has 18+ years of diverse techno-functional exposure spread across roles in commercial banking, digital transformation, risk management, technology, pre-sales & B2B product management. He has innovated products in the domains of Lending, Credit Risk, Collateral Management, GRC and Model Risk Management. He has extensively worked on AI models in Credit and Fraud Risk Management space. BCT Digital’s products have been recognized by leading analysts such as Chartis. He has also worked with banking regulators as part of sandbox testing. He is a qualified Chartered Accountant, an Associate Member of CPA (Australia), and holds an MBA from IIM Indore. https://www.linkedin.com/in/kasrangan/
  • 3. 3 Agenda  AI Models: Usage in Fintech Products  Key Functional & Technical Challenges  Key challenges of deploying models at scale: Indian context  Key Regulatory Concerns: Bias and Explainablity of AI Models  Product Management across AI Model Lifecycle: Development |Deployment |Usage  3 Pillars for AI Models Product Management: Model Governance| Model Validation | Challenger Models  References for further study
  • 4. 4 AI Models: Usage in Fintech Products Area Illustrative AI Use Cases Lead Generation • Personalized Marketing • Tracking customer’s buying preference over a period • Prioritizing leads for sales channels for maximizing conversion rates Campaign Management • Identifying most effective campaign technique Improving wallet share • Personalized offers for cross-selling • Optimal investment strategies for wealth management Credit Scoring • Credit Decisioning for BNPL/Instant loan companies • Identifying repayment patterns • Non-traditional sources such as Social Media KYC • Image recognition for ID cards • OCR for digitalization of manually filled-up forms Risk Management • Predictive Analytics for identifying risk of default • Suggesting risk management strategies such as hedging Fraud Risk/AML • Suspicious Transactions, Funds Diversion Customer Servicing • AI based Chat Bots • Contextual e-mail/voice-based responses Back office (Operations) • Automating repeated tasks such as form filling/data entry • Automated workflows, MISMIS AI Models are increasingly replacing human judgement and effort
  • 5. 5 AI Models Adoption: Key reasons for tremendous increase  High velocity/volume of transactions: AI helps in increasing the speed of decision making  Overcoming human limitations in identifying patters through cognition  Information overload: Beyond processing capacity of routine algorithms  Need for improving models which can reduce frauds/AML incidents  Need to automate very routine activities: Cost/Time Efficiency; boosting human productivity  Increasing digital reach (UPI/Apps): No human factor involved in the value stream  Decreasing cost of compute power (Memory & Processing)  Easy Deployment: Ready-built platforms for AI Models deployment (E.g. AWS offers off- the-shelf AI services for image/speech recognition, chatbots, personalization of websites, Fraud Detection etc.)
  • 6. 6 ..But the devil lies in the details  An AI Model is not a standalone entity within a product. Intended success of an embedded AI model depends on factors such as:  Infrastructure (E.g., Hardware, Processing Power, Connectivity)  Model Algorithm  Modeler’s expertise  Quality of code  Quality of data  Product Configurations, Parameters (E.g., Thresholds, logical operators)  Regulatory Compliance  Consequently, AI models need to be carefully monitored at every stage of their lifecycle Image Credit: https://home.paynearme.com/blog/beyond-innovation-what-it-takes-to-be-successful-with-ai/
  • 7. 7 Case Study XYZ, a fintech company, disburses unsecured loans to customers for the lower/mid-income segment. Its products embed AI Models in key areas such as origination, credit scoring, fraud risk and customer-servicing. Ravi, the company’s Chief Data Scientist, is a worried man these days as there have been increasing instances of : - Stretched Payments/Defaults by customers flagged as ‘good’ during lead generation - Low lead conversion rates despite extensive social media campaigns - Frauds going undetected - Customer issues such as improper responses by chatbots - Auditors raising concerns on the transparency and efficacy of the models Following this, he appointed a third-party consultant to investigate into the matter. The consultant did a thorough analysis of all the AI Models and presented the senior management with the following findings (explained in the next few slides):
  • 8. 8 AI Models: Key Functional Challenges for a PM/DS Dev/Production Data Mismatch • Development of ML models happens in carefully cleaned models while real- life data may be completely different • Finding: XYZ’s model was developed on anonymized data obtained from a third-party vendor and was not representative of the company’s primary customer segments • Implication: Unstable/out-of- sync models Explainability • We can observe the input and the output; however, we may not be able to explain the relationship/connec tion between them • Finding: XYZ could not explain how changing income levels impacted the models, over a period of time • Implication: Why and how the output has come from the input is not known Changing Anything Changes Everything (CACE) • Various components of a ML model (inputs, model & parameters) are entangled (i.e., they do not have a separate identify • Finding: XYZ used 15 different variables including non-traditional data points such as smart phone usage • Implication: Any alteration/data fluctuation in the input may impact the performance of the model Undeclared Customers • Output of a ML model may be silently used by unintended/undecla red customers (such as other systems). • Finding: Single model was being used across business segments • Implication: May negatively influence model’s behavior (E.g. False Positives) Unstable Data Dependencies • Finding: Input to a ML model may be consumed from another source, which is unstable (such as Social Media) • Implication: Unstable/out-of- sync models
  • 9. 9 AI Models: Key Technical Challenges  Data Management  Managing data gaps, quality, governance and privacy  Real Time/Near Real Time Computations  Latency in infrastructure, penetration of internet/smartphone etc. may impact real time decisioning  Processing and queuing large data  Frequent updation of scenarios  Scenarios/use cases changes over a period (E.g.: New types of frauds emerge); incorporating them existing models would be a challenge  Managing multiple deployment approaches  Models may have to be deployed on-cloud/locally/webservice, depending on the specific need. Managing multiple deployment approaches is a huge challenge  Version Control  Difficulty in managing different versions of models, say across products, model lifecycle and customer segment
  • 10. 10 AI Models: Key challenges from an Indian Context  Lack of industry level data for model enrichment & validation  Lack of shared data repositories such as industry consortium (excepting Credit Bureau); access to certain databases are restricted to banks/NBFCsequate data standards  Inadequate Model Governance Regulations  Inadequate regulatory push to maintain model standards  Lack of clean data  Usage of assumptions, proxies to fill up data gaps  Data Privacy  Lack of effective control over usage of PII data  Fast changing demographics  Data not representative of the sample on which it was developed; model drift may happen quickly (For example, income levels have undergone tremendous change in the last 5 years)
  • 11. 11 Key Regulatory Concerns: Explainability Explainability is influenced by Type of decision & End-user of the explanation Source Deloitte https://www2.deloitte.com/us/en/insights/industry/financial-services/explainable-ai-in-banking.html
  • 12. 12 Key Regulatory Concerns: Explainability  Three components of Explainability  Transparency: Ease of understanding  Traceability of different model parameters/components  Questionability: Questioning back ‘why it is so’ or ‘what if’  Approaches to improving Explainability  Not making it more complicated than required  Trade off between explainability and predicted outcome  Improving explainability of individual components and their relationship (E.g. Individual Condition Expectations Plots)  Improving explainability of significant components and their collective impact on the overall model
  • 13. 13 Key Regulatory Concerns: Bias  Bias unfairly restricts a segment from participation (For example, denial of credit)  Regulators like RBI has highlighted bias as one of the key risks of AI Models  There are various laws which explicitly prohibit such bias  For example, the US Equal Credit Opportunity Act prohibits discrimination on the basis of race, color, religion, national origin, sex, marital status, age or receipt of public assistance  Sources of bias in an AI Model could be  Training Data: For example, heavily skewed data  Algorithm: For example, overweighing/double counting certain parameters  Interpretation: Caused due to judgements based on past preferences/judgements  Human oversight on the models at all stages is a very important factor to reduce bias  Understanding and removing bias is most complex as it involves understanding of data, data science and social science “The manifestation of unfair or discriminatory behaviour in an AI/ML model’s predictions, influenced by biased training data or the algorithm itself, leading to unequal treatment of different groups”
  • 14. 14 Explainablity/Bias: Illustrations • Discrimination based on color: https://www.cnbc.com/2023/06/23/ai-has-a-discrimination- problem-in-banking-that-can-be-devastating.html • AI Models such as ChatGPT has ‘invented hypothetical case studies’ (https://apnews.com/article/artificial-intelligence-chatgpt-courts- e15023d7e6fdf4f099aa122437dbb59b) • Amazon was found using a hiring algorithm that favoured male candidates against female candidates based on certain keywords like “executed”, usually found in resumes of male candidates (https://www.reuters.com/article/us-amazon-com-jobs-automation-insight- idUSKCN1MK08G/) • Google’s ad algorithm was displaying more high paying jobs to males than to females (https://www.washingtonpost.com/news/the-intersect/wp/2015/07/06/googles-algorithm- shows-prestigious-job-ads-to-men-but-not-to-women-heres-why-that-should-worry-you/)
  • 15. 15 AI Models: Product Management across lifecycle Model Retirement Model Ideation Model Design & Development Independent Validation Testing (E.g.: Sandbox, Pilot, UAT) Model Deployment & Usage Ongoing Validation Calibration Challenger Models Initial Model Development Stage Continuous Model Validation Model Governance Model Documentation AI Model Lifecycle management refers to the process of managing models across its lifecycle i.e., from Development, Usage till the stage of Retirement. AI Models have a significantly larger domino effect.
  • 16. 16 AI Models: Best PM Practices - Development Stage  Prior to new model development following factors to be considered:  Choice of In-house vs Third Party Models  Calibrate Existing vs Develop New Models  Technology choice for Model Development platform: Carefully consider factors such as overall architecture, functional capabilities, technology lock-in, any technical debt overhang  Choice of model after considering various factors such as data availability, quality & appropriateness  Trade-off between sophistication of model, data availability and model soundness  Model development environment to be sanitized and demarcated to mix up data/models with production environment  Pre-Production Testing: Pilots, Limited Release & Stabilization; Followed by refinement/calibration before final release  Model release to authorized after rigorous testing phases (In-house & External Validation)  Product design to factor be made flexible to factor in scenario change through parameterization, configuration etc.  Buy-in of Engineering teams is a must before model finalization, as it needs to be rolled-out eventually
  • 17. 17 AI Models: Best PM Practices - Deployment Stage  Strict product level controls, at levels, between models in production and those in other life stages  This can be achieved through fine-grained Role Based Access  Strict control on code-base  Version control through repositories such as GIT  Linkage to source systems (real-time/off-line)  Data pipelines to be clearly established and managed  Do not over-engineer than required (Example: Too many transformations on-the-fly)  Data validation to be undertaken periodically, depending on criticality (E.g. Models can have inbuilt check to ensure data fields from source systems have not undergone a change by the provider)  Must Have: Data Traceability (from where and when did the data originate)  Real-Time/Near Real Time Computation  Latency: Real world issues in network/connectivity  Queuing/Bunching of data  Off-line storage/computation
  • 18. 18 AI Models: Best PM Practices – Usage Stage  Type I/Type II Errors: True flags not available immediately for retraining/recalibrating the model  Flagging (False Positives/True Negatives must be done at the earliest and models re-trained)  Drift: Caused due to changing patterns of data due to factors such as changing demographics, customer preferences etc.  Drift may be gradual, sudden or intermittent  Changing input data patterns; very different from the development sample  Human oversight is a must, especially when quality of data is not up-to-the mark  Backtest models periodically. If required, models need to be calibrated/modified  Daily health check on various key parameters; any abnormal fluctuations can be immediately noticed
  • 19. 19 AI Models Management: Three Pillars Model Governance Data Governance Standards for model design and development Documentation for internal/third party models Change Management Model Materiality Model Inventory Model Ownership Automated process control over development and implementation Feedback loop and workflow remediation Vendor Model Management Model Validation Validation Calendar depending on Model Materiality Standardised validation process & Automation Thresholds and Guardrails for deviations Documentation Assessment of model overrides Remediation of issues Model Issue Management Creating feedback loops Challenger Models Parallel running of Champion/Challenger Models Monitor major deviations Benchmark performance Measure against different set of assumptions Care to be taken to ensure comparability of models
  • 20. 20 To Summarize  These days, with ready made entablements, deploying ML models is easy but difficult to maintain  ML models are not meant for solving problem once and forgetting about it; it needs constant attention  When model is not performing, deciphering what went wrong is tedious  There is no better substitute for rigorous Model Lifecycle Management and Model Risk Management, embedded as part of Product Management “Success in creating AI would be the biggest event in human history. Unfortunately, it might also be the last, unless we learn how to avoid the risks”: Stephen Hawking (https://www.cam.ac.uk/research/news/the-best- or-worst-thing-to-happen-to-humanity-stephen-hawking-launches-centre-for-the-future-of)
  • 21. 21 References/Further Reading • What is an AI Model? https://www.ibm.com/topics/ai-model • Hidden Technical Debt in Machine Learning Systems: https://proceedings.neurips.cc/paper/2015/file/86df7dcfd896fcaf2674f757a2463eba-Paper.pdf • Opening the black box of machine learning: https://www.fca.org.uk/insight/explaining-why- computer-says-no • The Equal Credit Opportunity Act [ECOA], 15 U.S.C. 1691 et seq. prohibits creditors from discriminating against credit applicants on the basis of race, color, religion, national origin, sex, marital status, age, because an applicant receives income from a public assistance program, or because an applicant has in good faith exercised any right under the Consumer Credit Protection Act • Federal Reserve: Guidance on Model Risk Management https://www.federalreserve.gov/supervisionreg/srletters/sr1107.htm • European Central Bank: Guide to Internal Models https://www.bankingsupervision.europa.eu/legalframework/publiccons/pdf/ssm.pubcon230622_g uide.en.pdf • EU AI Act: https://www.europarl.europa.eu/news/en/headlines/society/20230601STO93804/eu- ai-act-first-regulation-on-artificial-intelligence

Editor's Notes

  1. https://www.ibm.com/blog/shedding-light-on-ai-bias-with-real-world-examples/