"Embarking on a journey to design a real-time fraud barrier in the fintech domain brought forth both challenges and revelations. This session aims to share our narrative of leveraging Apache Kafka for constructing a robust fraud detection mechanism, while reflecting on the design missteps and how they steered the evolution of a more refined, effective solution.
We will unfold our initial approach, the unforeseen challenges encountered due to certain design choices, and how a pivot in our architectural blueprint to incorporate fast-moving variables led to a more resilient fraud prevention framework. The discussion will detail our experiences with Kafka Streams for real-time data processing and Kafka Connect for seamless data integration, while shedding light on the bad practices that were replaced to enhance our fraud detection capabilities.
Key Takeaways:
Architecting a real-time fraud barrier using Apache Kafka, and an honest reflection on the initial design missteps.
Evolving the design to incorporate fast-moving variables for prompt fraud detection and mitigation.
Utilizing Kafka Streams and Kafka Connect more effectively by overcoming initial design flaws.
Practical lessons learned from design missteps, and actionable takeaways to avoid common pitfalls in real-time fraud prevention system design.
By sharing our journey, complete with the challenges faced, lessons learned, and strategies employed, this session aims to provide attendees with a rich understanding of not only how to construct a real-time fraud prevention framework using Kafka, but also what common pitfalls to avoid. The narrative will underscore the significance of iterative design and continuous learning in developing a robust, scalable, and real-time responsive fraud prevention system."
5. Payment Ecosystem
● Innovative Payment Solutions
○ Pay in 4
○ Pay later
○ Financing
○ Direct Payments and Flexibilities
● Banking infrastructure
6. Customer Centric Approach
● Payment Ecosystems
○ Flexibility, Freedom and Trust
● Critical role of Data Integrity
● Real time processing
○ Data curation
○ Seamless Customer Experience
7. Content
01 Frauds in Fintech
Need for real time ?
02 Scenario 1
03 Scenario 2
Promotional Abuse
04 Real Time Platform
Challenges
05 Future of Fraud
06 Conclusion
Synthetic Identity theft
14. What ? How ?
● Combination of real and fake
information
● Prolonged Fraud Lifecycle
○ Not straightforward Identity theft
○ Might build creditworthiness
● Detection Challenges
○ Hybrid nature
○ Fraud Indicators
15. What's a Variable ?
● Data points for a genre of data
○ Real time
○ Batched
● To solve our current problem
○ Let us take `incoming_debt` as an
example
● Kafka being the backbone for the
orchestration and creation using
streams
16. Incoming Debt ?
● Ideal behaviour for a user
● Detect Fraud while it's happening
● Behavior of a fraudulent account ?
○ Maybe initial credibility
○ Bursts of purchase behaviour
○ No payments or settlements
○ Lots of minimal or exorbitant
transactions ?
17.
18.
19.
20.
21.
22.
23.
24.
25.
26. Prevent and Detect
● Understand order/transaction
behaviours
● Anomaly detection
● Market agnostic
○ Underrated
● Movement in User Incoming debt
for a window
28. What ? How ?
● Promotions are a double edged sword
● Exploits offers
○ Bots or Individuals
○ Undermine marketing efforts and
customer experiences
● Unchecked Exploitation
○ Losses
○ Customer trust ?
29. We need some Real timeness
● First line of defense
● Track multiple events
○ Promotional campaigns
○ Transactions
○ Clickstream
○ Account Activities
32. Mitigation of Promotion Abuse
● Analysis of curated and understood
variables
○ Code redemptions per IP
○ Rapid creations of new accounts
○ Intersecting Account details
● ML models to be trained on new
variables and dimensions
○ Refine and improve
33. Future of Fraud ?
● Requirements
○ Automated alerts
○ Variable creation
○ Swift and Scalable responses
across the customer base
○ Anomaly/Pattern Detection
○ Market Agnostic
■ EU, AP, US, CA . . .
34. Vision for Real Timeness
● Broader Approach
● Extendable
○ Maybe a user curated approach
○ Reuse concepts to create better
risk variables
○ Risk based on
■ Merchants
■ Users
■ Payment Behaviour
○ Analytics
38. Challenges
● Engineering meets Analytics
● Multi Region, more markets and data
sources
● Data consistency (Upstream)
● Discovery and curation of market
agnostic logics and data freshness
● Calculation of historic values for
models (Hybrid maybe ?)
39. ● Real Time Processing
○ Not only in detection of Fraud
○ Improving customer experience
● Potential in Viewing and Experimenting
with new Data Sets
● Reduction of cost
● Faster Reaction time
● Increase Security and Reliability of the
organization
Conclusion