This tech talk deals with how we leveraged Spark Streaming and Spark Machine Learning models to build & operationalize real-time credit card approvals for a banking major. We plan to cover ML capabilities in Spark and how a typical ML pipeline looks like.
We are going to talk about the domain and the use case of how a major credit card provider is using spark to calculate card eligibility in real-time. We’re also going to share the challenges faced by the current system and how spark is a good fit to solve these kinds of problems.
We will then take a deep dive on the different tools that were used to design the solution and the architecture of the system. Here, we will also be sharing of how a spark based workflow was created to address various aspects like reading from Kafka, parsing, data enrichment, model selection, model scoring, rule execution to conclude the recommended output.
Finally, we’re also going to talk about the key challenges, learning and recommendations when building such a system and taking it to production.