1. © 2021 Thoughtworks | Confidential
Real time insights and AI for
better products, customer
experience and resilient
platform
Balvinder Khurana and Sushant Joshi
www.agilemumbai.com

2. Who we are
2
Balvinder Khurana
Principal Consultant
Data Architect and
Global data community lead
Sushant Joshi
Product Principal
@sushantjoshi
https://sushant-joshi.medium.com
Balvinder has 15 years of experience in building large-
scale custom software and big data platform solutions for
complicated client problems. She has extensive experience
in Analysis, Design, Architecture,
and Development of Web based Enterprise systems and
Analytical systems using Agile practices like Scrum and XP.
Balvinder currently works as a Data Architect and Global
Data Community Lead for Thoughtworks
Sushant is a Product Principal at ThoughtWorks. His work
includes working with clients to assess product-market-fit,
create goal aligned roadmaps and product delivery. He
brings in his ever-curious mindset, business knowledge,
and interdisciplinary thinking to solve problems that form
our surroundings. His primary focus area is - product
discovery - through which he helps address key product
risks in the early stages of the product
Sushant is passionate about Indian digital ecosystems. He
is working with Indian companies to create better products.

3. Start your day with the your business dashboard!
3

4. Start your day with the your business dashboard!
4

5. Problem Landscape
5
Retail Bank
KYC - Compliance
to Customer Service
The Ask

6. Few examples:
How much amount was disbursed yesterday
in Mumbai?
How many car loans were sanctioned this
week?
What’s the health of APIs and underlying
systems for last 3 hours?
What are we
trying to achieve
6
“Data intelligence delivering business
value” What are the reasons
for drop offs?
Is it system or user?
Are my APIs
overloaded?
Average time for
disbursement
Opportunity loss in the
Funnel at different
stages
Login
Offers
Risk
Checks
Sanction
2FA
Disbursement
Which offers are
attractive?
How many customers
are moving further after
viewing offers
How long it takes to
send the OTP and
customer action?
Do users need alternate
mechanism?

7. Explosion of personas and explosion of requirements
7
Revenue Generated
Price Sensitivity
Number of Users
Customer micro-
Segment
Business
Deployment status
Load on a service
Downtime for Service
Developers
Service Availability
Service Traceability
Routing
Security team
Insights
What can I
understand?
Data Scientists
Customer 360
Customer Propensity
Customer-product fit
Customer facing
executives
7

8. 8
Isolated Solutions
Web
application
Mobile
application
User click
stream
Social media
Market data
System level
metrics
Customer
support
Competitor
data
Logs
Real time monitoring of all
infrastructure components and
service issues monitored using
Prometheus and charts created
on Grafana. Many other tools
like EFK stack, kafka etc. are
used.
Developers/
IT Support
Understand
system health
and avoid failures
Periodic data is provided
(sometimes manually) after
pulling out of tools like Kafka,
GTM in form of excel.
Product
Owners
No drop-outs,
all journeys
should be
completed
Business/
C-level execs
Data pulled out on-demand
(manually) and shared via
email/excel sheets.
How is my business
performing, where
are the leakages

9. 9
Business
● Siloed
○ Systems
○ Tools
● Different
○ Targets
○ Maturity
○ Objectives
People & Process
● Dependent
○ On a central data team
● Manual effort
○ Lack of standard
processes
○ Duplicate effort
● Different
○ Tech Stack, Architecture
& tools
○ Data granularity, formats
& architectures
○ Data exploration Scopes
● Siloed
○ Data (Storage)
○ Business Units
Data & Technology
Limitations and pain points

10. 10
● Consolidation
● Coordination
● Manual synthesis
● Low confidence
Piecemeal Solutions

11. 11
Shift in mindset
Solutions of
Yesterday
created the problems of
Today
Photo by Garidy Sanders on Unsplash

12. 12
Persona and
business domains,
wants/requirement
s
Levers of mindset shift
Traditionally data is looked from the weekly or for monthly leadership review.
So someone requests and presents what needs to be presented
From requests to predefined datasets serving insights & exploration
Design,
Data discoverability
& presentability
Product Mindset
12
© 2021 Thoughtworks

13. Bringing it all
together - The
“Superdata”
Solution
13

14. 14
14
What it is really? Make it quick and easy to explore a
hypothesis (business or technical),
accept or disprove it, and move on
to find the root cause.
Platform which enables people to
use their skills, extend their senses,
support their intuitions.
The Superdata solution a.k.a
Command center

15. 15
Data Platform
Cloud DW
Data Lake
(Cloud Object
Storage)
arts
Data
Marts
ODS
Tech Solution
Dashboards
Data Service
API’s
Reports
Data Ingestion &
Integration
Batch
Ingestion
Unstructured
Source
Ingestion
API Ingestion
Streaming
Ingestion
Orchestration Service
15
Source
Systems
Bank
applications
data
Tele channels
data
Physical
data from
Branches
Social media
data
Partners
data
15
15
DevOps/DataOps
Data Governance
Data Catalog Data Quality Security
Business Events
Data Transformation & Ml
ELT
Stream
Processing
ML
Toolkits*
Deep
Learning*
ETL
*Future Scope

16. ©ThoughtWorks 2021 Commercial in Confidence
SOURCE
SYSTEMS
Tele channels
data
Physical data
from branches
Social media
data
Partners data
Bank
applications
data
Whitelisted,
one way hashed data
CONFIGURABLE
SECURITY
LAYERS
Data filter
service
GEMALTO
CCKM
GEMALTO
HSM
TOKEN
LOOKUP
STORE
DATA INGESTION /
INTEGRATION ON LAYER
DATA TRANSFORMATION
LAYER
DATA CONSUMPTION LAYER
Cloud DW
Data Lake
(Cloud Object
Storage)
arts
Data
Marts
ODS
ODS
Cloud Native Encryption at Storage
DATA PLATFORM
DATA INGESTION /
INTEGRATION ON LAYER
DATA TRANSFORMATION
LAYER
DATA CONSUMPTION LAYER
PII / SPDI Layer (Only for PII fields)
Data Encryption
Component
Data Decryption Component
Tokenization Layer (Only for PII fields)
Tokenizer De Tokenizer
Gemalto
Virtual HSM
KMS
VPC
Security in Layers

17. Data Quality Framework
Intermediate Data Quality
Fit for Purpose Data Quality
Ad-hoc Analysis
Data Discovery
Metadata Service
Repository &
Indexing Service
Ownership of DQ
Fit for Purpose Data Quality
Purpose
Fit for Purpose Data Quality
Purpose
Baseline Data Quality / Sensible Defaults
Metrics
Definition
Rules
Authoring
Rules
Execution
Engine
KPIs and
Dashboards
Metrics
Definition
Continuous data quality improvement*

18. Dashboards
Data Service
API’s
Reports
18
Tech Stack
Data Platform
Cloud DW
Data Lake
(Cloud Object
Storage)
Data
Marts
ODS
Data Ingestion &
Integration
Batch
Ingestion
Unstructured
Source
Ingestion
API Ingestion
Streaming
Ingestion
Orchestration Service
Data Transformation & Ml
ELT
Stream
Processing
ML
Toolkits
Deep
Learning
ETL
Source
Systems
Bank
Applications
Data
Tele Channels
Data
Physical
Store Data
Social Media
Data
Partners
data
DevOps/DataOps
Data Governance
Data Catalog Data Quality Security
Business Events

19. 19
19
Data Platform
Cloud DW
Data Lake
(Cloud Object Storage) arts
Data Marts
ODS
Dashbo
ards
Data
Service
API’s
Report
s
Data Ingestion &
Integration
Batch Ingestion
Unstructured Source
Ingestion
API Ingestion
Streaming Ingestion
Orchestration Service
Data Transformation & Ml
ELT
Stream
Processi
ng
ML Toolkits
Deep
Learning
ETL
Source
Systems
Bank Applications
Data
Tele Channels Data
Physical
Store Data
Social Media Data
Partners
data
DevOps/DataOps
Data Governance
Data Catalog Data Quality Security
Business Events
Serve Data as a Product
Auto
Loan/Cam
paign
Customer
Personal
Loan
/Network
analysis
Credit
Card/Finan
ce
Social
Media
Customer
Profile
Domain driven
data boundaries
The boundaries cut across
the platform - from source
to consumption!

20. Principles guiding building blocks of Data Mesh
20
Domain ownership Data as a
product
Self-serve data
infrastructure
Federated
computational
governance
{G} {G} {G}

21. 21
Use cases served through platform
● Increased self-service
● Anomaly detection and alerting
● Personalization and Nudges
● Domain driven boundaries for data
● Adaptive journey completions

22. Impact and learnings
22

23. 23
Business
● Responsive
○ Real Time
○ Self service
○ Responding to customer
behavior quicker
● Insights
○ Stimulating
○ Proactive
Process & People
● Transparent
○ Democratization
○ Standardized processes
○ Data driven process
planning
● Empowered
○ Own, create and analyse
○ Touch multiple business
aspects
● Governance & Quality
○ Accurate
○ Secure
● Technology
○ Resilient
○ Rapidly evolving
○ Loosely coupled
○ Configuration driven
Data & Technology
Impact
“Can we ask for this data” to “Can we pull
up this data”
“we are not able to track numbers, since
last couple of hours dashboard is showing
the same numbers”

24. 24
Our Learning
● Data-as-a-first-class-citizen
● Extend domain boundaries to data
platform
● Avoid tight coupling with upstream
and downstream
● Timely scaling considerations

25. Thank you
Sushant Joshi
Twitter: @sushantjoshi
https://sushant-joshi.medium.com
Balvinder Khurana
LinkedIn: https://www.linkedin.com/in/balvinder-khurana/
https://khuranabalvinder.medium.com/
25
www.agilemumbai.com

26. Size, Scale and Complexity
26
Web & Apps
Tele channels
Physical stores
Digital Ecosystem
partners
Kiosks, agents etc
Sales &
marketing
operations
Products
Technology
Risk &
compliance
Strategy
Enterprise
functions
Social media
Retail Loans
Retail
Deposits
...
Regulators
Investors
Aggregators
Partners
Product
based
Customer
segment
based
Channel
Corporate
Customer
risk
Fraud
Underwriting
Liquidity
HR
Employee
Facilities
Legal
How customers interact with
the organisation
Constraining and driving
forces

27. Data Product
Data Platform Architecture
Quantum
Fundamental unit of architecture
Self-Serve Data
Product
Domain
Polyglot Data
Output Ports
Polyglot Data
Input Ports
Discoverable
Addressable
Self-describing
Trustworthy
Interoperable
governed by global open
standard
Secure
enforce globally configured
access control at each data
product output port
Control
Ports
Stats
Logs, metrics
Self
discovery
Management
Defined and Monitored silos
27