34. Fast Data from Legacy to Cloud
The battle to overcome the gravity
Marzo ’19
Antonello Mantuano
Head of Software Engineering
antonellomantuano
@manant74
35. Cerved – the Data Driven Company
42
Credit
Information
Credit
Management
Marketing
Solutions
LEAD GENERATION
CREDIT COLLECTION
DATA PROVIDING
& MARKETING
ANALYSIS
CREDIT INFORMATION
CREDIT SCORING
BAD CREDITS EVALUATION
We are deeply passionate about data. Our data enables
various financial services, from credit risk analysis to marketing
solutions to managing non-performing loans and bad debt.
•1M companies sitesWeb Data
•4M info from open data setOpen Data
•70M payment
•60M scoring
Cerved Data
•70M Real EstateProperty
•20M companies
•16M shareholders
Company Data
2.600
Persons
34.000
Customers
40M€
In Data &
Technologies
30 M
Decisions
1.400 TB
Of Data
36. Why Cerved in Cloud?
43
TIME-TO-MARKET
Rapid implementation for
basic services.
Benefits of
Cloud for Cerved
PRIVACY & SEGREGATION
Manage customers data in secure
mode
AVAILABILITY
Services available 7x24
SCALABILITY
Infrastructure quickly
adaptable to the load
37. The Data Gravity
44
As data accumulates, it begins to have gravity. This Data Gravity pulls
services and applications closer to the data.
- Dave McCrory, 2010
DATA
Services Apps
Latency
Throughpu
t
This attraction (gravitational force) is caused by the need for
services and applications to have higher bandwidth and/or lower
latency access to the data.
38. 45
Data Ecosystem in Cerved
Sourcing
Liv.2
Sourcing
Liv. 1
REPOS
SYNTH
Mondo Dati Lince
Dati
clienti
NCA
ERG
EBS
HUB
DWH
MBD
Teradata
Tabula
Mongo4
DW
DB4You
XPCH 2
MATCHNEMO
Quaes
tio
LUDO
Tabula
(su
AWS)
Aracne
G4U
MBD1
R3
Pragma
Splunk CDR
Mambo
CAS
Dedalo
ELK
CSS
CR-RIBA
(Payline)
39. How to overcome Gravity?
How to lift to the cloud with Data Gravity?
40. Cerved Data in Cloud Architecture
Cerved
DBs
CDC
DBs Operational
Online Data
OLTP Processes
Batch
Hadoop DataLakeNoSql
Tabula
Cloud DB
DynamoDB RDS S3
Streaming is the new ETL
CDC
Producer
Raw Events
Aggregator
Basic Events
Aggregator
HL Events
Aggregator
NoSql
Ingestion
Synk Connector
For Cloud
Hadoop
Ingestion
Stream Processing
Streaming is the Anti-Gravity
41. Cerved API: a Data In Cloud use case
Cerved
DBs
CDC
DBs Operational
Online Data
OLTP Processes
Batch
Hadoop DataLakeNoSql
Tabula
Cloud DB
DynamoDB RDS S3
CDC
Producer
Raw Events
Aggregator
Basic Events
Aggregator
HL Events
Aggregator
NoSql
Ingestion
Sync
Connector to
CloudHadoop
Ingestion
Stream Processing
Back End
AWS Lambda Spring Boot API Gateway
Front End
ReactJs Redux Swagger
42. The Results of API in Cloud
49
SLA
API available 7x24x365
99,998% in January 2019
PERFORMANCE
High scalability with quickly
adaptability to load
COSTS
With AWS Lambda, DynamoDB,
S3, ecc… the cost of infrastructure
grows with the load
DATA SYNC
Data are continuously updated in
near real time mode
43. Future use case of Data in Cloud
Cerved
DBs
CDC
DBs Operational
Online Data
OLTP Processes
Batch
Hadoop DataLakeNoSql
Tabula
Cloud DB
DynamoDB RDS S3
CDC
Producer
Raw Events
Aggregator
Basic Events
Aggregator
HL Events
Aggregator
NoSql
Ingestion
Sync
Connector
To CloudHadoop
Ingestion
Stream Processing
Back End
AWS Lambda
API Gateway
EMR
SageMaker
AWS Kinesis or
Managed
Streaming for
Kafka
Data
Scientist in
Cloud
Real Time
Apps
DR &
Backup
Use Cases
44. THANK YOU
Moving Fast Data in cloud creates a new gravity for
new and innovative apps and services
Antonello Mantuano
Head of Software Engineering
antonellomantuano
@manant74