Near real-time data pipelines in the Telecommunication industry need to collect data from multiple source operational systems and transform them into an accessible unified data model. There is a dramatic increase in data volume, speed, and types of data and an increasing demand to minimize the lag of having the data available for Operation and Analytical data stores on the cloud. Amdocs' data pipeline process and transform billions of mobile subscriber events every day. Enabling this type of transformation requires a reliable DB engine that provides consistent low latency response time with high throughput to serve as a data cache. In this talk, Chief Architect at Amdocs Technology will provide a quick overview of the unique transformation process and the process for selecting our target architecture. He’ll review the considerations for the selection of their cache DB as well as the obstacles they had to overcome in the process. He will also share the benefits of the selection, and detail how they successfully integrated ScyllaDB into their Data Management platform. To watch all of the recordings hosted during Scylla Summit 2022 visit our website here: https://www.scylladb.com/summit.

1. Reinventing Data
Management on the Cloud for
Modern Telecom Providers
Hili Shtein
Chief Architect for Amdocs Data and AI Platform

2. Hili Shtein
■ Leading Architecture for Amdocs Data & AI Platform for the past
5 Years
■ Lead the architecture of Big Data and Event processing for the
last 12 Years
■ Over 30 years at global operating telecom solution providers
Chief Architect for Amdocs Data and AI Platform
YOUR COMPANY
LOGO HERE

3. Problem Statement
Build and maintain an ODS/EDW that can provide required diagnostic, predictive and actionable
(prescriptive) quicker and cheaper
■ Many sources of data
■ Hard to get the data from all sources and align them
■ Ensure quality – technical/Business
■ Ensure data freshness (e.g. Digital contextual scenarios, data for API layer)
■ Supplying data to different departments for different use-cases
■ Hard to organize the data to be effective with reports/insight - takes experience, time
■ How to scale with the growing amount of data and remain cost effective
■ There are changes in source all the time

4. Source
Replicator
Orchestrator
+
Transformer
Source
Cache
Source
Schema RT / Batch
Aggregations
ODS Ingestion
• ODS
• Smart EDW
Usage(TS)
DPI /
Probe
Invoicing
CRM
AR
Ordering
Billing
CM
SOM
USM
Optima
C1/D1
RTB
Other 3rd
party
apps
Modelled
Schema
Enrichment:
KPIs, AI Insights,
Profiles, Alerts
On-Pre
m Cloud
…
Files
RDBMS
Collector
(Oracle /
Postgres)
Kafka
Couchbase
Collector
CDC / Bulk
(Attunity /
Golden Gate)
Subs /
Clients
App API
Sources
Collectio
n
Transformatio
n
Targets
• Operational
• Analytical
• Business
High Level Data Flow
DB
Apps
Integration

5. Data & AI Platform –unique value: example
Designed for CSP
complex
Transformation rules
A “simple” sub plan change
becomes a storm over a
dozen
- Faster time to value
- Significantly lower costs
- Future-proof with Amdocs
systems

6. Complex transformation example
Transformation
Replicato
r
Transform
er
Data
Store
Collection
Billing
CRM
OMS
Sub
11
AGR
11
OA
22
aLDM
Subscriber
aLDM
Subscriber
ADDR
12
aLDM
Subscriber
aLDM
Subscriber

7. Real Time Transformation – complex real time model
■ Record completion for CDC sources -
No additional load on source applications
■ Filter un-changed records for large JSON documents
Helps reducing data volume and improve data freshness
■ Managed No-SQL storage to power transformations
Schema creation, version handling, data retention
■ Performs in-memory (SQLite) transformations
enables - Complex queries in real-time
Built-in capabilities to handle typical challenges with different sources
■ Minimal work for ETL engineer – focus on core logic
of transformation – no pipelines
Entity level metrices for easy understand on business
■ Kafka lag based auto-scaling to ensure data
freshness even during extreme peaks (end-of-week
biling cycles)

8. Realtime transformation
Collector
Cache
DB
Kafka
Collector
Collector
Replicator
Transform
Kafka
Replicator
Streams of Master Data are not
partitioned (to maintain order)
1 Partition per Topic (Data Source)
Kafka
Transformer
Collector
1. Retrieve changed entity
2. Update change and persist with column
history
3. Determine context
4. Publish to Kafka (with Context and
timestamp) – partitioned
Streams of Transient Data are
partitioned
Multiple partitions per Topic (Data
Source)
1. Load all context entities (effective to
timestamp) using indexes / search
2. Apply transformations
3. Publish to aLDM context to Kafka

9. Evaluation Creteria for Cache DB Selection
■ Tech Alignment
■ TCO (License / BOM / Bottom Line)
■ Non Functional / technical requirements
■ Refactoring effort
■ Install base
■ Scale
■ Vendor locking
■ Querying tools
■ Availability of K8S operator
■ Ability to get managed cloud service
■ Satisfactory Proof of Concept execution results

10. Proof of Concepts Finalists
■ Cassandra – Already in use in our stack
■ Leading in-memory data structure store
■ Advanced SQL enables Object based store
■ ScyllaDB

11. Evaluation matrix for Cache DB Selection
Criteria Cassandra Cont #1 Cont #2 ScyllaDB
Tech Alignment ✔✔ 🗶 🗶 ✔
TCO (License / BOM / Bottom Line) (#1 / #2 ) #4/#3 #3/#3 #2/#1
Non Functional / technical requirements
#4 #2 #1 #3
Refactoring effort
#2 #1 #1 #2
Install base XL L M S
Scale ✔- ✔ ✔ ✔✔
Vendor locking #2 #1 #1 #2
Querying tools ✔ ✔✔ ✔✔✔ ✔
K8S operator ✔(unofficial) ✔ ✔ ✔(Beta)
Ability to get managed cloud service ✔ ✔ ✔ ✔(lim)
Passed POC execution 🗶 🗶 🗶 ✔

12. Thank you!
Stay in touch
Hili Shtein
Hili.Shtein@Amdocs.com