Document showing case studies on Application of AccelData

1. Case studies for Application of Acceldata:
Acceldata Case study - TrueDigital:
Synopsis:
Employing Acceldata helped TrueDigital (part of TrueGroup an integrated
telecommunications and digital service provider, based in Thailand) to grow 400% (5x)
Data Cluster Growth with Data Observability
Client Company Overview:
 TrueDigital was founded in 1990.
 TrueDigital is one of Thailand’s largest communications companies
 TrueDigital has operations in Thailand, Indonesia, and the Philippines.
 TrueDigital develops solutions for digital media, data analytics, and IoT
Context:
 True Digital is one of the core businesses of True Group, an integrated
telecommunications and digital service provider, based in Thailand.
 The company is a leader in the development and integration of artificial
intelligence, big data, blockchain, cloud, Internet of Things (IoT), and robotics
solutions.
 As the company expanded its business operations across Southeast Asia, they
built a unique ecosystem of digital platforms and solutions to address the needs
of consumers, merchants, and enterprises.
 The data operations team at True Digital quickly discovered that they needed
visibility into their data pipelines so they could scale growth and ensure the
reliability of their data.
Business Problems TrueDigital was trying to solve:
True Digital runs a large environment of Hadoop clusters that power several aspects
of the business including:
• Improve Customer experience by measurement of Quality of Experience (QoE): This is
a measure of customer satisfaction. QoE can be service-specific, or a measure of overall
satisfaction across all services (video, voice, text).

2. • Abuse detection: SIM boxes can be used fraudulently and in violation of the fair use
policy of the SIM cards issued by the telecom operators
• Monetization and upselling opportunities: Accumulating customer behavior data
across the network enables the Analytics team to identify preferences that can be
turned into product recommendations and opportunities for increased monetization.
Challenge faced by TrueDigital:
 As the company grew and along with it so did its data volume which grew
exponentially, so did the operational challenges of managing this data and
maintaining optimal analytics performance.
 In response to the challenge of growing data, True Digital’s Analytics team
deployed hundreds of nodes/servers across multiple clusters to address storage
and processing requirements of the volume, velocity, and variety of data.
 The company was in the process of setting up a new cluster with larger nodes,
better specs, and more processing resources as more data was generated on
the network.
 As part of a major migration process, the team had to onboard applications
from the old cluster to the new one. Once the flow of execution was changed
to the new cluster, the team discovered that despite having more capacity,
processing slowed dramatically. As a result, new business processes could not
be supported.
 The Analytics team couldn’t identify issues with the hardware, resource manager
configuration, or Spark application code. Performance issues in complex,
interconnected data systems are difficult to isolate. Without multidimensional
data observability, True Digital couldn’t resolve its data flow issues or achieve
the pace and scale of data operations required to support analytics-based
business goals.
How Acceldata solved TrueDigital’s problems:
 The data flow issue could have been attributed to one or multiple layers of the
processing environment which would suggest a correlation between Yarn
Metrics, Spark Metrics, Infrastructure, User, and Concurrency. The first step,
therefore, was to figure out where the issues existed.
 After evaluating TrueDigital’s environment, the Acceldata team identified
specific points within data pipelines that were preventing a normal flow of data
into new clusters.
 To address the problem, Acceldata Pulse was implemented which immediately
enabled end-to-end observability into the True Digital Analytics environment,
along with additional data-related information in the Pulse dashboard that
enabled the True Digital team to improve data-related decision making.

3.  Pulse categorically showed the path to figuring out the right OS parameters,
the right Yarn parameters, the right OS parameters, fixed the calendar days on
which the jobs should be run, the time of day that streaming jobs should be
ideally executed.
 After these improvements to the configuration, code changes, and setup were
implemented, Pulse was then put to the task of continuous observation,
alerting, and remediation of issues across Hadoop clusters and their data
sources.
 Pulse delivered operational alerts which were domain-specific and related to
use cases specific to True Digital. These include cases where massive amounts
of streaming data operate at high velocity and require an alignment in capacity
in order to effectively process all of the data.
 Because these alerts are specific in what they address, and are unique to True
Digital’s needs, the Analytics team has dramatically reduced the alert noise that
used to deliver false positives and other erroneous, or useless, information.
Acceldata Pulse:
 Acceldata Pulse enables IT professionals and data engineers to gain greater
visibility across hybrid data environments by connecting data sources, defining
metrics, observing thresholds, and monitoring results from a customizable
dashboard.
Results:
Within the first year of using Acceldata Pulse, True Digital has been able to realize
these, among other, benefits:
 Data environment increased from 35 to 200+ nodes.
 Clusters processing more than 2x data.
 Improved stability has allowed the Analytics team to support multiple
 lines of business.
 Eliminated engineering involvement in daily operational issues freeing
resources to focus on expanding business use cases.
 Replaced expensive commercial solutions with open-source technologies
Key Take Aways:
 Using Acceldata Pulse has enabled True Digital to focus on key business use
cases while optimizing the underlying data infrastructure.
 The level of automation and cross-sectional visibility provided by Acceldata has
allowed True Digital to keep costs under control, move away from the predatory
pricing of other OEMs which provide remote Hadoop support, and enabled the
company to be self-reliant with their technology investments.

4.  As a result, True Digital has moved away from expensive commercial Hadoop
Support, is capable of managing and maintaining its cluster environment, and
has saved over $1 million USD.
Acceldata Case study -PhonePe:
Synopsis:
Acceldata helped PhonePe/Walmart to Scale their data infrastructure by 2000% and
saved $5 Million for PhonePe.
Overview of Problem:
PhonePe faced scaling and performance issues on the open-source Online Transaction
Platform (OLTP) and Online Analytical Platform (OLAP) they employed.
Overview of Solution brought by Acceldata:
Acceldata Pulse helped PhonePe monitor HBase, Spark, and Kafka to distinguish
between infrastructure issues and seasonal and campaign-based anomalies.
Introduction to PhonePe:
PhonePe is a Walmart subsidiary that provides more than 350 million consumers
across India with the ability to send and receive money, make payments at more than
ten million physical and online retail stores, use ATMs and invest in mutual funds and
other securities.
Challenge faced by PhonePe:
 PhonePe uses a variety of open-source data technologies, such as Apache
Hbase, HDFS, Kafka, Spark, and Spark Streaming, to run their high-volume, real-
time payments and cash transfer platform , handling around 400 million
transactions per month.

5.  With hundreds of millions of customers and millions of merchants on the
system, PhonePe’s Data Warehouse cluster must be highly performant, reliable
and transparent, which includes the ability to accurately report on system and
business performance to internal and external stakeholders 24/7.
PhonePe’s Scaling Problem:
 As PhonePe’s business grew explosively in 2018-19, the company embarked on
a massive data infrastructure expansion in terms of both scale and new
technologies. The company needed to increase the size of its Hadoop
infrastructure to support tens of millions of new consumers and millions of new
merchants who were rapidly adopting the service, all while adding Hive LLAP,
Spark 3.x and Druid to the platform, technologies that were needed to support
new products and business requirements.
 Even in the early stages of this infrastructure expansion, the technology team
experienced tremendous pressure on system performance and reliability. Key
engineers spent the majority of their time firefighting problems and searching
for causes behind data application issues and infrastructure failure instead of
focusing on increasing scale and new capabilities as required by the business.
 PhonePe’s Chief Reliability Officer, Burzin Engineer, quickly realized that his
team needed tools to improve visibility into every aspect of the company’s data
operations. Without more advanced tools that matched the sophistication of
his core open-source technologies, PhonePe’s critical data initiative would fail,
jeopardizing the company’s growth prospects and business success.
The Acceldata Solution:
 After gaining an understanding of PhonePe’s objectives and challenges with
Burzin Engineer and the PhonePe team, Acceldata demonstrated how its Pulse
data observability tool could provide real-time monitoring of Hbase, Hive, and
Spark data pipelines.
 Acceldata Began Delivering Value in 24 Hours the PhonePe team implemented
Pulse in less than a day and immediately began to identify problems with HBase
region servers and tables that were under pressure.
 Acceldata Pulse helped PhonePe distinguish between HBase cluster issues
caused by hardware or poorly designed tables and anomalies resulting from
seasonal and campaign-related surges.
 PhonePe had previously tried to use open-source and other commercially
available tools, like HBase Console and Ambari in addition to building single
metric Grafana dashboards, for root cause analysis but found that they were

6. insufficient. HBase Console, for example, only provided aggregated information
and required significant time and analysis from highly experienced data
engineers before it delivered useful intelligence.
 In contrast, Pulse directs users to the problem’s root cause quickly and clearly
through automated alerts and easy-to-read dashboards. In many cases, Pulse
even recommends fixes to solve the problem.
Key result points:
In the first 18 months of using Acceldata Pulse, PhonePe has been able to realize these,
among other, benefits:
 Scale data infrastructure rapidly from 70 to more than 1500 Hadoop nodes;
more than 2000% growth.
 Deliver 99.97% availability across its Hadoop infrastructure.
 Eliminate day-to-day engineering involvement and firefighting on outages and
performance degradation issues.
 Support multi-cluster data and workload management with uniform
configurations.
 Upgrade systems and migrate to new applications and nodes with no
performance degradation.
 Reduce data warehouse costs by 65%, while eliminating the need for expensive
commercial data warehousing licenses.