Download to read offline
Inside Big Data Inteview With Ashwin - Build and Operate Data Products SMALL.pptx
- 1. Build and Operate GreatData Products Using Data Observability Rohit Rajeeva CTO and Co-founder, Acceldata
- 2. © 2023 —Confidential & Proprietary Acceldata Overview 2 Founded 2018 Campbell (HQ), Bangalore, Singapore. 170+ Employees $95+ MM RAISED Insight, Lightspeed, March Capital, Industry Ventures, Sorenson, Sanabil & Emergent Ventures 3X GROWTH Observing 300+ PB data. Highest CSAT Ratings Key Customers Data Observability Platform Leadership Team Build and manage data products at scale by ensuring reliability, eliminating operational blind spots & reducing spend to achieve high ROI on your data investments.
- 3. © 2023 —Confidential & Proprietary Common Data Challenges 3 Data Sprawl Tech Sprawl Talent Shortage
- 4. © 2023 —Confidential & Proprietary Operational Blind Spots Continue Unabated 4 ML & AI DATA APPLICATIONS DATA SOURCES DATA PIPELINE ORCHESTRATION DASHBOARDS RAW LANDING ZONE CONSUMPTION ZONE ENRICHED ZONE STREAMING APPS RDBMS FILE / OBJECT ON-PREM DATA LAKE … Data Architecture Schema Drift Thruput & Latency Perf. Trending & RCA Data Reconciliation Data Quality Data Drift Pipeline Health Spend
- 5. © 2023 —Confidential & Proprietary Enterprise Data Observability for Your Data Stack 5 Users Pipelines Compute Reliability Enterprise Data Observability Optimize capacity, data processing, cost optimization, FinOps governance. Improve data quality, reconciliation, determine schema drift, data drift. Identify issues with transformation, events, applications, alert and provide insights. Real-time insights for data engineers, scientists, administrators and platform engineers.
- 6. © 2023 —Confidential & Proprietary Data Observability: Cost-Value Optimization 6 ML & AI DATA APPLICATIONS DATA SOURCES DATA PIPELINE ORCHESTRATION DASHBOARDS RAW LANDING ZONE CONSUMPTION ZONE ENRICHED ZONE STREAMING APPS RDBMS FILE / OBJECT ON-PREM DATA LAKE … Recency & Redundancy $ Data Quality Automation $ Over-provisioning $ Utilization & Cost $ Design trade-offs $ Data Architecture
- 7. © 2023 —Confidential & Proprietary Acceldata Data Observability Platform Reliable Scale Optimise Data Users (Engineers) Gain end-to-end visibility Improve throughput Align costs & benefits Increase data trust/reliability Increase data consumption Automate data validation Prevent outages & achieve SLAs Scale data processing Optimize resources & costs Data Executives Data Engineers Admin/ Platform Engineer Multi-layered Data Observability Hybrid Cloud Data Systems On-Premises Integrations Cloud Integrations Users
Editor's Notes
- #2 First of all, thank you for the opportunity to share and discuss a new category “Data Observability and how Acceldata’s “Enterprise Data Observability” helps solve complex data challenges and pain points. I'm GeeBee from Acceldata. Our vision at Acceldata is to help enterprise data teams in particular to build and operate great data products. I will spend few minutes walking you through it.
- #3 Acceldata is a 4 year startup that is the market leader in a new category called data observability. We were founded in US and India, and we have over 170 employees now. We have teams in US, Europe, Asia, representing all major functions. We have been fortunate to have raised capital from some of the leading investment and venture capital firms. What makes our investors unique is that they're not only professional investors that pick the top startups and technologies to invest and all of them have operating experience and many of them have extensive data and analytics experience. We have had a great start. Our first customer was onboarded in less than six months after the company got started. Our customers are among the who’s who in the Enterprise space representing financial institutions, analytics companies, mobile operators, healthcare, fintech and technology companies worldwide. Here is a short list of customers who agreed to go public with us and that list of customers is much longer than this. We have customers such as Oracle, Dun and Bradstreet, PubMatic, PhonePe which is now part of Walmart as some of our large enterprise customers. We offer to our customers an Enterprise data observability platform to build and manage data products at scale by ensuring reliability, eliminating their blind spots and reducing their spend to achieve high ROI on their data investments. Unlike many companies in this space, or in the data space, we have a hardcore data DNA. Our founders come from leading data and analytics companies. We've gone through the pain points we've experienced the pain points before we built our platform. Our data background and success with data products is more comprehensive than any company in the data industry. Our team, across all functions – engineering, sales, marketing, customer success has spent many years in Data Management, data analytics, machine learning AI, cloud solving some of the more complex problems that ever existed.
- #4 Data Sprawl Modernize or perish Lots of bad data Unable to realize business outcomes Tech Sprawl Too many tools Complexity Cost overrun Talent Shortage Scarce data & engineering experience Right experience Limited industry know-how These are some of the common data challenges that enterprises experience. We are categorized into three broad buckets here. One is what we call in data sprawl. There's just way too much data. Sometimes people refer to this as a data deluge problem, which is massive amounts of data. How do you make sense? How do you analyze this? How do you use data? Sets strategically? How do you package your data as a product or provide analytics on top of your data? What's good data? What's bad data? The other challenge why many enterprises fail is there's just way too many tools. There are too many IT and security tools and common for enterprises to have 35 plus data related tools One of the critical challenges is talent shortage. It's very common for an enterprise to have hundreds of openings related to different data roles. In fact, one of the companies we're looking at there about 1600 data engineer positions, good luck finding them, and good luck finding the right people. Good luck training them. Good luck, bring them up to speed with the complexity of tools today. The industry needs a better way.
- #5 If you're a data engineer or an architect or data practitioner, you may be thinking, what does this mean for me? Our customers tell us that their operational blind spots continue to increase, even though there are a lots of tools. This diagram is a generic concept of data ecosystem or a data product. You have data sources on one side and then you process them, enrich them, put them in some sort of a data warehouse or equivalent in cloud or hybrid, and then you, orchestrate it across so that it's ready for consumption. Click. This is, of course, the ideal scenario, but the reality is that data teams experience numerous problems on a daily basis. These are few of the challenges that we hear as pain points we hear from our install base. How do you address that? There needs to be a better way - enter data observability.
- #6 Acceldata offers enterprise data observability for your data stack. Enterprise Data observability has four layers., the compute layer, reliability, pipelines and users. Compute and infrastructure layer of observability not only helps you optimize your capacity and data process the data but also helps you optimize cost providing what's in the industry is called financial operations governance. We call it spend intelligence that is providing Insight and Analytics of who's Where are you spending your dollars? How is that being analyzed? Who's using that either malt, malt performing queries? How do you go about doing that? That's what we provide insight to. We have a different approach to data reliability. Many tool vendors and some of the data observability solution providers talk about data quality. We think that focusing just on data quality, is not sufficient. You need to look at things like data reconciliation, can you determine the schema drift Can you solve the data drift problem? Reliability is important. Just focusing on quality is not sufficient. Enterprises have many pipelines. Think of them as workflows. Can your observability solution identify issues and events and alert on that, but can it help you identify where the problems are and help with the remediation? An often-overlooked aspect of observability is providing the real time insights across your data teams. It's not just about solving a data engineers problem. What about the platform engineers? What about the architect? What about the person who's responsible for the budget and spending hundreds of 100,000s of dollars or euros or other? What insights does the chief data officer need?
- #7 How does Acceldata data observability help you? We offer enterprise data observability for your data stack, no matter whether you use a cloud native platform or if you are using hybrid cloud. We support a combination of on prem or cloud native solutions, or platforms, as well as self serve cloud models. It solves the pain points that you and other enterprises face. We solve the pain points directly - out of the box, it's automated. We provide you the ability to detect, investigate, remediate, and we provide deep insights, provide best practices.