Deepak Goyal works at Walmart Labs in Bengaluru, India on the Customer Backbone team. He previously worked at Microsoft on knowledge graphs. He presented on using Verdict, an approximate query processing tool, to speed up data analytics and make it more resource efficient at Walmart. Verdict allows queries to be run on a sample of data rather than the entire dataset to get fast, good approximations. It can reduce query costs and speeds up data exploration without changing applications or SQL engines.

1. Approximate Query Processing
using Verdict
Deepak Goyal @WalmartLabs

2. About me
Interests
• Distributed Processing and Database platforms
Present
• Customer Backbone, Walmart Labs, Bengaluru
Past
• Knowledge Graph, Bing, Microsoft, Hyderabad
• B. Tech. in CSE, International Institute of Information Technology, Hyderabad

3. Data Analytics
• Processing and exploitation of cleansed data generates information.
• Data analysis converts information to intelligence.
• Exploring various forms of data.
• Searching for new data insights.
• Hence, making better business decisions.

4. Email marketing @Walmart
• Email marketing
Dedicated email campaigns for specific customers based on intelligence
gathered from data analytics.
• Customer Segmentation
Dividing a broad business market into smaller targeted audience segments for
email advertising.
• Segmentation base as a combination the following facets for example
Demographics age, gender, income, etc.
Geographic country, state, city, postal code, etc.
Psychographics lifestyle, social, personality, etc.

5. The Problem: Slow and Costly Data Analytics
Contributing Factors
• Limited cluster resources
• Long running analytic jobs
• Large number of short/medium length analytic jobs
• Large volumes of data
• Slow response times
implies Slow data analytics

6. The solution: Verdict
• A next generation approximate query processor
• Built upon the theories of approximate query processing (AQP)
• Based on the novel architecture of AQP-as-a-middleware
• Can reliably estimate many important statistics from a small fraction
of the entire data
• Exploits the state-of-the-art techniques from statistics
• Paves the way from slow exact results to fast resource-efficient good
approximates

7. AQP-as-a-middleware

8. Runs all on SQL-based engines

9. Approximate Query Processing using Verdict
• Reduces the query processing cost
• Fewer resources
• No changes to the application
• Compatible with all SQL engines
• Highly accurate
• Faster results
• Enable continuous data exploration

10. Notes and References
• Verdict is developed primarily by the database group at the University
of Michigan
• Links
• http://verdictdb.org/
• https://github.com/mozafari/verdict
• Discuss further
• Mailing List: verdict-user@umich.edu
• Walmart folks
• Deepak Goyal deepak.goyal@walmartlabs.com
• Giridhar Addepalli gaddepalli@walmartlabs.com
• Chirag Singla cchirag@walmartlabs.com

11. Thank you.
Q&A