Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Approximate Query Processing

AQP-as-a-middleware

  • Login to see the comments

  • Be the first to like this

Approximate Query Processing

  1. 1. Approximate Query Processing using Verdict Deepak Goyal @WalmartLabs
  2. 2. About me Interests • Distributed Processing and Database platforms Present • Customer Backbone, Walmart Labs, Bengaluru Past • Knowledge Graph, Bing, Microsoft, Hyderabad • B. Tech. in CSE, International Institute of Information Technology, Hyderabad
  3. 3. Data Analytics • Processing and exploitation of cleansed data generates information. • Data analysis converts information to intelligence. • Exploring various forms of data. • Searching for new data insights. • Hence, making better business decisions.
  4. 4. Email marketing @Walmart • Email marketing Dedicated email campaigns for specific customers based on intelligence gathered from data analytics. • Customer Segmentation Dividing a broad business market into smaller targeted audience segments for email advertising. • Segmentation base as a combination the following facets for example Demographics age, gender, income, etc. Geographic country, state, city, postal code, etc. Psychographics lifestyle, social, personality, etc.
  5. 5. The Problem: Slow and Costly Data Analytics Contributing Factors • Limited cluster resources • Long running analytic jobs • Large number of short/medium length analytic jobs • Large volumes of data • Slow response times implies Slow data analytics
  6. 6. The solution: Verdict • A next generation approximate query processor • Built upon the theories of approximate query processing (AQP) • Based on the novel architecture of AQP-as-a-middleware • Can reliably estimate many important statistics from a small fraction of the entire data • Exploits the state-of-the-art techniques from statistics • Paves the way from slow exact results to fast resource-efficient good approximates
  7. 7. AQP-as-a-middleware
  8. 8. Runs all on SQL-based engines
  9. 9. Approximate Query Processing using Verdict • Reduces the query processing cost • Fewer resources • No changes to the application • Compatible with all SQL engines • Highly accurate • Faster results • Enable continuous data exploration
  10. 10. Notes and References • Verdict is developed primarily by the database group at the University of Michigan • Links • http://verdictdb.org/ • https://github.com/mozafari/verdict • Discuss further • Mailing List: verdict-user@umich.edu • Walmart folks • Deepak Goyal deepak.goyal@walmartlabs.com • Giridhar Addepalli gaddepalli@walmartlabs.com • Chirag Singla cchirag@walmartlabs.com
  11. 11. Thank you. Q&A

×