Big Data Decision-Making


Published on

Gayatri Patel, eBay, presents at the Big Analytics 2012 Roadshow
The wonders of what data can do for an organization is measured in the productivity and competitiveness of their team's decisions. Some believe more data is the key. Agreed...but good decisions require more than just deriving intelligence from big data. In this dynamic market, the need to socialize and evolve ideas with other teams, quickly correlate information across sources, and test ideas to fail fast early are strong enablers to gain competitive footing. eBay¹s analytic and technology advancements garners insights and approaches that continue to help our employees tell their "data stories" and make better decisions.

  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Big Data Decision-Making

  1. 1. Extreme Decision- Making at eBay Gayatri Patel
  2. 2. We All Make Decisions…
  3. 3. Our Decision Makers “I love to connect the dots to “I love to find unique find out anomalies that have not what is been discovered yet!” driving our growth and why!” “I love to monitor the health of our global operations with confidence and tell great stories to our BoD”
  4. 4. Is Customer Brandimproving with last Businessmonths’ new Motors Marketingcampaign from our Manager Motors Mobile Application? What is the latest revenue projection for our coupon campaigns targeted to enhance Fashion Search for large retailer listings?
  5. 5. 5
  6. 6. Source: McKinsey Global Institute“Big data: The next frontier for innovation, competition, and productivity”
  7. 7. What Makes it Happen?Lots & Lots & Lots of Dataat eBay
  8. 8. BIG Data VVC >50 TB/day new data >100k data elements >100 Trillion pairs of information>100 PB/day Processed >50k chains of logic >7500 business users & analysts Active/Active turning over a TB every second 24x7 x365 Millions of queries/day Always online 99.98+% Availability Near-Real-time 9
  9. 9. Who Makes it Happen withExtreme Analytics at eBay?
  10. 10. Researcher Analyst “I love to $ “I love to find unique $$ connect anomalies that have not the dots been discovered yet!” to find out whatPM & BM is driving our $$$ “I love to monitor growth the health and why!” of our global operations with confidence and tell Management & great Executives stories to $$$$ our BoD” Decision Makers…
  11. 11. Capacity Facilities Customer Procurement Planning Support Fraud Finance SearchOperations Strategy Marketing Corporate Legal Shipping Supply Chain HR IT Independent Data Marts = Silo Intelligence…
  12. 12. 500+ 150+ 5-10 concurrent users concurrent users concurrent users Analyze & Report Discover & Explore Structured Semi-structured Unstructured SQL SQL++ JAVA / CProduction Data Warehousing Contextual-Complex Analytics Structure the Unstructured Large Concurrent User-base Deep, Seasonal, Consumable Data Sets Detect Patterns Enterprise-class System Low End Enterprise-class System Commodity Hardware System EDW Singularity Hadoop 6+PB 40+PB 20+PB Data Storage & Processing Platforms
  13. 13. There’s No Technology Silver Bullet
  14. 14. What is Why is it How to take happening?* happening?* action?Harmony Symphony Melody Informative Enlightened Actionable Opportunistic Measurements Reasonings Recommendations Decisions • Iterate…Test-What If…Collaborate-Share • Trust • Timely • Quality • Context Key Questions to Make Informed Decisions
  15. 15. Informative Opportunistic Measurements Decisions Harmony-Class Insights Harmony Vertical (Customer/Inventory) Harmony Health – Finance & Marketing 50,000 ft• Cross-Function Metrics• Business Performance Summaries• Personalize views: • Dimensions • Attributes• Deep Dive to Summarized Explore Cube Data Harmony Search & Harmony Trading & Trust Behavior Quick…Easy…”Personalizable”…”Business-blessed”
  16. 16. Informative Enlightened Opportunistic Measurements Reasonings Decisions Harmony Trading Harmony Verticals Harmony-class Insights 50,000 ft Symphony-class Symphony Shipping Symphony Fashion Insights Symphony Enthusiast 10,000 ft Symphony Transaction• Product Performance• Drill Downs• What-If/Ad-Hoc Explore Cube Explore Cube Quick…Easy…”Personalizable”…”Product-blessed”
  17. 17. Data Customer Inventory Trading Search Enabling Services… … • Dashboards • Summary Tables • Reports • Aggregate Tables • Views • In-Memory Caches • Metrics • Data Feeds • Measurements The DataHub Portal • Data Files • Base Tables Front End Back End Analytic Assets Analytic Assets ODW EDW Singularity Hadoop Analytic Assets
  18. 18. Health Product Customer Inventory Behavior Search Trading Trust Product Product Product Product Product ProductHarmonyFront End (4 only)SymphonyFront End (1-3) Common Dimensional ModelSymphonyBack-End Insights Products
  19. 19. Q1 Q2 Q3 Q4 Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec Health Harmony Harmony Health v3.2 Harmony Health v3.4 Harmony Health v3.6 Insights Health v3.0 3.0 4.0 Symphony v3.3 Symphony v3.5 Symphony v3.7 Symphony v3.9 Trading Symphony Symphony v0.11 Symphony v1.3 Symphony v1.5 Symphony v1.13 Symphony v1.17 Insights 1.7 Symphony Symphony Symphony v1.15 Harmony 1.9 v1.11 Trading 1.0 Harmony Trading v1.2 Harmony Trading v1.4 Harmony Trading v1.6 2.0 v1.0Trust Insights Symphony v0.9 Symphony v1.3 Symphony v1.5 Symphony v1.7 Symphony v1.9 … • Extends value over time • Adapt & make available quickly • Continuous quality Product Roadmap
  20. 20. Extreme Analytics at eBayAdvanced Analytic Infrastructures
  21. 21. Designing for the Unknown>85% of eBay analytical workload is NEW & UnknownThe metrics you know are cheapThe metrics you don’t know are expensive – but high in potential ROIExploration & Testing are core pillars of an analytics- driven organization 2222
  22. 22. Hypothesis Ideas Assumptions Algorithm Pattern Guesses Simulation BehavioralMore Better BetterTests Tests Decisions Ownership & Confidence Attribution Machine Advanced Learning Experimentation: Build From Idea Up 23
  23. 23. Platform Metrics For Sample Queries
  24. 24. Business Centric Technology CentricExecutives & Management Executives & Management Executives & Management Analysts Executives & Management PM & BU Managers * PM & BU Managers PM & BU Managers Scientists & Researchers * PM & BU Managers Analysts Analysts * Analysts Engineers * Scientists & Researchers Scientists & Researchers Engineers * Engineers Engineers Desktop Self- Rapid-fire BI & Statistical Purpose Built Enterprise BI service Exploration Modeling Applications* Primary authoring group BI Tools & Platforms
  25. 25. Self Service Analytics
  26. 26. 30
  27. 27. Thank You Gayatri