Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Extreme Analytics @ eBay
Evolution of GovernedSelf Service Analytics
Agenda
•eBay Today
•Big Data @ eBay
•The HOW?
•Q&A
PRESENTATION TITLE GOES HERE 2
OUR
BUSINESS
Most Powerful
Selling Platform
For business
sellers: the
potential to drive
profitable sales
and build a brand
For consume...
EBAY INC AT A GLANCE
$2.1B
Revenue in Q1 2016
$20.5B
GMV in Q1 2016
162M
Global Active Buyers
57%
International
revenue
Q1...
EBAY MARKETPLACE AT A GLANCE
$19.6B
GMV in Q1 2016
9.5M
New listings added via
mobile per week
300M
Searches each day
63%
...
VELOCITY STATS
US
3 car parts or accessories are sold every
A smartphone is sold every
A dress is sold every
1 sec
4 sec
6...
MOBILE VELOCITY STATS
US
A woman’s handbag is sold every
A car or truck is sold every
An action figure is sold every
10 se...
THREE KEY TRENDS ARE
REDEFINING COMMERCE
Smart CommerceSeamless Commerce True Global Commerce
SEAMLESS
COMMERCE
TRUE GLOBAL
COMMERCE of eBay’s business
is international57%
of commercial
sellers engage in
exporting*
95%
languages
8
*Se...
SMART
COMMERCE
Identify an
interesting set of
candidate items,
trends, events, etc.
Personalize
the results
Inspiration
at...
PRESENTATION TITLE GOES HERE 13
Volume
Variety
Velocity
VALUE
PRESENTATION TITLE GOES HERE 17
PRESENTATION TITLE GOES HERE 18
Big Data @ eBay
BIG Data VVC
20
>50 TB/day new data
>100 PB/day
>100 Trillion pairs of information
Millionsof queries/day
>7500
business u...
21
TECHNOLOGY
EDW
Analytics
Application
Analysts and Data
Scientists
Management Integrators Business
Owners
Application
Se...
22
eBay has one of the largest most active
data platforms in the world.
eBay has one of the largest most active data
platf...
Product Families
23
CURATED
DATA
PLATFORMS
SQL/RM/SAS/R
TXN
IN
V
BEH
CUS
DATA
EXPLORATION
&
COLLABORATI
ON:
ALATION
ADVANC...
24
FIVE STAGES
24
4
3
2
1
BEHAVIORALDATA
PLATFORM
ANALYTICAL
APPLICATION
PLATFORM
COLLABORATIVE
IDEATION
PLATFORM
AGILEDAT...
25
Agile Data Warehousing
EDWs + VDMs1
Semi-Structured SQL++Structured SQL
Low End Enterprise-class System
Contextual-complex analytics, deep,
seasonal, consumab...
page
27
Biggest complexity drivers are
 Maintaining separate databases
 weekly/daily/hourly data transfers
 Data incons...
PRESENTATION TITLE GOES HERE 28
...the wrong way
Data Marts in the Cloud
Customer
Customer
Customer
Customer
Customer
Prod...
PRESENTATION TITLE GOES HERE 29
Virtual Data Marts
Customer
Product Transactio
n
Behavior
Virtual
DataMart Virtual
DataMar...
30
Deep Data Platforms
Hadoop + ddDBMS2
Semi-Structured SQL++Structured SQL
Low End Enterprise-class System
Contextual-complex analytics, deep,
seasonal, consumab...
32
The Data Hub
Collaborative Analytics3
33
Collaborative Analytics
Compose
Write and discover queries with ease; understand and reuse
code
easily; drives time and...
Wiki + metadata repository
Alation SQL Assistant
Metadata repository
+ +
Storytelling
Mixing textual analysis with graphs
...
35
The App Platform
Analytics Application Platform4
ENTERPRISE DATA PLATFORM
36
Data Warehouse Data Streams
Batch
Humans
Sets of data
Streams
Systems
Sets of data
Data Servic...
DQRecon
Data Processing Ecosystem
37
Curated
Streams
Applications
Data Services
ApplicationAnalytics
Data
Scientists
Analy...
38
ENTERPRISE
DATA STREAMS AND SERVICES
39
Automated Decision Support
Signal Detection @ Scale5
Automated Signal Detection
40
Prediction – anomaly signal detection
Massively scalable and automated signal
detection and ...
41
ANALYTICS IN EBAY
Measure Everything
Embedded in our daily life
Bottom-up & Top-down
Think and Live Analytics
Always
Bu...
page
43
IKEA Job Interview
Please have a seat
page
44
Analytics at eBay
Go use data
45
The Diverse User Community
page
46
Diverse User Community
Data
Scientists
Financial
Planning &
Analytics
Site Analysts
Business
Analysts
Consumers
On...
47
The Analytics Environment at eBay
 Direct SQL access
 User datasets
 MicroStrategy
 Tableau
 Web based App
 1000+...
48
“We can’t solve problems
by using the same kind of
thinking we used when
we created them.”
• - Albert Einstein
51
Organizing for Success
Governed Self Service0
Self-service Strategy changes
everything…
52
The data user experience is….
Incoherent
Isolated
Disjointed
Uncertain
Consol...
Comprehensive & Documented -- Self-directed Experience
Insights Hub
ONE portal , ONE framework, ONE analytics app Store
Ta...
DATA GOVERNANCE
54
Business Glossary – Managed articles about logic and language.
Knowledge: What should it be?
Data Asset...
DATA GOVERNANCE
55
Business GlossaryData Asset Certification Well Managed
What we do: Data knowledge management and data
s...
56
Value
Generation
Governed Self Service0
57
Organizing for Success
Purified Data Science6
Data Prep
Data Science
58
59
Data
ScienceData
+
60
A COMPLETE VIEW OF OUR CUSTOMERS
Behavior Demographics & Interests
AttitudeValue to eBay
61
 DATA SCIENCE
Data Data
Science
Business
ImpactData Data Science Data Science Data Science
Business
Impact
Insights
Cu...
62
CONVERSION MODEL
User Category Probability
111602**** 1564** 10.1%
111602**** 1562** 6.54%
111602**** 1569** 5.67%
1116...
Thanks!
• ALEX LIANG
• hliang@ebay.com
• http://www.linkedin.com/in/alexlianghu
Extreme Analytics @ eBay
Extreme Analytics @ eBay
Upcoming SlideShare
Loading in …5
×

Extreme Analytics @ eBay

1,728 views

Published on

Extreme Analytics @ eBay

Published in: Technology
  • Be the first to comment

Extreme Analytics @ eBay

  1. 1. Extreme Analytics @ eBay Evolution of GovernedSelf Service Analytics
  2. 2. Agenda •eBay Today •Big Data @ eBay •The HOW? •Q&A PRESENTATION TITLE GOES HERE 2
  3. 3. OUR BUSINESS
  4. 4. Most Powerful Selling Platform For business sellers: the potential to drive profitable sales and build a brand For consumer sellers: an easy way to declutter, sell and make money A partnership not a competition Best Choice Providing the greatest selection of inventory for our buyers From new, everyday items to rare and unique goods And incredible deals only found on eBay Most Relevance A shopping experience that is simple, data-driven and personalized Enabling buyers to easily find, compare and purchase items they need and want Highlighting the unique value that eBay brings OUR STRATEGY
  5. 5. EBAY INC AT A GLANCE $2.1B Revenue in Q1 2016 $20.5B GMV in Q1 2016 162M Global Active Buyers 57% International revenue Q1 2016 data $9B Mobile Volume 314M App downloads
  6. 6. EBAY MARKETPLACE AT A GLANCE $19.6B GMV in Q1 2016 9.5M New listings added via mobile per week 300M Searches each day 63% Transactions that ship for free (in US, UK, DE) 79% Items sold as new Q1 2016 data ~900M Live listings One of the world’s largest and most vibrant marketplaces
  7. 7. VELOCITY STATS US 3 car parts or accessories are sold every A smartphone is sold every A dress is sold every 1 sec 4 sec 6 sec UK A necklace is sold every A make-up product is sold every A Lego product is sold every 10 sec 3 sec 19 sec GERMANY A truck or car is sold every A pair of women’s jeans is sold every A video game is sold every 5 min 4 sec 11 sec AUSTRALIA A pair of men’s sunglasses is sold every A home décor item is sold every A car or truck part is sold every 1 min 12 sec 4 sec
  8. 8. MOBILE VELOCITY STATS US A woman’s handbag is sold every A car or truck is sold every An action figure is sold every 10 sec 5 min 10 sec UK A tablet is sold every A cookware item is sold every A car is sold every 1 min 6 sec 2 min GERMANY A pair of women’s shoes is sold every A watch is sold every A tire or car part is sold every 20 sec 48 sec 35 sec AUSTRALIA A piece of jewelry is sold every A baby clothing item is sold every A motorcycle part is sold every 12 sec 46 sec 51 sec
  9. 9. THREE KEY TRENDS ARE REDEFINING COMMERCE Smart CommerceSeamless Commerce True Global Commerce
  10. 10. SEAMLESS COMMERCE
  11. 11. TRUE GLOBAL COMMERCE of eBay’s business is international57% of commercial sellers engage in exporting* 95% languages 8 *Sellers with $10,000 or more/year in sales
  12. 12. SMART COMMERCE Identify an interesting set of candidate items, trends, events, etc. Personalize the results Inspiration at scale!
  13. 13. PRESENTATION TITLE GOES HERE 13
  14. 14. Volume Variety Velocity VALUE
  15. 15. PRESENTATION TITLE GOES HERE 17
  16. 16. PRESENTATION TITLE GOES HERE 18
  17. 17. Big Data @ eBay
  18. 18. BIG Data VVC 20 >50 TB/day new data >100 PB/day >100 Trillion pairs of information Millionsof queries/day >7500 business users & analysts >50k chains of logic 24x7x365 99.98+%Availability turning over a TB every second Active/Active Near-Real-time >100kdata elements Always online Processed >1.5 x 1012 new records/day
  19. 19. 21 TECHNOLOGY EDW Analytics Application Analysts and Data Scientists Management Integrators Business Owners Application Servers Data Processing Clusters Aggregation & Summarization Visualization & Reporting ClicktoInsights Kylin OVER Billings EVENTS FROM 162M EBAY BUYERS CAPTURED, TRANSFORMED, SYNTHESIZED TO PROVIDE ACTIONABLE INSIGHTS
  20. 20. 22 eBay has one of the largest most active data platforms in the world. eBay has one of the largest most active data platforms in the world with a diverse set of users.
  21. 21. Product Families 23 CURATED DATA PLATFORMS SQL/RM/SAS/R TXN IN V BEH CUS DATA EXPLORATION & COLLABORATI ON: ALATION ADVANCED DATA TOOLS on iHub: NOUS,GRO,DNA ,M1, SPA, SD, CC, Adv … ENTERPRISE DATA SERVICE/Feed: RETAIL SIGNAL,IDENTI TY, SS, SH… REALTIME DATA STREAM: TORA
  22. 22. 24 FIVE STAGES 24 4 3 2 1 BEHAVIORALDATA PLATFORM ANALYTICAL APPLICATION PLATFORM COLLABORATIVE IDEATION PLATFORM AGILEDATA WAREHOUSE AUTONOMOUS DECISIONING PLATFORM 5
  23. 23. 25 Agile Data Warehousing EDWs + VDMs1
  24. 24. Semi-Structured SQL++Structured SQL Low End Enterprise-class System Contextual-complex analytics, deep, seasonal, consumable datasets Production data warehousing, large concurrent user base Discover & Explore Analyze & Report Enterprise-class System Unstructured JAVA / C Structure the unstructured, detect patterns Commodity Hardware System Singularity HadoopTeradata Enterprise Data Warehouse DISCOVER & EXPLOREANALYZE & REPORT 26
  25. 25. page 27 Biggest complexity drivers are  Maintaining separate databases  weekly/daily/hourly data transfers  Data inconsistencies  Data duplication  Increased complexity  Loss of centralized viz & control DMs A data mart cannot be cheap enough to justify its existence
  26. 26. PRESENTATION TITLE GOES HERE 28 ...the wrong way Data Marts in the Cloud Customer Customer Customer Customer Customer Product Customer Product Customer Product Customer Product Trx Customer Product Trx Customer Product Trx Customer Product Trx
  27. 27. PRESENTATION TITLE GOES HERE 29 Virtual Data Marts Customer Product Transactio n Behavior Virtual DataMart Virtual DataMart Virtual DataMart Virtual DataMart Virtual DataMart Virtual DataMartVirtual DataMart Virtual DataMart Virtual DataMart
  28. 28. 30 Deep Data Platforms Hadoop + ddDBMS2
  29. 29. Semi-Structured SQL++Structured SQL Low End Enterprise-class System Contextual-complex analytics, deep, seasonal, consumable datasets Production data warehousing, large concurrent user base Discover & Explore Analyze & Report Enterprise-class System Unstructured JAVA / C Structure the unstructured, detect patterns Commodity Hardware System Singularity HadoopTeradata Deep-Data Platforms DISCOVER & EXPLOREANALYZE & REPORT 31 Behavioral Data Centric
  30. 30. 32 The Data Hub Collaborative Analytics3
  31. 31. 33 Collaborative Analytics Compose Write and discover queries with ease; understand and reuse code easily; drives time and savings. Catalog & Govern Document and discover data and concepts; structured and crowd sourced tagging of content in a stewarded environment. Answers Fast, trusted answers for everyone; search for analytic products (metrics, reports, KPIs). Forensics Insightful IT and operational data to expose and eliminateredundancies Experts / Stewards Govern Simple Data Management Analyst Compose Better,Faster Queries Business Users Answers Google for your Data IT Forensics Intelligence about your data
  32. 32. Wiki + metadata repository Alation SQL Assistant Metadata repository + + Storytelling Mixing textual analysis with graphs WHAT IT’S LIKECOLLABORATION TOOL 2013 2009 2014 AnswerHub Discussion forum moderated by support DataHub + for data2010 COLLABORATION JOURNEY 2014
  33. 33. 35 The App Platform Analytics Application Platform4
  34. 34. ENTERPRISE DATA PLATFORM 36 Data Warehouse Data Streams Batch Humans Sets of data Streams Systems Sets of data Data Services Services Applications Specific calls Populated Used by How Enterprise Populated Used by How
  35. 35. DQRecon Data Processing Ecosystem 37 Curated Streams Applications Data Services ApplicationAnalytics Data Scientists Analysts BU/PD Leaders Site DBs Real-Time Data Sources External Data Sources ETL Enterprise Data Warehouse Deep Data Analytics Platform Hadoop Engineers Stream Processing Caching DOE DQFirewall Buyers/Sellers
  36. 36. 38 ENTERPRISE DATA STREAMS AND SERVICES
  37. 37. 39 Automated Decision Support Signal Detection @ Scale5
  38. 38. Automated Signal Detection 40 Prediction – anomaly signal detection Massively scalable and automated signal detection and prediction  Phase 1: Signal detection  Phase 2: Root Cause analysis
  39. 39. 41 ANALYTICS IN EBAY Measure Everything Embedded in our daily life Bottom-up & Top-down Think and Live Analytics Always But know when to avoid Analysis Paralysis! Analytics DNA
  40. 40. page 43 IKEA Job Interview Please have a seat
  41. 41. page 44 Analytics at eBay Go use data
  42. 42. 45 The Diverse User Community
  43. 43. page 46 Diverse User Community Data Scientists Financial Planning & Analytics Site Analysts Business Analysts Consumers One-off Analysis Descriptive, Predictive & Prescriptive Modeling Experimentation & Mining Standard Reports Dashboards Hadoop R/SAS/SQL on Teradata Excel Tableau MicroStrategy , Diverse Needs& Diverse Tools
  44. 44. 47 The Analytics Environment at eBay  Direct SQL access  User datasets  MicroStrategy  Tableau  Web based App  1000+ files  10,000+ tables  5000+ reports  10,000+  100+ named apps  Tough to find the right metrics and reports  Hard to build new metrics and reports  Impossible to know which metrics and reports are correct vs old
  45. 45. 48
  46. 46. “We can’t solve problems by using the same kind of thinking we used when we created them.” • - Albert Einstein
  47. 47. 51 Organizing for Success Governed Self Service0
  48. 48. Self-service Strategy changes everything… 52 The data user experience is…. Incoherent Isolated Disjointed Uncertain Consolidates all knowledge about data for “Just-in- Time” use Unifies a consistent set of Data Products on the hub Makes it easy to find and trace the path from Business Insights and summaries to the underlying SQL, metrics and metadata Delivers transparency and build trust with Data Governance and Stewardship
  49. 49. Comprehensive & Documented -- Self-directed Experience Insights Hub ONE portal , ONE framework, ONE analytics app Store Targeted & Simplified -- Self-service Experiences SQL Writer Search Collaberation Knowledge Management Subject Matter Expert (SME) Directory and Subject Domain pages Business Metrics Glossary Certified data assets, endorsements, descriptions. MoreDetailedMoreSummary TechnicalAnalysisBusinessInsight Self Service Strategy, Governed Exploration for Analysis and Business Insight
  50. 50. DATA GOVERNANCE 54 Business Glossary – Managed articles about logic and language. Knowledge: What should it be? Data Asset Certification Trust: Is this the right view? Who says so? As of when? Well Managed – Quality checks, release notes, load updates Trust: Is it ok to use RIGHT NOW?
  51. 51. DATA GOVERNANCE 55 Business GlossaryData Asset Certification Well Managed What we do: Data knowledge management and data stewardship Goals: • Demystify our data warehouse of tens of thousands of datasets • Increase trust in data by increasing transparency • Save analysts’ time and reduce their opportunities for error
  52. 52. 56 Value Generation Governed Self Service0
  53. 53. 57 Organizing for Success Purified Data Science6
  54. 54. Data Prep Data Science 58
  55. 55. 59 Data ScienceData +
  56. 56. 60 A COMPLETE VIEW OF OUR CUSTOMERS Behavior Demographics & Interests AttitudeValue to eBay
  57. 57. 61  DATA SCIENCE Data Data Science Business ImpactData Data Science Data Science Data Science Business Impact Insights Customer Insights used to make decisions and set strategy Predictive Models Models that predict outcomes to achieve optimal targeting Segments New ways to assess value and attitudes of our customers DNA
  58. 58. 62 CONVERSION MODEL User Category Probability 111602**** 1564** 10.1% 111602**** 1562** 6.54% 111602**** 1569** 5.67% 111602**** 3564** 4.33% 111602**** 1397** 1.19% 111602**** 3877** 1.11% 111602**** 9282** 1.01% 111602**** 3607** 0.91% 111602**** 1040** 0.81% 111602**** 1564** 0.76% 111602**** 1040** 0.66% 111602**** 4250** 0.01% 111602**** 5235** 0.01% • Cart data • Watch data • Mobile watch • Search pages • Browse data • Purchase history Models
  59. 59. Thanks! • ALEX LIANG • hliang@ebay.com • http://www.linkedin.com/in/alexlianghu

×