• Save
MicroStrategy World 2014: Scaling MicroStrategy at eBay
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

MicroStrategy World 2014: Scaling MicroStrategy at eBay

on

  • 482 views

eBay has one of the largest data warehouses in the world! See how the BI Platform team at eBay had to rethink and rebuild their system architecture and processes in order to support the ever-growing ...

eBay has one of the largest data warehouses in the world! See how the BI Platform team at eBay had to rethink and rebuild their system architecture and processes in order to support the ever-growing data volume and scalability needs of their developers and users.

Statistics

Views

Total Views
482
Views on SlideShare
480
Embed Views
2

Actions

Likes
0
Downloads
3
Comments
0

1 Embed 2

https://www.linkedin.com 2

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Today’s eBay isn’t what it used to be. Many people think of us only as an auction site. But that perception hasn’t kept up with reality. The reality is that more than 70% of what is sold on eBay is new merchandise, available for purchase immediately.
  • At eBay we believe that commerce will change more in the next 3 years than it has in the past 20.  It’s consumer driven, and technology enabled. And it’s being led by mobile.
  • First, consumers are using tablets and smart phones as a mission control deviceResearchInspirationCoupons or other discountsChattingWeatherAnd then they’re buying, straight from their mobile devices.
  • It’s clear, retail and commerce are fundamentally changing – and technology is the driving force. We expect this will cause the $10 Trillion commerce market to be turned on its head in the years to come.
  • In 2013, our mobile business continued accelerate.There were 120 million mobile applications downloads; by the way our iPhone and iPad apps are now available in 8 languages and in 190 countries. eBay generated $13B in mobile GMV in 2012 and $22 billion mobile GMV for 2013.
  • When you put all of this together, our business is actually pretty simple – eBay is about connecting people to the things they need and love.
  • Start off with some Data StatisticsGreater than 12,000 of Named UsersGreater than 55,000 chains of logicGreater than 150,000 Data elements Millions of queries run on our Platforms everydayGreater than 40 Terabytes of Backup per hourGreater then 3.5 Trillion rows in our largest table100TB of New Data everyday100 PB a Day of Physical IO
  • 98 nodehundreds of small Oracle databases on an Hourly basisAn older 128 node system is used as it’s DR.    It is managed for high availability.   Software releases are adopted after enough time has been given to shake out bugs. Hardware components are all enterprise class for performance and reliability. Singularity 256 nodeTeradata system with 36 PB of spinning disksUser behavior data; A/B testing Software releases have been adopted very aggressively to benefit from new features earlier in the process. Since the workload is not quite as diverse as the EDW, the exposure to bugs is not as significant.The previous production generation serves as a DR system. 20PB Hadoop system Structured and Unstructured Data.   Pattern Detection and behavioral data.
  • Started in AccessCouple years later moved to Oracle,Informatica and Business ObjectsFirst Teradata system around 2002Migrated to MicroStrategy in 2003SAS added to the capabilities in 2004VDMs 2006Tableau 2009Hadoop gained momentum a couple years ago
  • Close to 800GB of cubes
  • Heavy user of cubes due to Teradata performanceAt the beginning of 2012 we had 256GB of RAM
  • Some cube processing was taking 18+ hoursLoading cubes from shared storage was taking too long
  • Backend network connects to Teradata and carries NFS traffic
  • Also exploring use for vanity URLs
  • External process built with SDKRuns every 5 minutes and writes to database tableOutput formatted using Freeform SQL and HTML tags
  • EM only provides visibility into sever-side processing
  • JavaScript plugin to measure client-side performance--Time--Browser--Web server--Page rending time--Report/dashboard nameCombine with internal user data to determine location
  • Includes log agent for log4jUsed within plugins and custom widgets - Application - Function - User - Execution Time (ms) - Was it an error? - Message (note)
  • Web and Mobile on Windows due to SSO requirements
  • Couldn’t achieve performance requirements using NAS or SANLocal SSD performed better
  • All flash storage bubble on 10Gbps network
  • 2 billion row limit for Intelligent Cubes (240GB cube)Linear scalabilityReduce dependency Sub 5 second response time
  • Same datasets available for MicroStrategy and TableauCan also be leverage in other analytic applications like SAS and RNo 2 billion row limit

MicroStrategy World 2014: Scaling MicroStrategy at eBay Presentation Transcript

  • 1. HOW BIG IS BIG: SCALING MICROSTRATEGY AT EBAY Business Intelligence Architect Tim Case
  • 2. TODAY’S TOPICS •eBay Overview •Challenges & Solutions •Plans for 2014 •Extending MicroStrategy SCALING MICROSTRATEGY AT EBAY 2
  • 3. 128M Active users 7 out of 10 Items sold are new 7 out of 10 Items sold are fixed price EBAY MARKETPLACES
  • 4. A new retail is emerging. It’s consumer driven and technology enabled.
  • 5. THE PHONE IS BECOMING A MISSION CONTROL DEVICE
  • 6. People want to get what they want, when they want it, wherever they are.
  • 7. 2012 Mobile GMV $13B $22B in 2013 Countries 190 Application downloads 190M EBAY LEADS IN MOBILE
  • 8. every 10 min.a car or truck is bought via mobile every 1 min.a tablet is bought via mobile every 15 sec.a ladies handbag is bought via mobile EBAY MOBILE VOLUME IS STAGGERING
  • 9. We connect people to the things they need and love.
  • 10. DATA STATISTICS SCALING MICROSTRATEGY AT EBAY 10
  • 11. DATA PLATFORMS SCALING MICROSTRATEGY AT EBAY 11
  • 12. 52N Co-existent 5300/5350/5380 (30 TB /50 TB Max) ‘99 ’01 ‘11’03 ‘05 ‘07 ‘09 Access DB & MS Excel Reports (3 MB) Informatica Oracle DW & Business Objects (500 GB) Campaign Mgmt DMs: FADE, RFM, Power Sellers, VCRU… (4 TB) Teradata 12N 5300 (8 TB) Dual System Primary: 64N 5400 Secondary: 60N 5380 Auto Gen ETL SOA Virtual Data Marts (VDM) Tactical Workload Member Insight ABC (RAM) Singularity Connected Commerce Cloud (Internal) Hadoop and NoSQL MicroStrategy SAS MAX Portal Ab Initio Tableau ANALYTICS EVOLUTION SCALING MICROSTRATEGY AT EBAY 12
  • 13. MICROSTRATEGY BY THE NUMBERS 64 Projects 5,200 Users 440,000 Sessions 15,000,000 Jobs 70,000 Reports 240 Cubes SCALING MICROSTRATEGY AT EBAY 13
  • 14. MICROSTRATEGY ARCHITECTURE SCALING MICROSTRATEGY AT EBAY 14
  • 15. CHALLENGES SCALING MICROSTRATEGY AT EBAY 15
  • 16. INTELLIGENT CUBE GROWTH SCALING MICROSTRATEGY AT EBAY 16 PROBLEM •No available memory (RAM) for additional Intelligent Cubes
  • 17. INTELLIGENT CUBE GROWTH SCALING MICROSTRATEGY AT EBAY 17 256 1024 2048 2011 2012 2013 RAM (in GB)PROBLEM •No available memory (RAM) for additional Intelligent Cubes SOLUTION •Increased the RAM to physical limit of servers (2TB)
  • 18. INTELLIGENT CUBE PROCESSING PROBLEM •Intelligent cube processing taking too long and missing SLAs SCALING MICROSTRATEGY AT EBAY 18
  • 19. INTELLIGENT CUBE PROCESSING PROBLEM •Intelligent cube processing taking too long and missing SLAs SOLUTION •Upgraded to a 10Gbps connection to backend network •Added additional server to cluster •Balanced jobs across nodes Cube 05 Cube 04 Cube 03 Cube 02 Cube 01 Cube Processing Times (in seconds) Before After SCALING MICROSTRATEGY AT EBAY 19
  • 20. PROBLEM •Moving servers between environments caused issues URL REWRITE SCALING MICROSTRATEGY AT EBAY 20
  • 21. PROBLEM •Moving servers between environments caused issues SOULTION •URL Rewrite rules URL REWRITE SCALING MICROSTRATEGY AT EBAY 21 <rule name="test" stopProcessing="true"> <match url="^asp/Main.aspx” /> <conditions> <add input=“{QUERY_STRING}” type=“Pattern” pattern=“(.*)server=[^&]+(.*)” </conditions> <action type="Redirect" url="bix/asp/Main.aspx? {C:1}server=ISERVER01{C:2}" /> </rule>
  • 22. PROBLEM •Users need to be able to gauge data freshness at a glance CUBE MONITOR SCALING MICROSTRATEGY AT EBAY 22
  • 23. PROBLEM •Users need to be able to gauge data freshness at a glance SOULTION •Custom Cube Monitor application CUBE MONITOR SCALING MICROSTRATEGY AT EBAY 23
  • 24. PROBLEM •No way to measure end-to-end user experience PAGE LOAD TIMES SCALING MICROSTRATEGY AT EBAY 24
  • 25. PROBLEM •No way to measure end-to-end user experience SOULTION •Plugin to measure client-side performance PAGE LOAD TIMES SCALING MICROSTRATEGY AT EBAY 25
  • 26. PROBLEM •No way to measure end-to-end user experience SOULTION •Plugin to measure client-side performance •Email Alerts PAGE LOAD TIMES SCALING MICROSTRATEGY AT EBAY 26
  • 27. PROBLEM •Need a low friction way to log anything MEASURE EVERYTHING SCALING MICROSTRATEGY AT EBAY 27
  • 28. PROBLEM •Need a low friction way to log anything SOULTION •HTTP Log Service MEASURE EVERYTHING SCALING MICROSTRATEGY AT EBAY 28 http://api.bix.corp.ebay.com/LogService/Log ?app=TEST &function=TestFunction &user=tcase &executiontime=9999 &error=FALSE &note=This%20is%20a%20sample
  • 29. PROBLEM •Difficult to monitor multiple applications •No single point-of-view CROSS-PLATFORM INSIGHT SCALING MICROSTRATEGY AT EBAY 29
  • 30. PROBLEM •Difficult to monitor multiple applications •No single point-of-view SOULTION CROSS-PLATFORM INSIGHT SCALING MICROSTRATEGY AT EBAY 30
  • 31. SPLUNK SCALING MICROSTRATEGY AT EBAY 31
  • 32. SPLUNK SCALING MICROSTRATEGY AT EBAY 32
  • 33. PLANNED IN 2014 SCALING MICROSTRATEGY AT EBAY 33
  • 34. PROBLEM •Managing Windows and Linux servers increases complexity MIXED ENVIRONMENT SCALING MICROSTRATEGY AT EBAY 34
  • 35. PROBLEM •Managing Windows and Linux servers increases complexity SOULTION •Standardize on Linux MIXED ENVIRONMENT SCALING MICROSTRATEGY AT EBAY 35
  • 36. PROBLEM •Shared storage is a single point of failure SHARED STORAGE SCALING MICROSTRATEGY AT EBAY 36
  • 37. PROBLEM •Shared storage is a single point of failure SOULTION •Flash-based “storage bubble” SHARED STORAGE SCALING MICROSTRATEGY AT EBAY 37
  • 38. PROBLEM •Widespread impact during planned and unplanned downtime PLATFORM STABILITY SCALING MICROSTRATEGY AT EBAY 38
  • 39. PROBLEM •Widespread impact during planned and unplanned downtime SOULTION •Sharded architecture – Physical servers – Linux cgroups – Internal cloud (C3) PLATFORM STABILITY SCALING MICROSTRATEGY AT EBAY 39
  • 40. PROBLEM •High memory requirements •Duplicated data •Scalability issues ANALYTIC DATA STORE SCALING MICROSTRATEGY AT EBAY 40
  • 41. PROBLEM •High memory requirements •Duplicated data •Scalability issues SOULTION •Analytic data layer ANALYTIC DATA STORE SCALING MICROSTRATEGY AT EBAY 41
  • 42. EXTENDING MICROSTRATEGY SCALING MICROSTRATEGY AT EBAY 42
  • 43. METRICS EXPLORER •Integrated with internal DataHub •Leverages MicroStrategy Visual Insight •Access to Intelligent Cubes for quick analysis •Warehouse reports also available for deeper analysis •Can save and share results SCALING MICROSTRATEGY AT EBAY 43
  • 44. METRICS EXPLORER SCALING MICROSTRATEGY AT EBAY 44
  • 45. METRICS EXPLORER SCALING MICROSTRATEGY AT EBAY 45
  • 46. METRICS EXPLORER SCALING MICROSTRATEGY AT EBAY 46
  • 47. METRICS EXPLORER SCALING MICROSTRATEGY AT EBAY 47
  • 48. ELEMENT CACHING SERVICE •Improve prompt performance – Critical for Metrics Explorer •Daily and weekly schedules •Developer self-service SCALING MICROSTRATEGY AT EBAY 48
  • 49. ELEMENT CACHING SERVICE SCALING MICROSTRATEGY AT EBAY 49
  • 50. ELEMENT CACHING SERVICE SCALING MICROSTRATEGY AT EBAY 50
  • 51. ELEMENT CACHING SERVICE SCALING MICROSTRATEGY AT EBAY 51
  • 52. ELEMENT CACHING SERVICE SCALING MICROSTRATEGY AT EBAY 52
  • 53. WEB BASED COMMAND MANAGER (WBCM) •No need to install Command Manager •Ability to trigger events via REST service •Simple workflows supported •Simplified user management •Empowers developers SCALING MICROSTRATEGY AT EBAY 53
  • 54. WEB BASED COMMAND MANAGER SCALING MICROSTRATEGY AT EBAY 54
  • 55. WEB BASED COMMAND MANAGER SCALING MICROSTRATEGY AT EBAY 55
  • 56. WEB BASED COMMAND MANAGER SCALING MICROSTRATEGY AT EBAY 56
  • 57. WEB BASED COMMAND MANAGER SCALING MICROSTRATEGY AT EBAY 57
  • 58. SELF SERVICE MIGRATION TOOL (SSMT) •Developer-focused tool for object management •Enables software development lifecycle (SDLC) •Built-in automated testing •Improved visibility into object migration •Leverages System Manager SCALING MICROSTRATEGY AT EBAY 58
  • 59. SELF-SERVICE MIGRATION TOOL SCALING MICROSTRATEGY AT EBAY 59 1. Get Object Manager access 2. Generate package using Object Manager 3. Upload package 4. Import package o Update Schema o Purge Object Cache o Purge Element Cache o Email Notifications
  • 60. OData •Industry-standard Open Data Protocol •Get data in XML or JSON format •Consume MicroStrategy data from your tool of choice: – Excel – Tableau – Visualization libraries (D3.js, Highcharts) SCALING MICROSTRATEGY AT EBAY 60
  • 61. OData http://localhost:8080/owind.svc/Categories/Description http://localhost:8080/owind.svc/Categories/$count http://localhost:8080/owind.svc/Categories?$orderby=CategoryName desc&$top=4 http://localhost:8080/owind.svc/Categories?$filter=CategoryID eq 8 SCALING MICROSTRATEGY AT EBAY 61
  • 62. WRAP UP •Challenges & Solutions •Plans for 2014 •Extending MicroStrategy SCALING MICROSTRATEGY AT EBAY 62
  • 63. QUESTIONS? tcase@ebay.com @timrcase http://www.linkedin.com/in/timrcase/