• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Openstack and eBay
 

Openstack and eBay

on

  • 5,013 views

 

Statistics

Views

Total Views
5,013
Views on SlideShare
4,905
Embed Views
108

Actions

Likes
5
Downloads
302
Comments
0

4 Embeds 108

http://blog.zenoss.com 104
http://web1.conversationminer.com 2
http://twitter.com 1
https://twitter.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • eBay Inc. has two strong core businesses … that offer significant potential for growth and profitability. And we have a consistent track record of delivering on our commitments.We are a more customer-focused company.We are accelerating innovation across the board.And we have a unique combination of assets that position us well to capitalize aggressively on the inflection point we are seeing in the external environment.eBay/marketplaces around the world: North America: eBay.com and eBay.ca EU: eBay.co.uk eBay.de eBay.fr Etc. Asia: eBay.in eBay.au Korea: GMarketCore adjacency businesses: PayPalStubHub
  • Top 8 pools represent 1900 machines (Nov 2009). By defining a cutoff at 800 machines , it means that we would need to cloud burst 36800 VM/hours to cover the peak.And using $1/h on Azure(excluding bandwidth) = $36,800 / month1900 – 800 = 1100 machines x $1000 = $1.1M /monthImprove power efficiencyServer Consolidation (Sharing)Reduce DR capacity by implementing pre-emptionClicks 1, 2, 3: Align costs to utilizationData center efficiencyEnergy-proportional computingClick 4: “Fill out the valleys”Reuse spare capacity for experimentation, …Offer Infrastructure as a serviceClick 5: Chop off peaksClick 6: Move the peaks into the valleys (smooth out)Cloudburst (borrow capacity to address peaks)
  • Tiered approach more efficient, cost-effective, and reliable.Equate to flying airplane with 8 engines when 2 will do. Overcompensates, costs too much, reduces efficiency.Can allocate power/cost of other 6 engines to other applications at tiered levels, based on business need. Avoid spending the capital up front when you build lower-level tiers/Tier IIEvaluate systems based on level of business-critical applications/requirements.Place applications based on their resilience to failure. EXAMPLE: Search – a business-critical function – points immediately at another center if there is an issue, as opposed to a more time-consuming process to migrate a DB from one location to another. The search nodes can be in Tier II, the DB needs to stay in Tier IV (more sensitive to failures). We believe that 70%+ of our applications can reside in Tier II (half the capex, half the opex). Today, all of them live in Tier IV.  Assign top tier operations – redundancy, cost, reliability – to mission-critical applications.Assign lower-level operational structure to less-critical applications.
  • Future state : based on utility cost, datacenter and infrastructure metrics, traffic will be directed to the most economical location while achieving SLA. consider utility companies incentives to shift traffic and power down or idle down infrastructure.
  • VISUALLY CONSISTENT – UPDATE LABELS – ADD AXESBalance: all in-house; all outsourceInflection points to choose some distributionLines rep various choice; dots rep various modelseBay’s method: Adopt an hybrid cloud approach (part on premise, part off premise) Use the on premise infrastructure for core business critical applications and infrastructure. Convert site to a cloud infrastructure to enable agility and increase average utilization Use the off premise cloud to offload capacity for peak demand, and new/experimental projects. Reduce on premise footprint.Anticipating double-digit transaction per second (TPS) increases YoY.Clarify TPS definition: Combination of internal and external factors, including processing, analytics, as well as site traffic and commerce transactions.As TPS increases, need options for addressing cloud bursting:Advantage: Adjust computing power to applications’ varying demandEnables higher resource utilization due to statistical multiplexing of loadEnables short time to market for new app to capture opportunities when they are identifiedGreen line: Servers operated by eBay Estimated ~ 5228 servers / 13130 CU’s Variety of HW types and virtualization status (Nov 2010)Cost based on 2 yr leaseComplete control; tried and known environmentParameters: Server cost per hourChange in the number of servers over timeBlue line: Internal cloud: Resources dedicated to a single operator who incurs constant cost per server regardless of actual loadTan line: External cloud: use and pay for server only when neededUsually pay by hour of reserved operationLight blue line: Hybrid cloud: A combination of internal and external clouds (part on-premise; part off-premise)Use the on premise infrastructure for core business critical applications and infrastructure.Convert site to a cloud infrastructure to enable agility and increase average utilizationUse the off premise cloud to offload capacity for peak demand, and new/experimental projects. Reduce on premise footprint.Internal cloud “stamps” 2 yr lease. Fixed cost independent of utilizationQuickly deploy and change appsShare load dynamically with self-operated serversParameters: number of stamps and installation/operation datesExternal cloud Paid for by the hour.Quickly turned on to carry traffic when exceeds capacity of DC-based serversParameters: hourly cost of server unit
  • As infrastructure is commoditized, eBay can decrease cloud costs as much as possible by working with external industry leaders.Allows eBay to focus on strengths of running world’s largest online marketplace: Business intelligence Commerce expertise Customer focus Reliability Security Global presence

Openstack and eBay Openstack and eBay Presentation Transcript

  • Neal Sample
    VP Architecture, Platform Products, Developers Program
  • eBay’s global commerce platform
  • eBay marketplace = economy of scale
    9
    23
    2
    2Bpage views/day
    23Mlines of code
    9Petabytesof data storage
    94
    200
    $62
    94M active users
    200Mlive listings
    $62B2010 gross merchandise volume
    6,000
    75
    6,000 application servers
    75B database calls/day
    Commercial data warehouse 100x larger than Library of Congress
  • Cloud bursting: ↓ costs by ↑ efficiency
    2000
    1800
    1600
    1400
    1200
    1000
    800
    600
    400
    200
    0
    Fixed Cost
    Datacenter efficiency
    Cloud Burst
    Energy-proportional computing
    Excess Capacity
    Ideal
    Remove the peaks
    Request for Data
    Current Traffic
    text
    text
    text
    text
    text
    text
    text
    text
    Timeline
  • Datacenter efficiency
    TIER II
    2(N+1)
    TIER IV
    N+1
    Datacenter tier = level of redundancy
    Blend of Tier IV & Tier II
    Minimum 30% reduction in OpEx cost per MW enables cost effective operations/expansion.
    50% reduction in CapEx for new lower-tier datacenters (Tier IV vs. Tier II)
    Burst Capacity (DC and Cloud)
  • Energy-proportional computing
    Traffic
    KWh cost / incentives / forecasts
    Meta
    Scheduler
    BMS metrics
    SLA
    Models
    Utilization (headroom)
    Availability
    Cloud
    Bursting
    SLC
    PHX (Tier 4)
    PHX (Tier2)
    Direct traffic to the most economical location / optimize placement within DC
  • Financial model structure
    Hourly demand(in average TPS)
    Hourly demand(in average TPS)
    Historical (from ODW) or predicted
    Historical (from ODW) or predicted
    Define a standard CU
    Define a standard CU
    Hourly computing units (CU) required
    Max TPS per CU
    Hourly computing units (CU) required
    Max TPS per CU
    Based on server inventoryand growth forecast
    Based on server inventoryand growth forecast
    Hourly CUs neededin external cloud
    Number of CU in internal cloud
    Hourly CUs neededin external cloud
    Number of CU in internal cloud
    Negotiated price with public cloud vendor
    Negotiated price with public cloud vendor
    Hourly cost of CUin external cloud
    Hourly total cost
    Hourly cost of CU in internal cloud
    Hourly cost of CUin external cloud
    Hourly total cost
    Hourly cost of CU in internal cloud
    HW/SW lease, networking, space, power, laborPredict future costs trends
    HW/SW lease, networking, space, power, laborPredict future costs trends
    Sum cost over period
    Sum cost over period
    Evaluate tradeoffs by varying size of internal cloud, and costs
    Evaluate tradeoffs by varying size of internal cloud, and costs
  • Cost components
    Datacenter
    People
    Total cost per CU/h : $1.07
  • TSP ↑ = more attractive options for cloud bursts
    All self operated CUs
    TPS and FE Compute Units (CU) 24-Month Projects
    Load served by external cloud
    Historical TPS hourly data from ODW
    CU decreasing by
    Lease expiration
    Load served by self operated CU
  • Cost-benefit analysis
    DC CUs Impact on Project Annual Cost 9/2010 – 8/2011
    DC CU Hr Cost = 1.07, 30 TPS per CU
    60,000,000
    Cloud cost ($/hour)
    Total Cost ($)
    0.5
    0.8
    1
    40,000,000
    1.1
    1.2
    1.5
    2
    3
    4
    20,000,000
    0 DCs
    $20M
    0 DCs
    $30M
    3K DCs
    $40M
    3.5K DCs
    $45M
    4K DCs
    $47M
    4.8K DCs
    $52M
    5K DCs
    $55M
    0
    2000
    4000
    6000
    8000
    DC CUs
  • Focus for the future
    Business Intelligence
    • Commerce expertise
    • Customer experience
    • Reliability
    • Security
    • Etc.
    Business Intelligence
    Increased Focus
    Infrastructureas Commodity
    Infrastructureas Commodity
    LowerCost
  • Financial Cost Model for the Hybrid Cloud