Rakuten's Private Cloud

2,758 views

Published on

Explains the origin of Private Cloud at Rakuten, how we built it, promote it, and operate it.

Published in: Technology, Business

Rakuten's Private Cloud

  1. 1. Rakuten’s Private Cloud Jonathan Levine http://www.rakuten.co.jp/
  2. 2. 2 What’s Rakuten?
  3. 3. Rakuten Facts • Founded in 1997 by Hiroshi Mikitani • Over 9,000 staff worldwide • Rakuten Ichiba and Rakuten Travel are largest businesses; we have a total of more than 50 services • Domestically, more than 130 million products offered by more than 130,000 providers
  4. 4. Corporate History - Domestic1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013
  5. 5. Corporate History – Global1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013
  6. 6. Domestic Marketplace Sales Growth Source: Rakuten *: Domestic E-Commerce = Mall (Fixed-price, Group purchase), Mobile, Business, Golf, Books, Auction (Flea market), download, tickets, Internet Supermarket, Check Out, Off-track betting, toto, ShowTime, Media rental, Dining , Salon (added from Q2/12), and Mart and Kenko.com (added from Q3/12)
  7. 7. International Marketplace Sales Growth Source: Rakuten *: Marketplace = Taiwan Rakuten Ichiba + Tarad Premium Mall + Lekutian + Buy.com marketplace + PriceMinister marketplace + Rakuten Belanja Online + Rakuten Deutschland + Play.com marketplace + Rakuten Brasil marketplace (from Q2/12).
  8. 8. 8 Why is Rakuten Ichiba Different?
  9. 9. Our Mission Empowerment
  10. 10. 10 Unique E-Commerce Model Rakuten’s Empowerment Model is Unique and Difficult to Copy
  11. 11. 11 Typical E-Commerce Site
  12. 12. 12 Rakuten’s E-Commerce Site
  13. 13. 13 Why Cloud?
  14. 14. My First Computer
  15. 15. Moore’s Law Source: Wikipedia
  16. 16. My Current “Computer” <
  17. 17. 17 Server CPU Utilization, by Service 新サービス系サーバー別  CPU稼働率(2009年3月) 0 10 20 30 40 50 60 70 80 90 100 wfbtool wplaza misml wdeli wenq wmoney wdigi wincu woubo wiseek ddin wnikki btmail bplaza wisdir bdigi wspace ddeli wtmail bfbtool wapitic wpsp wgame wwoman apdelicti wportalgl bnikki wdigiadm wmile wdeliimg bmedia asearchr wbrtic wportaladm mtic wgameapp bauto サーバー群 CPU使用率(%) CPU使用率最大値 CPU使用率平均値 10%
  18. 18. Domestic Marketplace Sales Growth Source: Rakuten
  19. 19. 19 Cloud at Rakuten
  20. 20. 20 Step 1: Virtualization Virtual Server Pool Server Server Shared Storage Hard Disk Storage Network Old: 1 physical == 1server New:1 physical == n servers System Benefits of Virtualization •Faster Deployment Time for New Servers •Automatically Handle Downtime •Improve Equipment Utilization •Easy Replacement of Hardware OS App OS App OS App OS App OS App Virtual Server Pool
  21. 21. 21 “Pcloud” production Environment Server Farm: 104 Physical Storage Farm: 14 Physical iSCSI
  22. 22. 22 Private Cloud Progress – Stage 1 VMs
  23. 23. Why so Slow?
  24. 24. Technology.
  25. 25. 25 “RIaaS" production Environment Fabric Network (10G, FCoE) 3PAR Storage Blade Server VMware
  26. 26. 26 Private Cloud Management Tool ・VM management ・Server Setup (with Chef) ・DNS setting ・SLB control
  27. 27. Not Just Technology.
  28. 28. Not Just Technology. It’s an Internal Business.
  29. 29. Source: Amazon re:Invent
  30. 30. • Tools • Training • Support • Roadmap • Communication == Product Management
  31. 31. RIaaS is crossing over the Chasm!!! We are here! Early Adopter Early Majority Late Majority Proof of solution Willing to use Step by Step Test -> Production Case study Sensitive Communication Word of mouth Interactive Passive SLA No Risk No Return Reasonable requirement Demanding Priority Speed Experience Balanced Risk/Return Stability PCloud RIaaS
  32. 32. 32 Private Cloud Progress – Stage 2 Jul-10 Aug-10 Sep-10 Oct-10 Nov-10 Dec-10 Jan-11 Feb-11 Mar-11 Apr-11 May-11 Jun-11 Jul-11 Aug-11 Sep-11 Oct-11 Nov-11 Dec-11 Jan-12 Feb-12 Mar-12 Apr-12 May-12 Jun-12 Jul-12 Aug-12 Sep-12 Oct-12 Nov-12 Dec-12 Jan-13 Feb-13 Mar-13 VMs
  33. 33. 33 Case Study
  34. 34. 34 Exciting Event …?
  35. 35. 35 2005/1/1 2005/6/1 2005/11/1 2006/4/1 2006/9/1 2007/2/1 2007/7/1 2007/12/1 2008/5/1 2008/10/1 2009/3/1 2009/8/1 2010/1/1 2010/6/1 2010/11/1 2011/4/1 2011/9/1 2012/2/1 2012/7/1 2012/12/1 SuperSale Network Effect
  36. 36. 36 Private Cloud for SuperSale 0 1000 2000 3000 4000 5000 6000 Jul-10 Aug-10 Sep-10 Oct-10 Nov-10 Dec-10 Jan-11 Feb-11 Mar-11 Apr-11 May-11 Jun-11 Jul-11 Aug-11 Sep-11 Oct-11 Nov-11 Dec-11 Jan-12 Feb-12 Mar-12 Apr-12 May-12 Jun-12 Jul-12 Aug-12 Sep-12 Oct-12 Nov-12 Dec-12 Jan-13 Feb-13 Mar-13 VMs
  37. 37. 37 Next Steps
  38. 38. High Capacity Racks Patch Patch EqualLogic R610 R610 2 1 5 4 3 8 7 6 11 10 9 14 13 12 17 R610 16 15 R610 20 19 R610 18 27 R610 24 23 R610 25 R610 31 R610 30 28 35 34 33 R610 32 41 Patch 40 39 38 37 36 21 R610 22 26 29 R610 42 Patch 39 Nexus 2232PP 2U4N32 42 Nexus 2232PP 41 Nexus 2232PP 40 Nexus 2232PP 38 37 36 35 34 33 31 2U4N30 29 2U4N28 27 2U4N26 25 2U4N24 23 2U4N22 21 2U4N20 19 2U4N18 17 2U4N16 15 2U4N14 13 2U4N12 5 2U4N4 11 2U4N10 9 2U4N8 3 2U4N2 1 Management 48port switch 7 2U4N6 Standard Rack (3KVA) Hicap Rack (15KVA) 48 Nodes 9 Cables 12 Nodes 36 Cables
  39. 39. Site Recovery Manager for Migration Old DC New DC
  40. 40. 40 Global DC Expansion EU DC Japan DC U.S. DC
  41. 41. Private Cloud Roadmap • IaaS (Compute as a Service) improvements – Add APIs – Add App team access • DBaaS (Database as a Service) • STaaS (Storage as a Service) • PaaS (Platform as a Service)
  42. 42. 42 Final Thoughts
  43. 43. 43 •12 Employees •27,000,000 users •2 years
  44. 44. 44 •31 Employees •#16 site in US (as of May, 2012; source: Alexa) •30 months
  45. 45. 45 But…
  46. 46. • Technology is really important… • … but it isn’t everything
  47. 47. • Cost is really important… • … but it isn’t everything
  48. 48. • Make sure you can have enough scale to be competitive… • Then compete to win!
  49. 49. Thank you!

×