• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Big Data Behind the Scenes
 

Big Data Behind the Scenes

on

  • 753 views

Bret Piatt, General Manager of Apps@Rackspace, will talk about how his businesses use OpenStack, Hadoop, and custom built big data systems to deliver service provider scale and reliability systems. ...

Bret Piatt, General Manager of Apps@Rackspace, will talk about how his businesses use OpenStack, Hadoop, and custom built big data systems to deliver service provider scale and reliability systems. This talk will cover how Rackspace solves real world problems including high level architectures and operational metrics.

Statistics

Views

Total Views
753
Views on SlideShare
736
Embed Views
17

Actions

Likes
2
Downloads
12
Comments
0

2 Embeds 17

https://twitter.com 12
http://www.linkedin.com 5

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Sending the data to the computing system is becoming less and less efficient.1 Petabyte of data fits in a single data center cabinet today and takes 10 days to copy to the cabinet next to it over a 10Gbps network.
  • Powering the public site for SharePoint@Rackspace – we are a customer of our own SaaS servicesAlso gives us the ability to clone environment for development and testing of new features
  • Logs through Hadoop for 150+ GB per day of processed, indexed, searchable log data2 billion DNS queries per day1 billion SQL queries per day500+ GB of logs generated per day
  • Troubleshooting email services requires data from many sources in an aggregated view.
  • We manage the size of the Hadoop cluster for the incremental jobs to complete every 15 minutes. This gives our support organization near real-time information on complex system interactions.

Big Data Behind the Scenes Big Data Behind the Scenes Presentation Transcript

  • Big Data Behind the Scenes Delivering email, backup, & collaboration to millions Bret Piatt – General Manager, Apps@Rackspace Email: bret.piatt@rackspace.com / Twitter: @bpiatt
  • SAFE HARBOR STATEMENT • This presentation contains "forward-looking" statements that involve risks, uncertainties and assumptions. If the risks or uncertainties ever materialize or the assumptions prove incorrect, our results may differ materially from those expressed or implied by such forward-looking statements. All statements other than statements of historical fact could be deemed forward-looking, including, but not limited to, any projections of financial information; any statements about historical results that may suggest trends for our business; any statements of the plans, strategies, and objectives of management for future operations; any statements of expectation or belief regarding future events, potential markets or market size; technology developments; and any statements of assumptions underlying any of the items mentioned. • These statements are based on estimates and information available to us at the time of this presentation and are not guarantees of future performance. Actual results could differ materially from our current expectations as a result of many factors, including but not limited to: the unpredictable nature of our rapidly evolving market and quarterly fluctuations in our business; the effects of competition; and any adverse changes in our indirect channel relationships. These and other risks and uncertainties associated with our business are described in our quarterly and annual reports filed with the Securities and Exchange Commission at www.edgar.gov. We assume no obligation and do not intend to update these forward-looking statements. • This presentation outlines general information regarding our services and is for informational purposes only; all statements and information are provided “AS IS” and are presented without warranty of any kind, express or implied. Our product/services offerings are subject to change without notice. • Trademarks and Service Marks • Rackspace® and Fanatical Support® are service marks of Rackspace US, Inc. registered in the United States and other countries. OpenStack and OpenStack design are trademarks of OpenStack, LLC. Other trademarks and trade names appearing in this presentation are the property of their respective holders. We do not intend our use or display of other companies’ trade names, trademarks, or service marks to imply a relationship with, or endorsement or sponsorship of us by, these other companies. RACKSPACE® HOSTING 2 | WWW.RACKSPACE.COM
  • Who is Rackspace? AWARD WINNING 5,300+ RACKERS CUSTOMER SERVICE OFFICES IN 8 COUNTRIES HOSTING PORTFOLIO 200,000+ CUSTOMERS DEDICATED * CLOUD * HYBRID CUSTOMERS IN 120+ COUNTRIES GLOBAL FOOTPRINT PUBLIC CLOUD IN 6 REGIONS 10 DATA CENTERS WORLDWIDE ON 4 CONTINENTS LEADER IN GARTNER’S MAGIC QUADRANT MANAGED HOSTING * * * 4 TIME MICROSOFT * PARTNER OF THE YEAR * * SHAREPOINT LEADER MOST MVPS, BOOKS, & HOSTING* OPENSTACK FOUNDER FOUNDATION PLATINUM MEMBER RACKSPACE® HOSTING *Hosting leadership based on revenue from Microsoft Service Provider License Agreement (SPLA) SharePoint software | WWW.RACKSPACE.COM
  • Who am I? What is Apps@Rackspace? • Bret Piatt, General Manager – Apps@Rackspace – 15 years of service provider experience (AT&T and Rackspace) – Technical operations, architecture, and support background – Product, corporate development, and corporate strategy experience • Apps@Rackspace – the SaaS division of Rackspace – 350+ Rackers with major offices in San Antonio, TX; Blacksburg, VA; and Cincinnati, OH – Business email service with over 3,000,000 paid mailboxes – SharePoint collaboration service with over 1,000,000 managed seats – Data backup service with over 10,000,000,000 files protected RACKSPACE® HOSTING | WWW.RACKSPACE.COM 4
  • What infrastructure runs Apps@Rackspace? RACKSPACE® HOSTING | WWW.RACKSPACE.COM 5
  • Major Trend Keeping Me Up At Night • Storage doubles every 1.7 years • Network doubles every 2.4 years • Storage unit costs drop 40% per year • Network unit costs drop 26% per year Applications need to be redesigned over time to bring the code to the data or networking costs will become a much larger portion of my infrastructure spend RACKSPACE® HOSTING | WWW.RACKSPACE.COM 6
  • SharePoint@Rackspace Cloud computing powering business collaboration RACKSPACE® HOSTING | WWW.RACKSPACE.COM 7
  • Per User SharePoint – Key Requirements Development Goals • Rapid time to market • Easy capacity management • Scale to 10,000+ customers • Ability to start small (10 users) • Ability to grow big (100s of users) • Efficient infrastructure Product Offering • Full versions of SharePoint 2013 • Foundation, Standard, & Enterprise • Active Directory integration • Fully managed and supported RACKSPACE® HOSTING | WWW.RACKSPACE.COM 8
  • Per User SharePoint – Capacity On Demand • Built on OpenStack public cloud infrastructure • Cloud Networks • Cloud Servers • Cloud Block Storage • Cloud Load Balancers • Cloud Servers Vyatta Appliance (firewall) • Will use hybrid public/private at scale • RackConnect joining Public & Private • Private for workload optimized hardware • Public for capacity on demand RACKSPACE® HOSTING | WWW.RACKSPACE.COM 9
  • Rackspace Email The big data platform behind millions of mailboxes RACKSPACE® HOSTING | WWW.RACKSPACE.COM 10
  • Rackspace Email by the Numbers One billion SQL queries per day ~150 GB of indexed data per day Two billion DNS queries per day 500+ GB of logs per day 11 RACKSPACE® HOSTING | WWW.RACKSPACE.COM
  • Rackspace Email Log Sources for Hadoop RACKSPACE® HOSTING | WWW.RACKSPACE.COM 12
  • Rackspace Email Hadoop System RACKSPACE® HOSTING | WWW.RACKSPACE.COM 13
  • Jungle Disk Hybrid infrastructure ensuring reliable data backup RACKSPACE® HOSTING | WWW.RACKSPACE.COM 14
  • Goals for Jungle Disk Infrastructure Development Goals • Always online • Fast upload and download • Cost effective • Easy capacity management • Easy to troubleshoot • Transparent deployments Product Offering • Desktop and laptop backup • Server backup • Online storage • Windows, Mac, & Linux RACKSPACE® HOSTING | WWW.RACKSPACE.COM 15
  • How Jungle Disk Uses Cloud Services Public cloud for capacity on demand Dedicated Cloud for customized fixed capacity systems RACKSPACE® HOSTING | WWW.RACKSPACE.COM 16
  • RACKSPACE® HOSTING 17 | WWW.RACKSPACE.COM 17