• Like
  • Save
How Enterprises like The Weather Company use the AWS Cloud - Jinesh Varia & Bryson Koelher
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

How Enterprises like The Weather Company use the AWS Cloud - Jinesh Varia & Bryson Koelher

  • 1,183 views
Published

TIA Keynote 2013: Jinesh Varia, Technology Evangelist, AWS and Bryson Koehler, Executive Vice President and CIO, The Weather Company discuss how different enterprises use AWS today. They discuss 4 …

TIA Keynote 2013: Jinesh Varia, Technology Evangelist, AWS and Bryson Koehler, Executive Vice President and CIO, The Weather Company discuss how different enterprises use AWS today. They discuss 4 strategies and real customer stories that represent trends in how large-scale enterprises adopt cloud computing. For the first time, The Weather Company, discusses on stage their strategy and use of AWS.

Video available here: www.youtube.com/watch?v=crIxor6FqCg

Published in Technology , Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,183
On SlideShare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
0
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. How Enterprises Like The Weather Company Leverage the AWS Cloud Jinesh Varia Technology Evangelist, AWS @jinman Bryson Koehler EVP, CIO, The Weather Company @brysonkoehler 1
  • 2. 7 Years Young Amazon S3 launched: March 14th 2006
  • 3. Cloud
  • 4. Broad & Deep Services to Support Virtually Any Cloud Workload Compute Amazon EC2 Amazon EMR Amazon ELB Storage & CDN Networking Amazon Route 53 Amazon VPC AWS Direct Connect AWS Partner Network Amazon S3 Amazon Glacier Amazon EBS AWS Import/Exp AWS Premium Support Database Amazon RDS Amazon DynamoDB Amazon Elasticache Amazon RedShift AWS Professional Services App Services Management CloudSearch Amazon SWF Amazon SQS Amazon SNS Amazon SES Amazon Elastic Transcoder AWS IAM Amazon CloudWatch AWS Elastic Beanstalk AWS Cloudformation AWS Data Pipeline AWS OpsWorks AWS CloudHSM AWS Trusted Advisor AWS Marketplace AWS Training & Certification
  • 5. Trusted by Enterprises Around the World
  • 6. Used by Government Agencies & Educational Institutions Worldwide
  • 7. • $5.2B retail business Every day, AWS adds enough • 7,800 employees server capacity to power that • A whole lot of servers whole $5B enterprise
  • 8. Gartner Magic Quadrant for Cloud Infrastructure as a Service (August 19, 2013) “AWS is the overwhelming market share leader, with more than five times the compute capacity in use than the aggregate total of the other fourteen providers in this Magic Quadrant.” Gartner “Magic Quadrant for Cloud Infrastructure as a Service,” Lydia Leong, Douglas Toombs, Bob Gill, Gregor Petri, Tiny Haynes, August 19, 2013. This Magic Quadrant graphic was published by Gartner, Inc. as part of a larger research note and should be evaluated in the context of the entire report.. The Gartner report is available upon request from Steven Armstrong (asteven@amazon.com). Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings. Gartner research publications consist of the opinions of Gartner's research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.
  • 9. Bryson Koehler EVP and CIO 9
  • 10. B2B2C DIRECT CONSUMER ONLINE & APPS B2B TV GOV’T PRODUCTS WHERE THE WORLD GETS ITS WEATHER 61.6M #1 MOST DISTRIBUTED Cable Network* *Nielsen, October 2012. Unduplicated Monthly Uniques (weather.com + wunderground)* 72% visit 2x or more Daily** *comScore, Oct 2012 **Omniture, July 2012 100M+ App Downloads* #2 All Time Most Downloaded iPad App** 47.2M Unduplicated Monthly Uniques*** *Various app stores, TWC only, **App Store Rewind, 2011 ***Omniture Site Catalyst, August 2012. TWC Mobile & wunderground 10
  • 11. Did You Know? TWC Weather Services for Businesses World’s Leading Weather Media Company  - Power 500+ broadcast channels world-wide World’s Leading Aviation Weather Service  - Support 50,000+ flights daily with embedded forecasters, systems and tools  - 85% of major US airlines. 30% of Top 100 global. World’s Leading Energy Forecaster  - Over 300 power trading clients worldwide  - Hourly, daily, and seasonal trading Fastest Growing Insurance Weather Service  - 7 of the top 10 P&C insurance customers  - Connecting Marketing, Underwriting, & Claims with systems & tools 11
  • 12. Weather is Changing Warming winters in US • • • • 2012 was the hottest in recorded history for the US Last time January was colder than normal was 1983 In July 2012 61% of the country was in drought conditions 9.2MM acres charred in 2012; 3rd highest in 13 years Source: Time.com; Climate Central
  • 13. Weather changes… So must IT Huge projects required to support these business needs In the next 12 months TWC will: • Increase data points by 1,400% • Increase data velocity by 300% • Increase number of deployments by 2,000% • Increase number of TV feeds from 1 to 7 • Increase the amount of video content by 100% • Increase the amount of social content by 500% ... Cloud-Ready • Consolidate 13 Data Centers to 4 • 75% of workloads to IaaS Cloud • Move to agile development teams • New TV Transmission System (retire 2, build 1) • 575 APIs and 20 Data platforms to 1 platform • 2500+ new localization servers • New HR, Email, Finance, Inventory, Expense, etc.
  • 14. Enterprise Strategies For the Cloud 14
  • 15. Strategy Build All New Apps and Services in the Cloud 15
  • 16. Strategy 1: Build New Apps in the Cloud Faster to build Social Apps Mobile Streaming Consumer Apps Easier to manage Less expensive to run Distributed architectures for high availability Product Websites Global Web Sites Marketing Campaigns
  • 17. All of TWC’s new applications are being purposebuilt to run exclusively in the cloud MAX 3D Radar PilotBrief New Weather.com and Other Websites New Weather Analytics Systems New Aviation/Energy Forecasting Apps New Data Ingest & Publishing Apps Weather Data API Mobile Apps 17
  • 18. Speed and Agility: Rapidly developing, testing, deploying in the Cloud From 1 highly constrained team 1 release a month Core Development Team Extra Development Resources 8-10 decoupled Scrum teams 4 releases a day Contractor Team Remote Team
  • 19. Strategy 1: Nasdaq used AWS to Build a New Line of Business
  • 20. Strategy Augment On-Premises IT Resources with Cloud Capacity 20
  • 21. Customer Corporate Data Center Availability Zone A Customer Gateway Private Dedicated 10G DirectConnect VPN Gateway NAT Private Subnet Router Customer Branch Offices Internet Public Subnet Internet Gateway Amazon VPC Availability Zone B The Cloud-Ready Enterprise IT Network Architecture Object Store S3 NoSQL DB Service Email Service Queue Service DynamoDB SES SQS AWS Region
  • 22. Strategy #2 Augment On-Premises IT Resources with Cloud Capacity Use Cloud to Make On-Premises Apps Better 22
  • 23. Strategy 2: Use Cloud to Make On-Premises Apps Better App 1 Analytics App 2 App 3 … Corporate Data Centers App N Elastic Map Reduce Amazon Redshift AWS Storage Gateway Backup Amazon S3
  • 24. Strategy 2: S&P Capital IQ Uses AWS for Big Data Processing S3 Provides data to 4200+ top global investment firms Hadoop Cluster Launched Hadoop faster, Learned Hadoop faster
  • 25. Weather’s Big Data Analytics Platform for Enterprises and Government TWC’s Redshift Cluster now contains: WeatherFX CreativeFX Partner EDW Platform Amazon S3 AudienceFX … App N TWC Private Data Center EDW Amazon Amazon Redshift Redshift 600+ million records of Weather forecast data since March 2005 700+ million records of Current conditions observation data since April 2004 11+ billion records of Int. Civil Aviation Org. (ICAO) Observation data since 1970 TBs of Storm event data from from 1950 - 2012 Weather sentiment survey data ~1 Billion records Bulk loaded in 36 minutes!
  • 26. Weather is more severe 26
  • 27. Weather ⌃ is more accurate Day 1-3 Forecast – Temperature and Precipitation
  • 28. Weather forecast is more accurate 3, 5, 7 Day Forecast 1980-2010 100 90 Accuracy (%) 80 70 3 day 60 5 day 7 day 50 40 30 20 1980 1990 2000 2010 Source: NCEP 28
  • 29. Taking the World’s most accurate weather forecast into the future and moving it to the Cloud… Elastic Load Balancing Data Data API Caching, Notifications, Auth etc. Government Autoscaling Instances US-EAST Data Universities Research Facilities Data Private Suppliers or Corporates Multiple Sources of Data Weather.com + Weather Channel, etc Autoscaling Instances APAC Forecast Core Proprietary Forecasting Models Daily roll-ups, worded forecasts, indices, etc. in 49 languages Where the science happens Forecast at 10X points worldwide Weather Company’s Next Generation Forecast Platform Media Retail Agriculture Insurance Aviation Energy Trading Ground Transportation Marine Utilities Emergency Response Construction Outdoor Recreation Military Construction Travel & Hospitality B2B Customers 29
  • 30. Lots of Users Elastic Load Balancing Data Data API Caching, Notifications, Auth etc. Government Autoscaling Instances US-EAST Data Universities Research Facilities Data Private Suppliers or Corporates Multiple Sources of Data Weather.com + Weather Channel, etc Autoscaling Instances APAC Forecast Core Proprietary Forecasting Models Daily roll-ups, worded forecasts, indices, etc. in 49 languages Where the science happens Forecast at 10X points worldwide Weather Company’s Next Generation Forecast Platform Media Retail Agriculture Insurance Aviation Energy Trading Ground Transportation Marine Utilities Emergency Response Construction Outdoor Recreation Military Construction Travel & Hospitality B2B Customers 30
  • 31. Lots of Users Weather.com + Weather Channel, etc Data Government Autoscaling Instances US-EAST Data Universities Research Facilities Data Autoscaling Instances APAC Scaling based on demand Elastic scaling when huge surge of user requests Private Suppliers or Corporates Multiple Sources of Data Weather Company’s Next Generation Forecast Platform Media Retail Agriculture Insurance Aviation Energy Trading Ground Transportation Marine Utilities Emergency Response Construction Outdoor Recreation Military Construction Travel & Hospitality B2B Customers 31
  • 32. Lots of Users SEVERE WEATHER Scaling to provide the best accuracy Data Improved frequency of forecast updates, more locations, less latency Government Autoscaling Instances US-EAST Data Universities Research Facilities Data Autoscaling Instances APAC More Data Sources Private Suppliers or Corporates Multiple Sources of Data sophisticated weighting scheme Weather Company’s Next Generation Forecast Platform Weather.com + Weather Channel, etc. Media Retail Agriculture Insurance Aviation Energy Trading Ground Transportation Marine Utilities Emergency Response Construction Outdoor Recreation Military Construction Travel & Hospitality B2B Customers 32
  • 33. Strategy #2 Augment On-Premises IT Resources with Cloud Capacity Use Cloud to Make On-Premises Apps Better 33
  • 34. Strategy #2 Augment On-Premises IT Resources with Cloud Capacity Use Cloud to Make On-Premises Apps Better Build Cloud Apps that Integrate with On-Premises Apps 34
  • 35. Strategy 2: Cloud Apps that Integrate with On-Premises Apps AWS serves application content & data Integration to Samsung data centers for Authentication and Financial Transactions Hybrid App On-Premises Data Centers Saved $34 Million 85% savings versus traditional hosting
  • 36. Hybrid App: TWC Forecast Platform Integrates with Nationally Distributed Cable Network Legacy Star Distribution August 2012 Weather Forecast Platform Hybrid App integrates back to STARs TV Network National Coverage Star Legacy Area Served - August 2012 4000 Junior XL On-Premises Data Centers Local on The 8s 4000+ 36 STAR Boxes in US
  • 37. Strategy Migrate Existing Apps & Data to the Cloud 37
  • 38. Strategy 3: Migrate Existing Apps and Data to the Cloud 1/3 of servers migrated to AWS App On-Premises Data Centers Migrated SAP and SharePoint Workloads Saved £1.5 Million Saved $1M over 3 years 50% of Traditional hosting costs Migrated clinical trials simulations platform Simulations in 1.2hrs vs. 60hrs 64% reduction in costs
  • 39. Our First Real AWS Project : Migrate real-time radar mapping system wunderground.com to AWS Cloud Wunderground Radar and Maps 100 million hits a day One Billion data points per day
  • 40. 30,000 Personal Weather Stations Source: Wunderground, Inc. 2013
  • 41. Before Migration – Traditional IT Model doesn’t scale well Server Count Avg. CPU Load (110 Servers) HTTP Response Latency (~6000 ms) After Migration - Wunderground Radar App Server Count (from 110 to 170 Instances) Avg. CPU Load HTTP Response Latency (5-15ms)
  • 42. Strategy All-in Cloud 42
  • 43. Strategy 4: All-In 100s of applications supporting 33M+ global members 10,000s of EC2 instances in multiple regions & zones At peak consumes 1/3 of US Internet bandwidth
  • 44. is All-in TWC’s transformation into a large scale, global, big data platform for the world’s weather is founded on a All-In IaaS strategy.
  • 45. Strategy 4: TWC’s All-In Cloud Strategy We are using all 3 strategies simultaneously across the enterprise to get us there lightning fast! Strategy #1 Strategy #2 Strategy #3 What we are gaining with the cloud:  Transfer TB’s of weather data every day globally  Manage TB’s of digital content globally  Handle 1 TB/sec at peak  Handle 1 Billion requests per pay  Increase production changes by 20X  Decrease hosting costs by 30%+ 45
  • 46. Key Takeaway: Cloud isn’t an ‘All or Nothing’ Choice Strategy #1 Strategy #2 Strategy #3 Strategy #4 Build All New Apps for the Cloud Augment On-Premises IT resources with Cloud Capacity Migrate Existing Apps and Data to the Cloud All-in
  • 47. Thank You! Jinesh Varia Technology Evangelist, AWS jvaria@amazon.com @jinman Bryson Koehler EVP, CIO, The Weather Company bryson@weather.com @brysonkoehler 47