Running in the Cloud - First Belgian Azure project

  • 1,742 views
Uploaded on

 

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,742
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
0
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • Datacenter pairing!

Transcript

  • 1.
  • 2. Running in the CloudChronoRace
    Maarten Balliauw – RealDolmenE-mail: maarten.balliauw@realdolmen.comBlog: http://blog.maartenballiauw.beTwitter: @maartenballiauw
  • 3. Who am I?
    Maarten Balliauw
    Antwerp, Belgium
    www.realdolmen.com
    Focus on web
    ASP.NET, ASP.NET MVC, PHP, Azure, VSTS, …
    MVP ASP.NET
    http://blog.maartenballiauw.be
    http://twitter.com/maartenballiauw
  • 4.
  • 5. Agenda
    Cloud computing?
    Windows Azure?
    The ChronoRace case
    ChronoRace?
    Infrastructure
    The project
    Resources
    Q&A
  • 6. What’s in the cloud?
    Cloud Computing
  • 7. Cloud computing
    On-demand self service
    Standardized access
    Location independant resource pooling
    Rapid elasticity
    Pay per use
  • 8. IT as a Service
    Private
    (On-Premise)
    Infrastructure
    (as a Service)
    Platform
    (as a Service)
    You manage
    Applications
    Applications
    Applications
    You manage
    Runtimes
    Runtimes
    Runtimes
    Security & Integration
    Security & Integration
    Security & Integration
    Managed by vendor
    You manage
    Databases
    Databases
    Databases
    Servers
    Servers
    Servers
    Managed by vendor
    Virtualization
    Virtualization
    Virtualization
    Server HW
    Server HW
    Server HW
    Storage
    Storage
    Storage
    Networking
    Networking
    Networking
  • 9. What’s in the cloud?
    Windows Azure
  • 10. The Windows Azure Platform
    Windows Azure
    SQL Azure
    Windows Azure platform AppFabric
    Microsoft Codename “Dallas”
    Microsoft Codename “Sydney”
  • 11. Windows Azure
    Flexible application hosting
    Lights-out service management
    Provide code & service model, hit ENTER
    Storage at massive scale
    Blobs, tables, queues
    Compute
    Storage
    Management
  • 12. Windows Azure
  • 13. Storage options
    Blobs, tables, queues
    Windows Azure Drive (a.k.a. XDrive)
    Virtual NTFS volume that can be mounted
    .vhd format
    Use existing NTFS API’s
    Easier migration
    Stored on blob storage provides quick mount/unmount in other VM
  • 14. SQL Azure
    Relational database as a service
    Highly available, automatically maintained
    Extension of the SQL Server Data Platform
    Reporting
    Data Sync
    Business Analytics
    Database
  • 15. Windows Azure AppFabric
    Secure connections between services
    Across organizational boundaries
    Claims-based access control
    ServiceBus
    AccessControl
  • 16. Service Bus Overview
  • 17. Microsoft Codename “Dallas”
    Content brokerage and discovery platform
    Available as a CTP
    Microsoft Codename “Dallas”Information Services
  • 18. Microsoft Codename “Sydney” - Connectivity
    WINDOWS AZURE PLATFORM
    ENTERPRISE
    Data Service
    Service Bus
    Access Control Service
    Code Name “Project Sydney”
  • 19. North America Region
    Asia Pacific Region
    Europe Region
    N. Europe
    Sub-region
    N. Central – US
    Sub-region
    E. Asia
    Sub-region
    W. Europe
    Sub-region
    S. Central - US
    Sub-region
    S.E. Asia
    Sub-region
  • 20.
  • 21.
  • 22.
  • 23.
  • 24. What’s in the cloud?
    Running in the Cloud
    The ChronoRace case
  • 25. ChronoRace
  • 26. ChronoRace
    Delivering infrastructure for timing sports events
    Running, biking, triathlon, mountainbike, …
    Founded 10 years ago, Malmédy, 4 persons
    Timing of every participant
    Video footage of every participant
    PDF certificates for every participant
    Around 340 events per year!
    Belgium, Luxemburg, France, Germany, Netherlands, …
    Small events
    Large events
    Dwars door Brugge 8.000 participants
    Antwerp 10 Miles 17.000 participants
    20km of Brussels 30.000 participants
  • 27. What’s in the cloud?
    Infrastructure
    The ChronoRace case
  • 28. Current infrastructure
  • 29. Google Analytics from Antwerp 10 Miles 2010
  • 30. Big event…
    20km of Brussels upcoming! (30/05/2010)
    Expected 30.000 participants
    Downloading their certificate 30.000 hits processing a PDF
    Watching results online 30.000 hits*
    Having a video online 30.000 x +/- 10MB (= 292 GB)
    Watching their video 30.000 video views
    And showing off to family and friends… 30.000 x 10MB x 10 views (avg.) = 2,92 TB bandwidth in 1 day
    About 35 Mbps
    Over a 10 Mbps line…
    But there’s an option to a 100 Mbps line!
  • 31.
  • 32. How to overcome these predictable peaks?
    Make videos only available after a day or 2
    Many hits to the results page can be handled
    Videos online during week: traffic spread
    Invest in streaming?
    Costly, big capacity only needed for +/- 7 events per year
    The videos are not the only problem
  • 33. How to overcome these predictable peaks?
    A better WAN connection?
    Costly
    Currently the server can handle all traffic because of the 10 Mbps cap
    Buy additional hardware?
    Costly
    Estimated use of additional hardware is +/- 7 events per year
    14 days of required uptime
    342 days of money burning
  • 34. “Growing Fast“
    “On and Off “
    Inactivity
    Period
    Compute
    Compute
    Average Usage
    Usage
    Average
    Time
    Time
    • On & off workloads (e.g. batch job)
    • 35. Over provisioned capacity is wasted
    • 36. Time to market can be cumbersome
    • 37. Successful services needs to grow/scale
    • 38. Keeping up w/ growth is big IT challenge
    • 39. Complex lead time for deployment
    “Unpredictable Bursting“
    “Predictable Bursting“
    Compute
    Compute
    Average Usage
    Average Usage
    Time
    Time
    • Unexpected/unplanned peak in demand
    • 40. Sudden spike impacts performance
    • 41. Can’t over provision for extreme cases
    • 42. Services with micro seasonality trends
    • 43. Peaks due to periodic increased demand
    • 44. IT complexity and wasted capacity
    Workloadpatternsoptimalforcloud
  • 45. What’s in the cloud?
    The Project
    The ChronoRace case
  • 46. Where did it start…
    Microsoft salesguy is a runner
    Annoyed by the slow / failing website
    Decided to give a call
    ChronoRace interested in the cloud story
    RealDolmen chosen as partner
    Already investing heavily in Windows Azure for 1,5 year
    Mostly building PoC’s to see what fits / what not
    Experience
  • 47. Step 1: Identify the possible use case for cloud
    Obviously: peaks, mostly predicted
  • 48. Step 2: Identify possible pitfalls
    Current website
    VS2003 Web Site
    Database
    1 GB or 10 GB?
    PDF generation
    32 bit component (Azure is 64 bit)
    License key requires registry access
    Streaming
    Possible from Azure?
    Directly from storage?
    Scaling
    Not automatic on Azure
    Deployment
    Site is +/- 300 MB, deploy this every time?
    Pricing
    Is this all worth the migration?
  • 49. Current web site
    VS2003 Web Site
    Should be easy!
    File -> New -> Cloud
    Add a web application
    Copy the VS2003 web site in
    Right click -> Convert to web application
    Done!
  • 50.
  • 51. Current web site
    VS2003 Web Site in a VS2010 Web Application
    Would not compile (no namespaces)
    Migration took some time
    Add namespace
    Restructure classes
    Throw out stuff that was no longer used
    Actually migration allowed us to get a cleaner application
  • 52. Database
    Current database size: around 2,5 GB
    Squeeze stuff into 1 GB?
    Data retention
    Restructure database?
  • 53. Database
    Decided to go for 10 GB version
    Migration
    Using open-source: SQL Azure Migration Wizardhttp://sqlazuremw.codeplex.com
    Worked surprisingly good!
    2 stored procedures could not be migrated
    Test migration took 7 hours initially!
    Don’t migrate on your home connection 
  • 54. Database
    Current website at www.chronorace.be runs on SQL Azure
    As easy as changing the connection string!
    After installing SQL Server 2008 R2 client
    After installing SQL Server 2008 R2 management objects
  • 55. Database
    Backups?
    Do you NEED backups on Azure anyway???
    Not supported in current version, backups manually via SQL Azure Migration Wizard to local SQL Server
    One of the upcoming SQL Azure versions will support database backups
  • 56. PDF generation
    Using “WebSuperGoo ABCpdf”
    Requires registry access for the license key
    Not allowed on Azure
    We built a small PoC in Development Fabric which worked!
    Deployment to production learned that this was false optimism 
    Current version: 32 bits
    Azure: 64 bits
    We tried to run the non-licensed version on Azure
    Did not work: author has disabled 64 bits support
  • 57. PDF generation
    64 bits version…
    Decided to keep this on the current web server for now
    Migrate to an open-source PDF component in the future
  • 58. Streaming
    Currently a Flash player
    Downloads chunks of +/- 1 MB
    Should work on blob storage!
    Cheap storage
    Cheap traffic
    CDN support
    API supports downloading chunks
  • 59. Streaming
    Getting a chunck took a few seconds for each video…
    What if a participant watches the video 10 times?
    Decided to cache the video to compute
    During the advert that is shown
    Next view of the video is instant!
    Demo: http://bit.ly/d8koz6
  • 60. Scaling
    Done by doing this:
    Would be awesome to have this:
    Not possible!
    “When” should it scale?
    “How” should it scale?
    “Who” / “What” is responsible for scaling?
    <Instancescount="2" />
    <InstancesminInstances="3" maxInstances="10" />
  • 61. Auto-Scaling
    Auto scaling would be a big benefit
    During events all 4 people are busy
    It is the cloud after all? The scale-up/down thing?
    We decided to build something for that
    Application should be able to monitor itself
    Only one instance should be responsible for that
  • 62. Auto-Scaling – Scaling logic
    Scaling logic provider uses sensor data to suggest an action (up/fast-up/down/stable)
    To implement per application
    Just a suggestion!
    Scaling logic provider uses sensor data to suggest an action (up/fast-up/down/stable)
    To implement per application
    Just a suggestion!
  • 63. Auto-Scaling
    Uses all the standard API’s of Windows Azure
    Master election  Role Environment
    Performance counters  Diagnostics API
    Queue length  Storage API
    Scaling (changing # instances in config)  Management API
  • 64. Auto-Scaling in ChronoRace
    Sensor based on concurrent video streams
    Scales between 2 and 20 instances automatically
    Hard to verify! The Windows Azure portal is not willing to disclose the number of instances during scale.
    MMC Snap-In does give useful info!http://code.msdn.microsoft.com/windowsazuremmc
  • 65. Deployment
    Site is +/- 300 MB, deploy this every time?
    Deployment through portal seems to take forever
    Uploading 300 MB for a very small change is crazy
    We decided to move images and static content to blob storage and serve it from there
    Code and dynamic content are on Windows Azure. Making it a 3,5 MB application.
  • 66. Deployment
    Deployment can be done in 2 ways
    “Upgrade”
    “VIP swap”
    Website will mostly run on 1 or 2 instances
    Case 1  upgrade will bring the site down!
    Process: use VIP swap
  • 67. Pricing – Current pricing
    Server lease for 2 servers
    Including 10 Mbps bandwidth
    +/- 450 EUR / month
  • 68. Pricing – Windows Azure
    Storage
    30 GB required x $ 0,15 = $ 4,50 / month
    SQL Azure
    10 GB database x $ 99,99 = $ 99,99 / month
  • 69. Pricing – Windows Azure
    Compute
    On average 1 instance = $ 43,20 / month
    Worst case 15 days 3 instances = $ 129,60 / month
    Traffic
    3 TB x $ 0,15 / GB = $ 450,00 / month*
    *will probably be less
  • 70. Pricing – Windows Azure
    Estimated worst-case scenario would cost:
    Current setup: 450 EUR / month
    Windows Azure worst-case: 590 EUR / month
  • 71. What’s in the cloud?
    Summary
    The ChronoRace case
  • 72. ChronoRace summary
    Pitfalls were identified and mostly tackled
    Except for the PDF generation
    Most hours spent with VS2003 web site to VS2010 web app
    Database migration
    One test run of 7 hours (unattended)
    Final migration took 1 hour
    Development using Windows Azure Storage API
    Max. 1 day spent
    Website on Azure will be put to test next weekend:
    20 km of Brussels is D-Day
    Confident that it will not explode this time
  • 73. Resources
    Where do I find more info?
  • 74. Resources
    Windows Azurehttp://www.azure.com
    Steve Marxhttp://blog.smarx.com/
    PDChttp://microsoftpdc.com
    My bloghttp://blog.maartenballiauw.be/category/Azure.aspx
    RealDolmen blogs is running on Azure!http://www.realdolmenblogs.com
    RealDolmen will be happy to work with you in identifying if cloud computing is beneficial to your applications.
    Stop by our booth for a chat!
  • 75. AZUG.BE – Azure User Group Belgium
    Focus on the Azure Services Platform
    Community-driven
    Focus on architecture
    And development
    And the business value of Azure
    www.azug.be
  • 76. Questions
    Possibly Answers
    Q&A Session also later today,check your session schedule
    or stop by our booth.
  • 77. Maarten Balliauwhttp://blog.maartenballiauw.be
    http://twitter.com/maartenballiauw
    THANK YOU!