Running in the Cloud - First Belgian Azure project

Running in the CloudChronoRace
Maarten Balliauw – RealDolmenE-mail: maarten.balliauw@realdolmen.comBlog: http://blog.maartenballiauw.beTwitter: @maartenballiauw

Who am I?
Maarten Balliauw
Antwerp, Belgium
www.realdolmen.com
Focus on web
ASP.NET, ASP.NET MVC, PHP, Azure, VSTS, …
MVP ASP.NET
http://blog.maartenballiauw.be
http://twitter.com/maartenballiauw

Agenda
Cloud computing?
Windows Azure?
The ChronoRace case
ChronoRace?
Infrastructure
The project
Resources
Q&A

What’s in the cloud?
Cloud Computing

Cloud computing
On-demand self service
Standardized access
Location independant resource pooling
Rapid elasticity
Pay per use

IT as a Service
Private
(On-Premise)
Infrastructure
(as a Service)
Platform
(as a Service)
You manage
Applications
Applications
Applications
You manage
Runtimes
Runtimes
Runtimes
Security & Integration
Managed by vendor
You manage
Databases
Databases
Databases
Servers
Servers
Servers
Managed by vendor
Virtualization
Virtualization
Virtualization
Server HW
Server HW
Server HW
Storage
Storage
Storage
Networking
Networking
Networking

Windows Azure

The Windows Azure Platform
Windows Azure
SQL Azure
Windows Azure platform AppFabric
Microsoft Codename “Dallas”
Microsoft Codename “Sydney”

Windows Azure
Flexible application hosting
Lights-out service management
Provide code & service model, hit ENTER
Storage at massive scale
Blobs, tables, queues
Compute
Storage
Management

Windows Azure

Storage options
Blobs, tables, queues
Windows Azure Drive (a.k.a. XDrive)
Virtual NTFS volume that can be mounted
.vhd format
Use existing NTFS API’s
Easier migration
Stored on blob storage provides quick mount/unmount in other VM

SQL Azure
Relational database as a service
Highly available, automatically maintained
Extension of the SQL Server Data Platform
Reporting
Data Sync
Business Analytics
Database

Windows Azure AppFabric
Secure connections between services
Across organizational boundaries
Claims-based access control
ServiceBus
AccessControl

Service Bus Overview

Microsoft Codename “Dallas”
Content brokerage and discovery platform
Available as a CTP
Microsoft Codename “Dallas”Information Services

Microsoft Codename “Sydney” - Connectivity
WINDOWS AZURE PLATFORM
ENTERPRISE
Data Service
Service Bus
Access Control Service
Code Name “Project Sydney”

North America Region
Asia Pacific Region
Europe Region
N. Europe
Sub-region
N. Central – US
Sub-region
E. Asia
Sub-region
W. Europe
Sub-region
S. Central - US
Sub-region
S.E. Asia
Sub-region

Running in the Cloud
The ChronoRace case

ChronoRace

ChronoRace
Delivering infrastructure for timing sports events
Running, biking, triathlon, mountainbike, …
Founded 10 years ago, Malmédy, 4 persons
Timing of every participant
Video footage of every participant
PDF certificates for every participant
Around 340 events per year!
Belgium, Luxemburg, France, Germany, Netherlands, …
Small events
Large events
Dwars door Brugge 8.000 participants
Antwerp 10 Miles 17.000 participants
20km of Brussels 30.000 participants

Infrastructure
The ChronoRace case

Current infrastructure

Google Analytics from Antwerp 10 Miles 2010

Big event…
20km of Brussels upcoming! (30/05/2010)
Expected 30.000 participants
Downloading their certificate 30.000 hits processing a PDF
Watching results online 30.000 hits*
Having a video online 30.000 x +/- 10MB (= 292 GB)
Watching their video 30.000 video views
And showing off to family and friends… 30.000 x 10MB x 10 views (avg.) = 2,92 TB bandwidth in 1 day
About 35 Mbps
Over a 10 Mbps line…
But there’s an option to a 100 Mbps line!

How to overcome these predictable peaks?
Make videos only available after a day or 2
Many hits to the results page can be handled
Videos online during week: traffic spread
Invest in streaming?
Costly, big capacity only needed for +/- 7 events per year
The videos are not the only problem

How to overcome these predictable peaks?
A better WAN connection?
Costly
Currently the server can handle all traffic because of the 10 Mbps cap
Buy additional hardware?
Costly
Estimated use of additional hardware is +/- 7 events per year
14 days of required uptime
342 days of money burning

“Growing Fast“
“On and Off “
Inactivity
Period
Compute
Compute
Average Usage
Usage
Average
Time
Time
On & off workloads (e.g. batch job)
Over provisioned capacity is wasted
Time to market can be cumbersome
Successful services needs to grow/scale
Keeping up w/ growth is big IT challenge
Complex lead time for deployment
“Unpredictable Bursting“
“Predictable Bursting“
Compute
Compute
Average Usage
Average Usage
Time
Time
Unexpected/unplanned peak in demand
Sudden spike impacts performance
Can’t over provision for extreme cases
Services with micro seasonality trends
Peaks due to periodic increased demand
IT complexity and wasted capacity
Workloadpatternsoptimalforcloud

The Project
The ChronoRace case

Where did it start…
Microsoft salesguy is a runner
Annoyed by the slow / failing website
Decided to give a call
ChronoRace interested in the cloud story
RealDolmen chosen as partner
Already investing heavily in Windows Azure for 1,5 year
Mostly building PoC’s to see what fits / what not
Experience

Step 1: Identify the possible use case for cloud
Obviously: peaks, mostly predicted

Step 2: Identify possible pitfalls
Current website
VS2003 Web Site
Database
1 GB or 10 GB?
PDF generation
32 bit component (Azure is 64 bit)
License key requires registry access
Streaming
Possible from Azure?
Directly from storage?
Scaling
Not automatic on Azure
Deployment
Site is +/- 300 MB, deploy this every time?
Pricing
Is this all worth the migration?

Current web site
VS2003 Web Site
Should be easy!
File -> New -> Cloud
Add a web application
Copy the VS2003 web site in
Right click -> Convert to web application
Done!

Current web site
VS2003 Web Site in a VS2010 Web Application
Would not compile (no namespaces)
Migration took some time
Add namespace
Restructure classes
Throw out stuff that was no longer used
Actually migration allowed us to get a cleaner application

Database
Current database size: around 2,5 GB
Squeeze stuff into 1 GB?
Data retention
Restructure database?

Database
Decided to go for 10 GB version
Migration
Using open-source: SQL Azure Migration Wizardhttp://sqlazuremw.codeplex.com
Worked surprisingly good!
2 stored procedures could not be migrated
Test migration took 7 hours initially!
Don’t migrate on your home connection 

Database
Current website at www.chronorace.be runs on SQL Azure
As easy as changing the connection string!
After installing SQL Server 2008 R2 client
After installing SQL Server 2008 R2 management objects

Database
Backups?
Do you NEED backups on Azure anyway???
Not supported in current version, backups manually via SQL Azure Migration Wizard to local SQL Server
One of the upcoming SQL Azure versions will support database backups

PDF generation
Using “WebSuperGoo ABCpdf”
Requires registry access for the license key
Not allowed on Azure
We built a small PoC in Development Fabric which worked!
Deployment to production learned that this was false optimism 
Current version: 32 bits
Azure: 64 bits
We tried to run the non-licensed version on Azure
Did not work: author has disabled 64 bits support

PDF generation
64 bits version…
Decided to keep this on the current web server for now
Migrate to an open-source PDF component in the future

Streaming
Currently a Flash player
Downloads chunks of +/- 1 MB
Should work on blob storage!
Cheap storage
Cheap traffic
CDN support
API supports downloading chunks

Streaming
Getting a chunck took a few seconds for each video…
What if a participant watches the video 10 times?
Decided to cache the video to compute
During the advert that is shown
Next view of the video is instant!
Demo: http://bit.ly/d8koz6

Scaling
Done by doing this:
Would be awesome to have this:
Not possible!
“When” should it scale?
“How” should it scale?
“Who” / “What” is responsible for scaling?
<Instancescount="2" />
<InstancesminInstances="3" maxInstances="10" />

Auto-Scaling
Auto scaling would be a big benefit
During events all 4 people are busy
It is the cloud after all? The scale-up/down thing?
We decided to build something for that
Application should be able to monitor itself
Only one instance should be responsible for that

Auto-Scaling – Scaling logic
Scaling logic provider uses sensor data to suggest an action (up/fast-up/down/stable)
To implement per application
Just a suggestion!
Scaling logic provider uses sensor data to suggest an action (up/fast-up/down/stable)
To implement per application
Just a suggestion!

Auto-Scaling
Uses all the standard API’s of Windows Azure
Master election  Role Environment
Performance counters  Diagnostics API
Queue length  Storage API
Scaling (changing # instances in config)  Management API

Auto-Scaling in ChronoRace
Sensor based on concurrent video streams
Scales between 2 and 20 instances automatically
Hard to verify! The Windows Azure portal is not willing to disclose the number of instances during scale.
MMC Snap-In does give useful info!http://code.msdn.microsoft.com/windowsazuremmc

Deployment
Site is +/- 300 MB, deploy this every time?
Deployment through portal seems to take forever
Uploading 300 MB for a very small change is crazy
We decided to move images and static content to blob storage and serve it from there
Code and dynamic content are on Windows Azure. Making it a 3,5 MB application.

Deployment
Deployment can be done in 2 ways
“Upgrade”
“VIP swap”
Website will mostly run on 1 or 2 instances
Case 1  upgrade will bring the site down!
Process: use VIP swap

Pricing – Current pricing
Server lease for 2 servers
Including 10 Mbps bandwidth
+/- 450 EUR / month

Pricing – Windows Azure
Storage
30 GB required x $ 0,15 = $ 4,50 / month
SQL Azure
10 GB database x $ 99,99 = $ 99,99 / month

Compute
On average 1 instance = $ 43,20 / month
Worst case 15 days 3 instances = $ 129,60 / month
Traffic
3 TB x $ 0,15 / GB = $ 450,00 / month*
*will probably be less

Estimated worst-case scenario would cost:
Current setup: 450 EUR / month
Windows Azure worst-case: 590 EUR / month

Summary
The ChronoRace case

ChronoRace summary
Pitfalls were identified and mostly tackled
Except for the PDF generation
Most hours spent with VS2003 web site to VS2010 web app
Database migration
One test run of 7 hours (unattended)
Final migration took 1 hour
Development using Windows Azure Storage API
Max. 1 day spent
Website on Azure will be put to test next weekend:
20 km of Brussels is D-Day
Confident that it will not explode this time

Resources
Where do I find more info?

Resources
Windows Azurehttp://www.azure.com
Steve Marxhttp://blog.smarx.com/
PDChttp://microsoftpdc.com
My bloghttp://blog.maartenballiauw.be/category/Azure.aspx
RealDolmen blogs is running on Azure!http://www.realdolmenblogs.com
RealDolmen will be happy to work with you in identifying if cloud computing is beneficial to your applications.
Stop by our booth for a chat!

AZUG.BE – Azure User Group Belgium
Focus on the Azure Services Platform
Community-driven
Focus on architecture
And development
And the business value of Azure
www.azug.be

Questions
Possibly Answers
Q&A Session also later today,check your session schedule
or stop by our booth.

Maarten Balliauwhttp://blog.maartenballiauw.be
http://twitter.com/maartenballiauw
THANK YOU!

Running in the Cloud - First Belgian Azure project

by Maarten Balliauw on May 26, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

1 Embed 23

Statistics

Running in the Cloud - First Belgian Azure project — Presentation Transcript