• Save
AWS Customer Presentation - Mediafed
Upcoming SlideShare
Loading in...5

AWS Customer Presentation - Mediafed



Alan Williamson, Co-Founder of Mediafed presents at AWS Start-Up Event - London.

Alan Williamson, Co-Founder of Mediafed presents at AWS Start-Up Event - London.



Total Views
Views on SlideShare
Embed Views



6 Embeds 10

http://www.visualcv.com 3
http://www.slideshare.net 3 1
http://b09-ingelesa.blogspot.com 1
http://www.cemrun.com 1
http://staging.visualcv.com 1



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    AWS Customer Presentation - Mediafed AWS Customer Presentation - Mediafed Presentation Transcript

    • Amazon AWS & Mediafed Alan Williamson Director, MediaFed ltd Co-Founder, aw2.0 ltd the cloud experts
    • MediaFed Ltd
      • RSS Experts
      • Monetize feeds
      • Track feeds
      • Value Added feeds
      • Feed Compliance
      • Focused on Premium publishers
    • MediaFed’s work flow MediaFed Publishers MediaFed Services Millions of RSS Clients
    • Traditional Setup
      • Fixed number of ‘real’ strokeable servers
      • Professionally managed servers
      • Costing £xxx’s per month just for one server
    • Then things got interesting Started to grow FAST! http://www.flickr.com/photos/lunadirimmel /
    • Where did MediaFed need to scale?
      • Front end Web Servers
        • Servicing the requests for feeds, article views, and click thru’s
      • RSS Ad Servers
        • Processing the ad insertion’s and click-thru’s
      • Feed Crawlers
        • Retrieving the feed content, cleaning and conversion
      • Database
        • Storage of feed data, and statistical analysis, running to TB’s of data
      • The current hosting company was asked for a quote
      • It came as quite a shock to us
      • We needed servers FAST
      • Time Quoted 10 working days!
      • Need a modern approach
    • http://www.flickr.com/photos/nanimo/ THE BIG FLUFFY CLOUD
    • Which Cloud Provider …
      • … had experience with a number of cloud providers
      • … each one with their strengths and weaknesses
      • … different price/performance points
      • But …
    • It’s “Hug Amazon” day! http://www.flickr.com/photos/devcentre/74685966/
    • Amazon’s EC2 HOT features
      • Instance start up parameter passing
      • Instance snap shot
      • VERY FAST instance startup
      • Large ‘ free ’ disk space with each instance
    • Some EC2 problems …
      • No VLAN support
      • No ability to turn off public IP address
      • No reverse DNS
    • .. and instances do go down! But “ssssh” you didn’t hear it from me! http://www.flickr.com/photos/59953599@N00/2852302091/
    • Our Amazon Family Usage
      • Amazon EC2 usage
        • Runs the main infrastructure; the bread’n’butter
      • SimpleDB
        • Used for something quite special
      • S3
        • Long term storage of log files and DB snapshots
    • Usage of EC2 Instances
      • The majority of all our services are Java based
        • High performance custom built code; no J2EE!
      • Load balancing provided by nginx
      • Database provided by MySQL 5
    • Provisioning of instances
      • Manage our deployment as a single ‘stack’
        • /opt/mediafed/
      • All applications, scripts, binaries in there
      • Require no external, pre-installed software
      • Complete stack is held within SVN
        • First thing that happens when an instance boots up is to update itself with the latest from SVN
    • Keep it Simple
      • No multiple instances to worry about
      • Any instance can do any job in the system we require
      • Horrendously transportable!
      • Instances are ready to work as soon as they are powered up
    • Our goals …
      • Over reaching goal was to make it an SSH free zone
      • Misbehaving instances would simply be shut off and a new one powered up
      • We viewed the “instance” as a component that could be used; not as a server that it really was
    • Starting up …
      • From the time we issue the command to start up to it actually doing REAL work
        • Usually 3 to 4 minutes!!!!
    • EC2 #1 Killer Feature
      • Parameter passing into EC2 instances
      • We pass in a configuration string on startup that specifies what it will do:
        • ec2-xxxx -d “TYPE=WEB,ROLE=RSSPOOL3”
      • Instance runs a ‘boot script’ that then runs up the necessary service we require
    • But others services in the system need to know about the new instances ????? http://www.flickr.com/photos/yourdon/2770955000/
    • SimpleDB!
      • We use SimpleDB as a sophisticated internal ‘DNS’ or service directory
      • When a service is started, it registers its role with SimpleDB including all IP, Port and other information
      • Other services keep an eye on changes in this configuration and automatically update their own configuration files accordingly
    • Amazon SimpleDB as DNS
      • Allow us to add in new web servers into the farm for load balancing without doing a thing except start it up
      • Add in additional MySQL slaves/masters without touching anything
      • And of course, we can remove and shutdown an instance as easily
      • Big advantage, we can use the same SimpleDB technique for other cloud providers too!
    • Amazon is cheap!
      • Doubling up instances for redundancy is very cheap to do
      • Mediafed is running an on-demand system for a fraction of the cost it would have cost to run a traditional server room
      • Turn off servers at low points in the week, no cost, no hassle!
    • No more hardware hassles!
      • The previous hosting company for Mediafed DID NOT recover from a hard disk failure
      • Fortunately Mediafed had a back up
    • Some cloud advice … http://www.flickr.com/photos/digitalsextant/34811579
    • Not a cloud in sight ! Forget the marketing hype - Cloud Providers do go down http://www.flickr.com/photos/montanaraven/
    • … and no two providers are the same NO CLOUD STANDARD http://www.flickr.com/photos/heather-dietz/227295326/
    • So can you move the whole house? http://www.flickr.com/photos/seanlloyd/24764719/
    • Don’t buy into the hype
      • All the usual ‘out sourcing’ rules apply
        • Always look at the worse case scenario
        • Plan for the worse case scenario
        • Assume the worse case scenario will happen
        • Practice the worse case scenario
    • Remember … you can’t just nip down and reboot the server http://www.flickr.com/photos/johnniewalker/
    • But take a moment and …
      • Think of your servers as single use devices instead of the typical “LAMP”
      • Managing up to 10 servers via SSH isn’t a problem
        • is doing the same for 10, 20, 50+ servers really practical?
      • Hold off on that purchase order to Dell and talk to your software guys to see if you can re-jig current things to work within the ‘cloud platform’, or Linux-OnDemand
    • www.aw20.co.uk [email_address] UK +44 1556 568 103 / US +1 650 331 9758 the cloud experts – putting it to work