Clouds against
           the floods
           or how RoR saved the day in
           Australia




            Leonardo Borges


Monday, 7 November 2011
self slide
                          • ThoughtWorker in Sydney, Australia
                          • Programming languages enthusiast
                          • Blogger - leonardoborges.com
                          • Twitter blabbermouth @leonardo_borges
                          • Seldom OSS contributor - github.com/
                          leonardoborges




Monday, 7 November 2011
self slide
                                   • ThoughtWorker in Sydney, Australia
                                   • Programming languages enthusiast
                                   • Blogger - leonardoborges.com
                                   • Twitter blabbermouth @leonardo_borges
                                   • Seldom OSS contributor - github.com/
                                   leonardoborges

      ...and also
     • Heavy Metal guitarist
     • Singer
     • Overall adrenaline junkie
Monday, 7 November 2011
Let’s step back for a sec...



Monday, 7 November 2011
Heroku, an overview




Monday, 7 November 2011
Heroku, an overview
                          Basic concepts




Monday, 7 November 2011
Heroku, an overview
                          Basic concepts
                            • PaaS




Monday, 7 November 2011
Heroku, an overview
                          Basic concepts
                            • PaaS
                            • Git based deployment cycle




Monday, 7 November 2011
Heroku, an overview
                          Basic concepts
                            • PaaS
                            • Git based deployment cycle
                            • Dynos (web processes)




Monday, 7 November 2011
Heroku, an overview
                          Basic concepts
                            • PaaS
                            • Git based deployment cycle
                            • Dynos (web processes)
                            • Workers (background processes)




Monday, 7 November 2011
Heroku, an overview
                          Basic concepts
                            • PaaS
                            • Git based deployment cycle
                            • Dynos (web processes)
                            • Workers (background processes)
                            • Cron




Monday, 7 November 2011
Simplistic view of a web request
                                   request


                                   routing




                                    dyno




Monday, 7 November 2011
Simplistic view of a web request
                                               request


                                               routing




                          dyno   dyno   dyno    dyno     dyno   dyno   dyno


                          dyno   dyno   dyno    dyno     dyno   dyno   dyno




Monday, 7 November 2011
A typical setup




Monday, 7 November 2011
A typical setup
                          Environments




Monday, 7 November 2011
A typical setup
                          Environments
                            • staging, showcase and production




Monday, 7 November 2011
A typical setup
                          Environments
                            • staging, showcase and production




Monday, 7 November 2011
A typical setup
                          Environments
                            • staging, showcase and production




Monday, 7 November 2011
A typical setup
                          Environments
                            • staging, showcase and production

                          Branching strategy
                            • develop
                            • master (tagged release candidates)




Monday, 7 November 2011
But before you get too excited...



Monday, 7 November 2011
Catches & restrictions




Monday, 7 November 2011
Catches & restrictions
       • Read-only file system




Monday, 7 November 2011
Catches & restrictions
       • Read-only file system
       • 30 seconds cap on http requests




Monday, 7 November 2011
Catches & restrictions
       • Read-only file system
       • 30 seconds cap on http requests
       • The console hijacks the web
          processes




Monday, 7 November 2011
Catches & restrictions
       • Read-only file system
       • 30 seconds cap on http requests
       • The console hijacks the web
        processes
       • Processes limited to 300MB 512MB
        of RAM




Monday, 7 November 2011
Catches & restrictions
       • Read-only file system
       • 30 seconds cap on http requests
       • The console hijacks the web
        processes
       • Processes limited to 300MB 512MB
        of RAM
       • Busted IP based SSL solution


Monday, 7 November 2011
Catches & restrictions
       • Read-only file system
       • 30 seconds cap on http requests
       • The console hijacks the web
        processes
       • Processes limited to 300MB 512MB
        of RAM
       • Busted IP based SSL solution
       • ... and for Australia, latency

Monday, 7 November 2011
Troubleshooting




Monday, 7 November 2011
Troubleshooting
                          • NewRelic: essencial to spot
                           bottlenecks
                            • database
                            • external services - paypal,
                             memcached
                            • application
                          • Heroku logs



Monday, 7 November 2011
Performance




Monday, 7 November 2011
Performance
       • ORMs are dumb




Monday, 7 November 2011
Performance
       • ORMs are dumb
       • Offload stats from main DB




Monday, 7 November 2011
Performance
       • ORMs are dumb
       • Offload stats from main DB
       • Cache everything but not until you really
          need it




Monday, 7 November 2011
The new Heroku:
                          Celadon Cedar




Monday, 7 November 2011
The new Heroku:
                          Celadon Cedar
                          New features (most in public beta)




Monday, 7 November 2011
The new Heroku:
                          Celadon Cedar
                          New features (most in public beta)
                           • The procfile




Monday, 7 November 2011
The new Heroku:
                          Celadon Cedar
                          New features (most in public beta)
                           • The procfile
                           • One-off processes




Monday, 7 November 2011
The new Heroku:
                          Celadon Cedar
                          New features (most in public beta)
                           • The procfile
                           • One-off processes
                           • Node.js/Clojure/Java/Python...




Monday, 7 November 2011
The new Heroku:
                          Celadon Cedar
                          New features (most in public beta)
                           • The procfile
                           • One-off processes
                           • Node.js/Clojure/Java/Python...
                           • New HTTP stack




Monday, 7 November 2011
The new Heroku:
                          Celadon Cedar
                          New features (most in public beta)
                           • The procfile
                           • One-off processes
                           • Node.js/Clojure/Java/Python...
                           • New HTTP stack
                           • The foreman gem




Monday, 7 November 2011
The new Heroku:
                          Celadon Cedar
                          New features (most in public beta)
                           • The procfile
                           • One-off processes
                           • Node.js/Clojure/Java/Python...
                           • New HTTP stack
                           • The foreman gem
                           • Strive for purism




Monday, 7 November 2011
The new Heroku:
                          Celadon Cedar
                          New features (most in public beta)
                           • The procfile
                           • One-off processes
                           • Node.js/Clojure/Java/Python...
                           • New HTTP stack
                           • The foreman gem
                           • Strive for purism
                           • In the horizon (no ETA) : multi-region
                           availability



Monday, 7 November 2011
The Procfile




Monday, 7 November 2011
The Procfile
                          Run anything




Monday, 7 November 2011
The Procfile
                          Run anything
                          Rails

                          web:      bundle exec rails server -p $PORT
                          worker:   bundle exec rake jobs:work


                          Node.js

                          web:      node web.js


                          Clojure

                          web:      lein run -m demo.web

Monday, 7 November 2011
Multi-Region
                          Availability




Monday, 7 November 2011
Multi-Region
                          Availability
                          • Not here yet...




Monday, 7 November 2011
Multi-Region
                          Availability
                          • Not here yet...
                          • DIY cloud availability




Monday, 7 November 2011
Back in late 2010




Monday, 7 November 2011
Back in late 2010
                          • 75% of Queensland under water




Monday, 7 November 2011
Back in late 2010
                          • 75% of Queensland under water
                          • 35 dead / 9 missing




Monday, 7 November 2011
Back in late 2010
                          • 75% of Queensland under water
                          • 35 dead / 9 missing
                          • A$ 30 billion in damages




Monday, 7 November 2011
Appeal to the people




Monday, 7 November 2011
Appeal to the people
       • QLD Government to run a
          telethon on Channel Nine




Monday, 7 November 2011
Appeal to the people
       • QLD Government to run a
          telethon on Channel Nine

       • Dry run of existing system showed
          problems




Monday, 7 November 2011
Appeal to the people
       • QLD Government to run a
          telethon on Channel Nine

       • Dry run of existing system showed
          problems

       • Backup plan?


Monday, 7 November 2011
We can help!




Monday, 7 November 2011
We can help!
                          • ThoughtWorks offered help




Monday, 7 November 2011
We can help!
                          • ThoughtWorks offered help
                          • We were at the beach




Monday, 7 November 2011
We can help!
                          • ThoughtWorks offered help
                          • We were at the beach
                          • and had 48hs to deliver!




Monday, 7 November 2011
We can help!
                          • ThoughtWorks offered help
                          • We were at the beach
                          • and had 48hs to deliver!




Monday, 7 November 2011
What?!




Monday, 7 November 2011
What?!

       • Payment gateway integration




Monday, 7 November 2011
What?!

       • Payment gateway integration
       • Donation receipts by email
          (thousands of them)




Monday, 7 November 2011
What?!

       • Payment gateway integration
       • Donation receipts by email
          (thousands of them)

       • Ability to scale based on load


Monday, 7 November 2011
user
      What?!
                                                    varnish




                                                    app
       • Payment gateway integration
       • Donation receipts by email
          (thousands of them)

       • Ability to scale based on load   payment
                                                     DB
                                                                mail
                                          gateway             gateway



Monday, 7 November 2011
Challenges




Monday, 7 November 2011
Challenges
                          • SSQ’s infra-structure was a no go




Monday, 7 November 2011
Challenges
                          • SSQ’s infra-structure was a no go
                          • Had to come up with a solution for
                           email sending as well




Monday, 7 November 2011
Challenges
                          • SSQ’s infra-structure was a no go
                          • Had to come up with a solution for
                           email sending as well

                          • Skepticism




Monday, 7 November 2011
Choices




Monday, 7 November 2011
Choices

       • AWS




Monday, 7 November 2011
Choices

       • AWS
        - Cheaper but require ops




Monday, 7 November 2011
Choices

       • AWS
        - Cheaper but require ops
       • Heroku




Monday, 7 November 2011
Choices

       • AWS
        - Cheaper but require ops
       • Heroku
        - PaaS - no/less ops



Monday, 7 November 2011
Choices

       • AWS
        - Cheaper but require ops
       • Heroku
        - PaaS - no/less ops
        - As simple as a git push


Monday, 7 November 2011
Choices

       • AWS
        - Cheaper but require ops
       • Heroku
        - PaaS - no/less ops
        - As simple as a git push
        - Focus on your app

Monday, 7 November 2011
Choices

       • AWS
        - Cheaper but require ops
       • Heroku
        - PaaS - no/less ops
        - As simple as a git push
        - Focus on your app
        - Dozens of add-ons out of the box

Monday, 7 November 2011
Choices

       • AWS
        - Cheaper but require ops
       • Heroku
        - PaaS - no/less ops
        - As simple as a git push
        - Focus on your app
        - Dozens of add-ons out of the box
        - $0.00 upfront cost
Monday, 7 November 2011
Choices

       • AWS
        - Cheaper but require ops
       • Heroku
        - PaaS - no/less ops
        - As simple as a git push
        - Focus on your app
        - Dozens of add-ons out of the box
        - $0.00 upfront cost
Monday, 7 November 2011
One last ingredient...




                          Ben Barnard, Phillip Calçado and myself

Monday, 7 November 2011
One last ingredient...


                                                                    An awesome team, committed and
                                                                    having fun on a weekend :)




                          Ben Barnard, Phillip Calçado and myself

Monday, 7 November 2011
The telethon




Monday, 7 November 2011
The telethon

       • Live on channel 9 for 2 hours




Monday, 7 November 2011
The telethon

       • Live on channel 9 for 2 hours
       • SSQ’s app went down at the very
          start




Monday, 7 November 2011
The telethon

       • Live on channel 9 for 2 hours
       • SSQ’s app went down at the very
          start

       • The “backup” plan was now the
          main app

Monday, 7 November 2011
What we achieved




Monday, 7 November 2011
What we achieved
                          Despite all skepticism we...




Monday, 7 November 2011
What we achieved
                          Despite all skepticism we...

                          • Developed a production ready
                           donation app using Ruby on Rails




Monday, 7 November 2011
What we achieved
                          Despite all skepticism we...

                          • Developed a production ready
                           donation app using Ruby on Rails

                          • Leveraged PaaS to get ops out of
                           the way




Monday, 7 November 2011
What we achieved
                          Despite all skepticism we...

                          • Developed a production ready
                           donation app using Ruby on Rails

                          • Leveraged PaaS to get ops out of
                           the way

                          • Generated over A$ 2 million in
                           donations during the telethon



Monday, 7 November 2011
What we achieved
                          Despite all skepticism we...

                          • Developed a production ready
                           donation app using Ruby on Rails

                          • Leveraged PaaS to get ops out of
                           the way

                          • Generated over A$ 2 million in
                           donations during the telethon

                          • And over A$ 38 million since then

Monday, 7 November 2011
What we achieved
                          Despite all skepticism we...

                          • Developed a production ready
                           donation app using Ruby on Rails

                          • Leveraged PaaS to get ops out of
                           the way

                          • Generated over A$ 2 million in
                           donations during the telethon

                          • And over A$ 38 million since then
                                           ...all in a 48 hour effort
Monday, 7 November 2011
Anyone still thinks ‘the cloud’ isn’t for
                     serious stuff?


Monday, 7 November 2011
Thanks for listening!
                                  Questions?

                                    @leonardo_borges
                                  www.leonardoborges.com




            Leonardo Borges


Monday, 7 November 2011

Clouds against the Floods (RubyConfBR2011)

  • 1.
    Clouds against the floods or how RoR saved the day in Australia Leonardo Borges Monday, 7 November 2011
  • 2.
    self slide • ThoughtWorker in Sydney, Australia • Programming languages enthusiast • Blogger - leonardoborges.com • Twitter blabbermouth @leonardo_borges • Seldom OSS contributor - github.com/ leonardoborges Monday, 7 November 2011
  • 3.
    self slide • ThoughtWorker in Sydney, Australia • Programming languages enthusiast • Blogger - leonardoborges.com • Twitter blabbermouth @leonardo_borges • Seldom OSS contributor - github.com/ leonardoborges ...and also • Heavy Metal guitarist • Singer • Overall adrenaline junkie Monday, 7 November 2011
  • 4.
    Let’s step backfor a sec... Monday, 7 November 2011
  • 5.
  • 6.
    Heroku, an overview Basic concepts Monday, 7 November 2011
  • 7.
    Heroku, an overview Basic concepts • PaaS Monday, 7 November 2011
  • 8.
    Heroku, an overview Basic concepts • PaaS • Git based deployment cycle Monday, 7 November 2011
  • 9.
    Heroku, an overview Basic concepts • PaaS • Git based deployment cycle • Dynos (web processes) Monday, 7 November 2011
  • 10.
    Heroku, an overview Basic concepts • PaaS • Git based deployment cycle • Dynos (web processes) • Workers (background processes) Monday, 7 November 2011
  • 11.
    Heroku, an overview Basic concepts • PaaS • Git based deployment cycle • Dynos (web processes) • Workers (background processes) • Cron Monday, 7 November 2011
  • 12.
    Simplistic view ofa web request request routing dyno Monday, 7 November 2011
  • 13.
    Simplistic view ofa web request request routing dyno dyno dyno dyno dyno dyno dyno dyno dyno dyno dyno dyno dyno dyno Monday, 7 November 2011
  • 14.
    A typical setup Monday,7 November 2011
  • 15.
    A typical setup Environments Monday, 7 November 2011
  • 16.
    A typical setup Environments • staging, showcase and production Monday, 7 November 2011
  • 17.
    A typical setup Environments • staging, showcase and production Monday, 7 November 2011
  • 18.
    A typical setup Environments • staging, showcase and production Monday, 7 November 2011
  • 19.
    A typical setup Environments • staging, showcase and production Branching strategy • develop • master (tagged release candidates) Monday, 7 November 2011
  • 20.
    But before youget too excited... Monday, 7 November 2011
  • 21.
  • 22.
    Catches & restrictions • Read-only file system Monday, 7 November 2011
  • 23.
    Catches & restrictions • Read-only file system • 30 seconds cap on http requests Monday, 7 November 2011
  • 24.
    Catches & restrictions • Read-only file system • 30 seconds cap on http requests • The console hijacks the web processes Monday, 7 November 2011
  • 25.
    Catches & restrictions • Read-only file system • 30 seconds cap on http requests • The console hijacks the web processes • Processes limited to 300MB 512MB of RAM Monday, 7 November 2011
  • 26.
    Catches & restrictions • Read-only file system • 30 seconds cap on http requests • The console hijacks the web processes • Processes limited to 300MB 512MB of RAM • Busted IP based SSL solution Monday, 7 November 2011
  • 27.
    Catches & restrictions • Read-only file system • 30 seconds cap on http requests • The console hijacks the web processes • Processes limited to 300MB 512MB of RAM • Busted IP based SSL solution • ... and for Australia, latency Monday, 7 November 2011
  • 28.
  • 29.
    Troubleshooting • NewRelic: essencial to spot bottlenecks • database • external services - paypal, memcached • application • Heroku logs Monday, 7 November 2011
  • 30.
  • 31.
    Performance • ORMs are dumb Monday, 7 November 2011
  • 32.
    Performance • ORMs are dumb • Offload stats from main DB Monday, 7 November 2011
  • 33.
    Performance • ORMs are dumb • Offload stats from main DB • Cache everything but not until you really need it Monday, 7 November 2011
  • 34.
    The new Heroku: Celadon Cedar Monday, 7 November 2011
  • 35.
    The new Heroku: Celadon Cedar New features (most in public beta) Monday, 7 November 2011
  • 36.
    The new Heroku: Celadon Cedar New features (most in public beta) • The procfile Monday, 7 November 2011
  • 37.
    The new Heroku: Celadon Cedar New features (most in public beta) • The procfile • One-off processes Monday, 7 November 2011
  • 38.
    The new Heroku: Celadon Cedar New features (most in public beta) • The procfile • One-off processes • Node.js/Clojure/Java/Python... Monday, 7 November 2011
  • 39.
    The new Heroku: Celadon Cedar New features (most in public beta) • The procfile • One-off processes • Node.js/Clojure/Java/Python... • New HTTP stack Monday, 7 November 2011
  • 40.
    The new Heroku: Celadon Cedar New features (most in public beta) • The procfile • One-off processes • Node.js/Clojure/Java/Python... • New HTTP stack • The foreman gem Monday, 7 November 2011
  • 41.
    The new Heroku: Celadon Cedar New features (most in public beta) • The procfile • One-off processes • Node.js/Clojure/Java/Python... • New HTTP stack • The foreman gem • Strive for purism Monday, 7 November 2011
  • 42.
    The new Heroku: Celadon Cedar New features (most in public beta) • The procfile • One-off processes • Node.js/Clojure/Java/Python... • New HTTP stack • The foreman gem • Strive for purism • In the horizon (no ETA) : multi-region availability Monday, 7 November 2011
  • 43.
  • 44.
    The Procfile Run anything Monday, 7 November 2011
  • 45.
    The Procfile Run anything Rails web: bundle exec rails server -p $PORT worker: bundle exec rake jobs:work Node.js web: node web.js Clojure web: lein run -m demo.web Monday, 7 November 2011
  • 46.
    Multi-Region Availability Monday, 7 November 2011
  • 47.
    Multi-Region Availability • Not here yet... Monday, 7 November 2011
  • 48.
    Multi-Region Availability • Not here yet... • DIY cloud availability Monday, 7 November 2011
  • 49.
    Back in late2010 Monday, 7 November 2011
  • 50.
    Back in late2010 • 75% of Queensland under water Monday, 7 November 2011
  • 51.
    Back in late2010 • 75% of Queensland under water • 35 dead / 9 missing Monday, 7 November 2011
  • 52.
    Back in late2010 • 75% of Queensland under water • 35 dead / 9 missing • A$ 30 billion in damages Monday, 7 November 2011
  • 53.
    Appeal to thepeople Monday, 7 November 2011
  • 54.
    Appeal to thepeople • QLD Government to run a telethon on Channel Nine Monday, 7 November 2011
  • 55.
    Appeal to thepeople • QLD Government to run a telethon on Channel Nine • Dry run of existing system showed problems Monday, 7 November 2011
  • 56.
    Appeal to thepeople • QLD Government to run a telethon on Channel Nine • Dry run of existing system showed problems • Backup plan? Monday, 7 November 2011
  • 57.
    We can help! Monday,7 November 2011
  • 58.
    We can help! • ThoughtWorks offered help Monday, 7 November 2011
  • 59.
    We can help! • ThoughtWorks offered help • We were at the beach Monday, 7 November 2011
  • 60.
    We can help! • ThoughtWorks offered help • We were at the beach • and had 48hs to deliver! Monday, 7 November 2011
  • 61.
    We can help! • ThoughtWorks offered help • We were at the beach • and had 48hs to deliver! Monday, 7 November 2011
  • 62.
  • 63.
    What?! • Payment gateway integration Monday, 7 November 2011
  • 64.
    What?! • Payment gateway integration • Donation receipts by email (thousands of them) Monday, 7 November 2011
  • 65.
    What?! • Payment gateway integration • Donation receipts by email (thousands of them) • Ability to scale based on load Monday, 7 November 2011
  • 66.
    user What?! varnish app • Payment gateway integration • Donation receipts by email (thousands of them) • Ability to scale based on load payment DB mail gateway gateway Monday, 7 November 2011
  • 67.
  • 68.
    Challenges • SSQ’s infra-structure was a no go Monday, 7 November 2011
  • 69.
    Challenges • SSQ’s infra-structure was a no go • Had to come up with a solution for email sending as well Monday, 7 November 2011
  • 70.
    Challenges • SSQ’s infra-structure was a no go • Had to come up with a solution for email sending as well • Skepticism Monday, 7 November 2011
  • 71.
  • 72.
    Choices • AWS Monday, 7 November 2011
  • 73.
    Choices • AWS - Cheaper but require ops Monday, 7 November 2011
  • 74.
    Choices • AWS - Cheaper but require ops • Heroku Monday, 7 November 2011
  • 75.
    Choices • AWS - Cheaper but require ops • Heroku - PaaS - no/less ops Monday, 7 November 2011
  • 76.
    Choices • AWS - Cheaper but require ops • Heroku - PaaS - no/less ops - As simple as a git push Monday, 7 November 2011
  • 77.
    Choices • AWS - Cheaper but require ops • Heroku - PaaS - no/less ops - As simple as a git push - Focus on your app Monday, 7 November 2011
  • 78.
    Choices • AWS - Cheaper but require ops • Heroku - PaaS - no/less ops - As simple as a git push - Focus on your app - Dozens of add-ons out of the box Monday, 7 November 2011
  • 79.
    Choices • AWS - Cheaper but require ops • Heroku - PaaS - no/less ops - As simple as a git push - Focus on your app - Dozens of add-ons out of the box - $0.00 upfront cost Monday, 7 November 2011
  • 80.
    Choices • AWS - Cheaper but require ops • Heroku - PaaS - no/less ops - As simple as a git push - Focus on your app - Dozens of add-ons out of the box - $0.00 upfront cost Monday, 7 November 2011
  • 81.
    One last ingredient... Ben Barnard, Phillip Calçado and myself Monday, 7 November 2011
  • 82.
    One last ingredient... An awesome team, committed and having fun on a weekend :) Ben Barnard, Phillip Calçado and myself Monday, 7 November 2011
  • 83.
  • 84.
    The telethon • Live on channel 9 for 2 hours Monday, 7 November 2011
  • 85.
    The telethon • Live on channel 9 for 2 hours • SSQ’s app went down at the very start Monday, 7 November 2011
  • 86.
    The telethon • Live on channel 9 for 2 hours • SSQ’s app went down at the very start • The “backup” plan was now the main app Monday, 7 November 2011
  • 87.
    What we achieved Monday,7 November 2011
  • 88.
    What we achieved Despite all skepticism we... Monday, 7 November 2011
  • 89.
    What we achieved Despite all skepticism we... • Developed a production ready donation app using Ruby on Rails Monday, 7 November 2011
  • 90.
    What we achieved Despite all skepticism we... • Developed a production ready donation app using Ruby on Rails • Leveraged PaaS to get ops out of the way Monday, 7 November 2011
  • 91.
    What we achieved Despite all skepticism we... • Developed a production ready donation app using Ruby on Rails • Leveraged PaaS to get ops out of the way • Generated over A$ 2 million in donations during the telethon Monday, 7 November 2011
  • 92.
    What we achieved Despite all skepticism we... • Developed a production ready donation app using Ruby on Rails • Leveraged PaaS to get ops out of the way • Generated over A$ 2 million in donations during the telethon • And over A$ 38 million since then Monday, 7 November 2011
  • 93.
    What we achieved Despite all skepticism we... • Developed a production ready donation app using Ruby on Rails • Leveraged PaaS to get ops out of the way • Generated over A$ 2 million in donations during the telethon • And over A$ 38 million since then ...all in a 48 hour effort Monday, 7 November 2011
  • 94.
    Anyone still thinks‘the cloud’ isn’t for serious stuff? Monday, 7 November 2011
  • 95.
    Thanks for listening! Questions? @leonardo_borges www.leonardoborges.com Leonardo Borges Monday, 7 November 2011