REINVENTING RACKSPACE:
Agile-techture: Nimble Cloud Engineering

PRESENTED BY:

Wayne A. Walls
Enterprise Architect
Cloud Connect Chicago - Sept 2012


@waynewalls
REINVENTING RACKSPACE




           OPEN SOURCE
         CLOUD ON CLOUD
       CONTINUOUS DELIVERY
             OPEN API
WINNING THE NEXT ROUND
WINNING THE NEXT ROUND
WINNING THE NEXT ROUND
WINNING THE NEXT ROUND
REINVENTING RACKSPACE




           OPEN SOURCE
          CLOUD ON CLOUD
        CONTINUOUS DELIVERY
             OPEN API
OPEN SOURCE


                         • Software to make a cloud anywhere
                         • An ecosystem devoted to innovation
                         • Flexibility in deployment/features
                         • Standards for broad deployment
                         • No fear of lock-in




 Open Source Cloud computing will meet the needs of public and
 private Cloud providers by being simple and massively scalable.
OPEN SOURCE




   – 63 Public repos
    • OpenStack tools
    • Mobile Apps
    • Software Configuration Management (SCM)
    • Cloud Management Software
      – yagi
      – atom-hopper


          https://github.com/rackspace
OPEN SOURCE :   Polling for information in OpenStack



                    INTERWEBS



               AUTHENTICATION LAYER

                ADMIN           PUBLIC
                 API             API


                          SCHEDULER
                           NETWORK

1         MQ
                            VOLUME
                        IMAGE SERVICES   DB
OPEN SOURCE :         Polling for information in OpenStack



                           INTERWEBS



                      AUTHENTICATION LAYER


                        ADMIN      PUBLIC
                         API        API


                              SCHEDULER
                               NETWORK

1         MQ
                                VOLUME
                            IMAGE SERVICES   DB




               YAGI
OPEN SOURCE :         Polling for information in OpenStack



                           INTERWEBS



                      AUTHENTICATION LAYER

                                   PUBLIC
                        ADMIN
                                    API


                              SCHEDULER
                               NETWORK

1         MQ
                                VOLUME
                            IMAGE SERVICES   DB




                         ATOM/
               YAGI     ATOM PUB
OPEN SOURCE :         Polling for information in OpenStack



                           INTERWEBS



                      AUTHENTICATION LAYER

                                       PUBLIC
                        ADMIN
                                        API


                              SCHEDULER
                               NETWORK

1         MQ
                                VOLUME
                            IMAGE SERVICES      DB




                         ATOM/          ATOM
               YAGI     ATOM PUB       HOPPER
OPEN SOURCE :         Polling for information in OpenStack



                           INTERWEBS



                      AUTHENTICATION LAYER


                        ADMIN          PUBLIC
                                        API


                               SCHEDULER
                                NETWORK

1         MQ
                                 VOLUME
                             IMAGE SERVICES     DB




                         ATOM/          ATOM
               YAGI     ATOM PUB       HOPPER

                        REAL-TIME
                        PUBLISHER
OPEN SOURCE :         Polling for information in OpenStack



                           INTERWEBS



                      AUTHENTICATION LAYER

                        ADMIN       PUBLIC
                         API         API


                               SCHEDULER
                                NETWORK

1         MQ
                                 VOLUME
                             IMAGE SERVICES            DB




                                                            MONITORING

                         ATOM/          ATOM
               YAGI     ATOM PUB       HOPPER

                        REAL-TIME
                        PUBLISHER               RBA/
OPEN SOURCE :         Polling for information in OpenStack



                           INTERWEBS



                      AUTHENTICATION LAYER

                                    PUBLIC
                        ADMIN
                                     API


                               SCHEDULER
                                NETWORK                       USAGE


1         MQ
                                 VOLUME
                             IMAGE SERVICES            DB    BILLING


                                                            CUSTOMER


                                                            TICKETING


                                                            MONITORING

                         ATOM/          ATOM
               YAGI     ATOM PUB       HOPPER

                        REAL-TIME
                        PUBLISHER               RBA/
A VIBRANT ECOSYSTEM




15
REINVENTING RACKSPACE




           OPEN SOURCE
         CLOUD ON CLOUD
        CONTINUOUS DELIVERY
             OPEN API
A BIT OF RACKSPACE CLOUD HISTORY
     FirstGen Staging       FirstGen Production
     Rackspace Cloud          Rackspace Cloud




 $    $   $   $    $    $   $   $   $   $   $     $
YO DAWG I HEARD YOU
        LIKE CLOUD



SO I PUT A CLOUD IN YOUR CLOUD SO
 YOU CAN CLOUD WHILE YOU CLOUD18
BOOTSTRAPPING THE CLOUD




  api     scheduler   database




network   compute      queue
SEED ENVIRONMENT



   nova-api    nova-api       nova-api     nova-api                 quantum        glance


   rabbitMQ    nova-cells     nova-db      scheduler                melange        console


               API (Region) Cell                                       Region Services


  rabbitMQ       nova-db            rabbitMQ           nova-db        rabbitMQ       nova-db


  nova-cells                        nova-cells                        nova-cells


  scheduler                         scheduler                         scheduler



cell-0001           compute
                                 cell-0003                compute
                                                                    cell-0003            compute
SEED ENVIRONMENT


                                                          API (Region) Cell




                        Available Capacity
nova-api   nova-api     nova-aoi   nova-api    nova-api   nova-api     nova-aoi   nova-api    nova-api   nova-api     nova-aoi   nova-api




rabbitMQ   nova-cells   nova-db    scheduler   rabbitMQ   nova-cells   nova-db    scheduler   rabbitMQ   nova-cells   nova-db    scheduler
SEED ENVIRONMENT


                                                          API (Region) Cell




                        Available Capacity
nova-api   nova-api     nova-aoi   nova-api    nova-api   nova-api     nova-aoi   nova-api    nova-api   nova-api     nova-aoi   nova-api




rabbitMQ    Dev
           nova-cells   nova-db    scheduler      Staging
                                               rabbitMQ   nova-cells   nova-db    scheduler   rabbitMQ   Prod
                                                                                                         nova-cells   nova-db    scheduler
DEPLOYMENTS AND UPGRADES




                   Load Balancer




        nova-api     nova-api      nova-api
DEPLOYMENTS AND UPGRADES




                   Load Balancer




        nova-api     nova-api      nova-api
REINVENTING RACKSPACE




           OPEN SOURCE
         CLOUD ON CLOUD
       CONTINUOUS DELIVERY
             OPEN API
CONTINUOUS DELIVERY


From commit to staging ...


  Commit by
  community
      OR
   Rackspace
CONTINUOUS DELIVERY


From commit to staging ...


  Commit by
  community     Merge &
      OR        Dev Tests
   Rackspace


               14 minutes
CONTINUOUS DELIVERY


From commit to staging ...


  Commit by
  community     Merge &     Package &
                            Deploy to
      OR        Dev Tests      QA
   Rackspace


               14 minutes   17 minutes
CONTINUOUS DELIVERY


From commit to staging ...


  Commit by
  community     Merge &     Package &    Automated
                            Deploy to    Functional
      OR        Dev Tests      QA          Tests
   Rackspace


               14 minutes   17 minutes   15 minutes
CONTINUOUS DELIVERY


From commit to staging ...


   Commit by
   community    Merge &     Package &    Automated      Publish
                            Deploy to    Functional     Staging
      OR        Dev Tests      QA                      Packages
                                           Tests
   Rackspace


               14 minutes   17 minutes   15 minutes   30 seconds


... 47 minutes total elapsed time
TRACKING THE PIPELINE
DATA-DRIVEN DECISIONS
REINVENTING RACKSPACE




            OPEN SOURCE
          CLOUD ON CLOUD
        CONTINUOUS DELIVERY
             OPEN API
OPEN API




    API CLONING
OPEN API

   Open SDKs         Library Projects




               Supernova
               https://github.com/rackerhacker/supernova
REINVENTING RACKSPACE
REINVENTING RACKSPACE




   VIRTUAL

 OPEN SOURCE
 CLOUD ON CLOUD
 CONTINUOUS DELIVERY
 OPEN API
Thank You!

    @waynewalls


Visit us at booth #301
                   RACKSPACE® HOSTING   |   WWW.RACKSPACE.COM

Agile-Techture: Nimble Cloud Engineering at Rackspace

  • 1.
    REINVENTING RACKSPACE: Agile-techture: NimbleCloud Engineering PRESENTED BY: Wayne A. Walls Enterprise Architect Cloud Connect Chicago - Sept 2012 @waynewalls
  • 2.
    REINVENTING RACKSPACE OPEN SOURCE CLOUD ON CLOUD CONTINUOUS DELIVERY OPEN API
  • 4.
  • 5.
  • 6.
  • 7.
  • 8.
    REINVENTING RACKSPACE OPEN SOURCE CLOUD ON CLOUD CONTINUOUS DELIVERY OPEN API
  • 9.
    OPEN SOURCE • Software to make a cloud anywhere • An ecosystem devoted to innovation • Flexibility in deployment/features • Standards for broad deployment • No fear of lock-in Open Source Cloud computing will meet the needs of public and private Cloud providers by being simple and massively scalable.
  • 10.
    OPEN SOURCE – 63 Public repos • OpenStack tools • Mobile Apps • Software Configuration Management (SCM) • Cloud Management Software – yagi – atom-hopper https://github.com/rackspace
  • 11.
    OPEN SOURCE : Polling for information in OpenStack INTERWEBS AUTHENTICATION LAYER ADMIN PUBLIC API API SCHEDULER NETWORK 1 MQ VOLUME IMAGE SERVICES DB
  • 12.
    OPEN SOURCE : Polling for information in OpenStack INTERWEBS AUTHENTICATION LAYER ADMIN PUBLIC API API SCHEDULER NETWORK 1 MQ VOLUME IMAGE SERVICES DB YAGI
  • 13.
    OPEN SOURCE : Polling for information in OpenStack INTERWEBS AUTHENTICATION LAYER PUBLIC ADMIN API SCHEDULER NETWORK 1 MQ VOLUME IMAGE SERVICES DB ATOM/ YAGI ATOM PUB
  • 14.
    OPEN SOURCE : Polling for information in OpenStack INTERWEBS AUTHENTICATION LAYER PUBLIC ADMIN API SCHEDULER NETWORK 1 MQ VOLUME IMAGE SERVICES DB ATOM/ ATOM YAGI ATOM PUB HOPPER
  • 15.
    OPEN SOURCE : Polling for information in OpenStack INTERWEBS AUTHENTICATION LAYER ADMIN PUBLIC API SCHEDULER NETWORK 1 MQ VOLUME IMAGE SERVICES DB ATOM/ ATOM YAGI ATOM PUB HOPPER REAL-TIME PUBLISHER
  • 16.
    OPEN SOURCE : Polling for information in OpenStack INTERWEBS AUTHENTICATION LAYER ADMIN PUBLIC API API SCHEDULER NETWORK 1 MQ VOLUME IMAGE SERVICES DB MONITORING ATOM/ ATOM YAGI ATOM PUB HOPPER REAL-TIME PUBLISHER RBA/
  • 17.
    OPEN SOURCE : Polling for information in OpenStack INTERWEBS AUTHENTICATION LAYER PUBLIC ADMIN API SCHEDULER NETWORK USAGE 1 MQ VOLUME IMAGE SERVICES DB BILLING CUSTOMER TICKETING MONITORING ATOM/ ATOM YAGI ATOM PUB HOPPER REAL-TIME PUBLISHER RBA/
  • 18.
  • 19.
    REINVENTING RACKSPACE OPEN SOURCE CLOUD ON CLOUD CONTINUOUS DELIVERY OPEN API
  • 20.
    A BIT OFRACKSPACE CLOUD HISTORY FirstGen Staging FirstGen Production Rackspace Cloud Rackspace Cloud $ $ $ $ $ $ $ $ $ $ $ $
  • 21.
    YO DAWG IHEARD YOU LIKE CLOUD SO I PUT A CLOUD IN YOUR CLOUD SO YOU CAN CLOUD WHILE YOU CLOUD18
  • 22.
    BOOTSTRAPPING THE CLOUD api scheduler database network compute queue
  • 23.
    SEED ENVIRONMENT nova-api nova-api nova-api nova-api quantum glance rabbitMQ nova-cells nova-db scheduler melange console API (Region) Cell Region Services rabbitMQ nova-db rabbitMQ nova-db rabbitMQ nova-db nova-cells nova-cells nova-cells scheduler scheduler scheduler cell-0001 compute cell-0003 compute cell-0003 compute
  • 24.
    SEED ENVIRONMENT API (Region) Cell Available Capacity nova-api nova-api nova-aoi nova-api nova-api nova-api nova-aoi nova-api nova-api nova-api nova-aoi nova-api rabbitMQ nova-cells nova-db scheduler rabbitMQ nova-cells nova-db scheduler rabbitMQ nova-cells nova-db scheduler
  • 25.
    SEED ENVIRONMENT API (Region) Cell Available Capacity nova-api nova-api nova-aoi nova-api nova-api nova-api nova-aoi nova-api nova-api nova-api nova-aoi nova-api rabbitMQ Dev nova-cells nova-db scheduler Staging rabbitMQ nova-cells nova-db scheduler rabbitMQ Prod nova-cells nova-db scheduler
  • 26.
    DEPLOYMENTS AND UPGRADES Load Balancer nova-api nova-api nova-api
  • 27.
    DEPLOYMENTS AND UPGRADES Load Balancer nova-api nova-api nova-api
  • 28.
    REINVENTING RACKSPACE OPEN SOURCE CLOUD ON CLOUD CONTINUOUS DELIVERY OPEN API
  • 29.
    CONTINUOUS DELIVERY From committo staging ... Commit by community OR Rackspace
  • 30.
    CONTINUOUS DELIVERY From committo staging ... Commit by community Merge & OR Dev Tests Rackspace 14 minutes
  • 31.
    CONTINUOUS DELIVERY From committo staging ... Commit by community Merge & Package & Deploy to OR Dev Tests QA Rackspace 14 minutes 17 minutes
  • 32.
    CONTINUOUS DELIVERY From committo staging ... Commit by community Merge & Package & Automated Deploy to Functional OR Dev Tests QA Tests Rackspace 14 minutes 17 minutes 15 minutes
  • 33.
    CONTINUOUS DELIVERY From committo staging ... Commit by community Merge & Package & Automated Publish Deploy to Functional Staging OR Dev Tests QA Packages Tests Rackspace 14 minutes 17 minutes 15 minutes 30 seconds ... 47 minutes total elapsed time
  • 34.
  • 35.
  • 36.
    REINVENTING RACKSPACE OPEN SOURCE CLOUD ON CLOUD CONTINUOUS DELIVERY OPEN API
  • 37.
    OPEN API API CLONING
  • 38.
    OPEN API Open SDKs Library Projects Supernova https://github.com/rackerhacker/supernova
  • 39.
  • 40.
    REINVENTING RACKSPACE VIRTUAL OPEN SOURCE CLOUD ON CLOUD CONTINUOUS DELIVERY OPEN API
  • 41.
    Thank You! @waynewalls Visit us at booth #301 RACKSPACE® HOSTING | WWW.RACKSPACE.COM