Playing God in the clouds...
SJUG Aug 2010 @Atlassian



    Michael Neale
    michael.neale@gmail.com
    twitter.com/mic...
Tiny bit of background...



         Copyright © 2010 Opscode, Inc - All Rights Reserved   2
#define cloud



   Copyright © 2010 Opscode, Inc - All Rights Reserved   3
SaaS
 PaaS
IaaS <--

(NIST)
 Copyright © 2010 Opscode, Inc - All Rights Reserved   4
IaaS:
 Illusion of infinite
       servers
Low level resources
Requires Sysadmin/
       Devops
       Copyright © 2010 Ops...
IaaS - fire data centre



        Copyright © 2010 Opscode, Inc - All Rights Reserved   6
PaaS - fire the
 sysadmins


    Copyright © 2010 Opscode, Inc - All Rights Reserved   7
SaaS - fire everone else



        Copyright © 2010 Opscode, Inc - All Rights Reserved   8
#define devops



    Copyright © 2010 Opscode, Inc - All Rights Reserved   9
Infrastructure as code
     Useful tools:
        jclouds
          FOG
      deltacloud
     puppet/chef
        Copyrigh...
What Does Cloud
Provisioning Look Like?
        Copyright © 2010 Opscode, Inc - All Rights Reserved   11
Two Answers
	

       Copyright © 2010 Opscode, Inc - All Rights Reserved   12
You building a platform
 (building an image?)

        Copyright © 2010 Opscode, Inc - All Rights Reserved   13
You deploying it


     Copyright © 2010 Opscode, Inc - All Rights Reserved   14
Build your platform
Chef, puppet, by hand...
     snapshotting



         Copyright © 2010 Opscode, Inc - All Rights Reserved   16
Deploy to a (IaaS)
             platform




http://www.byteonic.com/2009/overview-of-java-support-in-google-app-engine/
Command line, web UI,
      or API



        Copyright © 2010 Opscode, Inc - All Rights Reserved   18
cloud + devops =
better, faster, more

       Copyright © 2010 Opscode, Inc - All Rights Reserved   19
“Enable the reconstruction of the
business from nothing but a source
code repository, an application data
   backup, and “...
Cloud Provisioning is
       Step 1




       Copyright © 2010 Opscode, Inc - All Rights Reserved   21
Cloud Provisioning is
       Step 1
    Provisioning



       Copyright © 2010 Opscode, Inc - All Rights Reserved   21
Cloud Provisioning is
       Step 1
    Provisioning
    Configuration


       Copyright © 2010 Opscode, Inc - All Rights ...
Cloud Provisioning is
       Step 1
    Provisioning
    Configuration
 Systems Integration
       Copyright © 2010 Opscode...
Cloudy
environments
Cloudy
    environments

• automate EVERYTHING
Cloudy
    environments

• automate EVERYTHING
• seed/snapshotting
Cloudy
     environments

• automate EVERYTHING
• seed/snapshotting
• safely update configuration
github jclouds/

              runs on the JVM (including       )

(with-compute-service [compute]
  (run-node "webserver"...
jclouds

open source
feels like java (and clojure)
portability between clouds
deal with web complexity
unit testability
th...
Cross cloud API
abstractions:




         Copyright © 2010 Opscode, Inc - All Rights Reserved   26
Cross cloud API
abstractions:
Generic binary storage
 BlobStore




              Copyright © 2010 Opscode, Inc - All Righ...
Cross cloud API
abstractions:
Generic binary storage
 BlobStore
Generic service provisioning
 ComputeService




         ...
Cross cloud API
abstractions:
Generic binary storage
 BlobStore
Generic service provisioning
 ComputeService

but you have...
jclouds-blobstore             github jclouds/jclouds


blobstore = new BlobStoreContextFactory().createContext(
          ...
jclouds-compute               github jclouds/jclouds


service = new ComputeServiceContextFactory().createContext(
       ...
jclouds                            github jclouds/jclouds


service = new ComputeServiceContextFactory().createContext(
  ...
Common demoniator




       Copyright © 2010 Opscode, Inc - All Rights Reserved   30
Common demoniator

 BlobStore: OK




           Copyright © 2010 Opscode, Inc - All Rights Reserved   30
Common demoniator

 BlobStore: OK

 ComputeService:
 Not all clouds created equal, you may need to dive down
 and “lock in...
Common demoniator

  BlobStore: OK

  ComputeService:
   Not all clouds created equal, you may need to dive down
   and “l...
Common demoniator

 BlobStore: OK

 ComputeService:
  Not all clouds created equal, you may need to dive down
  and “lock ...
To avoid lockin:




                   Copyright © 2010 Opscode, Inc - All Rights Reserved   31
To avoid lockin:

IaaS Cloud primitives (my opinion on low level)
 Volume management
 Server provisioning **
 Blob storage...
To avoid lockin:

IaaS Cloud primitives (my opinion on low level)
 Volume management
 Server provisioning **
 Blob storage...
To avoid lockin:

IaaS Cloud primitives (my opinion on low level)
 Volume management
 Server provisioning **
 Blob storage...
Bright future




          Copyright © 2010 Opscode, Inc - All Rights Reserved   32
Bright future

 Eucalyptus
 In house “private clouds”




                  Copyright © 2010 Opscode, Inc - All Rights Res...
Bright future

 Eucalyptus
 In house “private clouds”


 OpenStack
 Rackspace/NASA - comparable to EC2 in
 scalecapability...
Bright future

 Eucalyptus
 In house “private clouds”


 OpenStack
 Rackspace/NASA - comparable to EC2 in
 scalecapability...
Copyright © 2010 Opscode, Inc - All Rights Reserved   33
Thanks !

twitter.com/michaelneale
http://www.jclouds.org/
http://deltacloud.org/
http://www.openstack.org/




          ...
Thanks !

twitter.com/michaelneale
http://www.jclouds.org/
http://deltacloud.org/
http://www.openstack.org/




          ...
Sjug aug 2010_cloud
Upcoming SlideShare
Loading in...5
×

Sjug aug 2010_cloud

1,027

Published on

My Presentation to Sydney JUG in Aug 2010

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,027
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide














  • choose your products and standards used




  • Adam in his chapter of the Web Operations book:
    &quot;Enable the reconstruction of the business from nothing but a source code repository, an application data backup, and bare-metal resources&quot;








  • works in google app engine
    high performance
    thread safe
    enterprise ready
  • focused on semantic portability across clouds
    I want an image running ubuntu and don&amp;#x2019;t want to know the id
    absolute portability where possible, but expose vendor apis where needed
  • focused on semantic portability across clouds
    I want an image running ubuntu and don&amp;#x2019;t want to know the id
    absolute portability where possible, but expose vendor apis where needed
  • focused on semantic portability across clouds
    I want an image running ubuntu and don&amp;#x2019;t want to know the id
    absolute portability where possible, but expose vendor apis where needed


  • focused on semantic portability across clouds
    I want an image running ubuntu and don&amp;#x2019;t want to know the id
    absolute portability where possible, but expose vendor apis where needed
  • focused on semantic portability across clouds
    I want an image running ubuntu and don&amp;#x2019;t want to know the id
    absolute portability where possible, but expose vendor apis where needed
  • focused on semantic portability across clouds
    I want an image running ubuntu and don&amp;#x2019;t want to know the id
    absolute portability where possible, but expose vendor apis where needed
  • focused on semantic portability across clouds
    I want an image running ubuntu and don&amp;#x2019;t want to know the id
    absolute portability where possible, but expose vendor apis where needed
  • focused on semantic portability across clouds
    I want an image running ubuntu and don&amp;#x2019;t want to know the id
    absolute portability where possible, but expose vendor apis where needed



  • focused on semantic portability across clouds
    I want an image running ubuntu and don&amp;#x2019;t want to know the id
    absolute portability where possible, but expose vendor apis where needed
  • focused on semantic portability across clouds
    I want an image running ubuntu and don&amp;#x2019;t want to know the id
    absolute portability where possible, but expose vendor apis where needed
  • focused on semantic portability across clouds
    I want an image running ubuntu and don&amp;#x2019;t want to know the id
    absolute portability where possible, but expose vendor apis where needed
  • focused on semantic portability across clouds
    I want an image running ubuntu and don&amp;#x2019;t want to know the id
    absolute portability where possible, but expose vendor apis where needed
  • focused on semantic portability across clouds
    I want an image running ubuntu and don&amp;#x2019;t want to know the id
    absolute portability where possible, but expose vendor apis where needed
  • Sjug aug 2010_cloud

    1. 1. Playing God in the clouds... SJUG Aug 2010 @Atlassian Michael Neale michael.neale@gmail.com twitter.com/michaelneale www.michaelneale.net Copyright © 2010 Opscode, Inc - All Rights Reserved 1
    2. 2. Tiny bit of background... Copyright © 2010 Opscode, Inc - All Rights Reserved 2
    3. 3. #define cloud Copyright © 2010 Opscode, Inc - All Rights Reserved 3
    4. 4. SaaS PaaS IaaS <-- (NIST) Copyright © 2010 Opscode, Inc - All Rights Reserved 4
    5. 5. IaaS: Illusion of infinite servers Low level resources Requires Sysadmin/ Devops Copyright © 2010 Opscode, Inc - All Rights Reserved 5
    6. 6. IaaS - fire data centre Copyright © 2010 Opscode, Inc - All Rights Reserved 6
    7. 7. PaaS - fire the sysadmins Copyright © 2010 Opscode, Inc - All Rights Reserved 7
    8. 8. SaaS - fire everone else Copyright © 2010 Opscode, Inc - All Rights Reserved 8
    9. 9. #define devops Copyright © 2010 Opscode, Inc - All Rights Reserved 9
    10. 10. Infrastructure as code Useful tools: jclouds FOG deltacloud puppet/chef Copyright © 2010 Opscode, Inc - All Rights Reserved 10
    11. 11. What Does Cloud Provisioning Look Like? Copyright © 2010 Opscode, Inc - All Rights Reserved 11
    12. 12. Two Answers Copyright © 2010 Opscode, Inc - All Rights Reserved 12
    13. 13. You building a platform (building an image?) Copyright © 2010 Opscode, Inc - All Rights Reserved 13
    14. 14. You deploying it Copyright © 2010 Opscode, Inc - All Rights Reserved 14
    15. 15. Build your platform
    16. 16. Chef, puppet, by hand... snapshotting Copyright © 2010 Opscode, Inc - All Rights Reserved 16
    17. 17. Deploy to a (IaaS) platform http://www.byteonic.com/2009/overview-of-java-support-in-google-app-engine/
    18. 18. Command line, web UI, or API Copyright © 2010 Opscode, Inc - All Rights Reserved 18
    19. 19. cloud + devops = better, faster, more Copyright © 2010 Opscode, Inc - All Rights Reserved 19
    20. 20. “Enable the reconstruction of the business from nothing but a source code repository, an application data backup, and “... a credit card Copyright © 2010 Opscode, Inc - All Rights Reserved 20
    21. 21. Cloud Provisioning is Step 1 Copyright © 2010 Opscode, Inc - All Rights Reserved 21
    22. 22. Cloud Provisioning is Step 1 Provisioning Copyright © 2010 Opscode, Inc - All Rights Reserved 21
    23. 23. Cloud Provisioning is Step 1 Provisioning Configuration Copyright © 2010 Opscode, Inc - All Rights Reserved 21
    24. 24. Cloud Provisioning is Step 1 Provisioning Configuration Systems Integration Copyright © 2010 Opscode, Inc - All Rights Reserved 21
    25. 25. Cloudy environments
    26. 26. Cloudy environments • automate EVERYTHING
    27. 27. Cloudy environments • automate EVERYTHING • seed/snapshotting
    28. 28. Cloudy environments • automate EVERYTHING • seed/snapshotting • safely update configuration
    29. 29. github jclouds/ runs on the JVM (including ) (with-compute-service [compute] (run-node "webserver")) pis pis d a ud a lou clo o c t of k- in t -ou loc lock zero zero nodes = compute.runNodesWithTag(“webserver”, 1);
    30. 30. jclouds open source feels like java (and clojure) portability between clouds deal with web complexity unit testability thread-safe and scalable
    31. 31. Cross cloud API abstractions: Copyright © 2010 Opscode, Inc - All Rights Reserved 26
    32. 32. Cross cloud API abstractions: Generic binary storage BlobStore Copyright © 2010 Opscode, Inc - All Rights Reserved 26
    33. 33. Cross cloud API abstractions: Generic binary storage BlobStore Generic service provisioning ComputeService Copyright © 2010 Opscode, Inc - All Rights Reserved 26
    34. 34. Cross cloud API abstractions: Generic binary storage BlobStore Generic service provisioning ComputeService but you have low level access when need it Copyright © 2010 Opscode, Inc - All Rights Reserved 26
    35. 35. jclouds-blobstore github jclouds/jclouds blobstore = new BlobStoreContextFactory().createContext( “s3”, identity, credential).getAsyncBlobStore(); blob = blobstore.newBlob(key); blob.getMetadata().getUserMetadata().put("date", now()); blob.setPayload(snapshot()); blob.getPayload().setContentMD5(snapshotMD5()); blobstore.putBlob(containerName, blob) .addListener(publishAMQP(key),sameThreadExecutor());
    36. 36. jclouds-compute github jclouds/jclouds service = new ComputeServiceContextFactory().createContext( “ec2”, identity, credential).getComputeService(); template = service.templateBuilder().fastest().build(); template.getOptions().runScript(installGemsAndRunChef); nodes = service.runNodesWithTag(“webserver”, 5, template);
    37. 37. jclouds github jclouds/jclouds service = new ComputeServiceContextFactory().createContext( “rimuhosting”, user, password ).getComputeService(); template = service.templateBuilder().any().biggest().build(); template.getOptions().installPrivateKey(privateRSA) .authorizePublicKey(publicRSA) .runScript(installGemsAndRunChef); nodes = service.runNodesWithTag(“webserver”, 5, template);
    38. 38. Common demoniator Copyright © 2010 Opscode, Inc - All Rights Reserved 30
    39. 39. Common demoniator BlobStore: OK Copyright © 2010 Opscode, Inc - All Rights Reserved 30
    40. 40. Common demoniator BlobStore: OK ComputeService: Not all clouds created equal, you may need to dive down and “lock in” just a tiny bit.. I have ! Copyright © 2010 Opscode, Inc - All Rights Reserved 30
    41. 41. Common demoniator BlobStore: OK ComputeService: Not all clouds created equal, you may need to dive down and “lock in” just a tiny bit.. I have ! Still early days: Copyright © 2010 Opscode, Inc - All Rights Reserved 30
    42. 42. Common demoniator BlobStore: OK ComputeService: Not all clouds created equal, you may need to dive down and “lock in” just a tiny bit.. I have ! Still early days: No Volume abstractions, yes, no LBs, firewalls etc Copyright © 2010 Opscode, Inc - All Rights Reserved 30
    43. 43. To avoid lockin: Copyright © 2010 Opscode, Inc - All Rights Reserved 31
    44. 44. To avoid lockin: IaaS Cloud primitives (my opinion on low level) Volume management Server provisioning ** Blob storage / Bulk data** Firewall/access control Image construction *** Copyright © 2010 Opscode, Inc - All Rights Reserved 31
    45. 45. To avoid lockin: IaaS Cloud primitives (my opinion on low level) Volume management Server provisioning ** Blob storage / Bulk data** Firewall/access control Image construction *** ** This is the currently successfully abstracted ones - generic enough via various apis Copyright © 2010 Opscode, Inc - All Rights Reserved 31
    46. 46. To avoid lockin: IaaS Cloud primitives (my opinion on low level) Volume management Server provisioning ** Blob storage / Bulk data** Firewall/access control Image construction *** ** This is the currently successfully abstracted ones - generic enough via various apis *** plenty of “traditional” tools help here - chef ! Copyright © 2010 Opscode, Inc - All Rights Reserved 31
    47. 47. Bright future Copyright © 2010 Opscode, Inc - All Rights Reserved 32
    48. 48. Bright future Eucalyptus In house “private clouds” Copyright © 2010 Opscode, Inc - All Rights Reserved 32
    49. 49. Bright future Eucalyptus In house “private clouds” OpenStack Rackspace/NASA - comparable to EC2 in scalecapability Copyright © 2010 Opscode, Inc - All Rights Reserved 32
    50. 50. Bright future Eucalyptus In house “private clouds” OpenStack Rackspace/NASA - comparable to EC2 in scalecapability Copyright © 2010 Opscode, Inc - All Rights Reserved 32
    51. 51. Copyright © 2010 Opscode, Inc - All Rights Reserved 33
    52. 52. Thanks ! twitter.com/michaelneale http://www.jclouds.org/ http://deltacloud.org/ http://www.openstack.org/ Copyright © 2010 Opscode, Inc - All Rights Reserved 33
    53. 53. Thanks ! twitter.com/michaelneale http://www.jclouds.org/ http://deltacloud.org/ http://www.openstack.org/ Copyright © 2010 Opscode, Inc - All Rights Reserved 33

    ×