• Like
Groovy DevOps in the Cloud
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Groovy DevOps in the Cloud

  • 2,491 views
Published

This talk focuses on a set of tools to automate the infrastructure provisioning and testing on Amazon EC2 with the help of Groovy-based tools and libraries. We will explore how to leverage those to …

This talk focuses on a set of tools to automate the infrastructure provisioning and testing on Amazon EC2 with the help of Groovy-based tools and libraries. We will explore how to leverage those to create an infrastructure for building, configuring and testing the provisioning of boxes in the cloud - elegant and groovy.

Published in Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
2,491
On SlideShare
0
From Embeds
0
Number of Embeds
4

Actions

Shares
Downloads
32
Comments
0
Likes
10

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Groovy DevOps in the Cloud 01
  • 2. About us 02
  • 3. Luciano Fiandesio • • • • Bio: Developer, speaker, author, kitchen hacker Company: Aestas/IT (http://aestasit.com) E-mail: luciano@aestasit.com Linkedin: http://www.linkedin.com/in/lucianofiandesio 03
  • 4. Andrey Adamovich • • • • Bio: Developer, coach, speaker, author Company: Aestas/IT (http://aestasit.com) E-mail: andrey@aestasit.com Linkedin: http://www.linkedin.com/in/andreyadamovich 04
  • 5. What's this presentation about? Our take on: • • • • DevOps Software Provisioning Continuous Integration Continuous Delivery 05
  • 6. Technologies • • • • • AWS - http://aws.amazon.com Groovy - http://groovy.codehaus.org Gradle - http://gradle.org Jenkins - http://jenkins-ci.org Puppet - http://puppetlabs.com 06
  • 7. Developers + Operations = ? 07
  • 8. Silos 08
  • 9. Conflicts 09
  • 10. Risk 10
  • 11. Agile 11
  • 12. What is DevOps? 12
  • 13. Infrastructure as Code Automate the provisioning and maintenance of servers: • • • Build from source control Utilize Open Source tools Ensure testability 13
  • 14. Configuration propagation 14
  • 15. Changes No Manual Changes! 15
  • 16. Building a DevOps toolkit • • • • Automation is key We are JVM hackers Fragmented ecosystem Full-stack approach 16
  • 17. Sshoogr 17
  • 18. Sshoogr Features • • • Remote command execution File uploading/downloading Tunneling 18
  • 19. Sshoogr Example I 0 .r m t S s i n { 1 eoeeso 0. 2 ul='sr:531lclot22' r ue2642@oahs:22 0. 3 ee 'm-f/m/' xc r r tp* 0. 4 ee 'oc /a/okm.i' xc tuh vrlc/ypd 0. 5 rmtFl(/a/ycn'.et="nbe=re eoeie'vrm.of)tx ealdtu" 0 .} 6 19
  • 20. Sshoogr Example II 0 .r m t S s i n { 1 eoeeso 0. 2 sp{ c 0. 3 fo {lcli "bidi/plcto"} rm oaDr $ulDrapiain 0. 4 it {rmtDr'vrbadmi/plcto'} no eoei /a/e/oanapiain 0. 5 } 0 .} 6 20
  • 21. Sshoogr Example III 0 .d f r s l = e e ( o m n : ' u r b n m c d , 1 e eut xccmad /s/i/ym' 0. 2 fiOErr fle soOtu:fle alnro: as, hwupt as) 0 .i ( e u t e i S a u = 1 { 3 f rsl.xttts = ) 0. 4 rsl.uptecLn {ln eutotu.ahie ie > 0. 5 i (iecnan(WRIG) { f ln.otis'ANN') 0. 6 trwnwRniexeto(Wrig!" ho e utmEcpin"ann!!) 0. 7 0. 8 0 .} 9 21 } }
  • 22. Why Groovy? • • • • Groovy is perfect choice for scripting Very mature, concise syntax Extremely easy to produce DSL We wrote a book about it! 22
  • 23. Shameless plug 23
  • 24. Puppet 24
  • 25. What is Puppet? • • Puppet Labs https://puppetlabs.com 25
  • 26. Why Puppet? • • • • More mature than competition Large community Ruby-based DSL No need to learn Ruby ;) 26
  • 27. Puppet example 27
  • 28. Puppet provisioning 28
  • 29. Puppet provisioning 29
  • 30. Puppet provisioning 30
  • 31. Puppet provisioning 31
  • 32. Puppet state management 32
  • 33. Puppet state management 33
  • 34. Puppet state management 34
  • 35. Puppet modules 35
  • 36. Puppet modules 36
  • 37. Puppet modules 37
  • 38. Continuous Integration 38
  • 39. Why Jenkins? • • • De-facto standard Stable There is a plugin for that! 39
  • 40. Services Partner 40
  • 41. Gradle Example I 0 .t s u l a M d l s < { 1 ak podoue < 0. 2 rmtSsin{ eoeeso 0. 3 ee 'm-f/m/eozp xc r r tprp.i' 0. 4 sp{ c 0. 5 fo {lclie"{ulDr/eozp } rm oaFl $bidi}rp.i" 0. 6 it {rmtDr"ro"} no eoei /ot 0. 7 } 0. 8 .. . 41
  • 42. Gradle Example I 0. 1 .. . 0. 2 ee 'm-f/t/uptmdls xc r r ecppe/oue' 0. 3 ee 'ni /m/eozp- /t/uptmdls xc uzp tprp.i d ecppe/oue' 0. 4 } 0 .} 5 42
  • 43. Gradle Example II 0 .t s p p e A p y d p n s n u l a M d l s < { 1 ak uptpl(eedO: podoue) < 0. 2 rmtSsin{ eoeeso 0. 3 sp{ c 0. 4 fo {lclie"{ulDr/eu.p } rm oaFl $bidi}stpp" 0. 5 it {rmtDr"tp } no eoei /m" 0. 6 } 0. 7 ee 'uptapy/m/eu.p xc ppe pl tpstpp' 0. 8 0 .} 9 43 }
  • 44. Jenkins example 44
  • 45. In the meanwhile... • • • We started developing complex Puppet modules Modules needs proper testing ...on different platforms 45
  • 46. Do you test, right? • • • How to test this stuff? How to reuse a JUnit approach to testing? We wanted things to be SIMPLE! 46
  • 47. PuppetUnit 47
  • 48. PUnit • • • • Simple testing tool for verifying remote server state Uses sshoogr and JUnit Reuse reporting features of JUnit and Jenkins As simple as ... 48
  • 49. PUnit Example I 0 .c a s D r y n t l T s 1 ls ebIsalet 0. 2 etnsBsPpeItgainet{ xed aeuptnertoTs 0. 3 @eoe Bfr 0. 4 vi isaleb( { od ntlDry) 0. 5 apy"nld dry) pl(icue eb" 0. 6 } 0. 7 .. . 0 .} 8 49
  • 50. PUnit Example II 0 .@ e t 1 Ts 0 .d f v i e s r D r y u n n ( { 2 e od nueebRnig) 0. 3 cmad'evc drysau >dryttslg) omn(srie eb tts ebsau.o' 0. 4 asrTu flTx(/otdryttslg) setre ieet"ro/ebsau.o" 0. 5 .otis'eb' cnan(Dry) 0. 6 asrTu flTx(/otdryttslg) setre ieet"ro/ebsau.o" 0. 7 .otis'srnig' cnan(i unn.) 0 .} 8 50
  • 51. PUnit Example III 0 .@ e t 1 Ts 0 .d f v i e s r C n o n c ( { 2 e od nueaCnet) 0. 3 Tra.le(00) hedsep100 0. 4 ulaSrp( podcit) 0. 5 cmad'otdryd-eb-0910bnbni '+ omn(/p/eb/bdry1...-i/i/j 0. 6 'etaacitsl>dryetlg) tsDtSrp.q ebts.o' 0. 7 .. . 51
  • 52. PUnit Example III 0. 1 .. . 0. 2 / Ceki telgo teisr / hc f h o f h net 0. 3 / oeaincnan tewr ERR / prto otis h od RO. 0. 4 asrFle setas( 0. 5 "h srp sol rtrea latoeerr, Te cit hud eun t es n ro" 0. 6 flTx(/otdryetlg) ieet"ro/ebts.o" 0. 7 .otis'RO' cnan(ERR) 0. 8 ) 0. 9 .. . 52
  • 53. PUnit Example III 0. 1 .. . 0. 2 / Ceko dt ta wsisre it atbe / hc n aa ht a netd no al. 0. 3 asrTu( setre 0. 4 "h lgsol cnanaSLC rsl" Te o hud oti EET eut, 0. 5 flTx(/otdryetlg) ieet"ro/ebts.o" 0. 6 .otis'rn Ae' cnan(Gad v.) 0. 7 ) 0 .} 8 53
  • 54. PUnit 54
  • 55. Next problem? 55
  • 56. Scalability • • • How do we test on different OS? How do we run parallel tests on multiple architectures? How do we avoid selling our houses? 56
  • 57. Amazon Web Services 57
  • 58. Elastic Compute Cloud • • • • Mature Great API Virtual hardware variety OS variety 58
  • 59. EC2 Console 59
  • 60. Gramazon 60
  • 61. Gramazon • • Groovy based API for interacting with EC2 Integrates with the rest of the stack 61
  • 62. Gramazon Example I 0 .t s s a t n t n e t p : S a t n t n e { 1 ak trIsac(ye trIsac) 0. 2 kyae eNm 0. 3 scrtGop'lu-o euiyru codd' 0. 4 isacNm 'rmzncodd' ntneae gaao/lu-o 0. 5 saeieae'lu-ojo' ttFlNm codd.sn 0. 6 ai m 0. 7 isacTp '1mco ntneye t.ir' 0. 8 wiFrtr tu atoSat re 0 .} 9 62 'lu-o codd' 'm-f748 ai60e1'
  • 63. Gramazon Example II 0 .t s t r i a e n t n e t p : T r i a e n t n e { 1 ak emntIsac(ye emntIsac) 0. 2 saeieae'lu-ojo' ttFlNm codd.sn 0 .} 3 63
  • 64. Costs considerations Name Compute Units Memory Hourly Cost t1.micro 2 0.60 GB $0.02 m1.small 1 1.70 GB $0.06 c1.medium 5 1.70 GB $0.14 m1.medium 2 3.75 GB $0.12 m1.large 4 7.50 GB $0.24 c1.xlarge 20 7.00 GB $0.58 m3.xlarge 13 15.00 GB $0.50 64
  • 65. Free tier • 750 hours of Amazon EC2 Linux Micro Instance usage (613 MB of memory and 32-bit and 64-bit platform support) • Enough hours to run continuously each month 65
  • 66. Security considerations • Host operating system • • • • All accesses logged and audited Customer-generated keypairs Stateful firewall • • Individual SSH keyed logins Mandatory inbound firewall, default deny mode Signed API Calls • 66 Require X.509 certificate or secret AWS key
  • 67. Imgr 67
  • 68. Imgr • • A tool for building images Inspired by Packer 68
  • 69. Images? 69
  • 70. Supports • • Shell Puppet 70
  • 71. Configuration Example 71
  • 72. The big picture 72
  • 73. Aetomation 73
  • 74. Conclusions • • • • • Reuse your existing Java knowledge ...to build a bridge between DEVs and OPs Reuse development best practices for OPs Don't be afraid to try new technologies Automate! 74
  • 75. Reading material 75
  • 76. Groovy 2 Cookbook 76
  • 77. The Phoenix Project 77
  • 78. Continuous Delivery 78
  • 79. Programming Amazon EC2 79
  • 80. Gradle in Action 80
  • 81. Technologies to follow • • • • • • Vagrant - http://www.vagrantup.com/ Docker - https://www.docker.io/ Packer - http://www.packer.io/ Qemu - http://wiki.qemu.org/Main_Page jclouds - http://jclouds.apache.org/ Cloudbees - http://www.cloudbees.com/ 81
  • 82. One more thing... 82
  • 83. It's all Open Source! 83
  • 84. Source code • • • • • Sshoogr: https://github.com/aestasit/sshoogr Sshoogr Gradle: https://github.com/aestasit/sshoogr-gradle PUnit: https://github.com/aestasit/puppet-unit Gramazon: https://github.com/aestasit/gramazon Imgr: https://github.com/aestasit/imgr 84
  • 85. Seeking contributors! 85
  • 86. Questions? 86
  • 87. Paldies! Thank you! 87