Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Ninja tricks for groovy system scripting for JavaLand 2016

2,137 views

Published on

Are you a Java developer practicing automation? Do you write Bash, BAT, Python, Ruby, Perl, Lua scripts for system administration and scripting purposes? Stop! You don't have too. Since JVM ecosystem has a perfect answer - Groovy. With the release of version 2.4, Groovy is a mature and stable language that can be employed to deploy small scripts or to build large, mission-critical applications.

This talk focuses on some less-known features of the language that are going to make the life of a Java/Groovy developer more fun than ever.

In particular, this presentation will cover system administrator scripting toolkit that developers can take full advantage of: I/O operations, mini-DSLs, comparison to other languages like Python and Bash.

Published in: Technology
  • Be the first to comment

Ninja tricks for groovy system scripting for JavaLand 2016

  1. 1. 01
  2. 2. About me 02
  3. 3. Andrey Adamovich Bio: Developer, coach, speaker, author Company: Aestas/IT (http://aestasit.com) E­mail: andrey@aestasit.com Linkedin: http://www.linkedin.com/in/andreyadamovich Lanyrd: http://lanyrd.com/profile/andrey­adamovich GitHub: https://github.com/aadamovich SO: http://stackoverflow.com/users/162792/andrey­adamovich Twitter: @codingandrey, @aestasit • • • • • • • • 03
  4. 4. Exposed to operations Maintenance scripts (backup, cleaning, monitoring) Scheduled/CRON jobs (transfers, conversions, notifications) Automation tasks (deployment, packaging, testing) Web applications (status pages, back doors, dashboards) • • • • 04
  5. 5. Why Groovy? 05
  6. 6. Native system scripting Bash, Dash, Zsh, Fish, Csh etc. Bat, Cmd and, eventually, PowerShell (PoSH) • • 06
  7. 7. Arrays in BAT? 07
  8. 8. BASH vs. PoSH 08
  9. 9. Scripting languages Perl Python Ruby Lua Javascript ... • • • • • • 09
  10. 10. Multi­platform support 10
  11. 11. Does it work on Windows? 11
  12. 12. Operational simplicity 12
  13. 13. Maintainable code 13
  14. 14. Stable ecosystem 14
  15. 15. Rich extension library15
  16. 16. Groovy is just a JAR 16
  17. 17. Reuse JVM knowledge 17
  18. 18. sdkman.io 18
  19. 19. sdkman.io curl ‐s http://get.sdkman.io | bash sdk install groovy 2.4.6 sdk default groovy 2.4.6 sdk list groovy 01. 02. 03. 04. 19
  20. 20. PoSH GVM > posh‐gvm.psm1 cannot be loaded  > because the execution of scripts  > is disabled on this system. 01. 02. 03. 20
  21. 21. Speed up! 21
  22. 22. GroovyServ 22
  23. 23. Files, directories, and other animals23
  24. 24. Reading f = new File('test.txt') String textContent = f.text byte[] binaryContent = file.bytes 01. 02. 03. 24
  25. 25. Writing I f << 'hello, this is a test file' f.text = new URL('http://server/my_data.json').text file.bytes = [ 65, 66, 67, 68 ] as byte[] 01. 02. 03. 25
  26. 26. Writing II file.text = '''What's in a name? That which we call a rose By any other name would smell as sweet.''' file << 'What's in a name? That which we call a rosen' file << 'By any other name would smell as sweet.' 01. 02. 03.04. 05. 26
  27. 27. Writing III file.withWriter { Writer writer ‐>   writer << 'What's in a name? That which we call a rosen'   writer << 'By any other name would smell as sweet.' } file.withOutputStream { OutputStream stream ‐>   stream << 'What's in a name? That which we call a rosen'   stream << 'By any other name would smell as sweet.' } 01. 02. 03. 04. 05.06. 07. 08. 09. 27
  28. 28. Line­by­line def lines = f.readLines() lines.each { String line ‐>   println line } 01. 02. 03. 04. 28
  29. 29. eachLine and eachByte f.eachLine { String line ‐>   println line } file.eachByte { int b ‐>   ... } 01. 02. 03. 04. 05. 06. 29
  30. 30. Filtering I outputFile.withPrintWriter { writer ‐>   inputFile.eachLine { line ‐>     if (!line.startsWith('#')) {       writer.println(line)     }   } } 01. 02. 03. 04. 05. 06. 07. 30
  31. 31. Filtering II inputFile.filterLine(outputFile.newWriter()) { line ‐>   !line.startsWith('#') } 01. 02. 03. 31
  32. 32. Filtering III outputFile.withWriter { Writer writer ‐>   inputFile.withReader { Reader reader ‐>     reader.transformLine(writer) { String line ‐>       line.replaceAll('t', '  ')     }   } } 01. 02. 03. 04. 05. 06. 07. 32
  33. 33. Filtering IV inputFile.withReader { reader ‐>   reader.transformLine(outputFile.newWriter()) { line ‐>     line.replaceAll('t', '  ')   } } 01. 02. 03. 04. 05. 33
  34. 34. Filtering V outputFile.text = inputFile.text.replaceAll('t', '  ')01. 34
  35. 35. Walking the path 35
  36. 36. eachFile and eachDir f.eachFile { File file ‐>   ... } f.eachDir { File dir ‐>   ... } 01. 02. 03. 04. 05. 06. 36
  37. 37. Recursive walking I currentDir.eachFileRecurse { File file ‐>   println file.name } currentDir.eachDirRecurse { File dir ‐>   println dir.name } 01. 02. 03. 04. 05. 06. 37
  38. 38. Recursive walking II currentDir.traverse { File file ‐>   if (file.name.endsWith('.groovy')) {     println file.name   } } 01. 02. 03. 04. 05. 38
  39. 39. Recursive walking III currentDir.traverse(nameFilter: ~/.*.groovy/) { File file ‐>   println file.name } 01. 02. 03. 39
  40. 40. Recursive walking IV import static groovy.io.FileType.* ... currentDir.traverse(   type: FILES,   nameFilter: ~/.*.groovy/,   excludeNameFilter: ~/^C.*$/) { File file ‐>     println file.name } 01. 02. 03. 04. 05. 06. 07. 08. 40
  41. 41. Extending File I File.metaClass.safeDelete = {   if (exists()) {     if (isDirectory()) {       if (!deleteDir()) {         def msg = "Unable to delete a directory: ${name}"         throw new IOException(msg)       }     ... 01. 02. 03. 04. 05. 06. 07. 08. 41
  42. 42. Extending File II     ...     } else {       if (!delete()) {         def msg = "Unable to delete a file: ${name}"         throw new IOException(msg)       }     }   } } 01. 02. 03. 04. 05. 06. 07. 08. 09. 42
  43. 43. Extending File III File f = new File('test.txt') f.safeDelete() File d = new File('test_dir') d.safeDelete() 01. 02. 03. 04. 43
  44. 44. Base script 44
  45. 45. Base script I BaseScript.groovy : abstract class BaseScript extends Script {   static {     File.metaClass.safeDelete = {       if (exists()) {         ...   } } 01. 02. 03. 04. 05. 06. 07. 45
  46. 46. Base script II delete.groovy : new File('idonotexist.txt').safeDelete()01. 46
  47. 47. Base script III groovy ‐b BaseScript delete.groovy01. 47
  48. 48. External commands 48
  49. 49. External commands I def exitValue = "ls ‐l".execute().exitValue() if (!exitValue) {   println "Command failed with exit code: ${exitValue}" } 01. 02. 03. 04. 49
  50. 50. External commands II println "ls".execute().text() println "ls".execute().inputStream.eachLine { println it } "grep abc".execute().waitForOrKill(1000) 01. 02. 03. 50
  51. 51. External commands III Process process = "command".execute()    def out = new StringBuffer() def err = new StringBuffer() process.waitForProcessOutput(out, err) 01. 02. 03. 04. 51
  52. 52. External commands IV "less temp.txt".execute().pipeTo("grep error".execute()).text def proc1 = "less temp.txt".execute() def proc2 = "grep error".execute() proc1 | proc2 println proc2.text 01. 02. 03. 04. 05. 52
  53. 53. Arguments 53
  54. 54. CliBuilder def cli = new CliBuilder(usage:'ls') cli.a('display all files') cli.l('use a long listing format') cli.t('sort by modification time') def options = cli.parse(args) 01. 02. 03. 04. 05. 54
  55. 55. Defensive scripting 55
  56. 56. Do you like NPEs? 56
  57. 57. Even simple things can cause NPE Java: System.out.println(user.                      getAddress().                      getCity()) Groovy: println user.           address.             city 01. 02. 03. 01. 02. 03. 57
  58. 58. Safe navigation operator (?.) println user?.           address?.             city  01. 02. 03. 58
  59. 59. Elvis operator (?:) println user?.          address?.            city ?: "No idea" 01. 02. 03. 59
  60. 60. Closures for DSLs 60
  61. 61. Imaginary DSL transaction {   request {     id     = '2L'     name   = 'PUT'     amount = 25   }   request {     ...   } } 01. 02. 03. 04. 05. 06. 07. 08. 09. 10. 61
  62. 62. DSL: transaction() method def transaction(Closure cl) {   def processor = new RequestProcessor()   try {     processor.init()     cl.delegate = processor     cl()     sendRequests()   } finally {     processor.cleanup() }} 01. 02. 03. 04. 05. 06. 07. 08. 09. 10. 62
  63. 63. DSL: request() method class RequestProcessor {   def requests = []   def request(Closure cl) {     def request = new Request()     requests << request     cl.delegate = request     cl()   }   ... 01. 02. 03. 04. 05. 06. 07. 08. 09. 63
  64. 64. DSL: Request structure class Request {   String id   String name   String amount  } 01. 02. 03. 04. 05. 64
  65. 65. Data manipulation 65
  66. 66. XML import groovy.util.XmlSlurper def xmlSource = new File('shakespeare.xml') def bibliography = new XmlSlurper().parse(xmlSource) println bibliography.author bibliography.play               .findAll { it.year.toInteger() > 1592 }               .each { println it.title } 01. 02. 03. 04. 05. 06. 07. 66
  67. 67. JSON import groovy.json.JsonSlurper def reader = new FileReader('ui.json') def ui = new JsonSlurper().parse(reader) ui.items.each { println it.type } println ui.items[0]           .axes           .find {             it.fields.contains('y')           }.title 01. 02. 03. 04. 05. 06. 07. 08. 09. 67
  68. 68. Grab your stuff! 68
  69. 69. @Grab: import @Grab('org.apache.httpcomponents:httpclient:4.2.1') import org.apache.http.impl.client.DefaultHttpClient import org.apache.http.client.methods.HttpGet 01. 02. 03. 69
  70. 70. @Grab: variable @Grab('org.apache.httpcomponents:httpclient:4.2.1') def httpClient =   new org.apache.http.impl.client.DefaultHttpClient() 01. 02. 03. 70
  71. 71. @Grab: method @Grab('org.apache.httpcomponents:httpclient:4.2.1') def getHttpClient() {   new org.apache.http.impl.client.DefaultHttpClient() } 01. 02. 03. 04. 71
  72. 72. @Grab: class @Grab('org.apache.httpcomponents:httpclient:4.2.1') class Searcher {   def httpClient   Searcher() {     httpClient =       new org.apache.http.impl.client.DefaultHttpClient()   } } 01. 02. 03. 04. 05. 06. 07. 08. 72
  73. 73. @Grab: multiple @Grapes([    @Grab('org.apache.httpcomponents:httpclient:4.2.1'),    @Grab('org.ccil.cowan.tagsoup:tagsoup:1.2')]) class Searcher { ... } 01. 02. 03. 04. 73
  74. 74. @Grab: repository @GrabResolver(name='codehaus',               root='http://repository.codehaus.org/') class Searcher { ... } 01. 02. 03. 74
  75. 75. @Grab: exclude @GrabExclude(group='commons‐codec',              module='commons‐codec') class Searcher { ... } 01. 02. 03. 75
  76. 76. HTTP 76
  77. 77. HTTPBuilder: import @Grab(   group='org.codehaus.groovy.modules.http‐builder',   module='http‐builder',   version='0.6' ) import groovyx.net.http.* 01. 02. 03. 04. 05. 06. 77
  78. 78. HTTPBuilder: instantiation def baseUrl = 'http://api.duckduckgo.com' def queryString = 'q=groovy&format=json&pretty=1' def http = new HTTPBuilder(baseUrl) 01. 02. 03. 78
  79. 79. HTTPBuilder: request http.request(Method.POST) {   send ContentType.URLENC, queryString   response.success = { response, reader ‐>     println response.statusLine     println reader.text   }   response.failure = { response ‐>     println response.statusLine   } } 01. 02. 03. 04. 05. 06. 07. 08. 09. 10. 79
  80. 80. SQL 80
  81. 81. SQL: settings import groovy.sql.Sql def dbSettings = [   url: 'jdbc:hsqldb:hsql://localhost/cookingdb',   driver: 'org.hsqldb.jdbcDriver',   user: 'sa',   password: '' ] 01. 02.03. 04. 05. 06. 07. 08. 81
  82. 82. SQL: query def sql = Sql.newInstance(dbSettings) sql.eachRow('SELECT * FROM COOKBOOK') { cookbook ‐>   printf '%‐20s%sn',          cookbook.id,          cookbook[1] } 01. 02.03. 04. 05. 06. 07. 82
  83. 83. Groovy as a server (GaaS?) > groovy ‐l 4444 ‐e "println new Date()" & > telnet localhost 4444 hey groovy give me the date Wed Feb 04 10:03:23 EET 2015 01. 02. 03. 04. 83
  84. 84. Ratpack 84
  85. 85. Ratpack: import @Grab("io.ratpack:ratpack‐groovy:1.1.1") import static ratpack.groovy.Groovy.ratpack 01. 02. 85
  86. 86. Ratback: script body ratpack {   handlers {     get {       response.send "This is the app root (try: /date)"     }     get("date") {       response.send new Date().toString()     }   } } 01. 02. 03. 04. 05. 06. 07. 08. 09. 10. 86
  87. 87. Sshoogr 87
  88. 88. Sshoogr: import @Grab(   group='com.aestasit.infrastructure.sshoogr',   module='sshoogr',   version='0.9.16') import static com.aestasit.ssh.DefaultSsh.* 01. 02. 03. 04. 05. 88
  89. 89. Sshoogr: defaults defaultUser    = 'root' defaultKeyFile = new File('secret.pem') execOptions {   verbose      = true   showCommand  = true } 01. 02. 03. 04. 05. 06. 89
  90. 90. Sshoogr: connection remoteSession {   url = 'user2:654321@localhost:2222'   exec 'rm ‐rf /tmp/*'   exec 'touch /var/lock/my.pid'   remoteFile('/var/my.conf').text = "enabled=true" } 01. 02. 03. 04. 05. 06. 90
  91. 91. Sshoogr: multi­line content remoteFile('/etc/yum.repos.d/puppet.repo').text = '''   [puppet]   name=Puppet Labs Packages   baseurl=http://yum.puppetlabs.com/el/   enabled=0   gpgcheck=0 ''' 01. 02. 03. 04. 05. 06. 07. 91
  92. 92. Sshoogr: file copying remoteSession {   scp {     from { localDir "$buildDir/application" }     into { remoteDir '/var/bea/domain/application' }   } } 01. 02. 03. 04. 05. 06. 92
  93. 93. Sshoogr: command result def result = exec(command: '/usr/bin/mycmd',   failOnError: false, showOutput: false) if (result.exitStatus == 1) {   result.output.eachLine { line ‐>     if (line.contains('WARNING')) {       throw new RuntimeException("First warning: ${line}")     }   } } 01. 02. 03. 04. 05. 06. 07. 08. 09. 93
  94. 94. Sshoogr: shortcuts if (ok('/usr/bin/mycmd')) {   ... } if (fail('/usr/bin/othercmd')) {   ... } 01. 02. 03. 04. 05. 06. 94
  95. 95. Sshoogr: tunnels tunnel('1.2.3.4', 8080) { int localPort ‐>   def url = "http://localhost:${localPort}/flushCache"   def result = new URL(url).text   if (result == 'OK') {     println "Cache is flushed!"   } else {     throw new RuntimeException(result)   } } 01. 02. 03. 04. 05. 06. 07. 08. 09. 95
  96. 96. Sshoogr: prefix/suffix prefix('sudo ') {   exec 'rm ‐rf /var/log/abc.log'   exec 'service abc restart' } suffix(' >> output.log') {   exec 'yum ‐y install nginx'   exec 'yum ‐y install mc'   exec 'yum ‐y install links' } 01. 02. 03. 04. 05. 06. 07. 08. 09. 96
  97. 97. Summary 97
  98. 98. Take­aways Groovy scripts are portable. Groovy scripts are concise. Groovy scripts are powerful. Well, the name is groovy as well! • • • • 98
  99. 99. Danke schön! 99
  100. 100. Questions? 100
  101. 101. Happy scripting! 101

×