© Schalk W. Cronjé Greach 2014
Groovy VFS
Schalk W. Cronjé
@ysb33r
© Schalk W. Cronjé Greach 2014
Quickstart
def vfs = new VFS()
vfs {
cp “http://from.here/a.txt”, “sftp://to.there/b.txt”
}
© Schalk W. Cronjé Greach 2014
Groovy VFS
Current version: 0.5
Source: http://github.com/ysb33r/groovy-vfs
Wiki: https://g...
© Schalk W. Cronjé Greach 2014
Bootstrap
@Grapes([
@Grab( 'org.ysb33r.groovy:groovy-vfs:0.5' ),
// If you wantftp
@Grab( '...
© Schalk W. Cronjé Greach 2014
Create directory on remote server
vfs {
mkdir “ftp://a.server/pub/ysb33r”
}
© Schalk W. Cronjé Greach 2014
Directory Listing
vfs {
ls (“ftp://www.mirrorservice.org/sites”) {
println it.name
}
}
© Schalk W. Cronjé Greach 2014
Set Schema Options
vfs {
options {
ftp {
passiveMode true
userDirIsRoot false
}
}
}
© Schalk W. Cronjé Greach 2014
Modify Schema Option per Request
vfs {
ls (“ftp://www.mirrorservice.org/sites?vfs.ftp.passi...
© Schalk W. Cronjé Greach 2014
Set Schema Options at Initialisation
def vfs = new VFS (
'vfs.ftp.passiveMode' : true,
'vfs...
© Schalk W. Cronjé Greach 2014
Schema Options
- Most Apache VFS filesystem options supported
- Follows Groovy property-ove...
© Schalk W. Cronjé Greach 2014
Copying & Moving
vfs {
cp “http://a.server/myFile.txt”,
“file:///home/scronje/MyFile.downlo...
© Schalk W. Cronjé Greach 2014
Copy DSL
vfs {
cp from_url,
to_url,
overwrite : false,
smash : false,
recursive : false,
fi...
© Schalk W. Cronjé Greach 2014
Interactive Copy
vfs {
cp from_url,
to_url,
overwrite : { from,to ->
println “Overwrite ${t...
© Schalk W. Cronjé Greach 2014
Move DSL
vfs {
mv from_url,
to_url,
overwrite : false,
smash : false,
intermediates : true,...
© Schalk W. Cronjé Greach 2014
Download & Unpack
vfs {
cp “zip:ftp://a.server/with-archive.zip”,
“file:///home/scronje/unp...
© Schalk W. Cronjé Greach 2014
Working with Content
import java.security.MessageDigest
vfs {
MessageDigest md5 = MessageDi...
© Schalk W. Cronjé Greach 2014
Local Files
- Prefix with 'file://'
- Use java.io.File instance
- Use org.apache.commons.vf...
© Schalk W. Cronjé Greach 2014
Authentication
vfs {
// clear password
cp new File(“a.txt”),
“ftp://${username}:${password}...
© Schalk W. Cronjé Greach 2014
Future Ideas For Investigation
●
Java 7 NIO
– java.nio.file.*
– Wait for Apache VFS or go n...
© Schalk W. Cronjé Greach 2014
VFS Gradle Plugin
(experimental / incubating)
© Schalk W. Cronjé Greach 2014
Bootstrap
buildscript {
repositories {
jcenter()
}
dependencies {
classpath 'org.ysb33r.gra...
© Schalk W. Cronjé Greach 2014
Available as Project Extension
task ftpCopy << {
vfs {
cp “ftp://a.server/file.txt”, buildD...
© Schalk W. Cronjé Greach 2014
Future Ideas For Investigation
●
vfsTree
– Akin toproject.tarTree
– Is it even possible to ...
© Schalk W. Cronjé Greach 2014
Other Plugins ?
●
Grails & Griffon
– No plugin (yet)
●
Jenkins
– Time to consolidate variou...
© Schalk W. Cronjé Greach 2014
Groovy VFS
#groovyvfs
Schalk W. Cronjé
@ysb33r
http://github.com/ysb33r/groovy-vfs
Upcoming SlideShare
Loading in...5
×

Groovy VFS

1,245

Published on

A Groovy-based DSL for working with files on remote servers and other virtual filesystems.

This is the presentation on v0.5 I did at Greach 2014

Published in: Software
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,245
On Slideshare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
10
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Groovy VFS

  1. 1. © Schalk W. Cronjé Greach 2014 Groovy VFS Schalk W. Cronjé @ysb33r
  2. 2. © Schalk W. Cronjé Greach 2014 Quickstart def vfs = new VFS() vfs { cp “http://from.here/a.txt”, “sftp://to.there/b.txt” }
  3. 3. © Schalk W. Cronjé Greach 2014 Groovy VFS Current version: 0.5 Source: http://github.com/ysb33r/groovy-vfs Wiki: https://github.com/ysb33r/groovy-vfs/wiki Packages: https://bintray.com/ysb33r/grysb33r/groovy-vfs Gradle Packages: https://bintray.com/ysb33r/grysb33r/vfs-gradle-plugin License: Apache 2.0 Twitter: #groovyvfs
  4. 4. © Schalk W. Cronjé Greach 2014 Bootstrap @Grapes([ @Grab( 'org.ysb33r.groovy:groovy-vfs:0.5' ), // If you wantftp @Grab( 'commons-net:commons-net:3.+' ), // If you want http/https @Grab( 'commons-httpclient:commons-httpclient:3.1'), // If you want sftp @Grab( 'com.jcraft:jsch:0.1.48' ) ]) import org.ysb33r.groovy.dsl.vfs.VFS
  5. 5. © Schalk W. Cronjé Greach 2014 Create directory on remote server vfs { mkdir “ftp://a.server/pub/ysb33r” }
  6. 6. © Schalk W. Cronjé Greach 2014 Directory Listing vfs { ls (“ftp://www.mirrorservice.org/sites”) { println it.name } }
  7. 7. © Schalk W. Cronjé Greach 2014 Set Schema Options vfs { options { ftp { passiveMode true userDirIsRoot false } } }
  8. 8. © Schalk W. Cronjé Greach 2014 Modify Schema Option per Request vfs { ls (“ftp://www.mirrorservice.org/sites?vfs.ftp.passiveMode=1”) { println it.name } } 220 Service ready for new user. USER guest 331 User name okay, need password for guest. PASS guest 230 User logged in, proceed. TYPE I 200 Command TYPE okay. CWD / 250 Directory changed to / SYST 215 UNIX Type: Apache FtpServer PASV 227 Entering Passive Mode (127,0,0,1,226,247) LIST 150 File status okay; about to open data connection. 226 Closing data connection.
  9. 9. © Schalk W. Cronjé Greach 2014 Set Schema Options at Initialisation def vfs = new VFS ( 'vfs.ftp.passiveMode' : true, 'vfs.http.maxTotalConnections' : 4 )
  10. 10. © Schalk W. Cronjé Greach 2014 Schema Options - Most Apache VFS filesystem options supported - Follows Groovy property-over-setter/getter convention Documentation: https://github.com/ysb33r/groovy-vfs/wiki/Protocol-Options
  11. 11. © Schalk W. Cronjé Greach 2014 Copying & Moving vfs { cp “http://a.server/myFile.txt”, “file:///home/scronje/MyFile.downloaded.txt” mv “ftp://that.server/myFile.txt”, “sftp://this.server/in-this-folder&vfs.sftp.userDirIsRoot= }
  12. 12. © Schalk W. Cronjé Greach 2014 Copy DSL vfs { cp from_url, to_url, overwrite : false, smash : false, recursive : false, filter : defaults_to_selecting_everything } Documentation: https://github.com/ysb33r/groovy-vfs/wiki/VFS-Copy-Behaviour
  13. 13. © Schalk W. Cronjé Greach 2014 Interactive Copy vfs { cp from_url, to_url, overwrite : { from,to -> println “Overwrite ${to}?” System.in.readLine().startsWith('Y') } }
  14. 14. © Schalk W. Cronjé Greach 2014 Move DSL vfs { mv from_url, to_url, overwrite : false, smash : false, intermediates : true, } Documentation: https://github.com/ysb33r/groovy-vfs/wiki/VFS-Move-Behaviour
  15. 15. © Schalk W. Cronjé Greach 2014 Download & Unpack vfs { cp “zip:ftp://a.server/with-archive.zip”, “file:///home/scronje/unpacked” } Warning: Large archives will cause slow-down due to performance bug in Apache VFS
  16. 16. © Schalk W. Cronjé Greach 2014 Working with Content import java.security.MessageDigest vfs { MessageDigest md5 = MessageDigest.getInstance("MD5") cat (“ftp://a.server/my-file.txt”) { strm → strm.eachByte(8192) { buffer, nbytes -> md5.update( buffer, 0, nbytes ) } } println md5.digest().encodeHex().toString() .padLeft( 32, '0' ) } // From a conversation on groovy-user ML
  17. 17. © Schalk W. Cronjé Greach 2014 Local Files - Prefix with 'file://' - Use java.io.File instance - Use org.apache.commons.vfs2.FileObject instance vfs { cp new File(“a.txt”), “ftp://to.server/pub” }
  18. 18. © Schalk W. Cronjé Greach 2014 Authentication vfs { // clear password cp new File(“a.txt”), “ftp://${username}:${password}@to.server/pub” // password encrypted with org.apache.commons.vfs2.util.EncryptUtil cp new File('b.txt'), 'ftp://testuser:{D7B82198B272F5C93790FEB38A73C7B8}@to.server/pub' } Warning: User with caution (security …) Enhancement: ISSUE-17 to provide better authentication DSL support
  19. 19. © Schalk W. Cronjé Greach 2014 Future Ideas For Investigation ● Java 7 NIO – java.nio.file.* – Wait for Apache VFS or go native Groovy? ● Direct SMB support? – Apache VFS Sandbox – jCIFS license is LGPL
  20. 20. © Schalk W. Cronjé Greach 2014 VFS Gradle Plugin (experimental / incubating)
  21. 21. © Schalk W. Cronjé Greach 2014 Bootstrap buildscript { repositories { jcenter() } dependencies { classpath 'org.ysb33r.gradle:vfs-gradle-plugin:0.5' classpath 'commons-net:commons-net:3.+' // ftp classpath 'commons-httpclient:commons-httpclient:3.1' // http/https classpath 'com.jcraft:jsch:0.1.48' // sftp } } apply plugin : 'vfs'
  22. 22. © Schalk W. Cronjé Greach 2014 Available as Project Extension task ftpCopy << { vfs { cp “ftp://a.server/file.txt”, buildDir } }
  23. 23. © Schalk W. Cronjé Greach 2014 Future Ideas For Investigation ● vfsTree – Akin toproject.tarTree – Is it even possible to createFileTree? ● Vfs Copy / Move Task – How to define @Input etc. correctly
  24. 24. © Schalk W. Cronjé Greach 2014 Other Plugins ? ● Grails & Griffon – No plugin (yet) ● Jenkins – Time to consolidate various protocol plugins Who wants to write one?
  25. 25. © Schalk W. Cronjé Greach 2014 Groovy VFS #groovyvfs Schalk W. Cronjé @ysb33r http://github.com/ysb33r/groovy-vfs
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×