A Groovy-based DSL for working with files on remote servers and other virtual filesystems. This is the presentation on v0.5 I did at Greach 2014

1. © Schalk W. Cronjé Greach 2014
Groovy VFS
Schalk W. Cronjé
@ysb33r

2. © Schalk W. Cronjé Greach 2014
Quickstart
def vfs = new VFS()
vfs {
cp “http://from.here/a.txt”, “sftp://to.there/b.txt”
}

3. © Schalk W. Cronjé Greach 2014
Groovy VFS
Current version: 0.5
Source: http://github.com/ysb33r/groovy-vfs
Wiki: https://github.com/ysb33r/groovy-vfs/wiki
Packages: https://bintray.com/ysb33r/grysb33r/groovy-vfs
Gradle Packages: https://bintray.com/ysb33r/grysb33r/vfs-gradle-plugin
License: Apache 2.0
Twitter: #groovyvfs

4. © Schalk W. Cronjé Greach 2014
Bootstrap
@Grapes([
@Grab( 'org.ysb33r.groovy:groovy-vfs:0.5' ),
// If you wantftp
@Grab( 'commons-net:commons-net:3.+' ),
// If you want http/https
@Grab( 'commons-httpclient:commons-httpclient:3.1'),
// If you want sftp
@Grab( 'com.jcraft:jsch:0.1.48' )
])
import org.ysb33r.groovy.dsl.vfs.VFS

5. © Schalk W. Cronjé Greach 2014
Create directory on remote server
vfs {
mkdir “ftp://a.server/pub/ysb33r”
}

6. © Schalk W. Cronjé Greach 2014
Directory Listing
vfs {
ls (“ftp://www.mirrorservice.org/sites”) {
println it.name
}
}

7. © Schalk W. Cronjé Greach 2014
Set Schema Options
vfs {
options {
ftp {
passiveMode true
userDirIsRoot false
}
}
}

8. © Schalk W. Cronjé Greach 2014
Modify Schema Option per Request
vfs {
ls (“ftp://www.mirrorservice.org/sites?vfs.ftp.passiveMode=1”) {
println it.name
}
}
220 Service ready for new user.
USER guest
331 User name okay, need password for guest.
PASS guest
230 User logged in, proceed.
TYPE I
200 Command TYPE okay.
CWD /
250 Directory changed to /
SYST
215 UNIX Type: Apache FtpServer
PASV
227 Entering Passive Mode (127,0,0,1,226,247)
LIST
150 File status okay; about to open data connection.
226 Closing data connection.

9. © Schalk W. Cronjé Greach 2014
Set Schema Options at Initialisation
def vfs = new VFS (
'vfs.ftp.passiveMode' : true,
'vfs.http.maxTotalConnections' : 4
)

10. © Schalk W. Cronjé Greach 2014
Schema Options
- Most Apache VFS filesystem options supported
- Follows Groovy property-over-setter/getter convention
Documentation:
https://github.com/ysb33r/groovy-vfs/wiki/Protocol-Options

11. © Schalk W. Cronjé Greach 2014
Copying & Moving
vfs {
cp “http://a.server/myFile.txt”,
“file:///home/scronje/MyFile.downloaded.txt”
mv “ftp://that.server/myFile.txt”,
“sftp://this.server/in-this-folder&vfs.sftp.userDirIsRoot=
}

12. © Schalk W. Cronjé Greach 2014
Copy DSL
vfs {
cp from_url,
to_url,
overwrite : false,
smash : false,
recursive : false,
filter : defaults_to_selecting_everything
}
Documentation:
https://github.com/ysb33r/groovy-vfs/wiki/VFS-Copy-Behaviour

13. © Schalk W. Cronjé Greach 2014
Interactive Copy
vfs {
cp from_url,
to_url,
overwrite : { from,to ->
println “Overwrite ${to}?”
System.in.readLine().startsWith('Y')
}
}

14. © Schalk W. Cronjé Greach 2014
Move DSL
vfs {
mv from_url,
to_url,
overwrite : false,
smash : false,
intermediates : true,
}
Documentation:
https://github.com/ysb33r/groovy-vfs/wiki/VFS-Move-Behaviour

15. © Schalk W. Cronjé Greach 2014
Download & Unpack
vfs {
cp “zip:ftp://a.server/with-archive.zip”,
“file:///home/scronje/unpacked”
}
Warning:
Large archives will cause slow-down due to performance bug in Apache VFS

16. © Schalk W. Cronjé Greach 2014
Working with Content
import java.security.MessageDigest
vfs {
MessageDigest md5 = MessageDigest.getInstance("MD5")
cat (“ftp://a.server/my-file.txt”) { strm →
strm.eachByte(8192) { buffer, nbytes ->
md5.update( buffer, 0, nbytes )
}
}
println md5.digest().encodeHex().toString()
.padLeft( 32, '0' )
} // From a conversation on groovy-user ML

17. © Schalk W. Cronjé Greach 2014
Local Files
- Prefix with 'file://'
- Use java.io.File instance
- Use org.apache.commons.vfs2.FileObject instance
vfs {
cp new File(“a.txt”), “ftp://to.server/pub”
}

18. © Schalk W. Cronjé Greach 2014
Authentication
vfs {
// clear password
cp new File(“a.txt”),
“ftp://${username}:${password}@to.server/pub”
// password encrypted with org.apache.commons.vfs2.util.EncryptUtil
cp new File('b.txt'),
'ftp://testuser:{D7B82198B272F5C93790FEB38A73C7B8}@to.server/pub'
}
Warning: User with caution (security …)
Enhancement: ISSUE-17 to provide better authentication DSL support

19. © Schalk W. Cronjé Greach 2014
Future Ideas For Investigation
●
Java 7 NIO
– java.nio.file.*
– Wait for Apache VFS or go native Groovy?
●
Direct SMB support?
– Apache VFS Sandbox
– jCIFS license is LGPL

20. © Schalk W. Cronjé Greach 2014
VFS Gradle Plugin
(experimental / incubating)

21. © Schalk W. Cronjé Greach 2014
Bootstrap
buildscript {
repositories {
jcenter()
}
dependencies {
classpath 'org.ysb33r.gradle:vfs-gradle-plugin:0.5'
classpath 'commons-net:commons-net:3.+' // ftp
classpath 'commons-httpclient:commons-httpclient:3.1' // http/https
classpath 'com.jcraft:jsch:0.1.48' // sftp
}
}
apply plugin : 'vfs'

22. © Schalk W. Cronjé Greach 2014
Available as Project Extension
task ftpCopy << {
vfs {
cp “ftp://a.server/file.txt”, buildDir
}
}

23. © Schalk W. Cronjé Greach 2014
Future Ideas For Investigation
●
vfsTree
– Akin toproject.tarTree
– Is it even possible to createFileTree?
●
Vfs Copy / Move Task
– How to define @Input etc. correctly

24. © Schalk W. Cronjé Greach 2014
Other Plugins ?
●
Grails & Griffon
– No plugin (yet)
●
Jenkins
– Time to consolidate various protocol
plugins
Who wants to write one?

25. © Schalk W. Cronjé Greach 2014
Groovy VFS
#groovyvfs
Schalk W. Cronjé
@ysb33r
http://github.com/ysb33r/groovy-vfs