Scripting GeoServer with GeoScript

4,785 views

Published on

GeoServer is a solid and mature implementation of a variety of OGC services including Web Feature Service, Web Map Service, Web Coverage Service, and Web Processing Service. Add to this a KML engine, integrated security framework, powerful styling language with SLD and this rich feature set makes GeoServer very appealing to the user. However it has always been somewhat lacking when it comes to the developer. Developing with GeoServer has a steep learning curve and requires expert knowledge to do simple tasks like writing new output formats, implementing new WPS processes, and adding custom filter functions. GeoScript to the rescue!

GeoScript adds spatial capabilities to popular languages such as Python, JavaScript, Scala, and Groovy. Scripting languages are the perfect tool for developers who want to do simple coding tasks quickly in a lightweight development environment. GeoScript builds on top of the very powerful GeoTools library to provide an interface to its capabilities though concise and easy to use API's. Recent extensions to GeoServer now allow developers to write components and plug-ins in the scripting language of their choice, using GeoScript as the engine for spatial functionality.

This presentation is geared toward developers who are interested in developing with GeoServer but not necessarily ready to get their hands dirty with low level Java. The talk will detail the various scripting hooks available and provide examples, complete with code, of how to write some simple plug-ins. Check out this presentation and you'll be developing with GeoServer in no time.

Published in: Technology

Scripting GeoServer with GeoScript

  1. 1. Scripting GeoServer with GeoScript Justin Deoliveira and Tim Schaub
  2. 2. GeoScript? <ul><ul><li>Spatial capabilities for scripting languages </li></ul></ul><ul><ul><ul><li>Groovy </li></ul></ul></ul><ul><ul><ul><li>JavaScript </li></ul></ul></ul><ul><ul><ul><li>Python </li></ul></ul></ul><ul><ul><ul><li>Scala </li></ul></ul></ul><ul><ul><li>Higher level api for GeoTools </li></ul></ul><ul><ul><ul><li>Convenient, concise, complete </li></ul></ul></ul><ul><ul><ul><li>Make easy things easy </li></ul></ul></ul><ul><ul><li>Faster development turnaround </li></ul></ul><ul><ul><ul><li>No recompilation/rebuilding </li></ul></ul></ul>
  3. 3. GeoScript
  4. 4. GeoScript
  5. 5. GeoScript ... import org.geotools.data.DataStore; import org.geotools.data.postgis.PostgisNGDataStoreFactory; import org.geotools.feature.FeatureCollection; import org.geotools.feature.FeatureIterator; import org.geotools.jdbc.JDBCDataStoreFactory; import org.opengis.feature.Feature; ... Map<String,Serializable> params = new HashMap<String, Serializable>(); params.put(JDBCDataStoreFactory.HOST.key, &quot;localhost&quot;); params.put(JDBCDataStoreFactory.PORT.key, 5432); params.put(JDBCDataStoreFactory.DATABASE.key, &quot;geoscript&quot;); params.put(JDBCDataStoreFactory.DBTYPE.key, &quot;postgis&quot;); params.put(JDBCDataStoreFactory.USER.key, &quot;jdeolive&quot;); PostgisNGDataStoreFactory factory = new PostgisNGDataStoreFactory(); DataStore pg = factory.createDataStore(params);  FeatureCollection features = pg.getFeatureSource(&quot;states&quot;).getFeatures(); FeatureIterator it = features.features(); try {   while(it.hasNext()) {       Feature f = it.next();       System.out.println(f.getProperty(&quot;STATE_NAME&quot;).getValue());   } } finally {   it.close(); } Java
  6. 6. GeoScript from geoscript.workspace import PostGIS pg = PostGIS('geoscript') for f in pg['states'].features():   print f['STATE_NAME'] Python JavaScript var PostGIS = require(&quot;geoscript/workspace&quot;).PostGIS; var pg = PostGIS(&quot;geoscript&quot;); pg.get(&quot;states&quot;).features.forEach(function(f) {     print(f.get(&quot;STATE_NAME&quot;)); });
  7. 7. GeoServer
  8. 8. GeoServer
  9. 9. Script Hooks - Data Formats <ul><li>Data Formats </li></ul><ul><ul><li>Drivers for spatial formats </li></ul></ul><ul><ul><li>Map to internal data model </li></ul></ul>
  10. 10. Script Hooks - Data Formats from geoserver import datastore from geoscript.layer import readJSON class GeoJSON(object):   @datastore('GeoJSON', 'GeoJSON', file=('GeoJSON file', str))   def __init__(self, file):     self.json = readJSON(file)   def layers(self):     return [self.json.name]   def get(self, layer):     return self.json GeoJSON DataStore
  11. 11. Script Hooks - Data Formats from geoserver import datastore from geoscript.layer import readJSON class GeoJSON(object):   @datastore('GeoJSON', 'GeoJSON', file=('GeoJSON file', str))   def __init__(self, file):     self.json = readJSON(file)   def layers(self):     return [self.json.name]   def get(self, layer):     return self.json GeoJSON DataStore
  12. 12. Script Hooks - Output Formats <ul><li>Output Formats </li></ul><ul><ul><li>Drivers for exchange formats </li></ul></ul><ul><ul><li>Map from internal data model </li></ul></ul>
  13. 13. Script Hooks - Output Formats from geoserver.format import vector_format @vector_format('property', 'text/plain') def write(data, out):   for f in data.features:     values = [str(val) for val in f.values()]     out.write('%s=%sn' % (f.id, '|'.join(values)) Property File Format states.1=MULTIPOLYGON (((37.51 -88.07, ... 37.51 -88.07)))|Illinois states.2=MULTIPOLYGON (((38.97 -77.00, ... 38.97 -77.01)))|District of Columbia states.3=MULTIPOLYGON (((38.56 -75.71, ... 38.56 -75.71)))|Delaware states.4=MULTIPOLYGON (((38.48 -79.23, ... 38.48 -79.23)))|West Virginia   .   .   . .../wfs?request=GetFeature&typename=topp:states&outputformat=property
  14. 14. Script Hooks - Process <ul><li>Processes </li></ul><ul><ul><li>Smarts of WPS </li></ul></ul><ul><ul><li>Simplicity of scripting </li></ul></ul>
  15. 15. Script Hooks - Process var Process = require(&quot;geoscript/process&quot;).Process; exports.process = new Process({     title: &quot;JavaScript Buffer Process&quot;,     description: &quot;Process that buffers a geometry.&quot;,     inputs: {         geom: {             type: &quot;Geometry&quot;,             title: &quot;Input Geometry&quot;         },         distance: {             type: &quot;Double&quot;,             title: &quot;Buffer Distance&quot;         }     },     outputs: {         result: {             type: &quot;Geometry&quot;,             title: &quot;Result&quot;         }     },     run: function(inputs) {         return {result: inputs.geom.buffer(inputs.distance)};     } }); JavaScript
  16. 16. Script Hooks - Filter Functions <ul><li>Filter Functions </li></ul><ul><ul><li>Filtering for WFS and WMS </li></ul></ul><ul><ul><li>Callable via SLD </li></ul></ul>from geosever.filter import function from geoscript.geom import Polygon @function def areaGreaterThan(feature, area):   return feature.geom.area > area Area Function
  17. 17. Script Hooks - Transactions Intercept WFS transactions with a wfs.js script in your data directory. exports.beforeCommit = function(details, request) {     LOGGER.info(&quot;beforeCommit&quot;);     var records = details[&quot;PreInsert&quot;] || [];          records.forEach(function(record) {         var feature = record.feature;         feature.geometry = feature.geometry.simplify(10);     }); }; JavaScript
  18. 18. Script Hooks - Web/HTTP <ul><li>&quot;apps&quot; </li></ul><ul><ul><li>Simple WGSI-like environment </li></ul></ul><ul><ul><li>Access to catalog/data </li></ul></ul>def app(environ, start_response):    start_response('200 OK', [('Content-type','text/plain')])    return 'Hello world!' Hello World App
  19. 19. Data Summary App from geoserver.catalog import Layer from StringIO import StringIO def app(env, start_response):   kvp = dict([tuple(kv.split('=')) for kv in env['QUERY_STRING'].split('&')])   layer = kvp['layer']   l = Layer(layer, store=None)   buf = StringIO()   buf.write('Layer: %sn' % l.name)   data = l.data   buf.write(' Format: %sn' % data.format)   buf.write(' Feature count: %dn' % data.count())   buf.write(' CRS/Projection: %sn' % data.proj.wkt)   b = data.bounds()   buf.write(' Bounds: (%f,%f,%f,%f)n' % (b.west, b.south, b.east, b.north))   buf.write(' Fields:n')   buf.write('n'.join(['   %s' % repr(fld) for fld in data.schema.fields]))   buf.write('n')   start_response('200 OK', [('Content-type','text/plain')])   return buf.getvalue()
  20. 20. Data Summary App from geoserver.catalog import Layer from StringIO import StringIO def app(env, start_response):   kvp = dict([tuple(kv.split('=')) for kv in env['QUERY_STRING'].split('&')])   layer = kvp['layer']   l = Layer(layer, store=None)   buf = StringIO()   buf.write('Layer: %sn' % l.name)   data = l.data   buf.write(' Format: %sn' % data.format)   buf.write(' Feature count: %dn' % data.count())   buf.write(' CRS/Projection: %sn' % data.proj.wkt)   b = data.bounds()   buf.write(' Bounds: (%f,%f,%f,%f)n' % (b.west, b.south, b.east, b.north))   buf.write(' Fields:n')   buf.write('n'.join(['   %s' % repr(fld) for fld in data.schema.fields]))   buf.write('n')   start_response('200 OK', [('Content-type','text/plain')])   return buf.getvalue()
  21. 21. Data Summary App from geoserver.catalog import Layer from StringIO import StringIO def app(env, start_response):   kvp = dict([tuple(kv.split('=')) for kv in env['QUERY_STRING'].split('&')])   layer = kvp['layer']   l = Layer(layer, store=None)   buf = StringIO()   buf.write('Layer: %sn' % l.name)   data = l.data   buf.write(' Format: %sn' % data.format)   buf.write(' Feature count: %dn' % data.count())   buf.write(' CRS/Projection: %sn' % data.proj.wkt)   b = data.bounds()   buf.write(' Bounds: (%f,%f,%f,%f)n' % (b.west, b.south, b.east, b.north))   buf.write(' Fields:n')   buf.write('n'.join(['   %s' % repr(fld) for fld in data.schema.fields]))   buf.write('n')   start_response('200 OK', [('Content-type','text/plain')])   return buf.getvalue()
  22. 22. Data Summary App from geoserver.catalog import Layer from StringIO import StringIO def app(env, start_response):   kvp = dict([tuple(kv.split('=')) for kv in env['QUERY_STRING'].split('&')])   layer = kvp['layer']   l = Layer(layer, store=None)   buf = StringIO()   buf.write('Layer: %sn' % l.name)   data = l.data   buf.write(' Format: %sn' % data.format)   buf.write(' Feature count: %dn' % data.count())   buf.write(' CRS/Projection: %sn' % data.proj.wkt)   b = data.bounds()   buf.write(' Bounds: (%f,%f,%f,%f)n' % (b.west, b.south, b.east, b.north))   buf.write(' Fields:n')   buf.write('n'.join(['   %s' % repr(fld) for fld in data.schema.fields]))   buf.write('n')   start_response('200 OK', [('Content-type','text/plain')])   return buf.getvalue()
  23. 23. Data Summary App Demo
  24. 24. Fusion Tables DataStore class GFT(object):   @datastore('GFT', 'Google Fusion Tables',                user=('User email', str), passwd=('Password', str))    def __init__(self, user, passwd):     token = ClientLogin().authorize(user, passwd)     self.ft = ftclient.ClientLoginFTClient(token)   def layers(self):     return [tbl.name for tbl in self.tables()]          def get(self, layer):     try:       return Layer(filter(lambda t: t.name == layer, self.tables())[0])     except IndexError:       raise Exception('No table named %s' % layer)   def tables(self):     tables = self.ft.query(SQL().showTables())     return [Table(self,*row.split(',')) for row in tables.split('n')[1:-1]]
  25. 25. Fusion Tables DataStore class GFT(object):   @datastore('GFT', 'Google Fusion Tables',                user=('User email', str), passwd=('Password', str))    def __init__(self, user, passwd):     token = ClientLogin().authorize(user, passwd)     self.ft = ftclient.ClientLoginFTClient(token)   def layers(self):     return [tbl.name for tbl in self.tables()]          def get(self, layer):     try:       return Layer(filter(lambda t: t.name == layer, self.tables())[0])     except IndexError:       raise Exception('No table named %s' % layer)   def tables(self):     tables = self.ft.query(SQL().showTables())     return [Table(self,*row.split(',')) for row in tables.split('n')[1:-1]]
  26. 26. Fusion Tables DataStore class GFT(object):   @datastore('GFT', 'Google Fusion Tables',                user=('User email', str), passwd=('Password', str))    def __init__(self, user, passwd):     token = ClientLogin().authorize(user, passwd)     self.ft = ftclient.ClientLoginFTClient(token)   def layers(self):     return [tbl.name for tbl in self.tables()]          def get(self, layer):     try:       return Layer(filter(lambda t: t.name == layer, self.tables())[0])     except IndexError:       raise Exception('No table named %s' % layer)   def tables(self):     tables = self.ft.query(SQL().showTables())     return [Table(self,*row.split(',')) for row in tables.split('n')[1:-1]]
  27. 27. Fusion Tables DataStore class GFT(object):   @datastore('GFT', 'Google Fusion Tables',                user=('User email', str), passwd=('Password', str))    def __init__(self, user, passwd):     token = ClientLogin().authorize(user, passwd)     self.ft = ftclient.ClientLoginFTClient(token)   def layers(self):     return [tbl.name for tbl in self.tables()]          def get(self, layer):     try:       return Layer(filter(lambda t: t.name == layer, self.tables())[0])     except IndexError:       raise Exception('No table named %s' % layer)   def tables(self):     tables = self.ft.query(SQL().showTables())     return [Table(self,*row.split(',')) for row in tables.split('n')[1:-1]]
  28. 28. Fusion Tables DataStore __types = {'string':str, 'number': float, 'location':Geometry} class Layer(object):     def __init__(self, tbl):       self.tbl = tbl       self.name = tbl.name       self.workspace = tbl.gft       self.proj = Projection('epsg:4326')       self.schema = Schema(tbl.name, [(col[0], __types[col[1]]) for col in tbl.schema()])     def bounds(self):       return reduce(lambda x,y: x.expand(y.bounds), self.features(), Bounds())     def features(self):       ...
  29. 29. Fusion Tables DataStore __types = {'string':str, 'number': float, 'location':Geometry} class Layer(object):     def __init__(self, tbl):       self.tbl = tbl       self.name = tbl.name       self.workspace = tbl.gft       self.proj = Projection('epsg:4326')       self.schema = Schema(tbl.name, [(col[0], __types[col[1]])          for col in tbl.schema()])     def bounds(self):       return reduce(lambda x,y: x.expand(y.bounds), self.features(), Bounds())     def features(self):       ...
  30. 30. Fusion Tables DataStore __types = {'string':str, 'number': float, 'location':Geometry} class Layer(object):     def __init__(self, tbl):       self.tbl = tbl       self.name = tbl.name       self.workspace = tbl.gft       self.proj = Projection('epsg:4326')       self.schema = Schema(tbl.name, [(col[0], __types[col[1]])          for col in tbl.schema()])     def bounds(self):       return reduce(lambda x,y: x.expand(y.bounds), self.features(), Bounds())     def features(self):       ...
  31. 31. Fusion Tables DataStore class Layer(object):     ...     def features(self):       rows = self.tbl.gft.ft.query(SQL().select(self.tbl.id))       rows = rows.split('n')[1:-1]       for row in rows:         vals = csv.reader([row]).next()         atts = []         for i in range(0, len(vals)):           val = vals[i]           fld = self.schema.get(i)           if issubclass(fld.typ, Geometry):             val = readKML(val)           atts.append(val)          yield Feature(atts, schema=self.schema)
  32. 32. Fusion Tables DataStore Demo
  33. 33. H2 Output Format @vector_format('h2', 'application/zip') def write(data, out):   dir = tempfile.mkdtemp()   # create the database and copy the features into it   db = H2(data.schema.name, dir=dir)   layer = db.create(schema=data.schema)   for f in data.features:     layer.add(f)   db.close()   # zip and ship   file = tempfile.mktemp()   zip = zipfile.ZipFile(file, 'w')   for root, dirs, files in os.walk(dir):     name = abspath(root)[len(abspath(dir)):]       for f in files:       zip.write(join(root,f), join(name,f), zipfile.ZIP_DEFLATED)   zip.close()   shutil.copyfileobj(open(file, 'r'), out)   # clean up   os.remove(file)   shutil.rmtree(dir)
  34. 34. H2 Output Format @vector_format('h2', 'application/zip') def write(data, out):   dir = tempfile.mkdtemp()   # create the database and copy the features into it   db = H2(data.schema.name, dir=dir)   layer = db.create(schema=data.schema)   for f in data.features:     layer.add(f)   db.close()   # zip and ship   file = tempfile.mktemp()   zip = zipfile.ZipFile(file, 'w')   for root, dirs, files in os.walk(dir):     name = abspath(root)[len(abspath(dir)):]       for f in files:       zip.write(join(root,f), join(name,f), zipfile.ZIP_DEFLATED)   zip.close()   shutil.copyfileobj(open(file, 'r'), out)   # clean up   os.remove(file)   shutil.rmtree(dir)
  35. 35. H2 Output Format @vector_format('h2', 'application/zip') def write(data, out):   dir = tempfile.mkdtemp()   # create the database and copy the features into it   db = H2(data.schema.name, dir=dir)   layer = db.create(schema=data.schema)   for f in data.features:     layer.add(f)   db.close()   # zip and ship   file = tempfile.mktemp()   zip = zipfile.ZipFile(file, 'w')   for root, dirs, files in os.walk(dir):     name = abspath(root)[len(abspath(dir)):]       for f in files:       zip.write(join(root,f), join(name,f), zipfile.ZIP_DEFLATED)   zip.close()   shutil.copyfileobj(open(file, 'r'), out)   # clean up   os.remove(file)   shutil.rmtree(dir)
  36. 36. H2 Output Format @vector_format('h2', 'application/zip') def write(data, out):   dir = tempfile.mkdtemp()   # create the database and copy the features into it   db = H2(data.schema.name, dir=dir)   layer = db.create(schema=data.schema)   for f in data.features:     layer.add(f)   db.close()   # zip and ship   file = tempfile.mktemp()   zip = zipfile.ZipFile(file, 'w')   for root, dirs, files in os.walk(dir):     name = abspath(root)[len(abspath(dir)):]       for f in files:       zip.write(join(root,f), join(name,f), zipfile.ZIP_DEFLATED)   zip.close()   shutil.copyfileobj(open(file, 'r'), out)   # clean up   os.remove(file)   shutil.rmtree(dir)
  37. 37. H2 Output Format Demo
  38. 38. Scripted WPS and  WFS Transaction Hooks Demo
  39. 39. Thanks! http://geoscript.org http://geoserver.org Questions?

×