State of GeoTools 2012 edition for Latinoware conference

1. State of GeoTools
1

2. About your Presenter
Jody Garnett LISAsoft
Activities LISAsoft
GeoTools PMC A great system integration
GeoServer PSC company helping our
uDig PSC customers make effective
use of open source spatial.
OSGeo Charter Member
OSGeo Incubation Chair
LocationTech 2

3. Thanks to Previous Presenters
Justin Deolivera Andrea Aime
(PMC) (PMC)
OpenGeo GeoSolutions
Bringing the best open source GeoServer/GeoTools core
geospatial software to developers, raster data
organisations around the management, map rendering,
world. spatial data processing
3

4. GeoTools
The Java GIS Toolkit
4

5. GeoTools - the Java GIS Toolkit
5

6. About GeoTools
Open Source 1994 - Present
LGPL Really old for Java
Open Development Started in Leads University
Public Process OSGeo Project in 2006
Anyone can Play Active / Diverse committers
6

7. History
1994 GeoTools (and GeoTool “Lite”)
Version 1 had more users then documentation
Solution was to fire the users ...
2002 GeoTools 2
We learned - GeoTools 2 leans on the OGC standards
OGC provides public standards
They are documentation we do not have to write
Benefit: Development was faster without inventing names
2010 GeoTools
Back to simply “GeoTools”
7

8. Traditional Diagram of Boxes
8

9. Why Mapping
9

10. Why Mapping
Map for Your
Mother!
(or Dad!)
10

11. Do your Parents Know what you Do?
11

12. UML bad / Map good
GeoServer Gallery by
Text Gérald Estadieu
12

13. State of GeoTools
2012
13

14. Community
261 Andrea Aime (GeoServer) - GeoSolutions
geotools-devel: 261 Jody Garnett (uDig) - Lisasoft
167 Simone(GeoServer) - GeoSolutions
414 members total 123 Justin Deoliveira (GeoServer) - OpenGeo
114 Michael Bedward (JAITools)
geotools-gt2-users: 66 Danielei (GeoServer) GeoSolutions
54 Ben Caradoc Davies - CSIRO
969 members total 51 Niels Charlier - CSIRO
32 moovida (JGrass) - Hydrologis
28 ang05a - CSIRO
2010 Stats tracking commits 25 groldan - (GeoServer) - OpenGeo
17 mpazos - Axios
15 victortey - CSIRO
30 committers 9 kengu - Norwegian Uni of Tech & Science
10 organisations 8 ischneider - OpenGeo
7 ianturton - University of Leeds
7 jesseeichar - CampToCamp
7 lmoran - Consultant
6 afabiani - GeoSolutions
2 alfonx (AtlasStyler) - WikiSquare 14

15. Summary
• Ohloh summary
Q: Jump in Size?
A: Move to SVN then
move to GIT
15

16. Issue Tracker Activity
Real Lock Down this
year for 8.0 Release!
Created vs resolved tickets (click image for update )
16

17. Name Change
Dropping the “2”
17

18. GeoTools 2 now called GeoTools
• What is in a Name?
• sed/GeoTools 2.8.0/GeoTools 8.0/
• What does this mean?
• Not very much - business as usual
• What does that really mean?
• It means there is no plans for a “GeoTools 3” rewrite :-)
• Confidence in our community
• Continue our commitment to stability
• Evolution handled with continuous small changes
18

19. Documentation
Its a feature - really
19

20. User Guide - Before
20

21. User Guide - After
21

22. Code Examples - Before (Wiki)
private synchronized static FeatureSource
buildMemoryFeatureSource(FeatureCollection coll, String typename,
FeatureType currentFeatureType, AttributeType newTypes[]){
FeatureType ftState=null;
MemoryDataStore MStore=null;
try {
MStore = new MemoryDataStore();
//---- current attributes
AttributeType currentTypes[] = currentFeatureType.getAttributeTypes();
Compile?
//---- new array of attributes = current array+ new attributes
AttributeType typesNew[] = new What Version?
AttributeType[currentTypes.length+newTypes.length]; When?
for(int i=0;i<currentTypes.length;i++){
typesNew[i] = currentTypes[i];
} All very good
for(int i=0;i<newTypes.length;i++){
typesNew[currentTypes.length+i] = newTypes[i]; questions with no clear
} answer.
ftState = FeatureTypeFactory.newFeatureType(typesNew, typename);
MStore.createSchema(ftState);
Iterator iterator = coll.iterator();
FeatureCollection newColl = FeatureCollections.newCollection();
Feature feature, newFeature;
Object[] objs;
try {
for( int count=0; iterator.hasNext(); count++) {
22
feature = (Feature) iterator.next();

23. Code Examples - After (Sphinx)
private synchronized static FeatureSource
SimpleFeatureSource
alter(SimpleFeatureCollection
collection,
String
typename,
buildMemoryFeatureSource(FeatureCollection coll, String typename,
SimpleFeatureType
featureType,
final
List<AttributeDescriptor>
newTypes)
{
FeatureType currentFeatureType, AttributeType newTypes[]){
try
{
FeatureType ftState=null;
MemoryDataStore MStore=null;
//
Create
target
schema
try {
SimpleFeatureTypeBuilder
buildType
=
new
SimpleFeatureTypeBuilder();
MStore = new MemoryDataStore();
buildType.init(featureType);
//---- current attributes
buildType.setName(typename);
AttributeType currentTypes[] = currentFeatureType.getAttributeTypes();
buildType.addAll(newTypes); Pulled Live from
//---- new array of attributes = current array+ new attributes
final
SimpleFeatureType
schema
=
buildType.buildFeatureType();
Source!
AttributeType typesNew[] = new
//
Configure
memory
datastore
AttributeType[currentTypes.length+newTypes.length];
final
MemoryDataStore
memory
=
new
MemoryDataStore();
memory.createSchema(schema); GeoTools Version at
for(int i=0;i<currentTypes.length;i++){
typesNew[i] = currentTypes[i];
collection.accepts(new
FeatureVisitor()
{
the Top of the Page!
}
public
void
visit(Feature
feature)
{
for(int i=0;i<newTypes.length;i++){
SimpleFeatureBuilder
builder
=
new
SimpleFeatureBuilder(schema);
typesNew[currentTypes.length+i] = newTypes[i];
}
builder.init((SimpleFeature)
feature);
ftState = FeatureTypeFactory.newFeatureType(typesNew, typename);
for
(AttributeDescriptor
descriptor
:
newTypes)
{
MStore.createSchema(ftState);
builder.add(DataUtilities.defaultValue(descriptor));
}
= coll.iterator();
Iterator iterator
FeatureCollection newColl = FeatureCollections.newCollection();
SimpleFeature
newFeature
=
builder.buildFeature(feature.getIdentifier().getID());
Feature feature, newFeature;
memory.addFeature(newFeature);
}objs;
Object[]
try {
},
null);
23
for( int count=0; iterator.hasNext(); count++) {

24. Code Examples - Example
Used for both Java
code examples and
also configuration files
such as maven
pom.xml.
24

25. Diagrams - Before (Visio)
25

26. Diagrams - After (Inkscape)
26

27. Class Diagrams - Before
27

28. Class Diagrams - After (Object Aid)
28

29. Tutorials - Before
29

30. Tutorials - After
30

31. Advanced Tutorials
Function
Finally a nice quick guide to how to create a function!
AbstractDataStore Tutorial - Property DataStore
The classic updated for GeoTools 8.
ContentDataStore Tutorial - CSVDataStore
New kid on the block; leaner; meaner and ready.
Welcome
Introduction for different developer communities.
Update
Have not used GeoTools in a while? Guide to updating.
31

32. Welcome Geomajas
32

33. Welcome GeoServer
33

34. Welcome uDig
34

35. Documentation Highlights
Details
- 291 “Pages”
- longest page is 36 pages printed out on paper
- Over 100,000 words
- Written from Feb through to May
- Accounts for every page of the wiki
New features
- Extensive FAQ
- whole project javadocs
- module by module reference
- plugins organised by module
- build instructions
35

36. Referencing
(do you know where your data is?)
36

37. Referencing
• Ellipsoids, datum, map projections
• Embedded EPSG database, over 5000 supported codes
(gt-epsg-hsql module)
• Reprojection utilities
Geometry g = ...;
CoordinateReferenceSystem wgs84 = CRS.decode("EPSG:4326");
CoordinateReferenceSystem utm32n = CRS.decode("EPSG:32632");
Geometry gtx = JTS.transform(g);
...
SimpleFetureCollection fc = featureSource.getFeature();
SimpleFeatureCollection fcTx =
new ReprojectingFeatureCollection(fc, utm32n);
37

38. Projections
Old projections
New
Aitoff
Albers Equal A.
CassiniSoldner Mollweide
Cylindrical Equidist.
Krovak Robinson
Lambert Conf.
Mercator
Oblique Mercator Eckert IV
Orthogonal
Polyconic
Stereographic Winkel Tripel
Transverse Mercator 38

39. Vector Plugins
Both direction and magnitude!
39

40. Vector Data Sources
• Application schema* • JDBC
• ArcSDE • DB2
• Aggregating * • H2 spatial
• CouchDb * • MySQL
• CSV * • Postgis
• DXF * • Oracle
• Excel • SQL Server
• Edigeo • Spatialite *
• PreGeneralized • Teradata *
• SimpleFeatureService * • Ingress
• Shapefile (directory of)
• WFS
name* = new entry
name = “unsupported”
40

41. JDBC DataStore "NG"
• JDBC stores:
• common base
• over 200 ready to use tests, just prepare the test data and
run
• Parametric SQL views: define a new data source as a raw
query
• Extensions for native joins
select ST_Union(the_geom) as geom
from countries
where REGION = %region%
41

42. Application schema
• Complex feature support
• Recent development by CSIRO
• Feature chaining, polymorphism
• Better performance (native DBMS joins), memory use
• Usable as vector data source, GML encoding, rendering
42

43. Raster Formats
pixel power
43

44. Raster Data Sources
• ArcGrid • imageio-ext based
• ArcSDE • AIG
• GeoTiff • DTED
• GRASS * • ECW
• GTopo30 • EnviHdr
• Image Mosaic • ErdasImg
• Image Mosaic JDBC * • EsriHdrJP2
• Image Pyramid • MrSID
• Image + World file • NITF*
• JP2 direct Kakadu • RPFTOC
• Matlab
• Image collection * name * = new entry
name = unsupported land
44

45. Non geo-referenced data
45

46. Image mosaic
46

47. Rendering
Gotta see it to believe it
47

48. Rendering
48

49. Layers
• FeatureLayer:
any vector + Style
(shapefile, spatial db, WFS, ...)
• GridCoverageLayer:
in memory raster grid
• GridReaderLayer:
dynamic raster read from disk
(leverage overviews and tiling)
• WMSLayer:
get maps from a WMS server
• DirectLayer: (new feature!)
roll your own custom paint method
49

50. StyleBuilder
• Style API is inspired by SLD 1.0 and SE 1.1:
• a style is a complex, deep object graph
• StyleBuilder provides a set of hierarchical builders
• Build a style easily, just pick the portion you need
Style style = new StrokeBuilder().color(Color.BLUE)
.width(3).dashArray(5, 2)
.buildStyle();
50

51. StyleBuilder
• StyleBuilder allows you to freely mix CQL expressions
• Provides “literate” methods allowing you to define your style
using your IDE autocomplete
fts = new FeatureTypeStyleBuilder();
fts.rule().filter("pop < 200000")
.polygon().fill().colorHex("#66FF66");
fts.rule().filter("pop between 200000 and 500000")
.polygon().fill().colorHex("#33CC33");
fts.rule().filter("pop > 500000")
.polygon().fill().colorHex("#009900");
51

52. Showing the Map Part 1
To start we get the data and create a Style
In this case we open a file and generate a style from the
schema
FileDataStore store =
FileDataStoreFinder.getDataStore(file);
SimpleFeatureSource featureSource =
store.getFeatureSource();
SimpleFeatureType schema = featureSource.getSchema();
Style style = SLD.createSimpleStyle( schema );
52

53. Showing the Map Part 2
Create Map and use JMapFrame to display
MapContent map = new MapContent();
map.setTitle("Quickstart");
Layer layer = new FeatureLayer(featureSource, style);
map.addLayer(layer);
JMapFrame.showMap(map);
53

54. Pluggable Mark Factory TTF
<Mark>
<WellKnownName>ttf://Wingdings#0xF054</WellKnownName>
<Fill>
<CssParameter name="fill">#000088</CssParameter>
</Fill>
</Mark>
54

55. Pluggable Mark Factory Slash
<Fill>
<GraphicFill>
<Graphic>
<Mark>
<WellKnownName>shape://slash</WellKnownName>
<Stroke>
<CssParameter name="stroke">0xAAAAAA</
CssParameter>
</Stroke>
</Mark>
<Size>16</Size>
</Graphic>
55

56. Pluggable Mark Factories Interface
public interface MarkFactory {
public java.awt.Shape getShape(
Graphics2D graphics,
Expression symbolUrl,
Feature feature) throws Exception;
}
org.geotools.renderer.style.WellKnownMarkFactory
org.geotools.renderer.style.TTFMarkFactory
org.geotools.renderer.style.ShapeMarkFactory
56

57. Geometry transform Arrow Head
Similar to SE 1.1 transformations, but open-ended,
<PointSymbolizer>
<Geometry>
<ogc:Function name="endPoint">
<ogc:PropertyName>the_geom</ogc:PropertyName>
</ogc:Function>
</Geometry>
<Graphic>
<Mark>
<WellKnownName>shape://arrow</WellKnownName>
<Fill/>
<Stroke/>
</Mark>
<Rotation>
<ogc:Function name="endAngle">
<ogc:PropertyName>the_geom</ogc:PropertyName>
</ogc:Function>
</Rotation>
</Graphic>
</PointSymbolizer>
57

58. Geometry transform Drop Shadow
<Geometry>
<ogc:Function name="offset">
<ogc:PropertyName>the_geom</ogc:PropertyName>
<ogc:Literal>0.00004</ogc:Literal>
<ogc:Literal>-0.00004</ogc:Literal>
</ogc:Function>
<Geometry>
58

59. Geometry transform Implementation
Based on functions, a pluggable extension point
public class FilterFunction_offset extends FunctionExpressionImpl
implements GeometryTransformation {
...
public Object evaluate(Object feature) {
Geometry geom = getExpression(0).evaluate(feature, Geometry.class);
Double offsetX = getExpression(1).evaluate(feature, Double.class);
if(offsetX == null) offsetX = 0d;
Double offsetY = getExpression(2).evaluate(feature, Double.class);
if(offsetY == null) offsetY = 0d;
if (geom != null) {
Geometry offseted = (Geometry) geom.clone();
offseted.apply(
new OffsetOrdinateFilter(offsetX, offsetY));
return offseted;
} else {
return null;
}
59

60. Rendering Transform Wind Map
Transform data before rendering. Contour extraction, wind
maps, geometry clustering. Pluggable, add your function!
<FeatureTypeStyle>
<Transformation>
<ogc:Function
name="gs:RasterAsPointCollection">
<ogc:Function name="parameter">
<ogc:Literal>data</ogc:Literal> u v
60

61. Rendering Transform Contour Extract
<FeatureTypeStyle>
<Transformation>
<ogc:Function name="gs:Contour">
<ogc:Function name="parameter">
<ogc:Literal>data</ogc:Literal>
</ogc:Function>
<ogc:Function name="parameter">
<ogc:Literal>levels</ogc:Literal>
<ogc:Literal>1100</ogc:Literal>
<ogc:Literal>1200</ogc:Literal> ....
<ogc:Literal>1700</ogc:Literal>
<ogc:Literal>1800</ogc:Literal>
</ogc:Function>
</ogc:Function>
</Transformation>
61

62. XML
(what could be as fun as rendering? not xml)
62

63. XML
Technology Parse Encode Supports
GeoTools SAX SAX,DOM Filter, GML, SLD
GeoTools DOM DOM Filter, GML, SLD
GeoTools Transform XML Filter, GML, SLD
JAXB SAX,DOM n/a
XDO
Filter, GML, SLD, WMS, WFS1.0,
(Schema Assisted SAX,DOM XML
XSD
Generation 1)
GTXML
Filter, GML, SLD, WMS, WFS1.0,
(Schema Assisted SAX,DOM XML
WFS 1.1, WPS, XSD, and more...
Generation 2)
63

64. Schema Assisted Parsing
GeoTools Specific Technology; makes use of the schema
information to minimise the code you need to write.
You teach GeoTools how to “bind” a object once; and it will
work in any schema that uses those objects.
64

65. Schema Assisted Encoding
The same approach is used when encoding objects; basically
using the schema as a cheat sheet to sort out what to emit
next.
65

66. Extensions
high value functionality built on top of GeoTools
66

67. Brewer
Wraps up the ColorBrewer for Java Developers; generating a
normal Style you can use in GeoTools; or export to other
applications.
FeatureTypeStyle style = StyleGenerator.createFeatureTypeStyle(
groups,
ff.propertName(“population”),
colors,
"Generated Style",
schema.getGeometryDescriptor(),
StyleGenerator.ELSEMODE_IGNORE,
0.95,
stroke);
67

68. Graph
Because getting there is half the fun.
BasicLineGraphGenerator graphGen = newBasicLineGraphGenerator();
for ( MultiLineString line : lines ) {
graphGen.add( line );
}
Graph graph = graphGen.getGraph();
EdgeWeigter weighter = new EdgeWeighter() {
public double getWeight(Edge e) {
SimpleFeature feature = (SimpleFeature) e.getObject();
Geometry geometry = (Geometry)
feature.getDefaultGeometry();
return gometry.getLength();
}
};
DijkstraShortestPathFinder pf =
new DijkstraShortestPathFinder( graph, start, weighter );
pf.calculate();
Path path = pf.getPath( destination );
68

69. Process
Still in Progress
69

70. Process
Annotation Based
No more filling in massive data structures; GeoTools will
generate based on annotations against a static method.
Bridge Functions and Process
Used for rendering transformations; allowing you much
greater control of geometry prior to display.
Massive influx of High Quality Processes
The full range of processes defined by GeoServer have
been back ported resulting in some of the best examples of how
to use GeoTools.
WPS Client
Still unsupported ... nothing to see here 70

71. Process Annotation Example
Tutorial Coming soon!
@DescribeProcess(title="Buffer",
description="Buffers a geometry using distance")
@DescribeResult(description="The buffered geometry")
static public Geometry buffer(
@DescribeParameter(name="geom",
description="The geometry to be buffered")
Geometry geom,
@DescribeParameter(name="distance",
description="The distance")
double distance) {
return geom.buffer(distance);
}
71

72. Application Schema
Graduated this year!
72

73. Application Schema
AuScope has completed their work on Application Schema.
This is an interesting module that is able to take information
from several DataStores and "map" the values into the correct
shape for publication against an application schema.
This is used when the data format used cannot be generated
and must exactly match that provided by a standards body or
expert community.
Thanks to Ben and the AuScope team for their dedication on
this very challenging problem.
73

74. Teradata DataStore
Another spatial database into the mix!
74

75. Terradata DataStore
Jesse Eichar (Camptocamp) took this module from
development, a quick unsupported beta period and to
supported status in record time.
75

76. EFeature
Mapping between Eclipse Modelling Framework
and GeoTools Feature Model.
76

77. EFeature
• Kenneth Gulbrandsoy has been an amazingly active
addition to the GeoTools community. Kenneth is a PHD
student from Norwegian University of Technology and
Science.
• EFeature datastore allows teams to leverage their Java
objects into the GeoTools dynamic feature system
• Adding Spatial capabilities to EMF models
• 98% finished
• Goal is to make it a supported plugin in GeoTools
• Docs are comming soon
77

78. Grass Raster reader
Created and Graduated this year!
78

79. Grass Raster Reader
Andrea Antonello has pulled together support for the GRASS
raster format and donated it to the GeoTools library (going
through the effort to match our quality assurance and
documentation standards).
79

80. Temporal
Merged into gt-main as core feature
80

81. Temporal
The temporal module has provided an enhanced range of time
and time range operations.
Justin has gathered this up into the gt-main module as one of
the cornerstones for WFS 2.0 Filter support.
after = ff.after(
ff.property("date"),
ff.literal(temporalInstant));
within = ff.toverlaps(
ff.property("constructed_date"),
ff.literal(period));
81

82. Query Improvements
Justin takes on WFS 2.0
82

83. Query Improvements
Join
Justin has sorted out how Joins will be represented - finally!
Function
Between Jody and Justin Functions now have excellent
runtime documentation. Arguments are now listed by name;
with specific type information.
Filter Evaluation over a Collection
Niels (AuScope) has sorted out this one allowing you to
control how many matches are required (one, any, all) when
comparing groups of values.
Temporal
As mentioned temporal filter support has been added;
seriously increasing the number of operations available.
83

84. Swing and SWT
(Getting your GUI on)
84

85. Swing
The much loved gt-swing module has been used for years to
make the GeoTools tutorials visual. At long last, it is looking to
graduate to supported status.
Michael Bedward has been a
star contributor this year
cleaning up the code base,
making it testable, all the
while answering questions
on the email list.
85

86. Swing
Major clean up of the internals!
JMapPane rewritten
Ported to Layer, FeatureLayer, RasterLayer etc...
DirectLayer support for Animation / map decorations
Nice clean background rendering
Ability to add new InfoTool classes via plugin
Support for Internationalisation
Supported Status?
Docs and Test Coverage improving by the day
Coming Soon: Concurrent rendering
86

87. SWT
A new addition to the project is a port of the gt-swing code to
an easy to use component for SWT.
Andrea Antonello (HydroloGIS) was responsible for this work
and it looks like gt-swing and gt-swt will be able to share the
same data models as time progresses.
87

88. What is Past?
Reflections on departed modules
88

89. Recently Dropped
JDBCDataStore and PostGIS-Versioned
The initial proof of concept JDBC datastore created in 2003
has finally been replaced. We had a good 2 year deprecation
cycle to allow migration.
GeoAPI
The "opengis" interfaces have been folded back into the
library allowing us to take responsibility, improve javadocs
and dive into filter improvements for WFS 2.0 work.
89

90. What is Next?
Ideas on the future of GeoTools
90

91. Future Plans
Process, Process, Process
Web Processing Service is finally attracting funding, with it
comes a lot of new process ideas, implementations and
directions. Hold on it is going to be a wild ride!
Prep for Java 7 try-with-resource
Update our API to mark which items are “Closable”
FeatureCollection as a Result Set
For Java 5 we needed to prevent FeatureCollection
extending java.util.Collections - so that iterators could be
closed.
We are completing this work by removing the deprecated
method names (add, remove, etc...)
This will allow FeatureCollection to be a simple result set.
91

92. Future RnD Ideas
OSGi Bundles
Branch started to explore this topic with contributors from several
projects.
Android
Jody had a chance to look at what it would take to port GeoTools
to Android. If you are interested in doing the work talk to us at the
Code Sprint.
3D
A number of teams hook GeoTools to 3D engines each year.
Anyone want to do it as part of the community?
Geometry Curves and 3D
Tisham Dhar (CSIRO) evaulated several alternatives. GeoTools
has components of ISO 19107 provided by SYS Technologies and
the of University Fachhochschule Koeln.
MIL2525B
Another classic we revisit each year - any takers? 92

93. Open Development
Is so much more than open source
93

94. Get Involved - Open Development
GeoTools provides clear “time boxed” procedures allowing you to
plan your community involvement.
Contribute
Clean patch + test + description = contribution
Request for Change
Open RFC procedure, time boxed allowing commit three days
after submitting (we have to meet deadlines as well).
Release
Public release process - make a release to match your projects
schedule and deadlines.
Procedures are in place just add developers (or money). 94

95. Get Involved - Unsupported Area
Unsupported is a scratch pad used to work on new ideas.
Easy Commit Access
Send email and read the developers guide; a PMC member
will help you with the paper work.
Unsupported does not mean unloved
You get commit access to your module, and your work
deployed to maven nightly, and published each release.
Formal review only when Ready
Ask for a QA / Docs check at any time and graduate to a
formal supported part of the library.
95

96. Predictable Release Cycle
• The “wait” for GeoTools 8 was far too long.
• GeoTools is switching to a 6 month release cycle:
• Stable
• monthly releases
• New features that do not change the API or Stability
• Master
• 1-4: open for change proposals
• 5: release candidate
• 6: final release!
96

97. State of GeoTools
Any Questions? Any time?
97

98. Upstream Projects
Thanks! We don't Do this alone
98

99. Upstream Projects
JTS Topology Suite (JTS)
Provides the excellent Geometry support we enjoy in
GeoTools. Thanks Martin!
imageio-ext
Provides geospatial raster formats to the Java Advanced
Imaging project we use for coverages. Thanks Simone and
GeoSolutions.
jaitools
Another extension to Java Advanced Imaging this time
thanks to Michael Bedward. Really fun for processing and
generating rasters.
99