Your SlideShare is downloading. ×
0
Do your Learnings
pramsey geo.com
Are you ready to get all nerdy?
... my name is Paul Ramsey
I work for Boundless, the ope...
You are
technologists?
generalist
generalist
generalist
integrator
integrator
integrator
builder
builder
builder
And I'm g...
is loved by
me
generalist
generalist
generalist
integrator
integrator
integrator
builder
builder
builder
So,
I'm going to ...
you go to the tourist centre when you arrive,
and youget matched to a local resident who can rent
you a room.
I got matche...
Manfred
his veterinary program tested students on EVERYTHING
they learned,
all at once, in the final exams.
So he was spend...
“Learnings,
I had to do
many learnings
today.”
"learnings, I had to do many learnings today".
I've kept that with me for a...
what tools to learn?
The way we choose to solve problems
is usually dictated by the tools we already understand.
So,
one o...
Do your learnings.
Many learnings.
The reason I have so much affection for FME
is that it was one of the first tools I lear...
data is not a solid
It's a big mass, and you chip away at it a little at a time,
making incremental changes bit by bit.
Ad...
data is a liquid
When you understand data is a liquid,
your definitions of what is a "big change" or a "lot of
work" change...
good tools integrate
And if they are hard to integrate,
it's not your fault,
it's the tool's fault.
Don't blame yourself.
...
GUI FILENAME SourceDataset Drawing_Files(*.dwg)|*.dwg|Drawing_Files(*.dxf)|*.dxf|All_files(*.*)|*.* Input AutoCAD
File:
DW...
good tools integrate
run
engine
generate
mapping
file
When you can completely configure your engine with a
single text file, ...
good tools integrate
run
engine
generate
mapping
file
mapmaker.pl
list of
mapsheets
DGN
file
I also learned Perl on that pro...
Do your learnings.
Many learnings.
More learnings. Each learning made me
a little more powerful as a technologist,
and eac...
I wonder if GIS data
will fit in there?
spatial database
It was so useful, it made me wonder, "hmmm, is it
possible to put ...
postgis is born
Heard of ?
How many of you have heard of PostGIS?
postgis is born
Use ?
How many of you have used PostGIS?
postgis is born
adds
GIS
types
to
So, for those of you who are new to this: PostGIS is a
spatial extension for the open so...
postgis is born
is
to
In terms of features and performance,
if you want a product analogue, it's comparable to
Oracle Spat...
postgis is born
is
to
and as bacon is to a cheese burger.
It’s part of a larger whole,
but it makes it so much better and ...
postgis is born
Early Adopters ~2001
The City of Boston and Fulton County, in Georgia
were using PostGIS within a couple y...
Corporations
Corporate users include...
Governments
Government users include...
postgis grows up
2001 20062002 2003 2004 2005
OSSIM
As PostGIS moved from the margins to the center, the
kinds of software...
postgis grows up
2001 20062002 2003 2004 2005
PostGIS
v0.1
“fme” on
postgis-users
first request
for FME format support
So,
...
2001 20062002 2003 2004 2005
OSSIM
In the spring of 2003 Safe starts PostGIS development,
and that year the FME becomes th...
good tools teach
@SQL()
@TCL()
Integration
GNU tools
Docbook XML
C/C++
CVS/SVN/GIT
So, working with FME taught me useful t...
to share
to modify
to share
modifications
free
free
free
open
source
is
In fact, I've found doing open source so interestin...
$$$
Consumers of proprietary software have gotten used to
the idea that
"paying money for access to the bits" is the only ...
What if
were open source?
What would Safe Software be like, if the FME were
open source?
What would they sell? Where could...
Usage: ogr2ogr [--help-general] [-skipfailures] [-append] [-update]
[-select field_list] [-where restricted_where]
[-progr...
What if were open source?
for
workbench
open
engine
My favourite imaginary open source FME scenario
is one where the core ...
But
is not open source
So that was fun, but just imaginary, at least for the
FME.
For open source software, though,
busine...
open source tools
- PostGIS I've mentioned, and manipulating and
querying data in SQL is a wonderful skill to have, since
...
open source tools
- TileMill is a really handy tool for generating
cartographically attractive tile sets and base maps
ope...
open source tools
OGR
for vector
format translation
and transformation
for raster
format translation
and transformation
fo...
engineering mindset
reusability is king
The open source mind-set also places a high premium
on re-usability.
One of the re...
Wait!
is full of open source!
Hey, the FME is already open source! It's chock full of it!
Like any sensible, modern softwa...
has a lot of stuff
• Cartography
MapNik
• Programming Languages
Python
TCL
• Filtering
Logical
Spatial
• GIS
Overlay
Inter...
Expert users
can be trapped
by their own
expertise
But these expert PostGIS users are trapped by their
expertise.
Learning...
owned by
Richard Stallman’s
“Java Trap”
If you build your
tools on a closed
platform, you don’t
really control your
tools....
Do your learnings.
Many learnings.
>>>> No matter how comfortable we get with our
tools, it's important to remember that t...
our tools define
our horizons
FME can be a great starting point. But don't stop there.
data is a liquid
Learn that data is ...
learn SQL
Learn about talking to databases with SQLExecutor,
and then learn about writing your own SQL to answer
complex q...
learn web services
Learn about web services
and hook FME up to GeoServer and MapServer and
ArcGIS Server.
learn cartograph...
learn
something
new
hard
uncomfortable
Most importantly,
Learn
<x> something New
<x> something Hard
<x> something Uncomfor...
Do your Learnings
pramsey geo.com
Upcoming SlideShare
Loading in...5
×

FME UC 2014: Keynote from Boundless

933

Published on

Paul Ramsey, Geospatial Architect for Boundless

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
933
On Slideshare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
9
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Transcript of "FME UC 2014: Keynote from Boundless"

  1. 1. Do your Learnings pramsey geo.com Are you ready to get all nerdy? ... my name is Paul Ramsey I work for Boundless, the open source geospatial company and I’m a developer of the PostGIS open source spatial database I am a technologist generalist generalist generalist integrator integrator integrator builder builder builder So in mixed company I describe myself as a technologist. Because I'm not just a computer programmer, I'm a generalist, an integrator, I like to put things together than haven't been put together before, I'm a system builder.
  2. 2. You are technologists? generalist generalist generalist integrator integrator integrator builder builder builder And I'm guessing, that in a room full of FME users, drawn from around the world, that might be a common description. is loved by technologists generalist generalist generalist integrator integrator integrator builder builder builder Because the FME is a great tool for integrators and system builders, it's systems glue, it papers over the ugly cracks and gaps that inevitably form when you try to jam two systems together.
  3. 3. is loved by me generalist generalist generalist integrator integrator integrator builder builder builder So, I'm going to be talking today about my experience with systems building, and learning new technology, and how FME has fit into that journey. Way back in 2000, before the first tech crash,.. ... I visited the CEBIT conference in Hanover, Germany. Now, Cebit is so big that Hanover doesn’t actually have enough hotel rooms to hold all the visitors, so over the years they developed a system of billeting:
  4. 4. you go to the tourist centre when you arrive, and youget matched to a local resident who can rent you a room. I got matched with Manfred, Manfred a German veterinary student, in his last year of studies. According to him (and I never verified this) rather than being tested on material progressively during the program,
  5. 5. Manfred his veterinary program tested students on EVERYTHING they learned, all at once, in the final exams. So he was spending a couple months learning a couple years worth of material, for his upcoming pass/fail exams. Manfred He was pretty stressed out. In the evening, after a hard day of conferencing, I'd ask what HE did that day, and he'd say,...
  6. 6. “Learnings, I had to do many learnings today.” "learnings, I had to do many learnings today". I've kept that with me for a long time. You have to DO YOUR LEARNINGS. Many learnings. Do your learnings. Many learnings. Because our LEARNINGS define the Boundaries of what we can conceive of as possible.
  7. 7. what tools to learn? The way we choose to solve problems is usually dictated by the tools we already understand. So, one of the most important decisions we can make (or can get made for us) as knowledge professionals is what tools to learn. what tools to learn? Good tools expand our horizons, gently introducing us to new concepts and new ways of solving problems. There are usually multiple ways to solve a problem with a good tool. You can usually see the gears turning inside a good tool. The FME is a good tool. I have a lot of affection for the FME, more than you'd perhaps expect, from some rabid open source guy.
  8. 8. Do your learnings. Many learnings. The reason I have so much affection for FME is that it was one of the first tools I learned at the start of my geospatial career, so it affected how I look at problems and their solutions, and it gave me lots of opportunities for learnings. data is not a solid The first important thing FME taught me was that data is not a solid. Because, when you look at data through the lens of operational data maintenance, it looks like a solid.
  9. 9. data is not a solid It's a big mass, and you chip away at it a little at a time, making incremental changes bit by bit. Add a record, edit a shape, modify a record. It doesn't change very fast, and big changes take a lot of work. data is a liquid When you look at data through the lens of the FME, you learn that data's not a solid, it's a liquid. You don't chip away at it, a record at a time, you POUR it in batch from receptacle to receptacle, changing the shape of it on the way, from round holes to square holes and back again.
  10. 10. data is a liquid When you understand data is a liquid, your definitions of what is a "big change" or a "lot of work" change. The bounds of what you understand is possible expand. Data is a liquid. That's a great learning I got early on from FME. good tools integrate The second important thing FME taught me was that it should be easy to integrate different technology parts. Good tools are easy to integrate.
  11. 11. good tools integrate And if they are hard to integrate, it's not your fault, it's the tool's fault. Don't blame yourself. So, when I started with FME, this was the public face of the company. get off my lawn! There was no graphical workbench, the way you made the FME “go” was to write mapping files. You describe your data transformation, and then invoke the engine to run it.
  12. 12. GUI FILENAME SourceDataset Drawing_Files(*.dwg)|*.dwg|Drawing_Files(*.dxf)|*.dxf|All_files(*.*)|*.* Input AutoCAD File: DWG_IN_DATASET "$(SourceDataset)" # ============================================================================ # The following GUI line prompts for a filename for the output DWG/DXF file. # The user input is stored in a macro, which is then used to define # the dataset to be written. GUI FILENAME DestDataset Drawing_Files(*.dwg)|*.dwg|Drawing_Files(*.dxf)|*.dxf|All_files(*.*)|*.* Output AutoCAD File: DWG_OUT_DATASET "$(DestDataset)" # ============================================================================ # When we're going autocad -> autocad, always use the source dataset as # the template file (But as of Feb 7 '99 we'll let the user override it if # they wish!) #DEFAULT_MACRO _TMPL $(SourceDataset) DEFAULT_MACRO _TMPL c:tempautocadautocad-entitiesone-point.dwg DWG_OUT_TEMPLATEFILE "$(_TMPL)" # ============================================================================ # And ask for the autocad release number -- it should be Release11, Release12, # or Release13. # We'll be kind to command line users and assume a default of Release12 DEFAULT_MACRO _REL Release2000 GUI TEXT _REL Output Autocad Release: DWG_OUT_VERSION $(_REL) # ========================================================================== # Pass bulge information from the source to the destination (Dec 13' 97) DWG_IN_STORE_BULGE_INFO yes # ========================================================================== # Pass spline information from the source to the destination (Jan 16' 02) DWG_STORE_SPLINE_DEFS yes # ========================================================================== # Don't explode dimensions (Mar 08' 02) DWG_IN_RESOLVE_DIMENSIONS no # ========================================================================== # Keep zero length arcs as arcs, rather than converting them to points (Oct 04' 02) DWG_IN_CONVERT_ZERO_LENGTH_ARCS_TO_POINTS no # ========================================================================== # Feb 02 '97 -- this will also do the 'right thing' with the attributes # Assume a default of storing attributes with the entities as extended # entity data. # You can replace 'extended_entity_data' as the definition for _ATTRKIND with: # (extended_entity_data insert_attributes external_attributes) # to change how the attributes are stored. DEFAULT_MACRO _ATTRKIND extended_entity_data DWG_AUTO_CREATE_LAYERS yes DWG_DEF DEFAULT ! autocad_color 10 ! autocad_linetype CONTINUOUS # ========================================================================== # Finally, just one transfer specification to do it all DWG_IN * DWG_OUT * autocad_attributes $(_ATTRKIND) get off my lawn! It's called a "declarative programing model". Describe your transformation in a mapping file, then invoke the engine. That's it, Write a mapping file: run the engine. get off my lawn! We translated our data using FME mapping files, uphill both ways, and we LIKED IT! We're a long way from the mapping file now, I guess. Any mapping file writers left in the audience? Get off my lawn!
  13. 13. good tools integrate run engine generate mapping file When you can completely configure your engine with a single text file, that makes it easy to integrate that engine into a larger system. The very first web program I wrote, back in 1997 when I was running a consulting company, good tools integrate run engine generate mapping file mapmaker.pl list of mapsheets DGN file was a web application that allowed map makers in our department to select a list of mapsheets and a list of layers they wanted, and output a single microstation file they could use to build a map product on. All the application did was take in the list of mapsheets and layers from a web form, spit out a mapping file, and run the FME on it. There was no need learn C++ APIs or complex data exchange protocols, the FME's declarative model made integration a trivial issue.
  14. 14. good tools integrate run engine generate mapping file mapmaker.pl list of mapsheets DGN file I also learned Perl on that project. More learnings. good tools integrate @SQL() @TCL() Q.E.D. Later on, the FME gave me a reason to learn TCL, to do coordinate manipulation during a transformation, and to learn SQL, another declarative language, to hook FME up to our departmental databases.
  15. 15. Do your learnings. Many learnings. More learnings. Each learning made me a little more powerful as a technologist, and each learning opened the doors to new learnings. spatial database Learning SQL and databases, I saw what great a integration point a database is: a format neutral place in the middle of the system that all kinds of languages can access easily.
  16. 16. I wonder if GIS data will fit in there? spatial database It was so useful, it made me wonder, "hmmm, is it possible to put GIS data in there?" And so in early 2001, our company started experimenting with GIS data in database. We tried simple stuff like BLOBS and relational schemas, and then we tried type extension using PostgreSQL. postgis is born Yes! Yes it does! And it turned out, that, yes, it was possible! PostGIS was the proof of that possibility. It was both really fast at answering GIS questions, and really convenient for storing GIS data. Since I have a geospatial cross section audience here, I'm interested to know:
  17. 17. postgis is born Heard of ? How many of you have heard of PostGIS? postgis is born Use ? How many of you have used PostGIS?
  18. 18. postgis is born adds GIS types to So, for those of you who are new to this: PostGIS is a spatial extension for the open source enterprise database PostgreSQL. postgis is born • Spatial types (point, line, polygon) • R-Tree spatial index • High-performance spatial functions • Geo-coder & routing engine • Raster data & analysis • LIDAR data & analysis <X> It supports all the ISO SQL/MM types <X> It has an auto-tuned R-Tree spatial index, and query planner integration <X> It has high-performance support for common spatial predicate tests <X> It has a geocoder and a routing package add-on <X> It has support for raster data and analysis in the database <X> It has support for lidar data and analysis in the database
  19. 19. postgis is born is to In terms of features and performance, if you want a product analogue, it's comparable to Oracle Spatial. PostGIS is to PostgreSQL as postgis is born is toS P A T I A L Oracle Spatial is to Oracle,
  20. 20. postgis is born is to and as bacon is to a cheese burger. It’s part of a larger whole, but it makes it so much better and tastier. postgis is born • Geometry column type • Spatial index on Geometry • distance(geometry, geometry) point_in_polygon(geometry, geometry) May, 2001 The first release of PostGIS was in May of 2001. It included the basic types and just a handful of functions. But nonetheless people started using it almost immediately.
  21. 21. postgis is born Early Adopters ~2001 The City of Boston and Fulton County, in Georgia were using PostGIS within a couple years of release. PostGIS User ~2001 PostGIS User ~2014 It’s over a decade later, now, and using PostGIS is about as cutting edge as using Linux, <X> which is to say, not very cutting edge at all.
  22. 22. Corporations Corporate users include... Governments Government users include...
  23. 23. postgis grows up 2001 20062002 2003 2004 2005 OSSIM As PostGIS moved from the margins to the center, the kinds of software that supported it also changed. The first software to support a PostGIS data source was MapServer the open source mapping engine. It supported it because WE wrote a connector. But other open source projects adopted PostGIS very quickly as a data source, it became the default data source. <X> When QGIS was launched, it supported two formats: shape files and PostGIS. Corporations Governments So, all these corporations and governments are now using PostGIS, it's just a standard tool, now. But back then, in 2003, how did we know we were catching on? By watching market demand for PostGIS support in third party software. So,
  24. 24. postgis grows up 2001 20062002 2003 2004 2005 PostGIS v0.1 “fme” on postgis-users first request for FME format support So, <X> The first release of PostGIS was in May of 2001. <X> The first reference to "FME" on the PostGIS users list occurred on July 27, 2001. <X> In January of 2002, my business partner wrote to Dale at Safe, telling him about PostGIS and wondering if Safe would support it. At that point we were already using PostGIS in production for data management, and we were already FME users, so having full FME support PostGIS would be very handy. postgis grows up in terms of PostGIS/PostgreSQL and FME, my hunch is that there is not a compelling commercial reason for Safe to do this, but (in my case anyway), a compelling "it is the right thing to be doing" reason ... Right now I'm thinking I'd just do this in my "spare time" such as it is. -- Dale Lutz, January 2002 Dale replied in terms of PostGIS/PostgreSQL and FME, my hunch is that there is <X> not a compelling commercial reason for Safe to do this Ow, cut to the quick! So, we know when we're relevant in the marketplace: it's when folks like Safe see a "compelling commercial reason" to support us. That doesn't happen for FME right away. But it did in the end.
  25. 25. 2001 20062002 2003 2004 2005 OSSIM In the spring of 2003 Safe starts PostGIS development, and that year the FME becomes the first proprietary third party software to support PostGIS as a data source. Fortunately, it's not the last. <X> First we pick up support from smaller GIS platforms that use niche differentiators like PostGIS support to appeal to the market. 2006 20112007 2008 2009 2010 ArcGIS Server postgis grows up But by the end, every major GIS vendor supported PostGIS as a data source. They had all found “a compelling commercial reason” to support PostGIS: their users demanded it.
  26. 26. good tools teach @SQL() @TCL() Integration GNU tools Docbook XML C/C++ CVS/SVN/GIT So, working with FME taught me useful things, like SQL and TCL and software integration. Similarly, working on PostGIS as an open source project has been professionally formative. <X> First, learning the tools of open source to support the project, * I learned the GNU toolchain and build tools to organize the code * I learned Docbook documentation systems for the first manual * I learned C and C++ for core development * I learned CVS and then SVN (and now GIT) revision control systems so I could collaborate with remote developers Do your learnings. Many learnings. So I've found that just doing "open source", has been an excellent driver of MORE LEARNINGS. It has made me learn more powerful tools and made me a more powerful technologist
  27. 27. to share to modify to share modifications free free free open source is In fact, I've found doing open source so interesting that I've made it into my career. I work on software that you can download and you can modify and you can give to others without restriction. Perhaps, for someone who gives his work away for free I look startlingly healthy and well-fed. Actually I don't give my WORK away, I just give my SOFTWARE away, which tends to generate lots of WORK, supporting the software.
  28. 28. $$$ Consumers of proprietary software have gotten used to the idea that "paying money for access to the bits" is the only viable model for a software business. But it's not, it's just one popular model, where the value is in access to bits. Training Consulting Support Cloud Deployment Build & Bundle Directed R&DFree But I work for a company, Boundless, that lives in the world of geospatial open source. The bits are free. <x> we provide training on our products, <x> we provide consulting on deployments and scaling, <x> we provide support for organizations deploying our products; insurance for when things go wrong, and assurance the software will continue to develop, <x> we keep project experts (like me) on staff, so we sell R&D and direct software development <x> we bundle and build best-of-breed open source into a single package, <x> and we provide special cloud and cluster ready packages of our software. There’s lots of things a software company can sell, other than the software itself. So, here's a fun question, since we're at FME UC, and Dale and Don are here:
  29. 29. What if were open source? What would Safe Software be like, if the FME were open source? What would they sell? Where could they add value to the core product? What kind of business models could they use? What if were open source? for new formats The simplest model is just selling access to expertise: want a new format, hire us to add one. That's an OK model, but it doesn't generate much revenue, <X>so Safe Software would probably still be Dale and Don in the basement.
  30. 30. Usage: ogr2ogr [--help-general] [-skipfailures] [-append] [-update] [-select field_list] [-where restricted_where] [-progress] [-sql <sql statement>] [-dialect dialect] [-preserve_fid] [-fid FID] [-spat xmin ymin xmax ymax] [-a_srs srs_def] [-t_srs srs_def] [-s_srs srs_def] [-f format_name] [-overwrite] [[-dsco NAME=VALUE] ...] dst_datasource_name src_datasource_name [-lco NAME=VALUE] [-nln name] [-nlt type] [-dim 2|3] [layer...]] Advanced options : [-gt n] [-clipsrc [xmin ymin xmax ymax]|WKT|datasource|spat_extent] [-clipsrcsql sql_statement] [-clipsrclayer layer] [-clipsrcwhere expression] [-clipdst [xmin ymin xmax ymax]|WKT|datasource] [-clipdstsql sql_statement] [-clipdstlayer layer] [-clipdstwhere expression] [-wrapdateline][-datelineoffset val] [[-simplify tolerance] | [-segmentize max_dist]] [-fieldTypeToString All|(type1[,type2]*)] [-fieldmap identity | index1[,index2]*] [-splitlistfields] [-maxsubfields val] [-explodecollections] [-zfield field_name] [-gcp pixel line easting northing [elevation]]* [-order n | -tps] What if were open source? And you can kind of see the end product of that model by examining the open source ogr2ogr tool, which has support for a decent number of formats, but has all the usability of a Soyuz space capsule What if were open source? Special Reader #1 Special Factory #1 Special Reader #2 Special Writer #1 open for add-ons core What about more nuanced models? Suppose the FME core was open source, but Safe selectively sold access to certain format read/writer writers or factories? The core is open, the add-ons are not.
  31. 31. What if were open source? for workbench open engine My favourite imaginary open source FME scenario is one where the core is all open source, and you can write mapping files and run them on the command-line to your hearts content. What if were open source? for workbench closed UI open engine But the fancy graphical workbench, that is packaged as a proprietary add-on. So you can have your FME in hardcore form open source, but if you want it to be really easy, you pay some money. The engine is open, the fancy user interface is not.
  32. 32. But is not open source So that was fun, but just imaginary, at least for the FME. For open source software, though, business models like these are driving the development of all kinds of interesting software. Do your learnings. Many learnings. And because it’s easy to access to tinker with, open source provides all kinds of opportunities to learn new things and expand your horizons. There's so many open source geospatial tools out there, which you can add to your skill set:
  33. 33. open source tools - PostGIS I've mentioned, and manipulating and querying data in SQL is a wonderful skill to have, since it also applies to other databases like Oracle and SQLServer open source tools - GeoServer and MapServer are really powerful web services engines, for generating maps and features on the fly over the web
  34. 34. open source tools - TileMill is a really handy tool for generating cartographically attractive tile sets and base maps open source tools - QGIS is a hugely powerful desktop GIS that not only provides a great multi-format editing and map making tool, but also integrates the algorithms from GRASS GIS, SAGA GIS and the Orfeo Tool Box image processing system.
  35. 35. open source tools OGR for vector format translation and transformation for raster format translation and transformation for LIDAR format translation and transformation - There's also great commandline utilities for manipulating data: <X> O.G.R. for vector formats and reprojection, <X> GDAL for raster formats and reprojections, and <X> PDAL for lidar data. And the best part is, they all are built with integration in mind, so you don't have to give up your existing tooling to use them. They are a lot like FME in that regard: open source tools play well with others. engineering mindset • “flexibility is good for me, must have that” • “integration is good for me, must have that” • “modularity is good for me, must have that” • “usability is good for others, will probably get to that” • “people like configuration options!” “add thousands of those!” Open source tools are built with an engineering mind- set, which explains both their positive attributes (like the flexibility, the integration, the modularity) and their negative attributes (like the half-hearted attempts at usability, and the tendency to provide an number configuration options).
  36. 36. engineering mindset reusability is king The open source mind-set also places a high premium on re-usability. One of the reasons we prize open source is that it allows us to reach into software and extract the useful bits to re-use elsewhere. Re-inventing wheels is not a virtue for open source developers. We don't need to own all the intellectual property, so we re-use the bits we need and build from there. • XML libexpat.1.dylib libxerces-c-3.1.dylib • Format Support libgdal_fme.dylib libogr_fme.dylib • Computational Geometry libgeos_fme.dylib libCGAL.8.0.0.dylib • Reprojection libproj.0.5.2.dylib • Format Support libmitab_fme.dylib libnetcdf_fme.dylib libsqlite_fme.dylib • Image Formats libpng_fme.dylib libtiff_fme.dylib • Language libtcl_fme.dylib • Rendering libmapnik_fme.dylib Before writing this talk, I opened up the FME and rooted around in the application folder, And, I found the following open source libraries, <x> XML libraries XPat and Xerces <x> Format support libraries, GDAL for rasters and OGR for vectors <x> Computational geometries libraries, CGAL and GEOS <x> Data format libraries, for MapInfo, NetCDF and SQLite <x> Image format libraries for PNG and TIFF <x> a Reprojection library, Proj4 <x> a Language library for TCL <x> and a Rendering library, MapNik for outputing cartography
  37. 37. Wait! is full of open source! Hey, the FME is already open source! It's chock full of it! Like any sensible, modern software development organization, Safe software is avoiding re-inventing wheels and spending their development time on places where they can actually add value. There's no value in writing XML handling, or image format support, or even map rendering logic anymore. So they don’t. There's value in bundling it all together, in providing a nice user interface and a visual ETL workflow. Do your learnings. Many learnings. >>> The FME developers have clearly been out, learning what options are available, so they can avoid re-inventing wheels. Doing their own learnings. The FME helped kick-start my early learnings about GIS, and gave me a good grounding in integrating multiple tools to solve problems. But that was a much simpler time.
  38. 38. has a lot of stuff • Cartography MapNik • Programming Languages Python TCL • Filtering Logical Spatial • GIS Overlay Intersection • Raster DEM Rasterization Transformation • More, more, more... DANGEROUS The FME has grown a lot since then. We just found a cartography rendering library hanging out inside it! It has two imperative programming engines (TCL and Python) embedded in it now. It has its own filtering systems, overlay system, geometry creation, TIN creation, rasterization engines, and more. It's a really good product, a really complete product. <X> And that can be dangerous. Because you can solve so many problems, without ever leaving the comfortable zone of your hard-won knowledge of FME. Why learn anything new? has a lot of stuff • SQL Programming Languages PL/PgSQL PL/Perl PL/Python • SQL Filtering Logical Spatial • SQL GIS Overlay Intersection Buffer • SQL Raster Overlay Raster-to-vector-to-raster • More, more, more... DANGEROUS I've seen the same thing in the world of spatial SQL, PostGIS users are writing incredible complex spatial computation routines using the internal PL/PgSQL language, which is simultaneously impressive and depressing. On the one hand, it's impressive what they can accomplish using only spatial SQL and PL/PgSQL. <X> On the other hand, it's depressing that they are doing it in this fairly narrow domain language, where the code is really hard to read and debug or reuse in any other domain. If they wrote their functions native to PostGIS in C, they would be easier to read, easier to re-use, easier to maintain.
  39. 39. Expert users can be trapped by their own expertise But these expert PostGIS users are trapped by their expertise. Learning a new programming tool, like C, means going back to basics before building up new expertise. They don't want to do the learnings. Really good tools can trap you intellectually, you have to keep up with the learnings to avoid it. Really good tools can also trap you another way.... The open source community around Java is long- standing, Java was born at the dawn of the open source explosion, and it grew a vibrant community of tool makers right from the start.
  40. 40. owned by Richard Stallman’s “Java Trap” If you build your tools on a closed platform, you don’t really control your tools. The platform owner does. Back when Sun owned the Java language outright, and the early Java open source community was growing up, the godfather of open source, Richard Stallman, warned about what he called the "Java trap". Basically if you build open tools on a closed platform, you run the risk of losing the platform, and with it, your hard-won tools. The same critique applies to knowledge build using any platform you don't have the right to redistribute: the knowledge can only be spread as far as the platform. “I appreciate open source, but I also see the value in proprietary software. However, more than half the students in my intro class don’t take any more GIS classes while they are at WSU. I think it would be valuable to them to learn how to use a GIS that is not going to expire on them or disappear when they upgrade their computers.” Rick Rupp On teaching QGIS in Intro to GIS http://www.palousegeo.com/2014/04/24/okay-this-should-be-different A teacher from Washington State University recently posted a blog about his decision to switch his Intro. to GIS course over completely to QGIS. <X> ... “I appreciate open source, but I also see the value in proprietary software. However, more than half the students in my intro class don’t take any more GIS classes while they are at WSU. I think it would be valuable to them to learn how to use a GIS that is not going to expire on them or disappear when they upgrade their computers.”
  41. 41. Do your learnings. Many learnings. >>>> No matter how comfortable we get with our tools, it's important to remember that they might not always be with us. We have to do the learnings. I might not always have a spatial SQL database handy. You might not always have the FME handy. Ishmael might not always have ArcGIS handy. our tools define our horizons We define our own limitations by what we choose to learn, and we define our organizational limitations by the freedom and encouragement we give to our employees to learn new things.
  42. 42. our tools define our horizons FME can be a great starting point. But don't stop there. data is a liquid Learn that data is a liquid, not a solid.
  43. 43. learn SQL Learn about talking to databases with SQLExecutor, and then learn about writing your own SQL to answer complex questions. learn Python Learn about imperative programming with PythonCaller, and expand that knowledge from FME, to ArcGIS, or to open source Python frameworks.
  44. 44. learn web services Learn about web services and hook FME up to GeoServer and MapServer and ArcGIS Server. learn cartography Learn about cartography with TileMill and build some web pages, and bring your cartographic knowledge back to FME and the MapNikRasterizer.
  45. 45. learn something new hard uncomfortable Most importantly, Learn <x> something New <x> something Hard <x> something Uncomfortable Do your learnings. Many learnings. Do your learnings. Thank you.
  46. 46. Do your Learnings pramsey geo.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×