Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
PreparingYour Rich GML Data for deegree-the ETL ChallengeJust van den Broeckedeegree Community Space 2012, BonnNovember 13...
About MeIndependent Open Source Geospatial ProfessionalTrailblazer OSGeo Dutch Local ChapterJust van den Broeckejust@justo...
THE DUTCH KADASTERGETS INSPIRED WITHdeegree day - nov 16, 2010
THE DUTCH KADASTERGETS INSPIRED WITHdeegree Community Space - nov 13, 2012
50+ DATASETSWMS/WFS/WCS/ATOM
METADATA+NL INSPIRE ACCESS POINT
PDOK	  –	  Open	  Source	  &	  Open	  StandardsApplicationsOGC/ISO/INSPIRE Web ServicesStorageConversionServicessETLFME?
OSGeo - Bolsena - 2010
BOLSENA2012
ALLES VORBEI ?BOLSENA2012
BOLSENA2012
PreparingYour Rich GML Data for deegree-the ETL ChallengeJust van den Broeckedeegree Community Space 2012, BonnNovember 13...
We have aProblem
The Rich GMLProblem
Rich GML = Complex Mess
INSPIREDutch National DSsAFIS-ALKIS-ATKIS..
The Streetname!
ComplexModelTransformations
MillionsofObjects
10s of Millionsof<Elements>
MultipleTransformationSteps
Solution isSpatial ETL
A.K.A.
ThankYoufor yourAttention!
But what about.......FOSS ?... Stetl?
FOSS ETL - High Level
FOSS ETL - Lower LevelBut Each Powerful by Itselfogr2ogr
FOSS ETL - DIY ? (No!)
FOSS ETL - How to Combine?=+ + ?ogr2ogr
FOSS ETL - Add Python to Equation=+ + ?( )ogr2ogr
=+ +Stetl( )ogr2ogr
Stetl=SimpleStreamingSpatialSpeedyETL
Process ChainInput Filter OutputgmlFilterStetl concepts
Speed: StreamingInput Filter OutputgmlStetl concepts
Speed: Going NativeInput Filter Outputgmlogr2ogr sETLsETLNative C Libs/ProgsCallsStetl concepts
Example: GML to PostGISReaderXMLSplitterogr2ogrgmlStetl concepts
Example: INSPIRE Model Transformogr2ogr XSLT WritergmlStetl concepts
Example: deegree Storeogr2ogr XSLTdeegreeWriterStetl concepts
Process Chain - How?Input Filters OutputStetl concepts
Example: XML to ShapeThe Source
Example: XML to ShapeFirst: XSLT Transform to GML
Example: XML to ShapeXMLInputXSLT ogr2ogr
Example: XML to ShapeThe SETL Chain Config FileProcessChainReaderXSLTogr2ogr
Example ComponentsInput Filters OutputStetl conceptsXMLFile XSLT GMLFileogr2gml GMLSplitter gml2ogrLineStream XMLValidator...
Data StructuresStetl concepts✴Components exchange Packet✴Packet contains data✴Data format:xml_line_streametree_docetree_fe...
deegree IntegrationStetl concepts✴InputDeegreeBlobstoreInput✴OutputDeegreeBlobstoreInputDeegreeFSLoaderOutputWFSTOutput
Cases✴INSPIRE Download Servicespublish to deegree store (WFS)GML files (for Atom Feed)✴National GML DatasetsGML to PostGIS
Case: Dutch AddressesSource<GML>sETL sETLdeegreeWFSsETLINSPIRE<GML>AtomFeedOtherINSPIREAddressesDutchAddresses+BuildingsDu...
ThankYou !stetl.orgwww.inspire-foss.org
Stetl: Preparing Rich GML Data for deegree - The ETL Challenge
Stetl: Preparing Rich GML Data for deegree - The ETL Challenge
Stetl: Preparing Rich GML Data for deegree - The ETL Challenge
Stetl: Preparing Rich GML Data for deegree - The ETL Challenge
You’ve finished this document.
Download and read it offline.
Upcoming SlideShare
NLExtract - Unlocking Dutch Open Geo-datasets - High Level Introduction
Next
Upcoming SlideShare
NLExtract - Unlocking Dutch Open Geo-datasets - High Level Introduction
Next
Download to read offline and view in fullscreen.

Share

Stetl: Preparing Rich GML Data for deegree - The ETL Challenge

Download to read offline

Presentation given at deegree Community Space, november 13, 2012, Bonn. More and more do we need to work with rich/complex GML. How can we tame this mess ? The deegree WMS/WFS server is very suitable to store and serve rich GML, but how do we get our data in ? What spatial ETL options do we have ? Basically this is a tutorial on Stetl, pronounce "staddle", with particular focus on deegree integration. NB Stetl used to be called sETL...

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all
  • Be the first to like this

Stetl: Preparing Rich GML Data for deegree - The ETL Challenge

  1. 1. PreparingYour Rich GML Data for deegree-the ETL ChallengeJust van den Broeckedeegree Community Space 2012, BonnNovember 13, 2012www.justobjects.nl
  2. 2. About MeIndependent Open Source Geospatial ProfessionalTrailblazer OSGeo Dutch Local ChapterJust van den Broeckejust@justobjects.nlwww.justobjects.nl
  3. 3. THE DUTCH KADASTERGETS INSPIRED WITHdeegree day - nov 16, 2010
  4. 4. THE DUTCH KADASTERGETS INSPIRED WITHdeegree Community Space - nov 13, 2012
  5. 5. 50+ DATASETSWMS/WFS/WCS/ATOM
  6. 6. METADATA+NL INSPIRE ACCESS POINT
  7. 7. PDOK  –  Open  Source  &  Open  StandardsApplicationsOGC/ISO/INSPIRE Web ServicesStorageConversionServicessETLFME?
  8. 8. OSGeo - Bolsena - 2010
  9. 9. BOLSENA2012
  10. 10. ALLES VORBEI ?BOLSENA2012
  11. 11. BOLSENA2012
  12. 12. PreparingYour Rich GML Data for deegree-the ETL ChallengeJust van den Broeckedeegree Community Space 2012, BonnNovember 13, 2012www.justobjects.nl
  13. 13. We have aProblem
  14. 14. The Rich GMLProblem
  15. 15. Rich GML = Complex Mess
  16. 16. INSPIREDutch National DSsAFIS-ALKIS-ATKIS..
  17. 17. The Streetname!
  18. 18. ComplexModelTransformations
  19. 19. MillionsofObjects
  20. 20. 10s of Millionsof<Elements>
  21. 21. MultipleTransformationSteps
  22. 22. Solution isSpatial ETL
  23. 23. A.K.A.
  24. 24. ThankYoufor yourAttention!
  25. 25. But what about.......FOSS ?... Stetl?
  26. 26. FOSS ETL - High Level
  27. 27. FOSS ETL - Lower LevelBut Each Powerful by Itselfogr2ogr
  28. 28. FOSS ETL - DIY ? (No!)
  29. 29. FOSS ETL - How to Combine?=+ + ?ogr2ogr
  30. 30. FOSS ETL - Add Python to Equation=+ + ?( )ogr2ogr
  31. 31. =+ +Stetl( )ogr2ogr
  32. 32. Stetl=SimpleStreamingSpatialSpeedyETL
  33. 33. Process ChainInput Filter OutputgmlFilterStetl concepts
  34. 34. Speed: StreamingInput Filter OutputgmlStetl concepts
  35. 35. Speed: Going NativeInput Filter Outputgmlogr2ogr sETLsETLNative C Libs/ProgsCallsStetl concepts
  36. 36. Example: GML to PostGISReaderXMLSplitterogr2ogrgmlStetl concepts
  37. 37. Example: INSPIRE Model Transformogr2ogr XSLT WritergmlStetl concepts
  38. 38. Example: deegree Storeogr2ogr XSLTdeegreeWriterStetl concepts
  39. 39. Process Chain - How?Input Filters OutputStetl concepts
  40. 40. Example: XML to ShapeThe Source
  41. 41. Example: XML to ShapeFirst: XSLT Transform to GML
  42. 42. Example: XML to ShapeXMLInputXSLT ogr2ogr
  43. 43. Example: XML to ShapeThe SETL Chain Config FileProcessChainReaderXSLTogr2ogr
  44. 44. Example ComponentsInput Filters OutputStetl conceptsXMLFile XSLT GMLFileogr2gml GMLSplitter gml2ogrLineStream XMLValidator WFS-Tdeegree* FeatureExtractor deegree*YourInput YourFilter YourOutput
  45. 45. Data StructuresStetl concepts✴Components exchange Packet✴Packet contains data✴Data format:xml_line_streametree_docetree_feature_arrayxml_doc_as_stringany
  46. 46. deegree IntegrationStetl concepts✴InputDeegreeBlobstoreInput✴OutputDeegreeBlobstoreInputDeegreeFSLoaderOutputWFSTOutput
  47. 47. Cases✴INSPIRE Download Servicespublish to deegree store (WFS)GML files (for Atom Feed)✴National GML DatasetsGML to PostGIS
  48. 48. Case: Dutch AddressesSource<GML>sETL sETLdeegreeWFSsETLINSPIRE<GML>AtomFeedOtherINSPIREAddressesDutchAddresses+BuildingsDutchGeocoderdeegreeblobstore
  49. 49. ThankYou !stetl.orgwww.inspire-foss.org

Presentation given at deegree Community Space, november 13, 2012, Bonn. More and more do we need to work with rich/complex GML. How can we tame this mess ? The deegree WMS/WFS server is very suitable to store and serve rich GML, but how do we get our data in ? What spatial ETL options do we have ? Basically this is a tutorial on Stetl, pronounce "staddle", with particular focus on deegree integration. NB Stetl used to be called sETL...

Views

Total views

927

On Slideshare

0

From embeds

0

Number of embeds

3

Actions

Downloads

9

Shares

0

Comments

0

Likes

0

×