XML-athon with Don and Dean

“XML-athon”
With Dean Hintz and Don Murray

Don Murray
Co-Founder
Dean Hintz
Senior FME Data Expert
Product Owner XML Technology
Presenters
Trent Kading
XML and FME
Expert on Q&A

Agenda
● Who we are and what do we do?
● XML & The FME Platform
● XML Highlights and Strategies
A dozen demos that just touch on what FME can do.
● CityGML, Simple Reading/Writing, Validation
● Using XSD documents to enhance XML Experience
● Metadata, OSM, Catalogs and Web API’s
● Complex XML & GML, Converting XML to JSON.
● Resources and Q&A

OUR MISSION
To help organizations
maximize the value of data.

FME Desktop FME Server FME Cloud
Build & Run Workflows Automate Workflows Hosted FME Server
FME® Integration Platform
Connect. Transform. Automate.
FME is the data integration solution with the best support for spatial data worldwide.

An extension for ArcGIS Pro and ArcMap developed by Safe Software.
FME Desktop functionality. Data Interop in 2.6 is exciting!!

Let’s Get Started and Have fun!!!

FME Supported Data Types - Slicing XML for ~ 20 years!
CAD
GIS
FME release by year
NumberofsupportedformatsinFME
1995
10500100300
20202015201020052000
TABULAR
DATABASE
RASTER
POINT
CLOUD
BIM
3D
WEB
XML/
JSON
CLOUD
BIG
DATA
IOT
BI
AR / VR
AI / ML
INDOOR
MAPPING
GAMING
XML Representations

FME support for XML is
Wide and Deep!

Let us know in the
chat what type of
XML data you are
working with.

All are XML! FME wraps formats to help you.
XMLGMLCityGML
Adds Value
GeometryTextures
Adds Value

USE MOST
SPECIFIC
FORMAT
FME supports many XML variants
and there are many ways to work
with a specific file..
The best way is almost always
using the specific format.
Reader
Reader
Reader

Demo:
General -> Specific Format Support
(CityGML)
1.

Most General Reader : XML Reader
Read CityGML by XML
reader
Feature Path:
Building
No geometry - must
process in workspace
-> GreenBuildingXML
example

● Read CityGML with
GML Reader (no
schema mode)
● Reads features,
geometries
● Ignores textures
since these are not
part of GML standard
● Auto reader selection
More Specific Reader: GML Reader

Read CityGML with
CityGML Reader
Note textures and
citygml properties
read (LOD)
Richest support
when most
specific R/W
chosen
Most Specific Reader: CityGML Reader

{"employees":[
{ "firstName":"John", "lastName":"Doe" },
{ "firstName":"Anna", "lastName":"Smith" },
{ "firstName":"Peter", "lastName":"Jones" }
]}

JSON Formats
Note: IMDF, CityJSON, TopoJSON - some of the newer formats added recently

XML Format Highlights
XML GML
KML OGCWFS
XSD based XML 3D: CityGML, Indoor GML
GMLJP2000 INSPIRE GML
Geodatabase - xml metadata AIXM
Geo RSS UK OS Mastermap
AEC/CAD/BIM: 12dXML, IFCXML GML in JP2000

FME’s XML Power
● XSD based XML Reading and Writing
● Standard XML Reader / Writer
● List Processing
● GML Reader / Writer
● WFS Reader - complex GML support
● Writer validation / XMLValidator
● XSLT, XFMaps
● XML Transformers
Most of these will be highlighted within the examples below
XML

Does Zipster Run on XML
or Love XML?
Zipster runs on machine code called:
Xtraordinary Machine Language
XML
Zipster Loves XML!
But....

XML Transformers - Favorites
XMLFragmenter / XMLFlattener
XMLValidator
XMLFormatter
XMLUpdater
ListExploder
Honorable mention:
● XMLXQueryExploder
Historical mention:
● XMLTemplater

XML Concepts
Tag <Course>
XML Element <Course>FME</Course>
Child element <Course><Id>101</Id>…
Element attribute <Course id=“101”>
GML Object
Object property
Object attribute
XML Schema - xsd's
Namespaces
<ps-f:geometry>

XML Data Structures: Object vs Relational
attribute1 attribute2 Active
Date_From
Active
Date_To
Coordinates
John Vancouver 11-22-99 12-11-09 -123.1,49.25
-122.9,49.15
June Surrey 02-25-05 9-15-10 -122.8,49.12
-122.5,49.0

XML Strategies
XML comes in many flavours so important to have a well
thought out approach to leverage the tools you have
and quickly get the data in the form you need
● Understand the type, logical structure and
meaning of your source data
● Understand your target data model and what
transforms are likely to be needed
● Choose the tools best suited
(Readers/Writers/Transformers) to do the work

XML Strategies: Reading
Use most specific format (Text or XML or GML or profile?)
● How to identify format type? (GML, XML, HTML, profile)
● Format clues? See namespace prefixes and header,
geometry types, xsd
● Use application schema if there is one (ask or search)
If XML not specific supported then use our generic XML tools:
● Don’t forget to inspect data first: Data Inspector, Notepad++
● Feature Paths
● XMLFragmenter / XMLFlattener
● List processing
● GeometryReplacer

XML Strategies: Writing
● Use most specific writer format (Text or XML or GML or profile?)
● Use XSD if it is available
● If XML not specifically supported then use our generic XML tools:
○ XMLTemplater
○ GeometryExtractor
○ List processing: ListRenamer (now supports manipulation of entire lists)
○ AttributeCreator, Aggregator to build lists
○ XMLFormatter and XMLValidator
○ Other writer parameters: namespace override
○ Destination schema: understand diﬀerences between xml data types
■ xml_buﬀer, xml_geometry, xml_xml
■ node level writing - new in 2019.2+

Arbitrary XML/No schema: UNESCO World Heritage Sites
● Inspect first to determine type of xml
● Read with Feature Path: /query/row
https://whc.unesco.org/en/list/xml/

● Nested XML sliced
into flattened
features at selected
node
● Parent.child
notation within
features for nested
attributes
UNESCO World Heritage Sites: Read with Feature Path

1. Add XML reader:
a. Dataset:
https://whc.unesco.org/en/list/xml/
b. Elements to match query/row
2. VertexCreator: longitude,
latitude
3. CoordSystemSetter:LL84
4. KMLPropertySetter
a. Name: site
b. Title: Heritage Sites
c. Content: short_description
5. Drop xml format fields
6. Fanout: category
UNESCO World Heritage Sites: XML to KML Workspace

Set kml_name,
description,
folder
Geometry and
attribute
schema
Illustrates how
FME supports
easy extraction
from XML to
client friendly
formats like KML
UNESCO World Heritage Sites: KML Result

Demo:
Writing Simple XML
(using XML Writer, XMLTemplater)
3.

Writing Flat XML
Database export with generic XML writer
Automatic schema definition based on
source or feature schema
Database export with generic XML writer

Default values for
most parameters
Make sure output
data and schema
not suppressed
Writing Flat XML: Workspace

FME generates namespaces as needed
Preview output in XML Editor
Writing Flat XML: Result

Writing Simple Flat XML: Result
Before FME 2020, could
validate using XSD but
had to read XML data with
feature paths and ignore
schema
Now output can be
checked with Data
Inspector using XSD

Write XML with XMLTemplater - Metadata
Insert attribute values using
fme:get-attribute{} functions
Write out XML document using Text File writer

XML Validator
Possible Values:
● None
● Syntax
● Syntax and Namespace
● Syntax and Schema
New in 2020:
● Multiple XSD support
Also useful to check if data you
receive is valid
Test to see if data you produce is
valid!

Demo:
XSD Driven Reading
(BeerXML)
5.

XSD Driven XML Reading - BeerXML
Read XML feature types based
on application schema and
feature paths (diﬀerent path).
Feature type attribute and
geometry definitions based
on XSD not data.
Similar to GML application
schema based reading
Prefix with namespace to
diﬀerentiate types

1. Set source XML dataset and
application schema paths
2. Set feature paths
(base:beer_xml)
3. Set XSD Types to be
features (all except units)
4. Max attributes per nested
attribute - becomes xml
fragment after that

Reading recipe additions (schemas from beerxml.com )

Demo:
XSD Driven Writing
(Emergency Alerts)
6.

● Cap 1.2 XML
● Write XML using Feature type
definitions based on XSD,
path.
● Set attribute values as you
would any typical FME format -
no templaters required.
● Similar to GML application
schema based writing
● Validation option
XSD Driven XML Writing: Common Alerting Protocol

http://docs.oasis-open.org/emergency/cap/v1.2/CAP-v1.2.html

XSD Driven XML Writing: CAP - Validation Pt 2
Missing attribute msgType:
ERROR |XML Validation: Error in ‘6.EmergencyAlerts_CAP1.2 XMLcapOut.xml' on line 27, column
13: 'element 'source' is not allowed for content model
'(identifier,sender,sent,status,msgType,source?,scope,restriction?,addresses?,code*,note?,referen
ces?,incidents?,info*)
Missing attribute ‘severity’:
ERROR |XML Validation: Error in ‘6.EmergencyAlerts_CAP1.2 XMLcapOut.xml' on line 26, column
13: 'element 'certainty' is not allowed for content model
'(language?,category+,event,responseType*,urgency,severity,certainty,audience?,eventCode*,effe
ctive?,onset?,expires?,senderName?,headline?,description?,instruction?,web?,contact?,parameter
*,resource*,area*)
Messages from open source Apache Xerces parsing library

OGC Open API: Features - WFS3 New, Updated Tutorials:
GML, XML, JSON
CityGML v3 GeoSciML*
S121 GML* CIM XML
ARML FXXM/WXXM*
List Processing Others?
What is Next for XML?
*Stay tuned!

Choose your XML
Adventure
Vote on:
● Arbitrary XML
● Catalogues
● Metadata

XSD Based XML R/W: ISO 19115 Metadata
XSD based reader /
writer new in FME 2020
Updates contact{0}

XSD Based XML R/W: ISO 19115 Metadata
Reader settings: Feature Path Writer settings: Feature Path

XSD Based XML: ISO 19115 Metadata: XMLTemplater

XSD Based XML R/W: ISO 19115
Metadata Result
source updated

OSM XML: Generating 3D Buildings Workflow
OSM comprehensive
global coverage for
base map data
Full access to rich
property sets

OSM XML: 3D Buildings Visualization
FME can read all
available OSM
outputs: XML, PBF,
Geopackage and
GeoJSON.
Each format has its
own properties
Data from:
https://extract.bbbike.org/

OGC CSW Client Workspace
Series of HTTPCallers send requests to CSW service for Entries, Details and then WFS features based on user selection

CSW Details URI: WFS Feature Retrieval S-121 GML

XML Resources
Tutorials & Articles
XML Tutorial
https://knowledge.safe.com/articles/29553/tutorial-getting-started-wit
h-xml.html
GML Tutorial - INSPIRE
https://knowledge.safe.com/articles/1321/eu-inspire-initiative-tutorial.
html
JSON Tutorial
h-json.html
Reading XML/GML
https://knowledge.safe.com/articles/888/reading-xmlgml.html
GML - Writing
https://knowledge.safe.com/articles/812/gml-writing-with-application-
schemas.html

Title
Join us tomorrow!
Keeping Cool, Calm, and Connected with FME 2020.1

Choose your XML
Adventure
Vote on:
● XML Web APIs
● OSM
● XML to JSON

Power of XML Reader: GreenBuilding XML
Custom reader
available on FME Hub
XML reader assembles
object attributes and
geometries to construct
BIM components
gbXMLStandard Single
Family Residential
2016.xml

ChapelHillOﬀice.xml

Custom format
just looks like any
other format
when reading
from Data
Inspector or a
workspace
Data from
gbxml.org

1.Explode to points elements 2.Generate point geometry 3. Build line, area and surfaces

Is there an easier way for Green Building XML?
hub.safe.com

XSLT Example: S-121 GML to HTML
● Option to specify XSLT path on
GML or XML writer
● Example from the OGC
Maritime Boundaries Pilot
● XSLT takes output S-121 GML
and transforms it into HTML
regulatory text representation
● Net result is a fully automated
DFO/CHS geodatabase to HTML
via GML to support their
regulation submission
workflow

Power of Generic GML R/W: Complex GML
These next 3 examples show the power of FME to support reading and writing complex GML,
such as that used for scientific applications and complex regulatory models.
GeoSciML: Just drag and drop demo dataset into DI - GML reader automatically gets the xsd
from the ns header - make sure you set axis order to 1,2. This is Australian data - geoscience
australia.
WXXM: need to specify the application schema in the folder provided: wxxm.xsd
S-121 GML: No need to specify app schema. This is because the xsi:schemaLocation points
to the local application schema which in this case is s121.xsd. This needs to be in the same
folder as the dataset.

Power of Generic GML R/W: GeoSciML
Open standard
for geologic
data. Submitted
as an OGC
standard.
Aligned with
INSPIRE

Weather
Information
Exchange Model
Interoperable
meteorological
information
exchange covering
the needs of the
aviation industry
eurocontrol.int
faa.gov
Power of Generic GML: WXXM

S-121 GML from OGC
Maritime Boundaries
Pilot
Collaboration between
IHO and OGC
S-121 generated from
DFO/CHS geodatabase
using FME
ogc.org
charts.gc.ca
Power of Generic GML: S-121

XML/GML to JSON conversion / diﬀerences
GML schema driven directly from XSD.
Complex structures / nesting modelled
within the feature type definitions and
using parent / child ids.
Currently JSON writer requires the use
of JSONTemplater for complex
properties / nesting, such as for lists or
multi-part elements such as date
ranges.

JSON / GeoJSON Enhancements
New formats: CityJSON, TopoJSON, IMDF
update
Writing: New Feature Collection mode in
FME2020.1+. Choose how to assemble feature
types:
● Single feature collection
● Array of feature collection objects
● Object with feature type names as keys
Performance improvements
To come: nested schema writing. In meantime
use JSONTemplater to write to root objects

Preview GeoJSON on GitHub, geojson.io
Geojson is automatically rendered as an interactive map on GitHub
Use geojson.io to test your geojson with an external client

Let us know in the
chat…
[question]

Other Topics to Explore
● Metadata -> AI/ML - metadata tags for training
● XML manipulation: XML Extraction & Updating:
XMLUpdater, XQueryExtractor
● XFMaps and XSLT - XML reader scripting,
dynamic schema, style sheets
● OGC Support
○ Web Services: OGC Service hosting
● Geometry extraction/ composition
● JSON parsing
● Other formats: Indoor, BIM (IFCXML), AEC:
12DXML, CIM XML
What do you want to hear /
learn more about?
Go to Knowledge Base for more
info on most of these topics

Summary - Key Takeaways
● XSD XML - Powerful new reader / writer: e.g. metadata 19115
● GML - XSD mode - very flexible and continues to evolve
● FME as your no-code XML solution: many XML tools
embedded in FME (Validation, Formatting, SampleGenerator,
XSLT, XML Update)
● FME constantly evolving: OpenAPI/WFS 3 etc - keep
upgrading to get latest functionality and performance
● JSON patterns very similar to XML ones
● FME’s XML and JSON support critical for web integration and
open standards support: W3C, OGC, EU INSPIRE, ISO, ITRF
Reach out to
us for Help

Q&A
And suggestion sharing!
Please share your XML ideas
in the chat panel.

XML Resources
Tutorials & Articles
XML Tutorial
h-xml.html
GML Tutorial - INSPIRE
https://knowledge.safe.com/articles/1321/eu-inspire-initiative-tutorial.
html
JSON Tutorial
h-json.html
Reading XML/GML
https://knowledge.safe.com/articles/888/reading-xmlgml.html
GML - Writing
https://knowledge.safe.com/articles/812/gml-writing-with-application-
schemas.html
Keep the XML conversation going!
We want to know what else you want to see
in FME to make XML easier.
Please reach out on safe.com/live-chat or
post on safe.com/community

Demo Links
And supporting articles
CityGML
https://knowledge.safe.com/articles/904/writing-citygml-example.html
XMLUpdater
https://knowledge.safe.com/articles/868/using-the-xmlupdater-transfo
rmer-to-manipulate-met.html
OGC Maritime Boundaries S-121 GML Pilot
https://knowledge.safe.com/articles/112578/ogc-maritime-limits-and-b
oundaries-s-121-gml-pilot.html
Writing to nested properties in AIXM:
https://knowledge.safe.com/questions/2949/aixm-annotations.html
https://knowledge.safe.com/questions/47251/how-to-transform-aixm-
45-to-aixm-51-xml-geometry-g.html
WFS Troubleshooting:
https://knowledge.safe.com/articles/19452/troubleshooting-wfs-errors.
html
Keep the XML conversation going!
We want to know what else you want to see
in FME to make XML easier.
Please reach out on safe.com/live-chat or
post on safe.com/community

Thank you!
Get a free trial of FME or live chat at safe.com

XML-athon with Don and Dean

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to XML-athon with Don and Dean

Similar to XML-athon with Don and Dean (20)

More from Safe Software

More from Safe Software (20)

Recently uploaded

Recently uploaded (20)

XML-athon with Don and Dean