Generating XML schemas from a Logical Data Model (EDW 2011)

6,777 views

Published on

I gave this 30 minute presentation in April 2011 at Enterprise Data World in Chicago, United States.
I also have a copy of the script (also PowerPoint) I used for the PowerDesigner demonstration.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
6,777
On SlideShare
0
From Embeds
0
Number of Embeds
43
Actions
Shares
0
Downloads
197
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • I’m a modeller at heart, not an XML Guru, and not a developer.
  • Why use a canonical data model?In this presentation, I assume that the use of a canonical model is a given; we all see the need to manage our understanding and metadataSee Designing Canonicals for SOA: Bridging ER and XML Worlds Mehmet Orun & Jeff Pekrul, Genentechhttp://sfdama.org/Presentations/2007-02-07_Canonical_for_SOA_%20ER_XML.pdf
  • It’s possible to manage the content of XML Schemas by using a single master (canonical) XML Schema, which defines all the allowed types. The hierarchical nature of an XML Schema makes it impossible for a master schema to do any more than specify the possible building blocks for schemas; it cannot govern the ways in which those building blocks are assembled. A canonical data model managed as an XSD can only be a type library; it cannot possibly control the ways in which the types can be assembledMany of the challenges (see later) we face when generating XSDs also apply to an XML Master Schema
  • A stand-alone canonical XML schema is an island of metadata, providing no impact and lineage analysis capabilities
  • The dedicated XML model in PowerDesigner enables us to manage XML schema designs in an integrated manner; have a look at the options in the next slide
  • You can see how the XML Schema Model forms an integral part of the PowerDesigner model management philosophy.This slide only shows the ‘downward’ generation options; it also possible for a model to generate another model of the same type, or another model of a ‘higher’ type. E.g. a PDM can be generated from an XSM.
  • This is an OAGi XML Schema, reverse-engineered into PowerDesigner. The only change I’ve made is to move some symbols closer to each other. This schema inherits all the simple types from other schemas, which is why there aren’t any here.
  • The arrows show the navigation path through the model required to create the schema; note that some relationships are traversed from parent to child, others from child to parent. Also note that no attributes are extracted from the ‘Team Member’ entity; we use it as a route to the details of the Employee. The grey parts of the model are not used in this schema.
  • A more complex schemaThe arrows show the navigation path through the model required to create the schema; note that some relationships are traversed from parent to child, others from child to parent. Also note that no attributes are extracted from the ‘Team Member’ entity; we use it as a route to the details of the Employee.
  • Differences between SchemasAttribute and relationship optionality and cardinalityExcluding attributesFewer or more enumerationsDenormalising attributes and entitiesAttributes may appear in >1 element1 element may include attributes from multiple entitiesScalability of Process We can handle 5 XSDs, but what about 500 XSDs?
  • From PDM
  • Generating XML schemas from a Logical Data Model (EDW 2011)

    1. 1. Generating XML Schemas from a canonical model - a practical example<br />George McGeachie<br />Metadata Matters Limited<br />Enterprise Data World<br />Chicago<br />April 5th, 2010<br />
    2. 2. The ‘C’ word<br />Managing XSDs using a master (canonical) XML schema<br />The flexibility of PowerDesigner<br />The XML Schema Model in PowerDesigner<br />Which Canonical Model Should You Use?<br />Factors To Be Considered<br />Generating XML Models in Power Designer<br />Enterprise Data World, Chicago, April 5th 2011<br />2<br />
    3. 3. About Me<br />I’ve been involved with modelling and managing data and metadata for longer than I care to remember.<br />When I first came across XML Schemas, I was struck by their simplicity and versatility. When I saw how some people use them, I realised that the days of unmanaged COBOL copybooks had returned to haunt me.<br />This is my attempt to exorcise the XML demon.<br />Enterprise Data World, Chicago, April 5th 2011<br />3<br />
    4. 4. The ‘C’ word<br />It’s difficult to avoid the word ‘Canonical’ when discussing standard XML messages<br />To some, the message schemas are the ‘canonical’<br />To others, the underlying model is the ‘canonical’<br />When I use the word, I mean both of them<br />Enterprise Data World, Chicago, April 5th 2011<br />4<br />
    5. 5. Managing XSDs using a master (canonical) XML schema<br />An XML master schema must be hierarchical<br />It cannot easily show all the complex relationships between the data concepts<br />It cannot be flexible enough to support multiple views of the data<br />You need to understand XML to create, edit or read the model<br />Enterprise Data World, Chicago, April 5th 2011<br />5<br />
    6. 6. Managing XSDs using a master (canonical) XML schema<br />How would we integrate the metadata represented by a schema with our other metadata?<br />Reverse-engineer it into a repository or modelling tool?<br />How would we<br />Ensure consistency of XSDs with each other?<br />Enable impact analysis?<br />Manage the variations necessary between dependent schemas?<br />Enterprise Data World, Chicago, April 5th 2011<br />6<br />
    7. 7. The flexibility of PowerDesigner<br />Enterprise Data World, Chicago, April 5th 2011<br />7<br />PowerDesigner has a dedicated XML Schema Model<br />Several different approaches supported out of the box<br />Generating XSDs from a class model<br />Generating an XML Schema model from a class model or a physical data model, then generating XSDs<br />Reverse-engineering existing XML Schemas into a class model or XML Schema Model<br />Can then generate a logical or conceptual view<br />
    8. 8. Generation options in PowerDesigner<br />Enterprise Data World, Chicago, April 5th 2011<br />8<br />
    9. 9. The XML Schema Model in PowerDesigner<br />Enterprise Data World, Chicago, April 5th 2011<br />9<br />This can link to an XML model, or an XSD<br />Referenced<br />Elements (in another XSD)<br />Complex Type<br />Note the XML-specific object types<br />
    10. 10. XML-specific Model Objects<br />10<br />Enterprise Data World, Chicago, April 5th 2011<br />Here’s the detail of one of the sequences in the schema<br />Filter the displayed properties<br />Add additional items to the sequence<br />
    11. 11. The XML Model<br />Dedicated to modelling XML Schema<br />1 schema per XML model, traceable to source models<br />Mapping Editor – drag and drop mappings between models<br />Supports XSD, XDR and DTD<br />Multiple Namespaces supported<br />Can use 1 XML model for multiple schemas in the same target namespace<br />Enterprise Data World, Chicago, April 5th 2011<br />11<br />
    12. 12. The flexibility of PowerDesigner<br />Linkages are automatically maintained between models as you generate them<br />Additional mappings can be added between any two models<br />The generation process can be customised<br />You have control over the naming standards and data types used in each model<br />Impact and Lineage analysis is enabled by the generation and mapping links<br />Enterprise Data World, Chicago, April 5th 2011<br />12<br />
    13. 13. Generation Links<br />For each model, you can trace links to other models in the generation sequence<br />Enterprise Data World, Chicago, April 5th 2011<br />13<br />
    14. 14. Mapping Editor<br />Visually check mappings<br />Enterprise Data World, Chicago, April 5th 2011<br />14<br />
    15. 15. Impact of deleting XML Attribute - diagram<br />Enterprise Data World, Chicago, April 5th 2011<br />15<br />Dependencies between PDM tables<br />PDM Tables mapped to the Element<br />Our attribute<br />XML Element<br />Simple Type (no longer required)<br />Remember: this diagram traces back to the model the XML attribute was generated from; in this case, the XML model was generated from a PDM, so the impact analysis traces to the PDM<br />
    16. 16. Visible Links Between Models<br />Using the ‘project’ feature, we can see the links between models<br />We can also include external objects, and create our own links to them<br />An XSD and an HTML file in this case<br />Enterprise Data World, Chicago, April 5th 2011<br />16<br />
    17. 17. Which canonical model?<br />One (or more) that you can sell within your organisation<br />Standard models<br />Standard Universal Data Models<br />e.g. IBM, Teradata, Oracle, EWS Solutions, Len Silverston’s Universal Data Models<br />Industry Universal Data Models<br />e.g. standard messaging structures, such as from the OAG<br />See www.industrydatamodels.com <br />Some organisations use several of these, which will need to be cross-referenced<br />Your own model(s)<br />possibly used to extend one of the above<br />Enterprise Data World, Chicago, April 5th 2011<br />17<br />
    18. 18. A simple example of a relational canonical model<br />Enterprise Data World, Chicago, April 5th 2011<br />18<br />This is an amended version of the ‘Project Management’ PDM supplied with PowerDesigner<br />
    19. 19. The ‘Team Members’ schema<br />Enterprise Data World, Chicago, April 5th 2011<br />19<br />
    20. 20. The ‘Project’ schema<br />Enterprise Data World, Chicago, April 5th 2011<br />20<br />
    21. 21. Factors To Be Considered<br />Generating a Type Library vs. generating Schemas<br />Physical Design Challenges<br />Model Management Challenges<br />21<br />Enterprise Data World, Chicago, April 5th 2011<br />
    22. 22. Generating a type library<br />often straightforward to generate<br />no structure<br />manual control of re-use in schemas<br />"let the service people design the schemas"<br />no impact analysis across schemas<br />may be the only acceptable first step<br />low impact on work patterns<br />gains initial acceptance of the role of a canonical model<br />Enterprise Data World, Chicago, April 5th 2011<br />22<br />
    23. 23. Generating Individual Schemas<br />A 'container' for each schema<br />Model, Submodel, Subject Area, Package etc<br />defines scope<br />provides documentation<br />facilitates governance<br />Complete generation<br />Nothing changed post-generation<br />Requires high degree of control over generation process<br />Enterprise Data World, Chicago, April 5th 2011<br />23<br />
    24. 24. Generating Individual Schemas<br />all types can be local<br />no need to 'include' or 'import' a base schema<br />the 'standard' types are used to generate the schema, rather than being referred to by the schema<br />The standard types are in the canonical data model<br />potential for impact analysis<br />can choose when to update each individual schema<br />Enterprise Data World, Chicago, April 5th 2011<br />24<br />
    25. 25. Sample Message Schema<br />This was generated from the same data model as the type library; it only contains what the message actually needs<br />Enterprise Data World, Chicago, April 5th 2011<br />25<br />
    26. 26. Physical Design Challenges<br />Some of these challenges are similar to those we face when we use a single Logical Data Model for designing multiple database schemas<br />Differences between Schemas<br />Handling Sub-types and inherited attributes<br />Denormalising attributes and entities<br />Data Type and Name conversions<br />Scalability of Process <br />Automating & future-proofing the generation process<br />Enterprise Data World, Chicago, April 5th 2011<br />26<br />
    27. 27. Model Management Challenges<br />Multiple reference models require careful consideration<br />E.g. Using OAG messaging standard as a reference model, extending it where required<br />Provides a quick-start for defining schemas<br />Be careful what you remove; perhaps remove nothing<br />How do you identify the parts that are actually in use?<br />How do you know what extensions are needed?<br />Have to map the standard to your own models<br />Enterprise Data World, Chicago, April 5th 2011<br />27<br />
    28. 28. Generating XML Models in PowerDesigner<br />Two options<br />Generate XML Model using default options, from selected tables, and tinker the XML afterwards<br />Use XML Builder to design and generate almost exactly what is required, and then tinker the XML model<br />28<br />Enterprise Data World, Chicago, April 5th 2011<br />
    29. 29. Contact Me<br />Telephone: +44 (0) 20 8123 8756 (forwarded to mobile)<br />UK mobile: +44 (0) 794 293 0648<br />Skype: gmcgeachie<br />Twitter: metadatajunkie<br />Email: George.McGeachie @ MetadataMatters.com<br />Blog: http://metadatajunkie.wordpress.com/<br />Enterprise Data World, Chicago, April 5th 2011<br />29<br />
    30. 30. Thanks for listening<br />

    ×