Your SlideShare is downloading. ×
0
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
NIEM and XML for Architects and Developers
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

NIEM and XML for Architects and Developers

798

Published on

Maplesoft event in Ottawa

Maplesoft event in Ottawa

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
798
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
28
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. <Insert Picture Here>Delivering NIEM-aligned data servicesDavid Webber, Information Architect, Oracle Public SectorNIEM and XMLfor Architectsand Developers
  • 2. The following is intended to outline Oracle generalproduct direction. It is intended for informationpurposes only, and may not be incorporated into anycontract. It is not a commitment to deliver anymaterial, code, or functionality, and should not berelied upon in making purchasing decisions.The development, release, and timing of any featuresor functionality described for Oracle’s productsremains at the sole discretion of Oracle.Disclaimer NoticeSlide 2
  • 3. © 2012/13 Oracle Corporation – Restricted Distribution 3Agenda• Background - What are the business drivers and needs?• NIEM today – Technical Introduction• The Delivery Life Cycle – Lessons Learned• Software Delivery - Anatomy of a NIEM “IEPD”• Tooling to the Rescue!• Key Pillar of Dictionaries• Summary
  • 4. BACKGROUNDBusiness Drivers and Needs
  • 5. © 2012/13 Oracle Corporation – Restricted Distribution 5XML is everywhere…• PMIX – Prescription Drug information sharing• Elections Results Reporting• Pharmacy and Vaccines locating• COBie Buildings Information sharing• PESC – Student Post-Secondary Education• City 311 Services• Vehicle spare parts inventory• Public Safety information sharing• Healthcare services delivery coordination• eDevice control templates
  • 6. © 2012/13 Oracle Corporation – Restricted Distribution 6Public Sector Challenges• Cost of building information exchanges– Reduction of development sunk costs through minimizing coding tasks– Allowing rapid prototyping and proof of concepts (agile development)– Flexibility - dynamic adaptable templates instead of rigid fixed code– Reuse – templates for common systems / solutions easily shared and adapted• Government transparency and public information sharing– Structured content from data stores – e.g. election results reporting, monthly costreports, scheduled road maintenance; obvious metadata and semantics– Simple search requests on data – e.g. part numbers, codes, licenses lookup– Not intended for unstructured and textual content document delivery• Practical real world data sharing anywhere– Direct simple tools with short learning curve and plug and play deployment
  • 7. © 2012/13 Oracle Corporation – Restricted Distribution 7Technology Capability Requirements• Delivering on the Open Data Digital Strategy Vision for government• Open Standards and Open Source based• Plug and play with code-free templates and rapid development• Supports NIEM and Open XML exchanges• Leverages deployed technology today• Workflow integration• Secure messaging delivery and partner management• Delivered across-platform and device• End-to-end security & governance
  • 8. NIEM TODAYTechnical Introduction
  • 9. The NIEM FrameworkNIEM connects communities of people who share a common need toexchange information in order to advance their missions, and provides afoundation for seamless information exchange betweenfederal, state, local, and tribal agencies. Much more than a datamodel, NIEM offers an active user community as well as a technical andsupport framework.Support FrameworkTechnical FrameworkCommunityFormal GovernanceProcessesOnline RepositoriesMission-Oriented DomainsSelf-ManagingDomain StewardsData ModelXML Design RulesDevelopment MethodologyPredefinedDeliverables (IEPD)Tools for Developmentand DiscoveryEstablishedTraining ProgramImplementation SupportHelp Desk &Knowledge Center
  • 10. The NIEM Data ModelNIEM’s data model is a set of common, controlled, and approved XMLdata structures and definitions vetted through theFederal, State, Local, Tribal and Private Sectors.Data elements are organized into core anddomain-specific componentsCore components areused by multipledomains and can bedescribed bystructure, semantics, and definitionuniversallyDomain-specificcomponents arecontinually updated bysubject matter expertsthat are actual NIEMparticipants andindustry experts fortheir particular domainNIEM Naming andDesign Rules (NDR)specify how each ofthese components aredefined and utilized
  • 11. NIEM’s Domains ViewNEWEST DOMAINSNIEM Government ResourceManagement DomainNIEM Health DomainNIEM Human Services DomainDOMAINS IN PLANNINGNIEM AgricultureNIEM EducationNIEM Transportation11
  • 12. © 2012/13 Oracle Corporation – Restricted Distribution 12Available XML Dictionaries• NIEM 2.1 dictionaries– CBRN dictionary– Emergency dictionary– Family dictionary– Immigration dictionary– Infrastructure dictionary– Intelligence dictionary– Justice dictionary– Maritime dictionary– Screening dictionary– Trade dictionary– Immigration blueprint– NIEM core dictionaryAvailable from download sitedirect link:http://sourceforge.net/projects/camprocessor/filesXML XML XML XML XML XML+ includes spreadsheets and sample modelsNote: Those marked in bold are model style dictionaries with recursive components.
  • 13. © 2012/13 Oracle Corporation – Restricted Distribution 13NIEM uses W3C Schema CollectionXSD Collection*Code list xsdNamespace* OptionalXML Schemaxsd subset *Constraint xsd*Extension xsd*Restriction xsdIndustry DomainschemasCode list xsdLEGEND:Exchange xsdNIEMExchangeSchema
  • 14. © 2012/13 Oracle Corporation – Restricted Distribution 14NIEM Past and Present Challenges• Vertical domain vocabulary for DHS and DOJ• One dozen domain dictionaries• Technology limited - built using XSD schema• Core components highly contextual to DHS/DOJ• Surprisingly – no actual logical models of information!• Gap with semantic technologies integration• Limited data content rules and code lists• Mapping automation support missing• Multi-year development life cycles
  • 15. © 2012/13 Oracle Corporation – Restricted Distribution 15NIEM is still improving…• Collections of complex XML Schema• Verbose components• Embedded context in names• Currently facing significant scaling challenges– Inconsistencies; too much manual management; slow lifecycles• Dictionary technology incubating• Enhanced code lists mechanism incubating• UML profile is evolving initiative with OMG– Attempt to marry modelling techniques and XSD Schema syntax• Policy and Security mechanisms incubating• Need to embrace enabling tools for rapid expansion ofdomains and community collaboration
  • 16. DELIVERY LIFECYCLEHow to build NIEM solutionsLessons learnedFoundation Pillars
  • 17. © 2012/13 Oracle Corporation – Restricted Distribution 17The 8 “D”s and NIEM• Design• Develop• Deploy• Document• Dictionaries• Discovery• Differentiate• Diagnose Repeatable, Reusable Process(Exchange Specification Lifecycle)NIEM IEPD Process*IEPD - Information Exchange Package Documentation
  • 18. ©2012 Oracle Corporation 18NIEM IEPDReportsDictionariesDiscoveryNIEM IEPD / Exchange Delivery LifecycleDiagnoseDifferentiateDesignDevelopValidatedTemplates /SchemaXSDSchemaXMLSamplesXMI / UMLModelsProductionResultsRequirementsUpdatesDocumentationDrag and DropVisualDesignerDocumentDeployExchangeTemplates 12345RequiredIEPD artifacts
  • 19. ©2012 Oracle Corporation 19Integrating into your existing processes and control• Critical to manage and steer use of NIEM• Ensure business outcomes match requirements• Lessons learned with NIEM• The earlier in the software development process that NIEM isintroduced the better the overall outcomes• Each step has deliverables – business plan, deliveryarchitecture, data model, alignment reports, schema, test resultsthat can be monitored in your SDLC*• Monitoring products reinforces reuse as best practice• Tools essential in review processes• Collaboration tools can coordinate artifacts acrossprojects and participants• Test bed tools and open APIs reduce integrationalignment costs *SDLC – Software Development Life Cycle
  • 20. SOFTWARE DELIVERYAnatomy of a NIEM “IEPD”It is not just all about the XSD schema
  • 21. ©2012 Oracle Corporation 21IEPD Components & Requirements<Exchange_Schema/><Extension_Schema/><Subset_Schema/>IEPD IEMIEPD MPDNIEM CoreSchema(s)DomainSchema(s)Main DocumentCatalogChange LogSample XMLInstanceIn order to be NIEM-conformant, the IEPD must adhere to:1. NIEM Conformance Document2. NIEM Naming and Design Rules (NDR) v1.33. NIEM Model Package Description (MPD) Specification v1.0
  • 22. Formal NIEM IEPD Packaging DetailsIEPD Artefact DescriptionExchange FilesSubset Schema (SSGT) Subset of the full NIEM schema—a compressed directory of schemas to distinguish from other schema sets.WantlistUser requirements -distinguishes user-required data components from components they depend on for conformance. Generated by and up-loaded to theSchema Subset Generation Tool (SSGT). This is an open specification and the SSGT is not required to create a wantlist, though it is easier.Exchange SchemaBase document schema that defines the XML root element and is generally named after the IEPD itself. Also known as the document schema, referenceschema, or root schema.Constraint Schema Constraints for separate constraint validation path—a compressed directory of schemas to distinguish from other schema sets.Extension Schema Specification for extended components—separate local name-spaces of components not contained in NIEM.Sample XML Instance Example instance(s) – may reference optional stylesheet.Stylesheet (generic) Example stylesheet for display of instance(s).DocumentationMaster DocumentationThe Master Document is the main document for which all of the context and details around the exchange are explained. This document includes, theoverview, as well as details surrounding the exchange, business drivers and requirementsExchange model XMIExchange model in standard open format (xmi, vsd, zargo) and standard open graphic (jpg, pdf, etc.) preferably a Unified Modeling Language (UML)model.Business RulesBusiness rules in one of the following formats: (1) plain or structured English, (2) written into master documentation, (3) Schematron or other formalbusiness rule language, or (4) generated by a development tool.Mapping to NIEM Mapping of domain components, tagged with constraints (i.e., cardinality, etc.) to NIEM components; preferably the Component Mapping Tool (CMT).Extended ComponentsComponents created because they were not in NIEM—may be part of mapping spreadsheet and include structure and definitions of new components. PreferCMT.Change log Record of cumulative changes from previous IEPD versions. The initial IEPD simple records its creation date.CatalogCatalog XML file A machine-readable list of artifacts in the IEPD in an open, portable and browser-displayable format (simple manual edit).Metadata XML file All metadata of owner and domain to be registered with the IEPD (simple manual edit).Generatedusing CAMNever UsedWordDocumentManualEntry
  • 23. ©2012 Oracle Corporation 23XML Technology ComponentsNIEM: need to managecomplete aspects of exchangedelivery, not just the structureand content data model
  • 24. ©2012 Oracle Corporation 24Swim Lane ViewUML Technologies XML TechnologiesW3C SchemaThis is what NIEM exchangesare delivered with on-the-wire!UMLXMIXSD NamespaceEnumerationsXSLTXQueryXPathXMLPatternsAssociationsCardinalityStereotypesInheritanceFacetsElementsAttributesContentComplex TypesAudience:Data Modellers / DesignersAudience:Software Developers andMiddleware EngineersAudience:General content deliveryand applications; contentengineers; rulesrepresentation systems;multimedia delivery;geospatial systemsObjectsWebHTMLJavaScriptJSONRESTSQLSOAPAudience:WebapplicationdevelopersOpen-XDX24
  • 25. TOOLING TO THE RESCUEAccelerating and Improving the NIEM process
  • 26. ©2012 Oracle Corporation 26IEPD Delivery Tasks / Tools MatrixCAM CAMVOpen-XDX(OPX)Exchange Schema Development XCanonical Model Development / Generation XExchange Schema Generation XNIEM Compliance Checking / Reporting XWSDL / JAXB Binding Generation XOperation Message Test Case Generation XProduction Services Message Generation XSQL Schema Mapping X XNIEM IEPD Artifact Generation X- Cross-reference spreadsheet X- UML model X- Business Rules Report X- XSD schema package XConformance Test Suite XRun time XML instance validation XIn order to be NIEM-conformant, the IEPD must adhere to:1. NIEM Conformance Document2. NIEM Naming and Design Rules (NDR) v1.33. NIEM Model Package Description (MPD) Specification v1.0
  • 27. © 2012/13 Oracle Corporation – Restricted Distribution 27CAM Toolkit for NIEM• Concepts behind the Content Assembly Mechanismapproach• Understanding the steps involved in top down /bottom up exchange design, testing and delivery• Applicability to NIEM and IEPD / MPD delivery* NIEM – National Information Exchange Model* IEPD – Information Exchange Package Documentation*MPD – Model Package Description
  • 28. © 2012/13 Oracle Corporation – Restricted Distribution 28CAM Templates – WYSIWYG ExchangeStructure RulesDocumentationCAM Templatehttp://www.oasis-open.org/committees/cam
  • 29. © 2012/13 Oracle Corporation – Restricted Distribution 29INGESTCAM and W3C Schema WizardsEXPORTXSLT toolsprocessentire XSDcollectionXSD CollectionStructureRulesDocumentation*Code list xsdNamespace* OptionalXML Schemaxsd subset *Constraint xsd*Extension xsd*Restriction xsdIndustry DomainschemasCode list xsdLEGEND:Exchange xsdExchangeSchemaCAM Template
  • 30. http://www.cameditor.org CAM Editor – Main Screens
  • 31. © 2012/13 Oracle Corporation – Restricted Distribution 31CAM Toolkit Features• Dictionary “Drag and Drop” of components to build exchanges• Visual editor and structure editing• Cross-platform deploy - Windows, Ubuntu/LINUX, Mac; 32/64bit• Mind map models – integration with Freemind tool• UML/XMI export – import to UML tools / ArgoUML• Code list table lookups – rule based and selectable• This allows people to fully integrate their exchange developmentfrom dictionaries through to working deployed exchangeschema.31
  • 32. © 2012/13 Oracle Corporation – Restricted Distribution 32XML Exchange Package Artifacts• Template Evaluation Report– Naming and Design Rules assessment– Interoperability factors review– Spell checking• Comparison to Dictionary– Reuse report, cross-reference spreadsheet, subset “wantlist”• Generate Schema Bundle with layout styles (e.g. NIEM)• Realistic test samples generation• Testing with runtime validation engine (CAMV)• Business information exchange documentation• Models – XMI and mind maps
  • 33. © 2012/13 Oracle Corporation – Restricted Distribution 33CAM Template Evaluation Report• The purpose of the CAM template evaluation report isto provide a suite of checks against a template thatare difficult, tedious or otherwise to determine bymanual inspection of a template structure and itsassociated rules and annotation details• The report covers the following areas:– alerts users to potential issues with their template andprovides suggestions for improving the functional details– addresses logical and conceptual issues that simple syntaxchecking alone does not flag– can be used as part of a formal review process to ensure abaseline for quality control
  • 34. © 2012/13 Oracle Corporation – Restricted Distribution 34CAM Toolkit Capabilities
  • 35. © 2012/13 Oracle Corporation – Restricted Distribution 35Online Tutorials – www.niemtrainingvideos.org
  • 36. KEY PILLAR OF DICTIONARIESIntroduction and Concepts
  • 37. © 2012/13 Oracle Corporation – Restricted Distribution 37The Semantic Framework and VisionXML/NIEMComponents inNeutralDictionaryRepresentationUMLTechnologiesXMLTechnologiesW3C SchemaToolsSemanticRepresentationsRulesTechnologiesHTML 5rendering toolsSecuritysolutionsMiddlewaresolutions
  • 38. © 2012/13 Oracle Corporation – Restricted Distribution 38Dictionaries: Key Pillar of Information SharingCanonical XML Components Dictionary1Component Associations and Couplings2Exchange Templates and Rules3W3C Schema and Model Representations4Delivery Control, Messaging, Security5Implementation Artifacts and Examples6Navigationand QueryToolsDomains Reuse LibraryComponentsAutomatic Rendering ToolsDeployment Environments and MiddlewareXMLUser InsertsRelationshipLookupsTemplateCatalogTesting Workbench, Rules Engine, Data Samples, Integration ETLSchemaXMLModels ComponentsCanonicalDictionaryCollections
  • 39. ©2012 Oracle Corporation 39Dictionaries Semantics Approach• Abstract dictionary representations with strongsemantics, rules and code lists support• Schema, models, documentation are generated fromdictionary definitions automatically• Contextual mechanisms allow linear number of truecore components – not exponential growth• Easier for practitioners to leverage with morepredictable and repeatable results• Enable domain use horizontally across government• ETL mapping automation support• Continuous collaboration environment
  • 40. ©2012 Oracle Corporation 40Developing domain dictionaries (EIEM)• Allows domains to manage their components libraries• Provides consistency for project development teams• Sets of NIEM consistent XML exchange components• Aligned to enterprise data stores• Optimized for reuse and interoperability• Save time and effort across the enterprise• Perennial question for developers – when should I use NIEMcomponents, and when our own local ones?• Are there components already available for that purpose?• Provide formal mechanisms and procedures to sharecomponents and collaborate across SDLC process• Provide external parties consistent data views*EIEM – Enterprise Information Exchange Model
  • 41. ©2012 Oracle Corporation 41EIEM/BIEC• Business Information ExchangeComponents (BIEC)• NIEM-conforming XML schema datacomponent definition that meets aparticular recurring businessrequirement for an enterprise• Enterprise Information ExchangeModel (EIEM)• NIEM-conforming schemas that definedata components to be reused inIEPDs developed by an enterprise;collection of enterprise BIECsorganized into a subset and one ormore extension schemas
  • 42. © 2012/13 Oracle Corporation – Restricted Distribution 42Managed Dictionaries – Authoritative SourcesSlide 42CentralizedDistributedschemaSFIS Values Library Service, NIEM, Metadata RegistryALLOWEDVALUESManagedDictionariesPublish
  • 43. SUMMARYNext Steps
  • 44. © 2012/13 Oracle Corporation – Restricted Distribution 44Why CAM and NIEM?“Can you give us a brief overview of how you expect ITprofessionals and line of business managers to useCAM*?”– Van Ristau, CTO DLT Inc.*CAM editor toolkit – http://www.cameditor.org and OASIS Content Assembly Mechanism standard – http://docs.oasis-open.org/cam
  • 45. © 2012/13 Oracle Corporation – Restricted Distribution 45Response• Line of business managers can use CAM to drive betteralignment to NIEM and faster more consistent andpredictable delivery of NIEM information exchanges• CAM provides tools that allow business analysts to workcollaboratively with software developers to build exchanges• Teams use CAM to perform the 8 “D’s” tasks whenimplementing NIEM exchanges• CAM automates the delivery of key XML artifacts for NIEMexchange services• XML validation can be deployed using the CAMV engine• Open-XDX provides rapid flexible open data web services
  • 46. © 2012/13 Oracle Corporation – Restricted Distribution 46Public Sector Open Data Services DeliveryOpen Platform Solutions• Visual Editor• SQL Browser• Dictionaries• Drag and Drop• Query/Response• Generate Data• Validate Data• Send Data• JAX-WS• soapUI• CAM Editor• Content Models• OER Integration• Web Forms UIOpen Data Test Bed Services DictionariesDBWebOpen XDXCAMVOpen-XDX / CAM EditorDictionarySecuritySQLXMLXMLSQLRepository• CAM Editor• Templates• Policy Rules• XACMLTemplatesXACML
  • 47. © 2012/13 Oracle Corporation – Restricted Distribution 47Technology Components and Features• Providing open data sharing– Open-XDX component accelerates deployment of interfaces• Aligning meaning and definitions– Collaboration using XML component dictionaries and NIEM• Formal information verification and content rules– CAMV rules engine provides runtime handling– Industry domain test bed facilities– Rules documentation and sharing• Business process integration– Oracle BPM and Forms combined with CAMV and Open-XDX• In progress – Security profiles and policy templates– Upcoming release of CAM Editor and XACML integration
  • 48. © 2012/13 Oracle Corporation – Restricted Distribution 48CAMVinteractivetestingCloud Services DeploymentOpen-XDXwebservicesXMLTechnologiesW3C SchemaToolsSemanticRepresentationsRulesTechnologiesSecuritysolutionsMiddleware solutionsCAMV testbed forindustriesRegistry forDictionariesSecurityProfileTemplatesWeb site and collaboration toolsPolicyRules
  • 49. CAMeditor.ORG Project StatisticsSNAPSHOT OFPROJECT ACTIVITIES135,000+ CAMeditor.orgpage visits to site165+ countries havedownloaded tools;30% of visitors are fromU.S.;800+ downloads weekly2000+ student views ofonline video trainingresources8 languages nowavailablewww.cameditor.orgwww.niemtrainingvideos.org
  • 50. ©2012 Oracle Corporation 50

×