Sdmx-EDI and Sdmx-ML


Published on

How implement a "data sharing model" and a pull mode using SDMX standards

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • <pagebreak> This diagram depicts one of the most important SDMX Artefacts, the Data Structure Definition (aka Key Family). NOTE: The three classes (DataSet, DataflowDefintion, Category) above the DSD class are not part of the Data Structure Definition, but are included in this diagram in order to show how the DSD is used by Dataflows that define DataSets and are linked to Categories within a Category Scheme. Three conceptual levels can be identified within the DSD diagram. The first level is the Structure level where the DSD is identified (id, version and other attributes not shown in this diagram). The second level comprises ComponentLists. These are conceptual groupings of components (i.e. Key, Groups). The third level includes the Components used by the DSD in order to define the structure of DataSet(s). Moreover, Components reference Concepts residing in ConceptSchemes and may have specific Roles within the DSD. Finally, a Dimension may be a MeasureType dimension, thus defining a Cross-Sectional Measure (XSMeasure class) for every Code included in the Codelist related to this Dimension.
  • <pagebreak> The SDMX standard specifies a single Information Model in order to describe what and how data and metadata can be exchanged in the context of SDMX. Based on this Information Model, SDMX defines two ways of representing its messages. Two different formats are available for expressing SDMX messages. The first format is SDMX-EDI and is equivalent to GESMES/TS. This format is based on an EDIFACT syntax and is Time-Series oriented. This means that only one type of observation throughout time can be carried in a single DataSet. Moreover, the DataSet messages have only one format. This format covers a subset of the SDMX-IM. For example, non time-series messages or reference metadata messages are not supported. The second format is SDMX-ML. It is an XML format that covers the whole SDMX-IM. The SDMX-ML format supports four different (although equivalent) formats, for data messages, in order to serve different purposes. It also supports reference metadata messages as well as messages for querying SDMX Web Services and Registry Interface messages. Two simple “rules” in order to go from the SDMX-IM to the SDMX-ML implementation are that: (concrete) classes become XML elements and their attributes become XML attributes. Of course, there are exceptions in these “rules”. In the SDMX-ML implementation there are XML elements that do not correspond directly to classes from the SDMX-IM and vice versa.
  • <pagebreak> The Cross-Sectional format is the only one capable of expressing non-time series DSDs. In case a TimeDimension is defined in the DSD, the Cross-Sectional measure is equivalent to the rest of the formats. In the namespace declarations of this message, a DSD specific XSD should be included in order to enable syntactical validation of the message. This XSD file can be downloaded from Eurostat’s SDMX Registry ( The layout of the Cross-Sectional type adopts the nesting used in the Generic format but expressing SDMX Artefacts like the Compact format. A major difference in this format is that the elements have different semantic than the previous formats. Although four levels are also defined in this format, only the <DataSet> element is used in the same way. The <Group> element is independent of the Groups included in the DSD. The <Section> element is different from the <Series> element in the sense that it does not include a time-series but a vertical slice across a time-series. Finally, the observation values are now included in DSD specific elements that correspond to the CrossSectionalMeasures defined in the DSD. The SDMX Dimensions and Attributes can be attached at any of the four aforementioned levels as defined in the DSD. Moreover, in the DSD more than one possible levels can be defined per SDMX Component. Of course, only one instance of each SDMX Component should exist in each combination of the four “level” elements. Apart from the capability of reporting data without Time, the Cross-Sectional format is also a useful message for reporting more than one measures at a single DataSet. Thus, in some cases the produced file is even smaller in size than the equivalent Compact message.
  • <pagebreak> The SDMX standard, in its 5 th document of specifications (, gives details on the interfaces that an SDMX compliant Registry should implement. Based on these specifications, Eurostat’s SDMX Registry has been developed and is now deployed in the European Commission’s production environment ( ). Eurostat’s SDMX Registry has been developed in order to be used a central repository for: Structural metadata: Code Lists, Concept Schemes, Data Structure Definitions, Metadata Structure Definitions, Category Schemes, Organisation Schemes, Hierarchical Code Lists Provisioning metadata: Data flows, Metadata flows, Provision Agreements The major interface of Eurostat’s SDMX Registry is a Web Service implementing the SDMX Registry Interface messages. These are SDMX-ML messages and are specified within the standard and by the following XSD: A Graphical User Interface (GUI) has been also implemented in order to enable human interaction of the World Wide Web. The user-authentication is realized using CIRCA accounts. A standalone tool implementing almost all functionality of the SDMX registry is the DSW (already presented).
  • Sdmx-EDI and Sdmx-ML

    1. 1. IPA 2007 - Tirana - INSTAT SDMX-EDI SDMX-ML Skopje 15 Jan 2009
    2. 2. <ul><li>Format that was used during starting period for external trade data </li></ul><ul><li>Later incorporated into Intrastat and Extrastat systems </li></ul><ul><li>There were utilized different media of transmission and passing data files: </li></ul><ul><ul><li>Tapes </li></ul></ul><ul><ul><li>CD-ROM </li></ul></ul><ul><ul><li>CDs </li></ul></ul><ul><ul><li>Diskettes </li></ul></ul><ul><ul><li>Files – by post </li></ul></ul>
    3. 3. <ul><li>First standard defined slowly for different domains in statistics </li></ul><ul><ul><li>GESMES/DSIS – still used for PRODCOM </li></ul></ul><ul><ul><li>GESMES/CB – Gesmes Central Banks </li></ul></ul><ul><ul><li>GESMES/TS – Gesmes Time Series </li></ul></ul><ul><ul><li>Version 3.0 has been used as background to next generation standard – SDMX-EDI </li></ul></ul>
    4. 4. <ul><li>Developed as continuation of the GESMES/TS Standard </li></ul><ul><li>Improved in the Version SDMX 2.0 </li></ul>
    5. 5. SDMX <ul><li>SDMX - S tatistical D ata and M etadata e X change. </li></ul><ul><li>Initiative of seven international organizations to build Standard for Data and Metadata eXchange </li></ul><ul><li>Using modern information technology, with an emphasis on aggregated data. </li></ul><ul><li>All definitions and tools are available on Internet: </li></ul><ul><li> </li></ul>
    6. 6. The need for a standard… <ul><li>Enhance electronic data and metadata exchange </li></ul><ul><li>Promote interoperability between different systems </li></ul><ul><li>Improve the quality of transmitted data (Timeliness & Punctuality, Accessibility & Clarity, Accuracy, Comparability) </li></ul>
    7. 7. SDMX SDMX is primarily focused on the exchange and dissemination of statistical data and metadata. We have normally two different approach to exchange data: PUSH and PULL
    8. 8. SDMX PUSH mode means that the data provider takes action to send the data to the party collecting the data. PULL mode implies that the data provider makes the data available via the Internet. The data consumer then fetches the data on his own initiative.
    9. 9. SDMX SDMX promotes a “ data sharing ” model to facilitate low-cost, high-quality statistical data and metadata exchange. Data Providers publishes the availability of data/metadata to Data Consumers and the latter are responsible for fetching the data/metadata at will. .
    10. 10. Data-Sharing Exchange YOU
    11. 11. NSI 1 2 3
    12. 12. Notes About Data-Sharing <ul><li>Data-sharing only works if there are standard formats </li></ul><ul><li>Like the Web itself, a data-sharing model relies on pull exchanges, not push exchanges </li></ul><ul><ul><li>Data consumers discover the data they need, and its location, and then go and get it </li></ul></ul><ul><ul><li>Data producers don’t have to send data </li></ul></ul>
    13. 13. An easy way to understand the SDMX-IM 10369
    14. 14. <ul><ul><li>What is the subject of the measurement? </li></ul></ul><ul><ul><li>What units does it measure in? </li></ul></ul><ul><ul><li>What country or geographical region, if any, does it apply to? </li></ul></ul><ul><ul><li>When was the measurement made? </li></ul></ul>An easy way to understand the SDMX-IM
    15. 15. <ul><li>Behind each of these questions is a particular idea, or &quot; concept &quot;, which is used to describe the data. </li></ul><ul><ul><li>the Subject is &quot;total population&quot; </li></ul></ul><ul><ul><li>the Unit of measure is &quot;thousands of people&quot; </li></ul></ul><ul><ul><li>the Country is “Germany&quot; </li></ul></ul><ul><ul><li>the Time is &quot;1 January 2001&quot; </li></ul></ul>An easy way to understand the SDMX-IM
    16. 16. <ul><li>We now know that our number represents the fact that the total population of Country “Germany” on 1 January, 2001, was 10,369,000. </li></ul>An easy way to understand the SDMX-IM
    17. 17. Data Set
    18. 18. Data Set: Structure
    19. 19. Data Set: Structure <ul><li>Computers need structure of data </li></ul><ul><li>Concepts and terms </li></ul><ul><li>Code lists </li></ul><ul><li>Data values </li></ul><ul><li>How these fit together </li></ul>Unit Multiplier Unit Topic Time/Frequency Country Stock/Flow
    20. 20. Structural Definitions Topic A Brady Bonds B Bank Loans C Debt Securities Country AR Argentina MX Mexico SA South Africa Stock/Flow 1 Stock 2 Flow Concepts TOPIC COUNTRY FLOW
    21. 21. Data Makes Sense SA,B,1,1999-06-30=16547 16457
    22. 22. Metadata <ul><li>Structural metadata must be associated with the data, otherwise it becomes impossible to identify, retrieve and browse the data. </li></ul>
    23. 23. Structures in the SDMX-IM <ul><li>Dimensions </li></ul><ul><li>Attributes </li></ul><ul><li>Measures </li></ul><ul><li>Groups </li></ul>Data Structure Definition (DSD) Category Category Scheme Code Code List Concept Concept Scheme Components Structure
    24. 24. <ul><li>The simplest explanation of a data structure definition is that it is a set of descriptor concepts, associated with a set of data, which allow us to understand what that data means. </li></ul>Data Structure Definition
    25. 25. DSD components <ul><li>Dimension (e.g. frequency, reference area ) : </li></ul><ul><li>Attribute (e.g. title, observation status ) : </li></ul><ul><li>Measure (e.g. turnover index, # of births, # of deaths ) : </li></ul><ul><li>Groups: </li></ul><ul><ul><li>Grouping of dimensions in order to attach group attributes (e.g. sibling group ) </li></ul></ul>
    26. 26. Data Structure Definition ………… ..………. Structure ………… ..... ComponentList ……………………… .... Component
    27. 27. SDMX V1 and V2
    28. 28. STS Sample Dataset EXAMPLE DATASET1
    29. 29. STS Sample Dataset Dimensions Measure Attributes Dimensions
    30. 30. STS DSD components Dataflow : STSRTD_TURN_M
    31. 31. Demography Sample Dataset EXAMPLE DATASET2
    32. 32. Demography Sample Dimensions Attributes Measures
    33. 33. Demography DSD components Dataflow : DEMOGRAPHY_RQ
    34. 34. IPA 2007 - Tirana - INSTAT Data Set Identifier Variables Form Description STSIND_PROD (_M, _Q) 110 I Production in industry STSIND_TURN (_M, _Q) 120, 121, 122 N, I Turnover in industry, total, domestic and non-domestic (total, Euro-zone, non-Euro-zone) STSIND_ORD (_M, _Q) 130, 131, 132 N, I New orders received in industry, total, domestic and non-domestic (total, Euro-zone, non-Euro-zone) STSIND_EMPL (_M, _Q) 210 N, I Number of persons employed, Number of employees, in industry STSIND_HOUR (_M, _Q) 220 N, I Hours worked in industry STSIND_EARN (_M, _Q) 230 N, I Gross wages and salaries in industry STSIND_PRIC (_M, _Q) 310, 311, 312, 340 I Output prices in industry, total, domestic market, non-domestic market (total, Euro-zone, non Euro-zone), import prices (total, Euro-zone, non-Euro-zone) STSCONS_PROD (_M, _Q) 110, 115, 116 I Production in construction, total, building construction, civil engineering STSCONS_ORD (_M, _Q) 130, 135, 136 N, I New orders received in construction, total, building construction and civil engineering STSCONS_EMPL (_M, _Q) 210, 211 N, I Number of persons employed, Number of employees, in construction STSCONS_HOUR (_M, _Q) 220 N, I Hours worked in construction STSCONS_EARN (_M, _Q) 230 N, I Gross wages and salaries in construction STSCONS_PRIC (_M, _Q) 310, 320, 321, 322 I Output prices in construction, construction costs, material costs, labour costs STSCONS_PERM (_M, _Q) 411, 412 N, I Building permits, number of dwellings or square metres of useful floor area STSRTD_TURN (_M, _Q) 120, 123 N, I Turnover in retail trade, value or deflated STSRTD_EMPL (_M, _Q) 210, 211 N, I Number of persons employed, Number of employees, in retail trade STSSERV_TURN (_M, _Q) 120, 123 N, I Turnover in repair and other services, value or deflated STSSERV_PRIC (_M, _Q) 310 I Outut prices in other services STSSERV_EMPL (_M, _Q) 210, 211 N, I Number of persons employed, Number of employees, in repair and other services STSSERV_CAR (_M, _Q) Number of car registrations STSOTHER_OTH (_M, _Q) Any other indicator not mentioned in the list above
    35. 35. IPA 2007 - Tirana - INSTAT Concept Mnemonic Concept Name Format Description Code list ADJUSTMENT Adjustment AN1 Code defining the adjustment of data such as working day or seasonally adjusted, etc. CL_ADJUSTMENT FREQ Frequency AN1 Frequency of the series (e.g. A, Q, M). CL_FREQ OBS_CONF Confidentiality flag AN1 Confidentiality status of the observation CL_OBS_CONF OBS_PRE_BREAK Pre-break observation value AN…15 Observation value if the reason of the &quot;break&quot; did not show up. [Conditional] OBS_STATUS Status flag AN1 S tatus of the observation, such as normal, estimated or provisional CL_OBS_STATUS OBS_VALUE Value AN…15 The value of the index. ORGANISATION Organisation AN3 Reporting/sending or receiving organisation used in the message administration section. CL_ORGANISATION REF_AREA Reference area AN2 Reporting Country in ISO code (The country, or geographical/political group of countries that the measured economic phenomenon relates to) CL_AREA_EE STS_ACTIVITY Economic Activity code AN6 NACE Rev. 1.1 & special STS aggregates CL_STS_ACTIVITY STS_BASE_YEAR Series variation in short-term stats context AN4 Concept to distinguish series variations in a short-term stats context CL_STS_BASE_YEAR STS_INDICATOR STS Indicator AN4 Type of indicator, such as production, turnover, etc. CL_STS_INDICATOR STS_INSTITUTION Institution originating STS dataflow AN1 Institution originating STS dataflow CL_STS_INSTITUTION TIME_FORMAT Time Format Code AN3 Technical use in message. TIME_PERIOD Time Period AN…35 The time period of the data.
    36. 36. IPA 2007 - Tirana - INSTAT Code List Mnemonic Code List Name Format CL_ADJUSTMENT Adjustment code AN1 CL_AREA_EE Country code AN2 CL_FREQ Frequency code AN1 CL_OBS_CONF Confidentiality flag AN1 CL_OBS_STATUS Observation status flag AN1 CL_ORGANISATION Organisation code list AN3 CL_STS_ACTIVITY STS Economic Activity code list AN6 CL_STS_BASE_YEAR Suffix in short-term stats context code list AN4 CL_STS_INDICATOR Indicators index code AN4 CL_STS_INSTITUTION Institution originating STS dataflow code list AN1
    37. 37. IPA 2007 - Tirana - INSTAT Value Description Variable PROD Production 110, 115, 116 TOVT Turnover (total turnover, non-deflated) 120 TOVD Turnover, domestic market (non-deflated) 121 TOVE Turnover, non-domestic market (non-deflated) 122 TOVV Turnover deflated (volume of sales) 123 TOVX Turnover, non-domestic market (non-deflated) (non-Euro-zone) 122 TOVZ Turnover, non-domestic market (non-deflated) (Euro-zone) 122 DEFL Deflator of sales 330 ORDT New orders received (total) 130, 135, 136 ORDD New orders received, domestic market 131 ORDE New orders received, non-domestic market 132 ORDX New orders received, non-domestic market (non-Euro-zone) 132 PRON Output prices for industry and services (total) 310 PRIN Output prices, domestic market 311 PREN Output prices, non-domestic market (can be approximated by unit value index , variable 313) 312, 313 PREX Output prices, non-domestic market (non-Euro-zone) 312 PREZ Output prices, non-domestic market (Euro-zone) 312 IMPR Import prices (total) 340 IMPX Import prices (non-Euro-zone) 340 IMPZ Import prices (Euro-zone) 340 EMPL Number of persons employed (can be approximated by number of employees, variable 211) 210, 211 HOWK Hours worked 220 WAGE Gross wages and salaries 230 PNUM Building permits, number of dwellings 411 PSQM Building permits: square metres of useful floor area 412 CSTI Construction costs (total) 320 CSTM Construction costs, material costs 321 CSTL Construction costs, labour costs 322 CSTO Output prices for construction (approximation for construction costs, variable 320) 310 CREG Car registrations (not in STS Regulation)
    38. 38. From SDMX-IM to messages SDMX-IM SDMX-EDI SDMX-ML
    39. 39. <ul><li>Based on a common Information Model </li></ul><ul><ul><li>SDMX-EDI (GESMES/TS) </li></ul></ul><ul><ul><ul><li>EDIFACT syntax </li></ul></ul></ul><ul><ul><ul><li>Time series oriented – One format for Data Sets </li></ul></ul></ul><ul><ul><li>SDMX-ML </li></ul></ul><ul><ul><ul><li>XML syntax </li></ul></ul></ul><ul><ul><ul><li>Four different formats for Data Sets </li></ul></ul></ul><ul><ul><ul><li>Easier validation (XML based) </li></ul></ul></ul>From SDMX-IM to messages
    40. 40. UNA:+.? ' UNB+UNOC:3+EUROSTAT+Unknown+060627:0000+IREF000001++GESMES/TS++++1' UNH+MREF000001+GESMES:2:1:E6' BGM+74' NAD+Z02+EUROSTAT NAD+MR+Unknown' NAD+MS+EUROSTAT' CTA+CC+:V. Patruno' DSI+SODI_IPI_PROD_M' STS+3+7' DTM+242:200601010000:203' DTM+Z02:200501200503:710' IDE+5+STS' GIS+AR3' GIS+1:::-' ARR++M:GR:W:PROD:NS0020:1:2000:200501200503:710:111.11:A:F:+222.22:A:F:+333.33:A:F:+444.44:A:F:+555.55:A:F:+666.66:A:F:+777.77:A:F:+888.88:A:F:+99.99:A:F:+123.45:A:F:+212.21:A:F:+234.56:A:F:' FNS+Attributes:10' REL+Z01+4' ARR+7+M:GR:W:PROD:NS0020:1:2000' IDE+Z10+COLLECTION' CDV+A' IDE+Z10+AVAILABILITY' CDV+A' UNT+26+MREF000001' UNZ+1+IREF000001' SDMX-EDI
    41. 41. SDMX-ML XML format for the exchange of SDMX-structured data and metadata.
    42. 42. <?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot;?> <!-- Created with SDMX Converter v2.1 --> <CompactData xmlns=&quot;; xmlns:sts=&quot;urn:sdmx:org.sdmx.infomodel.keyfamily.KeyFamily=ESTAT:STS:compact&quot; xmlns:xsi=&quot;; xsi:schemaLocation=&quot; SDMXMessage.xsd urn:sdmx:org.sdmx.infomodel.keyfamily.KeyFamily=ESTAT:STS:compact ESTAT_STS_Compact.xsd&quot;> <Header> <ID>SODI_IPI_PROD_M</ID> <Test>true</Test> <Name xml:lang=&quot;en&quot;>SDMX Tutorial Message</Name> <Prepared>2006-06-27T00:00:00</Prepared> <Sender id=&quot;EUROSTAT&quot;> <Contact> <Name xml:lang=&quot;en&quot;>V. Patruno</Name> <Department xml:lang=&quot;en&quot;>IT Dept</Department> <Role xml:lang=&quot;en&quot;>Maintainer</Role> <Email></Email> </Contact> </Sender> <DataSetAgency>EUSTAT</DataSetAgency> <DataSetID>SODI_IPI_PROD_M_02</DataSetID> <DataSetAction>Append</DataSetAction> <Extracted>2006-01-01T00:00:00</Extracted> <ReportingBegin>2005-01-01T00:00:00</ReportingBegin> <ReportingEnd>2005-03-31T00:00:00</ReportingEnd> </Header> <sts:DataSet> <sts:SiblingGroup REF_AREA=&quot;GR&quot; ADJUSTMENT=&quot;W&quot; STS_INDICATOR=&quot;PROD&quot; STS_ACTIVITY=&quot;NS0020&quot; STS_INSTITUTION=&quot;1&quot; STS_BASE_YEAR=&quot;2000&quot; UNIT=&quot;PC&quot; UNIT_MULT=&quot;0&quot; DECIMALS=&quot;2&quot; TITLE_COMPL=&quot;Elements of the full national etc.&quot;/> <sts:Series FREQ=&quot;M&quot; REF_AREA=&quot;GR&quot; ADJUSTMENT=&quot;W&quot; STS_INDICATOR=&quot;PROD&quot; STS_ACTIVITY=&quot;NS0020&quot; STS_INSTITUTION=&quot;1&quot; STS_BASE_YEAR=&quot;2000&quot; COLLECTION=&quot;A&quot; AVAILABILITY=&quot;A&quot; TIME_FORMAT=&quot;P1M&quot;> <sts:Obs TIME_PERIOD=&quot;2005-01&quot; OBS_VALUE=&quot;111.11&quot; OBS_STATUS=&quot;A&quot; OBS_CONF=&quot;F&quot;/> <sts:Obs TIME_PERIOD=&quot;2005-02&quot; OBS_VALUE=&quot;222.22&quot; OBS_STATUS=&quot;A&quot; OBS_CONF=&quot;F&quot;/> <sts:Obs TIME_PERIOD=&quot;2005-03&quot; OBS_VALUE=&quot;333.33&quot; OBS_STATUS=&quot;A&quot; OBS_CONF=&quot;F&quot;/> <sts:Obs TIME_PERIOD=&quot;2005-04&quot; OBS_VALUE=&quot;444.44&quot; OBS_STATUS=&quot;A&quot; OBS_CONF=&quot;F&quot;/> <sts:Obs TIME_PERIOD=&quot;2005-05&quot; OBS_VALUE=&quot;555.55&quot; OBS_STATUS=&quot;A&quot; OBS_CONF=&quot;F&quot;/> <sts:Obs TIME_PERIOD=&quot;2005-06&quot; OBS_VALUE=&quot;666.66&quot; OBS_STATUS=&quot;A&quot; OBS_CONF=&quot;F&quot;/> <sts:Obs TIME_PERIOD=&quot;2005-07&quot; OBS_VALUE=&quot;777.77&quot; OBS_STATUS=&quot;A&quot; OBS_CONF=&quot;F&quot;/> <sts:Obs TIME_PERIOD=&quot;2005-08&quot; OBS_VALUE=&quot;888.88&quot; OBS_STATUS=&quot;A&quot; OBS_CONF=&quot;F&quot;/> <sts:Obs TIME_PERIOD=&quot;2005-09&quot; OBS_VALUE=&quot;99.99&quot; OBS_STATUS=&quot;A&quot; OBS_CONF=&quot;F&quot;/> <sts:Obs TIME_PERIOD=&quot;2005-10&quot; OBS_VALUE=&quot;123.45&quot; OBS_STATUS=&quot;A&quot; OBS_CONF=&quot;F&quot;/> <sts:Obs TIME_PERIOD=&quot;2005-11&quot; OBS_VALUE=&quot;212.21&quot; OBS_STATUS=&quot;A&quot; OBS_CONF=&quot;F&quot;/> <sts:Obs TIME_PERIOD=&quot;2005-12&quot; OBS_VALUE=&quot;234.56&quot; OBS_STATUS=&quot;A&quot; OBS_CONF=&quot;F&quot;/> </sts:Series> </sts:DataSet> </CompactData>
    43. 43. ExAMplE <ul><li>The first part of the message contains information about the XML namespaces and schemas used </li></ul>
    44. 44. ExAMplE <ul><li>The next part of the message is the header, which identifies the particular data being sent in this case, including the sender and receiver. </li></ul>
    45. 45. ExAMplE <ul><li>The third part of the message contains the data values, or “observations” in SDMX terminology. </li></ul>
    46. 46. SDMX-ML: Six standard messages Fixed To query a database to obtain an SDMX-ML message as the result Query message 6 Derived from data structure definition message Exchange of many observation types in a data structure definition-dependent form Cross-sectional Data Message 5 Derived from data structure definition message For schema-based functions, such as validation, in a data structure definition-dependent form Utility Data Message 4 Derived from data structure definition message Exchange of large data sets in a data structure definition-dependent form Compact Data Message 3 Fixed Conveys data in a form independent of a data structure definition. It is designed for data provision on websites and in any scenario where applications receiving the data may not have detailed understanding of the data set's structure before they obtain the data set itself. Generic Data Message 2 Fixed Contains a data structure definition Structure Definition Message 1 Schema file Short description Name of message
    47. 47. Cross-Sectional Data Set <demo:DataSet REV_NUM = &quot;1&quot; TAB_NUM = &quot;RQFI05V1&quot; > < demo:Group COUNTRY = &quot;FI&quot; FREQ = &quot;A&quot; TIME = &quot;2005&quot; TIME_FORMAT = &quot;P1Y&quot; > < demo:Section DECI = &quot;0&quot; UNIT = &quot;PERS&quot; UNIT_MULT = &quot;0&quot; > < demo:ADJT OBS_STATUS = &quot;P&quot; SEX = &quot;F&quot; value = &quot;35&quot; /> < demo:DEATHST OBS_STATUS = &quot;P&quot; SEX = &quot;F&quot; value = &quot;23871&quot; /> < demo:LBIRTHST OBS_STATUS = &quot;P&quot; SEX = &quot;F&quot; value = &quot;28345&quot; /> < demo:NETMT OBS_STATUS = &quot;P&quot; SEX = &quot;F&quot; value = &quot;4187&quot; /> < demo:PJAN1T OBS_STATUS = &quot;P&quot; SEX = &quot;F&quot; value = &quot;2683230&quot; /> < demo:PJANT OBS_STATUS = &quot;P&quot; SEX = &quot;F&quot; value = &quot;2674534&quot; /> < demo:ADJT OBS_STATUS = &quot;P&quot; SEX = &quot;M&quot; value = &quot;131&quot; /> < demo:DEATHST OBS_STATUS = &quot;P&quot; SEX = &quot;M&quot; value = &quot;24057&quot; /> < demo:LBIRTHST OBS_STATUS = &quot;P&quot; SEX = &quot;M&quot; value = &quot;29400&quot; /> < demo:NETMT OBS_STATUS = &quot;P&quot; SEX = &quot;M&quot; value = &quot;4799&quot; /> < demo:PJAN1T OBS_STATUS = &quot;P&quot; SEX = &quot;M&quot; value = &quot;2572350&quot; /> < demo:PJANT OBS_STATUS = &quot;P&quot; SEX = &quot;M&quot; value = &quot;2562077&quot; /> < demo:ADJT OBS_STATUS = &quot;P&quot; SEX = &quot;T&quot; value = &quot;166&quot; /> < demo:DEATHST OBS_STATUS = &quot;P&quot; SEX = &quot;T&quot; value = &quot;47928&quot; /> < demo:LBIRTHST OBS_STATUS = &quot;P&quot; SEX = &quot;T&quot; value = &quot;57745&quot; /> < demo:NETMT OBS_STATUS = &quot;P&quot; SEX = &quot;T&quot; value = &quot;8986&quot; /> < demo:PJAN1T OBS_STATUS = &quot;P&quot; SEX = &quot;T&quot; value = &quot;5255580&quot; /> < demo:PJANT OBS_STATUS = &quot;P&quot; SEX = &quot;T&quot; value = &quot;5236611&quot; /> < /demo:Section> < demo:Section DECI = &quot;0&quot; UNIT = &quot;PURE_NUMB&quot; UNIT_MULT = &quot;0&quot; > < demo:DIV OBS_STATUS = &quot;P&quot; SEX = &quot;T&quot; value = &quot;13383&quot; /> < demo:MAR OBS_STATUS = &quot;P&quot; SEX = &quot;T&quot; value = &quot;29283&quot; /> < /demo:Section> < demo:Section DECI = &quot;3&quot; UNIT = &quot;PURE_NUMB&quot; UNIT_MULT = &quot;0&quot; > < demo:TFRNSI SEX = &quot;T&quot; value = &quot;1800&quot; /> < /demo:Section> < /demo:Group> </demo:DataSet>
    48. 48. SDMX-ML “Model-Driven” XML Approach
    49. 49. What Do You Need to Do? <ul><li>If you want to use SDMX to exchange data, what do you need to do? </li></ul><ul><li>Step 1 : Define a data structure definition (key family) </li></ul><ul><li>Step 2 : Create XML Schemas describing data formats </li></ul><ul><li>Step 3 : Use schemas in implementation </li></ul><ul><ul><li>Create data </li></ul></ul><ul><ul><li>Exchange data </li></ul></ul><ul><ul><li>Import data </li></ul></ul><ul><ul><li>Create deliverables (CSV, HTML, etc.) </li></ul></ul>
    50. 50. SDMX Registry <ul><li>A repository for keeping </li></ul><ul><ul><li>Structural metadata (e.g. CodeLists, ConceptSchemes, DSDs, MSDs) </li></ul></ul><ul><ul><li>Provisioning information (e.g. Dataflows, Provision Agreements) </li></ul></ul><ul><li>Repository is accessible via a Web Service accepting SDMX-ML messages </li></ul><ul><li>GUI for user interaction over the Web </li></ul><ul><li>Authentication using CIRCA accounts </li></ul><ul><li>DSW – “standalone” Java GUI </li></ul>
    51. 51. <ul><li> </li></ul>SDMX Registry
    52. 52. Query Message <ul><li>This is another special-purpose message. It is used to convey a query to a database which then returns an SDMX-ML message </li></ul><ul><li>For web services and database-driven applications </li></ul>
    53. 53. Query Message
    54. 54. Query SDMX Data SDMX-ML RSS WS NSI
    55. 55. Query SDMX Dati (SDMX-ML)‏ RSS WS
    56. 56. DB WS demogr DEMO SODI RSS script
    57. 57. Query SDMX Dati (SDMX-ML)‏ RSS WS demogr
    58. 58. Thank you for your attention Vincenzo Patruno: [email_address]