CORE ESSnet Report @MSIS 2011

891 views

Published on

Presentation of CORE Research Network (ESSnet) at MSIS (Management of Statistical Information Systems) 2011 - Luxembourg

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
891
On SlideShare
0
From Embeds
0
Number of Embeds
240
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

CORE ESSnet Report @MSIS 2011

  1. 1. CORE ESSnet(COmmon Reference Environment) progress report Carlo Vaccari Istat - ItalyMSIS Meeting - Luxembourg May 23-25 2011 1
  2. 2. Outline Introduction and history CORE objectives CORE where we are Architecture implementation Information model CORE and SDMX CORE and GSIMMSIS Meeting - Luxembourg May 23-25 2011 2
  3. 3. CORA ESSnet Financed by Eurostat under 2009 Statistical Workprogramme Countries involved: it (coordinator), ch, dk, lv, nl, no, se Duration: October 2009 - October 2010MSIS Meeting - Luxembourg May 23-25 2011 3
  4. 4. CORA Technical Architecture CORA Model: two dimensions Functional dimension Construction dimension Functional dimension Adoption of GSBPM 4.0 9 subprocesses of level 2 1 2 3 4 5 6 7 8 9 Specify Design Build Collect Process Analyse Disseminate Archive Evaluate NeedsMSIS Meeting - Luxembourg May 23-25 2011 4
  5. 5. Construction Dimension: Layers A domain of interest documented by Figures statistical products Time Series Statistical series over time Integrated or simple statistical product for a Statistic given time Population A population at a given time Unit A statistical unit at a given time Variable A statistical variable at a given time A logical representation of the value of a Value variableMSIS Meeting - Luxembourg May 23-25 2011 5
  6. 6. CORA Model Grid Statistical processes compliant to CORA model are intended to be designed by statisticiansMSIS Meeting - Luxembourg May 23-25 2011 6
  7. 7. After CORA…CORE! COmmon Reference Environment (CORE), financed by Eurostat under 2010 Statistical Workprogramme Countries involved: it (coordinator), fr, nl, no, pt, se Duration: December 2010 - January 2012MSIS Meeting - Luxembourg May 23-25 2011 7
  8. 8. CORE Principal Outcomes Environment for the definition and execution of statistical processes Definition of a process in terms of services selected from an available repository Execution of the composed workflowMSIS Meeting - Luxembourg May 23-25 2011 8
  9. 9. CORE Outcomes: Design CORA model → CORA information model Design of CORE services and processesMSIS Meeting - Luxembourg May 23-25 2011 9
  10. 10. CORE Outcomes: Implementation Selection of available middleware solutions for process execution Realization of an environment able to permit the execution of processes: Interfaces (GUIs) for defining CORE processes for statistical users Integration APIs Repository of integration layersMSIS Meeting - Luxembourg May 23-25 2011 10
  11. 11. CORE Outcomes: Testing Realization of processes starting from services implementing some GSBPM phase Evaluation of costs related to integration Prototype implementation (to be engineered)MSIS Meeting - Luxembourg May 23-25 2011 11
  12. 12. CORE Architecture (1) GUIs to support modelling of CORE processes according to the CORA grid Modeling & control flow constructs Drag & drop facilities for process design Global schema Implementation: we are evaluating the usage of an open process editor tool (Oryx - http://bpt.hpi.uni-potsdam.de/Oryx/WebHome )MSIS Meeting - Luxembourg May 23-25 2011 12
  13. 13. CORE Architecture (2) Process Runtime Controlled execution of services Implementation: integration of existing workflow solutions, currently in evaluation phase Service Runtime Integration APIs (in-out data transformation) Service execution Implementation: CSV and SQL data transformations are currently being implemented Service Repository Deployment of servicesMSIS Meeting - Luxembourg May 23-25 2011 13
  14. 14. CORE Information Model (1) First draft of CORE information model Del. 2.1 released: requirements for the model of the interface through which statistical services will communicate Information Model to be released, currently in discussion phaseMSIS Meeting - Luxembourg May 23-25 2011 14
  15. 15. CORE Information Model (2) Design Principles (in discussion): Rectangular data sets (rows & columns) Strong typing (data, rules, parameters) Dataset kinds (eg micro/aggreg) Free-style arguments (eg scripts tool dependent) Other (service arguments and infos)MSIS Meeting - Luxembourg May 23-25 2011 15
  16. 16. CORE and SDMX Both initiatives foster standardization CORE Focus on standardization of processes and data exchanges (mainly) intra-NSI SDMX Focus on standardization of processes and data exchanges (mainly) inter-NSIs (or between NSIs and international organizations)MSIS Meeting - Luxembourg May 23-25 2011 16
  17. 17. CORE and SDMX - 2 CORE Focus on all phases of statistical processes Both micro and macro data considered SDMX Focus (mainly) on dissemination phase Mainly macro data consideredMSIS Meeting - Luxembourg May 23-25 2011 17
  18. 18. Information Model Both propose an information model CORE information model Takes explicitly process dimension into account through GSBPM Data dimension SDMX information model Mainly focused on data dimensionMSIS Meeting - Luxembourg May 23-25 2011 18
  19. 19. CORA Information Model Service +belongs_to Layer level +contains 1 +implements +belongs_to n Constructor prescript +has +input Element n n output.belongs_to.level = input.belongs_to.level + 1 +output n Construct Object +represented_by 1 Figure Time series Statistic Population Unit VariableMSIS Meeting - Luxembourg May 23-25 2011 19
  20. 20. SDMX Data & Metadata Information Model Data or Metadata Structure Definition uses specific data or metadata can have child structure categories can be linked with Data or Data or categories from multiple category schemesMetadata Metadata Category Set conforms to business Flow rules of the data or metadata flow comprised can get data of subject from multiple or reporting data providers categories Data Provision Category Provider Agreement SchemeMSIS Meeting - Luxembourg May 23-25 2011 20
  21. 21. On Information Models Different abstraction levels CORE “Higher” modelling level E.g.: statistics as tabular data SDMX “Lower” modeling level E.g.: aggregated data set with dimensions, attributes and measuresMSIS Meeting - Luxembourg May 23-25 2011 21
  22. 22. Open Issues - 1 Can we use SDMX for micro and macro data exchanges in a CORE process? Need for mapping of information modelsMSIS Meeting - Luxembourg May 23-25 2011 22
  23. 23. On-Going Work - 1 CORE implementation scenario within Istat Main phases: Sample selection and allocation CORE wrapping of available SAS and R proceduresMSIS Meeting - Luxembourg May 23-25 2011 23
  24. 24. On-Going Work - 2 Design and implementation of CORE Integration APIs Possible in/out SDMX translations CORE TOOL SDMX IAPI TOOL IAPI SDMXMSIS Meeting - Luxembourg May 23-25 2011 24
  25. 25. Open Issues - 2 What about metadata? CORE: Data and metadata managed in the same way SDMX: Distinction between structural metadata and reference metadata Dedicated effort for metadata managementMSIS Meeting - Luxembourg May 23-25 2011 25
  26. 26. Collaboration between CORE/SDMX ESSnets CORE planned deliverable on “Feedbacks on SDMX Usage in CORE” Periodical meetings inside Istat between coordinators of the two ESSnets Exchanges of resources between the two ESSnetsMSIS Meeting - Luxembourg May 23-25 2011 26
  27. 27. CORE and GSIM GSIM: Generic Statistical Information Model deliverable from OCMIMF Operationalising a Common Metadata/Information Management Framework activity inside Statistical Network Ambiguity on the acronym: reference to “generic statistical information model” in CORE ESSnet proposal In March started activity to clarify relationships (thanks to J.P.Kent and A.Hamilton)MSIS Meeting - Luxembourg May 23-25 2011 27
  28. 28. CORE and GSIM First analysis and discussions: the deliverables from the two initiatives are complementary in intent and do not overlap in concept Necessary to avoid gaps and/or duplications and ensure the complementary relationshipMSIS Meeting - Luxembourg May 23-25 2011 28
  29. 29. CORE Information Model CORE will define a very generic information model (CORE-IM) for the interface through which statistical services will communicate with each other within the framework of the CORA model As a communication protocol, CORE-IM focuses on the “postal envelope” used when passing information between services, rather than focusing in detail on the information being communicated (ie what is inside)MSIS Meeting - Luxembourg May 23-25 2011 29
  30. 30. CORE and GSIM CORE-IM current hypothesis: to support a flag to indicate if the information being communicated is described within GSIM → without claiming to align the semantics of the content (eg, “classification”), but only to alert a consuming service which “understands” GSIM that it can relate the content to GSIMMSIS Meeting - Luxembourg May 23-25 2011 30
  31. 31. Complementary nature CORE-IM supporting semantic interoperability at a very high, abstract level (“here is an information object, along with the ‘envelope’ information about it”) where GSIM can provide greater semantic precision to a subset of information objects communicated using CORE CORE supporting communication between services → substantial interoperability benefits Information aligned with GSIM semanticsMSIS → further level of interoperability Meeting - Luxembourg May 23-25 2011 31
  32. 32. Complementary nature GSBPM reference model for statistical business processes GSIM reference model for information input to, used by and produced by those processes Models are independent → its possible to use one without the other CORE-IM recognizes and uses GSBPM and (hopefully) will do the same with regard to GSIM, giving them a potential contact pointMSIS Meeting - Luxembourg May 23-25 2011 32
  33. 33. Coordination in practice Need to maximize the extent to which these synergies are achieved in practice: Members in common (no, se) ABS leader for the OCMIMF and observer in CORE CORE members external reviewers for GSIM material CORE WP2 “co-ordination input” from the OCMIMF collaboration team in regard to deliverables Half day session at METIS workshop (October) presenting CORE and OCMIMF works to external metadata specialists Common documents in preparation ...MSIS Meeting - Luxembourg May 23-25 2011 33

×