2016 SDMX Experts meeting, SDMX system in the Banco de España, Eduardo Bollo
1. STATISTICS DEPARTAMENT
DIRECTORATE GENERAL ECONOMICS, STATISTICS AND RESEARCH
SDMX SYSTEM IN THE BANCO DE ESPAÑA
Eduardo Bollo
Specialist in Statistical Information Management and Dissemination Systems
EXPERT MEETING ON STATISTICAL DATA AND METADATA EXCHANGE
Aguascalientes,
17-20 October 2016
2. STATISTICS DEPARTMENT 2
INDEX
1. Main features of the BdE system
2. Time series creation
3. Statistical information exchange
4. Dissemination of statistical information
5. A step ahead in the short term: migration to the SDMX-ML 2.1 format
4. STATISTICS DEPARTMENT
MAIN FEATURES OF THE BDE SYSTEM
• BdE system relies on the SDMX standard in order to:
• Manage statistical information
• Automated time series creation
• Disseminate
• Statistical information search metadata (BIEST & $Tsearch)
• SDDS Plus (through INE)
• Exchange statistical information
• Data validation process prior to data delivery
4
5. STATISTICS DEPARTMENT
MAIN FEATURES OF THE BDE SYSTEM
• BdE FAME procedures and functions to produce SDMX data files or to
load SDMX data/metadata files.
• All this information (data/metadata) is stored in FAME
• Artifact definitions are loaded in CASE series in the metadata database
• Exchanged flows are stored in the exchanging (sending/receiving)
database
• SDMX structural definitions for both internal and exchanging time series
5
6. STATISTICS DEPARTMENT 6
• DSD code lists and DSD must be created in a FAME CASE series
• Rules for Collection (Observed), Decimals, Unit, Unit_multiplier are added
to a FAME CASE series where all those sibling attributes are included
• The creation process validates that the dimension values are reflected in
the code list, that the dimensions are well formed and that rules for
assigning sibling attributes exist.
TIME SERIES CREATION
SDMX Collection attribute Fame Observed
Code Description
A Average through period AVERAGED
B Beginning of period BEGINNING
E End of period END
H Highest in period HIGH
L Lowest in period LOW
M Middle of period AVERAGED
S Summed through period SUMMED
U Unknown FORMULA
V Other FORMULA
Y Annualised summed ANNUALIZED
7. STATISTICS DEPARTMENT 7
STATISTICAL INFORMATION EXCHANGE
• Centralized
• Used by the following departments
• Statistics
• Financial Stability
• Payment Systems
• Exchanges data with the institutions
• ECB, BIS, Eurostat, OECD, IMF
• Bidirectional: sends and receives information
• ECB, BIS
• Based on menus
• Automated
• Implemented in FAME
8. STATISTICS DEPARTMENT 8
STATISTICAL INFORMATION EXCHANGE
Tools developed in Java / Fame Time IQ and FAME 4GL
• Fame SDMX-EDI Writer / Fame SDMX-ML 2.1 Writer
• Automatically generates SDMX-EDI files with the information stored in FAME
databases
• Checks that the generated information complies with the Data Structure
Definitions
• Fame SDMX-EDI Reader / Fame SDMX-ML 2.1 Reader (almost finalized)
• Automatically loads SDMX-EDI files in FAME databases, including all the
artifacts declared in the file
• Automatically creates FAME time series with all the metadata hailed as
mandatory by the Data Structure Definitions
9. STATISTICS DEPARTMENT 9
STATISTICAL INFORMATION EXCHANGE
• Graphical user interface
• Dataflow-based security
Based on menus
Generates pre-sending reports and sends data!
10. STATISTICS DEPARTMENT
STATISTICAL INFORMATION EXCHANGE
Pre-sending report
Programa: $informe_envio
Organismo : ECB
Familia : ECB_BSI1, Balance Sheet Items
Tipo de envío : revisión
Frecuencia : mensual
Inicio: 13 ABRIL, 2004 14:23:42
CF_MESNNA30A1U40000Z01E ECB CF_MESNNA30A1U40000Z01E.STA ECB.STA CF_MESNNA30A1U40000Z01E.CNF ECB.CNF
----------------------- ---- --------------------------- ------- --------------------------- -------
2003:12 9900 9385 A A ND ND
2004:02 8674 7878 P P ND ND
10
$informe_envio compares the loaded
status, confidentiality and observation
value to those of the last sending. The
users are perfectly aware of what is going
to be sent.
11. STATISTICS DEPARTMENT 11
STATISTICAL INFORMATION EXCHANGE
• Everything goes fine
green background
• Any error
red background!!
• A message is sent to the user group
responsible for sending dataflow ECB_MIR1
to the ECB
Confirmation message
12. STATISTICS DEPARTMENT 12
STATISTICAL INFORMATION EXCHANGE
• Depending on the acknowledgement message received, a process
creates an HTML page and encloses it.
• A copy is then sent to the group of statisticians in charge of the data
flow
Acknowledgment message
13. STATISTICS DEPARTMENT
DISSEMINATION OF STATISTICAL INFORMATION
• Tools for searching statistical series through Intranet/Internet
• Intranet:
• $Tsearch: confidential databases (approx. 8,000,000 series)
• BIEST: all the information in public databases (approx. 170,000
series)
• Internet:
• BIEST: strictly publishable information (approx. 13,000 series)
• A comfortable way to access the information.
• It doesn’t require previous knowledge of the statistical DB structure.
• It doesn’t require any software installation for the client (BIEST).
13
14. STATISTICS DEPARTMENT
DISSEMINATION OF STATISTICAL INFORMATION
14
• Searching criteria
• Textual and advanced.
• Using SDMX DSD’s (*)
• By Topics
• Publications tree
• Dissemination flows /calendars
• Series code
• Most accessed series
• Series basket
15. STATISTICS DEPARTMENT 15
WHAT COMES NEXT: SDMX-ML 2.1
SDMX-EDI ECB
databases
Banco de
España
BIS
databases
Eurostat
databases
IMF
databases
OECD
databases
SDMX-ML
2.1
INE
databases
Today
SDMX-ML
2.0
16. STATISTICS DEPARTMENT 16
WHAT COMES NEXT: SDMX-ML 2.1
ECB
databases
Banco de
España
BIS
databases
Eurostat
databases
IMF
databases
OECD
databasesNSI
databases
Tomorrow
SDMX-ML
2.0
SDMX-ML
2.1