The document summarizes alternatives for implementing the SDMX standard at IBGE to expose aggregated data. It discusses: 1) Mapping dimensions from SDMX data structure definitions to expressions in IBGE's SIDRA database to identify available indicators. 2) Querying SIDRA to process and format results in the SDMX style for response. 3) As a secondary option, mapping to IBGE's BME microdata database to generate custom aggregates if not available in SIDRA. The document concludes SIDRA exposure would require less initial effort but mapping subjects to variables may not always be straightforward, requiring alternative approaches.

1. SDMX EXPERTS MEETING
Aguascalientes, Mexico
Alternatives for Implementing SDMX at IBGE
Luiz Antônio Vivacqua C. Meyer
luiz.vivacqua@ibge.gov.br
Outubro/2016

2. Agenda:
• Data Analysis and Dissemination Infrastructure
 Metadata Database
 BME Database
 Sidra Database
• Alternatives for exposing data according to
SDMX Standard
• The Logical Steps
 The Databases X SDMX Mapping
• Final Remarks

3. Metadata Goals:
• To provide a centralized metadata system to support
IBGE production areas and international statistical
agencies.
• To give more knowledge about data in order to make
IBGE data more visible and easy to understand.
• To inform users about enumeration methods, quality
and products regarding the surveys.
• To help other IT technologies like: Data WareHouse,
Dissemination Databases and Interoperability.
• Compliant to DDI standard
Contents:
•Surveys:
151
•Instances:
7.060
•Variables:
483.121
•Dictionaries:
1002
Metadata

4. BME - Multidimensional Statistical Database
The BME statistical information system is composed by a ROLAP
database and a set of tools that allow users to generate their own
aggregate data freely accordingly to their needs.
The database content is formed by IBGE surveys microdata.
Contents:
Records: 2,5 bilhões Storage: 2,5 TB
Variables: Coded: 13.400 Numeric: 5.250
Metadados
BME

5. The SIDRA statistical information system is composed by a Multidimensional
aggregated database for the dissemination of IBGE surveys tabular plans.
The database content is formed by IBGE surveys macrodata.
Contents:
4.100 tables - from 60 different surveys and censuses
1,3 trillion values
+- 70.000 user sessions each month
2 users interfaces: Web and REST API
SIDRA
SIDRA
Metadados
BME

6. Alternatives for exposing aggregated data:
• IBGE already has a convenient infrastructure that allows
users to efficiently access the surveys microdata,
metadata and aggregate data.
• The question is to how to determine the best way to
generate and to provide indicators to external users
using SDMX standard.

7. Alternatives for exposing aggregated data:
As mentioned by Efstratios Nikoloutsos in his report:
• “The OECD already proposes the use of SDMX-RI as the
component infrastructure to expose the disseminated
data via SDMX web services.”
• “Nevertheless, the current dissemination infrastructure,
due to the fact that there is an API in order to access
data, it cannot be used as-is for exposing data via the
SDMX-RI components.”
• “The problem is that the latter require an RDBMS in
order to fetch the disseminated data.”

8. A SIDRA Aggregate is formed by at least one variable in one or
more reference areas in a period of time.
An aggregate can also have up to 6 classifications.
Variable
ClassificationTime Aggregate
Reference
Area

9. Income R$
(Fact)
Income by
gender, city, education and economic activity
Multidimensional Model: (Star Schema)
City
(Dimension 1)
Gender
(Dimension 2)
Education
(Dimension 3)
Economic
Activity
(Dimension 4)

10. Alternatives:
Given the current infrastructure of the data dissemination
systems, our first choice is to use the SIDRA system to
expose the disseminated data. SIDRA already has many
indicators available. We just need to identify them.
In order to expose the data we have: (Logical Steps)
a) To map DSD Dimensions to SIDRA expressions
b) To verify if there is a SIDRA table with the requested
information.
c) To build a query.
d) To process the results.
e) To format the response in SDMX style.

11. Example: STLABOUR/LFACTTFE
Building the
mappingSIDRA
DSD
Files
Active population, Aged 15 and over, Females
• Which combinations of variables and
classifications can generate data for women
with age 15 and over?

12. SIDRA SYSTEM

13. SIDRA SYSTEM

14. SIDRA SYSTEM

15. The Result of the Mapping
The variable V4007 links
“população economicamente ativa”
to “pessoas com 15 anos ou mais”
LFACTTFE = T/4023/V/4007/C2/92957
Sex Female
SIDRA SYSTEM

16. Second Alternative:
Buiding the mapping from BME microdata database
BME
DSD
Files

17. BME SYSTEM

18. BME SYSTEM

19. SIDRA SYSTEM
Saving the query definition
Defining the filters

20. Answering a Request
BRA.LFAC.ST.M ?

21. Answering a Request
BRA.LFAC.ST.M ?

22. Answering a Request
Table 2041 = T/2041
BRA
N110/1
LFAC ST
V/891
ou
V/1000891
ou
V/894
V/891
LFAC
C58/0
Query SIDRA:
T/2041/V/891/C58/0/N110/1/P/ALL

23. Answering a Request
1)Data are read from SIDRA tables or from
BME tables
2)Formatting the response: The results have to
be mapped before returning
Units and time period have to be translated
into atributes and dimensions of the DSD.

24. Final Remarks:
DSD
Mapping
Databases
Map
Query
Engine
Converter
SDMX
File
Request

25. Final Remarks:
1. As short term, the use of SIDRA to expose the disseminated data using SDMX
would require less effort and time to implement.
2. Mapping a DSD subject to a SIDRA or BME expression may not be straightforward:
• It is not always easy to compare the meaning of a subject with the
meaning of a variable or a classification in both systems.
METADATA are important!
3. If there is no way to establish the mapping between a subject and a SIDRA
table, then two situations may occur:
a) The subject is not investigated by IBGE;
b) The subject is investigated but IBGE uses a set of different codes.
In this case, a solution may be:
To load microdata into BME with the desired codes;
To generate the result from BME database.
4. Besides the mapping approach, IBGE can also consider as an alternative to
generate SDMX files, the use of BME and SIDRA to produce a set of own DSDs.

26. Thank You!