Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

JDemetra+ Java Tool for Seasonal Adjustment

1,465 views

Published on

JDemetra+ is a tool for seasonal adjustment (SA) developed by the National Bank of Belgium (NBB) in cooperation with the Deutsche Bundesbank and Eurostat in accordance with the Guidelines of the European Statistical System (ESS). User support, training and methodological development is provided by the devoted Centre of Excellence on Seasonal Adjustment coordinated by INSEE, the French National Statistical Office.
JDemetra+ has been officially recommended, since February 2015, to the members of the ESS and the European System of Central Banks as software for seasonal and calendar adjustment of official statistics.

Published in: Data & Analytics
  • Be the first to comment

JDemetra+ Java Tool for Seasonal Adjustment

  1. 1. 1 JDemetra+ Java tool for Seasonal Adjustment Dominique Ladiray INSEE, France dominique.ladiray@insee.fr Dario Buono Eurostat, European Commission Dario.Buono@ec.europa.eu @darbuo Shared Tools for Computing with Data in Official Statistics STS 043 Monday 17 July, Room A 1.17 14:00-15:40
  2. 2. Outline • Time Series and Seasonality • What is JDEMETRA+ • Some History • Software Architecture • Seasonal adjustment methods • Some examples • The Seasonal Adjustment Centre of Excellence • Users Support
  3. 3. What is a Time Series? A Time Series is a sequence of measures of a given phenomenon taken at regular time intervals such as hourly, daily, weekly, monthly, quarterly, annually, or every so many years ..with at least 3 usual components • Trend/Cycle: the long term evolution of the series • Seasonal pattern: regular fluctuations observed during the year • Irregular: residual and random fluctuations
  4. 4. Cause of Seasonality Seasonality and Climate: variations of the weather/climate (seasons!) Seasonality and Institutions: social habits or to the administrative rules Indirect Seasonality: Seasonality that affects other sectors Why Seasonal Adjustment? To improve comparability: Over time & Across space Business cycle analysis Reduce noise to facilitate economic reading
  5. 5. It is about smoothing.. 5
  6. 6. Some Maths : decomposition models Usual Additive and Multiplicative Models More components: Outliers, Calendar Effects
  7. 7. What is JDEMETRA+? JDEMETRA+ is an Time Series Econometric tool for Seasonal Adjustment • developed by National Bank of Belgium and Bundesbank • supported by EUROSTAT and the European Central Bank Trend identification Outliers treatment Estimation of missing values Calendar Adjustment ARIMA modelling Benchmarking JDemetra+ provides a Java implementations of TRAMO-SEATS and of X12-ARIMA. It is based on the NetBeans platform, is developed under the EUPL license. 7
  8. 8. Some Historical Milestones 8 2002 Demetra • Program to compare X-12-ARIMA and TRAMO/SEATS (1997/98). • Integration of original software in a user-friendly application. • Lack of sufficient product development and handling of errors as a result of a loss of technical knowledge about software. 2010 Demetra+ • Developed in cooperation between Eurostat and the National Bank of Belgium. • Enables the implementation of the ESS Guidelines on SA. • Provides graphical interface and common input/output diagnostics for TRAMO/SEATS and X-12- ARIMA. • Includes complex technical solutions. Uses .NET technology and can be used only under Windows. 2015 JDemetra+ •Fortran codes re-written in JAVA. •Open source, platform independent. •Extensible graphical interface, based on the NetBeans platform (plugins). •Developed by the National Bank of Belgium, supported by the Deutsche Bundesbank for the X-11 part.
  9. 9. Anno Nobili 2015 ESS Guidelines on Seasonal Adjustment Introduced in 2009 and revised in 2015 http://ec.europa.eu/eurostat/documents/ 3859598/6830795/KS-GQ-15-001-EN- N.pdf List of options: • A (Recommended) • B (Acceptable) • C ( to be avoided) 9
  10. 10. Official Release of JD+ 2.0.0 Since the 2 of February 2015 JD+ is the official software to be used for Seasonal Adjustment within the European Statistical System for data to be used for Official Statistics Official joint ECB/Eurostat Methodological Note published at: http://www.cros- portal.eu/content/official-release- jdemetra-software-be-used-seasonal- adjustment 10
  11. 11. Software layout A rich graphical application (end-users) dedicated to Seasonal Adjustment 11
  12. 12. Software layout But in fact an advanced Java toolkit for time series processing (End users, production, IT- teams, researchers) 12
  13. 13. JDemetra+ characteristics Flexibility • Encompasses the leading SA algorithms and can evolve independently Versatility • Can be used in a rich graphical interface and/or be integrated in other. Reusability of modules the other circumstances: • Plug-in for temporal disaggregation • Outliers detection, estimation of missing values, Arima forecasts Extensibility • Additional plug-ins and modules do not change the core engines. • Efficient process of large datasets through: • JWSAcruncher, command line application that allows calling JDemetra+ from other applications; Web services and direct call to Java libraries. Open source 13
  14. 14. Architecture 14 In house developments JTsToolkit Core algorithms Peripheral modules External packages JDemetra+ plug-insNetBeans Third party plug-ins Jdemetra-core Jdemetra-app Cruncher
  15. 15. Seasonal Adjustment Methods SA methods Generic modules: •Analysis •Seasonality tests •Revision analysis •Sliding spans •I/O (common xml schema) •Graphical components •Charts •SI ratios... Tramo-Seats, X12-Arima... Model-based decomposition (canonical decomposition, structural models...) Signal extraction tools: •Estimation •Analysis •Graphical components Specific modules (X11...) Other filters (X11...) REGARIMA pre-processing REGARIMA modules: •Common model •Estimation tools •Automatic modelling routines •Analysis tools (residuals, forecasts...) •Graphical components
  16. 16. Input output
  17. 17. Graphs and diagnostics
  18. 18. Distributions
  19. 19. Trend/Seasonal Components with Forecasts
  20. 20. Forecast of original series
  21. 21. Plug-ins for Temporal Disaggregation
  22. 22. The Seasonal Adjustment Centre of Excellence • A “joint venture” between Eurostat, NSIs and CBs • SACE: Belgium, France, Italy, Latvia, Portugal, UK • Partners: Eurostat, BBk, OECD, ECB, IMF • SAUG: Partners + Denmark, Finland, Hungary, Lithuania, Luxembourg, Romania, Slovakia, Slovenia, Spain • 3-year contract: 03/10/2016 – 02/10/2019 22
  23. 23. On the Agenda • Knowledge sharing and Dissemination • Support to SA practitioners • User Group, Helpdesk, Documentation • Pre- release validation testing (JD+ 2.2.0 in 2017) • Plug-ins • Benchmarking, Quality, Analysis of revisions, Weekly and Daily data • Training, coaching and consultancy. 23
  24. 24. Who is using JD+? A difficult question: Downloads are anonymous Helpdesk Latvia, Ireland, Czech Republic, Fyrom, Finland, Germany, Greece, Hungary, Portugal, Spain, France, Italy, Malta, Serbia, Luxembourg, Romania, Cyprus, Austria, Denmark, Iceland, Netherlands, Norway, Slovenia, Sweden, Switzerland, Turkey But many other users: ECB, Eurostat, IMF, Algeria, Cameroon, Maroc, Senegal, Tunisia, UEMOA etc. 24
  25. 25. 25 User Support and Documentation The latest JDemetra+ released version can be downloaded at https://github.com/jdemetra/jdemetra-app/releases Java SE 8 or later versions are required. Modules, code and developers documentation and GitHub • https://github.com/jdemetra for the official modules • https://github.com/nbbrd for NBB resources All user documentation (JDemetra+ Quick Start, JDemetra+ User Guide, JDemetra+ Reference Manual) can be found here. In addition, beyond seasonal adjustment, the following prototype plug-ins are available for: • Temporal Disaggregation and Benchmarking • Quality/ Validation reporting • Revision Analysis • Nowcasting • Using JDEMETRA+ with R
  26. 26. JDEMETRA+ Helpdesk and trainings More info about the SACE http://ec.europa.eu/eurostat/cros/content/seasonal-adjustment- centre-excellence_en For your Helpdesk queries http://ec.europa.eu/eurostat/cros/content/ess-seasonal-adjustment- helpdesk_en For training opportunities visit http://ec.europa.eu/eurostat/web/ess/about-us/estp
  27. 27. Why using JDEMETRA+? 1. Implementation of ESS Guidelines on SA 2. Open source (JAVA) 3. Documented 4. Maintenance ensured (GITHUB) 5. Both x-12 and TRAMO/Seats 6. User Friendly 7. Users/Developers community (you will never walk alone!) 8. Helpdesk support 9. Trainings available 10. Free
  28. 28. …by the why who was Demetra? In Greek mythology it was believed that Demetra was the goddess of corn, grain, harvest, agriculture, and of fertility in general. She made the crops grow each year so was intimately associated with the seasons. Her daughter Persephone was abducted by Hades to be his wife in the underworld. In her anger at her daughter's loss, Demetra caused plants to die and the land to become desolate. Zeus, alarmed for the arid earth, sought for Persephone's return. However, because she had eaten while in the underworld, Hades had a claim on her. Therefore, it was decreed that Persephone would spend four months each year in the underworld. During these months Demetra would grieve for her daughter's absence, withdrawing her gifts from the world, creating winter. Her return brought the spring.
  29. 29. 29 Dominique Ladiray INSEE, France dominique.ladiray@insee.fr Thank you for your attention Dario Buono Eurostat, European Commission Dario.Buono@ec.europa.eu @darbuo
  30. 30. Some more info about the architecture and the State Space Framework 30
  31. 31. Algorithmic libraries (jtstoolkit) 31 Matrix computation Basic data handling Complex, polynomials Linear filters Function optimization Time series, calendars, regression variables... Basic statistics Utilities... Basic econometrics Arima, Ucarima VAR, Dynamic factor model Seats X11 State space framework Arima modelling RegArima Tramo Seasonal adjustment Structural models... Benchmarking, temporal disaggregation
  32. 32. REGARIMA modelling • Common definitions for Calendar variables, outliers, intervention variables, user variables... • Algorithms for likelihood estimation • Kalman filter (Tramo-like), • Ansley algorithm (Cholesky on banded matrix) • (modified) Ljung-Box algorithm (X12-like) • Equivalent results, different performances • JD+ uses Kalman filter • Up to 4 x faster than Ljung-Box • Ansley in specific cases (outliers detection) • Optimization procedure • Levenberg-Marquardt. Tramo-Seats, X12 and JD+ use slightly different variants. Model building (Reg. variables) Estimation of the model (likelihood, residuals) Estimation of the parameters (by ML)
  33. 33. Automatic model identification • Independent blocks (dynamically modifiable) • Specific implementation for Tramo-Seats, X12 Example: X12 modelling with outliers detection from Tramo Log/level Pre-test (seasonality...) Calendar effects... Outliers detection Final estimation Arima (diff. / Arma) Models comparison Model validation
  34. 34. Algorithms for signal extraction in JD+ • Wiener-Kolmogorov filters • Burman's algorithm, Maravall's analysis framework (Seats) • Kalman smoother • Koopman's initialization procedure (disturbance or ordinary smoother) • Matrix computation • McElroy 's formulae • Can be applied to any (valid) UCARIMA model • Results • Estimates: identical • Standard deviations: WK approach yields negligible differences (exception: quasi- unit roots in MA polynomial → large differences) 34
  35. 35. State space framework • Key solution for: • REGARIMA estimation • Signal extraction (Kalman smoother) • Alternative time series modelling (for SA or not) • Structural models... • Benchmarking • Cholette (including multi-variate extension) • Temporal disaggregation • Chow-Lin, Fernandez... • Multi-variate models • VAR, dynamic factor models, SUTSE… • JD+ provides an advanced implementation of SSF
  36. 36. State space framework (II) Models Atomic models: • Generic (time invariant or not) • Ar(i)ma • Ucarima • Basic structural • White noise • Random walk • ... Derived models: • Composite • Regression variables • Aggregation constraints • ... Algorithms Filtering: • Ordinary filter • Fast filter (Chandrasekhar) • Array filter (Kailath...) Diffuse initialization: • Koopman • Square root • Ad hoc Smoothing: • Ordinary • Disturbance • Fixed point Others: • Univariate handling of multi-variate models • Augmented Kalman filter (for reg. model) • Extended Kalman filter (for non linear models) • ... Likelihood evaluation: Prediction error decomposition

×