Modelers Datahub: Tool to manage model data
•
1 like•190 views
KAPSARC is an independent research center focused on energy economics, policy, technology and the environment. It conducts research collaboratively with international partners to find solutions for effective and productive use of energy to enable economic and social progress globally. The document discusses KAPSARC's DataHub, which is a tool for centralizing data from different systems, tracking changes, and providing API and UI access for researchers and their models.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Modelers Datahub: Tool to manage model data
Similar to Modelers Datahub: Tool to manage model data (20)
More from IEA-ETSAP
More from IEA-ETSAP (20)
Recently uploaded
Recently uploaded (18)
Modelers Datahub: Tool to manage model data
- 1. Dec 9th 2019 KAPSARC Modelers DataHub Pavithra Kumar Shetty Lead Web Developer, Energy information Management pavithra.shetty@kapsarc.org
- 2. 2 King Abdullah Petroleum Studies and Research Center (KAPSARC) conducts independent research and develops insights in collaboration with leading international research centers, public policy organizations, and industrial and government institutions. The center is focused on finding solutions for the most effective and productive use of energy to enable economic and social progress in the region and across the globe. ▪ Independent, non-profit, research institution ▪ Focus on energy economics, policy, technology, and the environment ▪ Global and local, has financial and administrative independence ▪ Located in Riyadh, Saudi Arabia KAPSARC: Energy Research Think-Tank, Independent Non Profit KAPSARC’s Mission Advance the understanding of energy economics and act as a catalyst for dialogue, charting a path to better welfare for societies, locally and globally
- 3. 3 Energy Systems and Macroeconomics Policy and Decision Sciences Productivity, Energy Demand, and Efficiency Mobility and Urban Infrastructure Energy Transitions Markets and Industrial Development Energy Information and Web Apps ❑Climate Change Policies and Governance ❑Future of Transport and Fuel Demand ❑Future of Global Oil and Natural Gas Markets ❑Regional Energy Markets ❑Electricity Sector Transitions ❑Model, Data and Tools ❑Evaluation of Public Investment Projects ❑Productivity and Economic Diversification ❑Energy and Economic Vulnerability KAPSARC Research Initiatives Climate and Environment
- 4. 4 Agenda Showcase features of DataHub Get feedback on future features 01 02 Questions03
- 5. 5 API and UI access to your data. Datahub - Motivation Centralize data from disparate systems Track the changes to the data Quick onboarding of data Open Source Data Versioning and Collaboration tool – git for databases
- 6. 6 Data versioning and collaboration toolDataHub Input Data Model
- 7. 7 Data versioning and collaboration toolDataHub Input Data
- 8. 8 Data versioning and collaboration toolDataHub Input Data
- 9. 9 Data versioning and collaboration toolDataHub Input Data
- 10. 10 Data versioning and collaboration toolDataHub Input Data
- 11. 11 Data versioning and collaboration toolDataHub Input Data
- 12. 12 Data versioning and collaboration toolDataHub Input Data
- 13. 13 Data versioning and collaboration toolDataHub Input Data
- 14. 14 Data versioning and collaboration toolDataHub Input Data
- 15. 15 Data versioning and collaboration toolDataHub Input Data
- 16. 16 Data versioning and collaboration toolDataHub Input Data Model
- 17. 17 Data versioning and collaboration toolDataHub Input Data Model Run different model versions against data versions.
- 18. 18 Datahub Datahub UI Automated Scripts Surveys On Board Data Datahub Flow Data Consumption Applications Analytics Datahub UI Readonly Models (KEM , VEDA) A P I A P I
- 19. 19 ETSAP– Datahub Integration (proposed) Datahub VEDA
- 20. 20 Features Onboard data easily 01 Edit / Manipulate the data 02 Keep track of different versions of the data 03 API ready 04 Branching (Ability to copy a data from any point in time) 05 Model Templates (GAMS , KTAF and more) 06 Fine grained permissions 07 Analytics (Metabase) 08
- 21. 21 UI Redesign Future Compare and Merge Data citing - Easily embed data / charts in any site Data Recipes with R/ Python/etc
- 22. 22 Custom code which can be used to change the default behavior of datahub. Recipes can be written in any modern programming languages like java , Python , R. Data Recipes Data Wrangling Custom Data importing Data validation Custom Export Build your own model within datahub Can be reused across users
- 23. 23 Models in DataHub KEM (Integration of GAMS) iTEM EPS KSA KTAF (Maritime Visualization Data) 10 more models (use datahub to just host data) GDX KAPSARC Datahub JSON API Modeler Machine GDX Importer Python Code embedded in GAMS code GDX Exporter Python Code embedded in GAMS code
- 24. 24 Models in DataHub KEM (Integration of GAMS) iTEM EPS KSA KTAF (Maritime Visualization Data) 10 more models (use datahub to just host data)
- 25. 25 Models in DataHub KEM (Integration of GAMS) iTEM EPS KSA KTAF (Maritime Visualization Data) 10 more models (use datahub to just host data)
- 26. 26 Models in DataHub KEM (Integration of GAMS) iTEM EPS KSA KTAF (Maritime Visualization Data) 10 more models (use datahub to just host data)
- 27. 27 Models in DataHub KEM (Integration of GAMS) iTEM EPS KSA KTAF (Maritime Visualization Data) 10 more models (use datahub to just host data)