KAPSARC: Energy Research Think-Tank, Independent Non Profit

1. Dec 9th 2019
KAPSARC Modelers DataHub
Pavithra Kumar Shetty
Lead Web Developer, Energy information Management
pavithra.shetty@kapsarc.org

2. 2
King Abdullah Petroleum Studies and Research Center (KAPSARC) conducts
independent research and develops insights in collaboration with leading
international research centers, public policy organizations, and industrial and
government institutions.
The center is focused on finding solutions for the most effective and
productive use of energy to enable economic and social progress in the
region and across the globe.
▪ Independent, non-profit, research institution
▪ Focus on energy economics, policy, technology, and the environment
▪ Global and local, has financial and administrative independence
▪ Located in Riyadh, Saudi Arabia
KAPSARC: Energy Research Think-Tank, Independent Non Profit
KAPSARC’s Mission
Advance the understanding of energy economics and act as a catalyst for dialogue, charting a path to better
welfare for societies, locally and globally

3. 3
Energy Systems
and
Macroeconomics
Policy and Decision
Sciences
Productivity,
Energy Demand,
and Efficiency
Mobility and
Urban
Infrastructure
Energy
Transitions
Markets and
Industrial
Development
Energy
Information and
Web Apps
❑Climate Change Policies and Governance
❑Future of Transport and Fuel Demand
❑Future of Global Oil and Natural Gas Markets
❑Regional Energy Markets
❑Electricity Sector Transitions
❑Model, Data and Tools
❑Evaluation of Public Investment Projects
❑Productivity and Economic Diversification
❑Energy and Economic Vulnerability
KAPSARC Research Initiatives
Climate and
Environment

4. 4
Agenda
Showcase features of DataHub
Get feedback on future features
01
02
Questions03

5. 5
API and UI access to your data.
Datahub - Motivation
Centralize data from disparate systems
Track the changes to the data
Quick onboarding of data
Open Source Data Versioning and Collaboration tool – git for databases

6. 6
Data versioning and collaboration toolDataHub
Input Data Model

7. 7
Data versioning and collaboration toolDataHub
Input Data

8. 8
Data versioning and collaboration toolDataHub
Input Data

9. 9
Data versioning and collaboration toolDataHub
Input Data

10. 10
Data versioning and collaboration toolDataHub
Input Data

11. 11
Data versioning and collaboration toolDataHub
Input Data

12. 12
Data versioning and collaboration toolDataHub
Input Data

13. 13
Data versioning and collaboration toolDataHub
Input Data

14. 14
Data versioning and collaboration toolDataHub
Input Data

15. 15
Data versioning and collaboration toolDataHub
Input Data

16. 16
Data versioning and collaboration toolDataHub
Input Data Model

17. 17
Data versioning and collaboration toolDataHub
Input Data Model
Run different model versions
against data versions.

18. 18
Datahub
Datahub UI
Automated Scripts
Surveys
On Board Data
Datahub Flow
Data Consumption
Applications
Analytics
Datahub UI
Readonly
Models (KEM , VEDA)
A
P
I
A
P
I

19. 19
ETSAP– Datahub Integration (proposed)
Datahub
VEDA

20. 20
Features Onboard
data easily
01
Edit / Manipulate
the data
02
Keep track of
different versions
of the data
03
API ready
04
Branching
(Ability to copy a data from any
point in time)
05
Model Templates
(GAMS , KTAF and more) 06
Fine grained
permissions 07
Analytics
(Metabase)
08

21. 21
UI Redesign
Future
Compare and Merge
Data citing - Easily embed data / charts in any site
Data Recipes with R/ Python/etc

22. 22
Custom code which can be used to change the default behavior of datahub. Recipes
can be written in any modern programming languages like java , Python , R.
Data Recipes
Data Wrangling Custom Data importing
Data validation Custom Export
Build your own model
within datahub
Can be reused across
users

23. 23
Models in DataHub
KEM
(Integration of GAMS)
iTEM
EPS KSA
KTAF
(Maritime Visualization Data)
10 more models
(use datahub to just host data)
GDX
KAPSARC Datahub
JSON
API
Modeler Machine
GDX Importer
Python Code embedded
in GAMS code
GDX Exporter
Python Code embedded
in GAMS code

24. 24
Models in DataHub
KEM
(Integration of GAMS)
iTEM
EPS KSA
KTAF
(Maritime Visualization Data)
10 more models
(use datahub to just host data)

25. 25
Models in DataHub
KEM
(Integration of GAMS)
iTEM
EPS KSA
KTAF
(Maritime Visualization Data)
10 more models
(use datahub to just host data)

26. 26
Models in DataHub
KEM
(Integration of GAMS)
iTEM
EPS KSA
KTAF
(Maritime Visualization Data)
10 more models
(use datahub to just host data)

27. 27
Models in DataHub
KEM
(Integration of GAMS)
iTEM
EPS KSA
KTAF
(Maritime Visualization Data)
10 more models
(use datahub to just host data)

28. 28
http://apps.kapsarc.org/datahub/
Demo