SlideShare a Scribd company logo
WP3 OpenGovIntelligence LOSD
Tools
NUIG
Mohamed Adel Rezk
Arkadiusz Stasiewicz
Dr. Adegboyega Ojo
Agenda
• Achieved / Ongoing Tools
• Assisted RDF Data Cube Schema Mapping
• Generic RDF Data Cube Builder
• RDF Data Cube Explorer
• RDF Data Cube geo-data supported Dashboard
• Planned Tools
• Real-Time LOSD Publishing Pipeline
• Marine Institute and Lithuanian Pilots’ Dashboards updates
• Collaboration Space and A Feedback Mechanism
2
LOSD Publishing pipeline [NUIG Approach]
3
Assisted RDF
Data Cube
Schema
Mapping
Generic RDF
Data Cube
Builder
RDF Data
Cube
Explorer
LOSD
Dashboards
[ET - MI]
LOSD Publishing pipeline [flow chart]
4
Schema Mapping Stage
Pilot / user dataset to RDF Data Cube Vocabulary Mapping
[Spreadsheet]
RDF Data Cube schema as rdf refine configuration file [json]
[Open refine] and [RDF refine ext.]
Pilot / user mapped RDF
Data Cube schema
[rdf/xml or ttl]
Pilot / user dataset [csv]
RDF Data Cube Builder Application
[Web App – Desktop – CMD - Web
service]
Pilot / user RDF Data Cube
[rdf/xml or ttl]
Data Cube Building Stage Data Cube Exploration Stage
JSON QB REST API implementation [java web service]
RDF Data Store [virtuso/f]
Data Cube
Explorer
[django -
wordpress]
ET dashboard
MI dashboard
Assisted RDF Data Cube Schema Mapping
• This tool/pipeline is created to ease the mapping process of government
statistical datasets into RDF according to the RDF data cube vocabulary.
• This tool integrates [spreadsheets, open refine, rdf refine, and rdf data cube] to
produce a generic LOSD mapping tool that fits any pilot or use case.
• The following demo shows the Schema Mapping pipeline steps/stages.
5
Assisted RDF Data Cube Schema Mapping
• Step 1 [Mapping Spreadsheet inputs]
6
Assisted RDF Data Cube Schema Mapping
• Step 1 [Mapping Spreadsheet inputs]
7
Assisted RDF Data Cube Schema Mapping
• Step 1 [Mapping Spreadsheet inputs]
8
Assisted RDF Data Cube Schema Mapping
• Step 1 [Mapping Spreadsheet inputs]
9
Assisted RDF Data Cube Schema Mapping
• Step 1 [Mapping Spreadsheet inputs]
10
Assisted RDF Data Cube Schema Mapping
• Step 2 [Loading Mapping Spreadsheet to Open Refine “+RDF refine ext.”]
11
Assisted RDF Data Cube Schema Mapping
• Step 2 [Loading Mapping Spreadsheet to Open Refine “+RDF refine ext.”]
12
Assisted RDF Data Cube Schema Mapping
• Step 2 [Loading Mapping Spreadsheet to Open Refine “+RDF refine ext.”]
13
Assisted RDF Data Cube Schema Mapping
• Step 2 [Loading Mapping Spreadsheet to Open Refine “+RDF refine ext.”]
14
Assisted RDF Data Cube Schema Mapping
• Step 3 [Applying predefined RDF Data Cube Schema to the input Spreadsheet]
15
Assisted RDF Data Cube Schema Mapping
16
• Step 3 [Applying predefined RDF Data Cube Schema to the input Spreadsheet]
Assisted RDF Data Cube Schema Mapping
• Step 3 [Applying predefined RDF Data Cube Schema to the input Spreadsheet]
17
Assisted RDF Data Cube Schema Mapping
• Step 3 [Applying predefined RDF Data Cube Schema to the input Spreadsheet]
18
Assisted RDF Data Cube Schema Mapping
• Step 3 [Applying predefined RDF Data Cube Schema to the input Spreadsheet]
19
Assisted RDF Data Cube Schema Mapping
20
• Step 3 [Applying predefined RDF Data Cube Schema to the input Spreadsheet]
Assisted RDF Data Cube Schema Mapping
• Step 3 [Applying predefined RDF Data Cube Schema to the input Spreadsheet]
21
Assisted RDF Data Cube Schema Mapping
• Step 3 [Extracting Schema mapping of the dataset in rdf/xml or ttl formats]
22
Assisted RDF Data Cube Schema Mapping
• Step 3 [Extracting Schema mapping of the dataset in rdf/xml or ttl formats]
23
Assisted RDF Data Cube Schema Mapping
• Step 3 [Extracting Schema mapping of the dataset in rdf/xml or ttl formats]
24
Assisted RDF Data Cube Schema Mapping
• Step 3 [Extracting Schema mapping of the dataset in rdf/xml or ttl formats]
25
Assisted RDF Data Cube Schema Mapping
• Git Repo: https://github.com/mohadelrezk/open-gov-RDFDataCube-schema-
creator
26
LOSD Publishing pipeline [flow chart]
27
Schema Mapping Stage
Pilot / user dataset to RDF Data Cube Vocabulary Mapping
[Spreadsheet]
RDF Data Cube schema as rdf refine configuration file [json]
[Open refine] and [RDF refine ext.]
Pilot / user mapped RDF
Data Cube schema
[rdf/xml or ttl]
Pilot / user dataset [csv]
RDF Data Cube Builder Application
[Web App – Desktop – CMD - Web
service]
Pilot / user RDF Data Cube
[rdf/xml or ttl]
Data Cube Building Stage Data Cube Exploration Stage
JSON QB REST API implementation [java web service]
RDF Data Store [virtuso/f]
Data Cube
Explorer
[django -
wordpress]
ET dashboard
MI dashboard
Generic RDF Data Cube Builder
• RDF Data Cube Builder is used to transform government statistical data [csv]
into RDF Data Cubes [rdf] based on the predefined mapping produced in the
previous stage.
• Cube Builder Application implemented in deferent interfaces Web App.,
Desktop App., Command Line and Web Service.
• The following demo shows the Cube Builder usages.
28
Generic RDF Data Cube Builder
• Web Application Interface [using the Web Service]
29
Generic RDF Data Cube Builder
• Web Application Interface [using the Web Service]
30
Generic RDF Data Cube Builder
• Web Application Interface [using the Web Service]
31
Generic RDF Data Cube Builder
• Web Application Interface [using the Web Service]
32
Generic RDF Data Cube Builder
• Web Application Interface [using the Web Service]
33
Generic RDF Data Cube Builder
• Web Application Interface [using the Web Service]
34
Generic RDF Data Cube Builder
• Web Application Interface [using the Web Service]
35
Generic RDF Data Cube Builder
• Web Application Interface [using the Web Service]
36
Generic RDF Data Cube Builder
• Web Application Interface [using the Web Service]
37
Generic RDF Data Cube Builder
• Web Application Interface [using the Web Service]
38
Generic RDF Data Cube Builder
• Web Application Interface [using the Web Service]
39
Generic RDF Data Cube Builder
• CMD Interface
40
Generic RDF Data Cube Builder
• Desktop Interface
41
Generic RDF Data Cube Builder
• Desktop Interface
42
Generic RDF Data Cube Builder
• Desktop Interface
43
Generic RDF Data Cube Builder
• Git repo:
https://github.com/OpenGovIntelligence/data-cube-builder
44
LOSD Publishing pipeline [flow chart]
45
Schema Mapping Stage
Pilot / user dataset to RDF Data Cube Vocabulary Mapping
[Spreadsheet]
RDF Data Cube schema as rdf refine configuration file [json]
[Open refine] and [RDF refine ext.]
Pilot / user mapped RDF
Data Cube schema
[rdf/xml or ttl]
Pilot / user dataset [csv]
RDF Data Cube Builder Application
[Web App – Desktop – CMD - Web
service]
Pilot / user RDF Data Cube
[rdf/xml or ttl]
Data Cube Building Stage Data Cube Exploration Stage
JSON QB REST API implementation [java web service]
RDF Data Store [virtuso/f]
Data Cube
Explorer
[django -
wordpress]
ET dashboard
MI dashboard
RDF Data Cube Explorer
• RDF Data Cube Explorer is a webserver that integrates [json QB API, RDF
Data Stores, Pivot table, Maps etc .. ] to produce explorations and
visualizations dashboards for LOSD previously created and loaded into RDF
Data Stores.
• Those Dash Boards are designed and implemented to meet the pilot use cases
requirements.
• The following demo shows the Cube Explorer abilities.
46
RDF Data Cube Explorer [Maritime Search and Rescue]
• Considering the sad incident in March 2017, the Cube Explorer would assist the
search and rescue efforts by pointing out possible search points for the remains
using wind and wave direction readings.
RDF Data Cube Explorer [Maritime Search and Rescue]
• Average Wind Direction in (+) or (–) 24 hours of the incident date (assuming the
incident happened on 12/09/2016) grouped by longitude and latitude points
using data sourced from [Irish Weather Buoy Network] data source.
• As shown in the next slide using our cube explorer rescue staff can choose the
dataset contains the relevant data and choose the visualization and aggregation
types, then they can filter out the data to keep only relevant dimensions (e.g.
longitude ) and measures (e.g. day) and the relevant values (e.g. 11,12, and
13/09/2016).
• From the visualizations the rescue staff can see that, during the incident.
RDF Data Cube Explorer [Maritime Search and Rescue]
Maritime Search and Rescue, avg. wind direction
RDF Data Cube Explorer [Maritime Search and Rescue]
Maritime Search and Rescue, mean wave direction
RDF Data Cube Explorer [Maritime Search and Rescue]
• Using the visualizations the rescue staff can see that, during the relevant time
period and geo-coordinates of the incident avg. wind direction was between (181.8
and 355 degrees) and mean wave direction was between (216.75 and 273
degrees). So that they will focus their efforts accordingly.
RDF Data Cube Explorer [Maritime Search and Rescue]
RDF Data Cube Explorer [Marine Renewable Energy]
• The installation process of new generators (wind or wave based) starts
with locating the most fitting spot that will guarantee the best operating
conditions for the maximum productivity and safety.
RDF Data Cube Explorer [Marine Renewable Energy]
• Using avg. wind speed around the year [for fixed location generators] or in
certain periods [for movable generators] it can be decided where and when to
install the generators.
RDF Data Cube Explorer [Marine Renewable Energy]
• Marine Renewable Energy, avg. wind speed {year based}
RDF Data Cube Explorer [Marine Renewable Energy]
Marine Renewable Energy, avg. wind speed {monthly based}
RDF Data Cube Explorer [Marine Renewable Energy]
• Using the visualizations we can see that,
• (1) Buoy doesn't get data for wind speed at the geo point (5.42, 58.48), the wind speed
sensor could be in need for replacement or maintenance.
• (2) Geo points (9.99, 54.99) and (10.54, 51.21) have the highest avg. wind speed around
the year.
• (3) Goe point (6.70, 51.69) have the highest avg. wind speed in November and geo point
(9.99, 54.99) have the highest avg. wind speed in August.
• So, “regarding other decision inputs” for fixed wind generators marine staff could place it at
the area around geo point (9.99, 54.99) and (10.54, 51.21).
• And for movable wind generators, best spot in August is (9.99, 54.99) and in November is
(6.70, 51.69).
• Same process will fit for the wave generators.
RDF Data Cube Explorer [Marine Renewable Energy]
RDF Data Cube Explorer [Maritime Tourism and Leisure]
• For setting up safe diving and swimming locations, and also avoiding dangerous
locations swimmer and surfers can use wave speed and height, wind speed, gust, sea
temperature and air temperature readings.
RDF Data Cube Explorer [Maritime Tourism and Leisure]
Maritime Tourism and Leisure, avg. sea temperature in August
RDF Data Cube Explorer [Maritime Tourism and Leisure]
Maritime Tourism and Leisure, avg. air temperature in August
RDF Data Cube Explorer [Maritime Tourism and Leisure]
RDF Data Cube Explorer [Generic Use Cases of the Cube Explorer Tool]
• Cube Explorer works by pointing it to a RDF Data Cube server, then it will fetch all
available cube data sets and offer the user a verity of RDF data cube visualizations and
filtering options.
• e.g. choosing the Foreign direct investment per capita RDF Data Cube for explorations
as shown below.
RDF Data Cube Explorer [Generic Use Cases of the Cube Explorer Tool]
• Cube Explorer offers a verity of visualization and exploration options e.g. Bar
charts, Heat Maps, Line chart, Table Bar Chart etc. .
RDF Data Cube Explorer [Generic Use Cases of the Cube Explorer Tool]
• e.g. Highlighting the Foreign direct investment per capita for the country of Lithuania.
• Visualizing the Data Cube using country wide, county, and municipality levels with
temporal filtering would surely assist decision makers to achieve more with this
information in hand.
RDF Data Cube Explorer [Generic Use Cases of the Cube Explorer Tool]
RDF Data Cube Explorer [Generic Use Cases of the Cube Explorer Tool]
RDF Data Cube Explorer [Generic Use Cases of the Cube Explorer Tool]
RDF Data Cube Explorer [Generic Use Cases of the Cube Explorer Tool]
• e.g. Highlighting the Average Monthly Earnings for the country of Lithuania.
• Visualizing the Data Cube using country wide, county, and municipality levels
with temporal filtering would surely assist decision makers to achieve more
with this information in hand.
RDF Data Cube Explorer [Generic Use Cases of the Cube Explorer Tool]
RDF Data Cube Explorer [Generic Use Cases of the Cube Explorer Tool]
RDF Data Cube Explorer [Generic Use Cases of the Cube Explorer Tool]
• e.g. Highlighting the Resident Population Demography for the country of
Lithuania.
• Visualizing the Data Cube using country wide, county, and municipality levels
with temporal filtering would surely assist decision makers to achieve more with
this information in hand.
RDF Data Cube Explorer [Generic Use Cases of the Cube Explorer Tool]
RDF Data Cube Explorer [Generic Use Cases of the Cube Explorer Tool]
RDF Data Cube Explorer [Generic Use Cases of the Cube Explorer Tool]
RDF Data Cube Explorer [Generic Use Cases of the Cube Explorer Tool]
• Git repo:
https://github.com/OpenGovIntelligence/data-cube-explorer
RDF Data Cube geo-data supported Dashboard
• This dashboard mainly designed and implemented to meet the Lithuanian pilot
use case requirements.
• The following demo shows the Data Cube geo-data supported Dashboard look.
77
RDF Data Cube geo-data supported Dashboard
• Lithuanian Data Cubes Geo based Visualization screen samples.
78
RDF Data Cube geo-data supported Dashboard
• Lithuanian Data Cubes Geo based Visualization screen samples.
79
RDF Data Cube geo-data supported Dashboard
• Lithuanian Data Cubes Geo based Visualization screen samples.
80
Planned
• Real-Time LOSD Publishing Pipeline
• Marine Institute / Lithuanian Pilot Dashboards updates
• Collaboration Space / Feed Back Mechanism
81
Real-Time LOSD Publishing Pipeline
• To target real time marine data use case [ search and rescue], we are planning
to design and implement a real time version of the LOSD publishing pipeline.
• Following a drafted vision of the above pipeline.
82
MI
servers
APIs
OGI RDF store
(Data of Curre
Week only){Weekly}
API calls
Cube
Annotation
Temporary
CSV DB
Cube
Builder
Push to
RDF
Store
Cursor
Tracking
Drop
outdated
data
Real-Time LOSD Publishing Pipeline
Questions
84

More Related Content

What's hot

Pilot Project for HDF5 Metadata Structures for SWOT
Pilot Project for HDF5 Metadata Structures for SWOTPilot Project for HDF5 Metadata Structures for SWOT
Pilot Project for HDF5 Metadata Structures for SWOT
The HDF-EOS Tools and Information Center
 
Multidimensional Scientific Data in ArcGIS
Multidimensional Scientific Data in ArcGISMultidimensional Scientific Data in ArcGIS
Multidimensional Scientific Data in ArcGIS
The HDF-EOS Tools and Information Center
 
The National Oceanographic Data Center’s NetCDF Templates
The National Oceanographic Data Center’s NetCDF TemplatesThe National Oceanographic Data Center’s NetCDF Templates
The National Oceanographic Data Center’s NetCDF Templates
The HDF-EOS Tools and Information Center
 
Geo Analytics Canada Overview - May 2020
Geo Analytics Canada Overview - May 2020Geo Analytics Canada Overview - May 2020
Geo Analytics Canada Overview - May 2020
GEO Analytics Canada
 
GDAL Enhancement for ESDIS Project
GDAL Enhancement for ESDIS ProjectGDAL Enhancement for ESDIS Project
GDAL Enhancement for ESDIS Project
The HDF-EOS Tools and Information Center
 
Data Are from Mars, Tools Are from Venus
Data Are from Mars, Tools Are from VenusData Are from Mars, Tools Are from Venus
Data Are from Mars, Tools Are from Venus
The HDF-EOS Tools and Information Center
 
HDFCloud Workshop: HDF5 in the Cloud
HDFCloud Workshop: HDF5 in the CloudHDFCloud Workshop: HDF5 in the Cloud
HDFCloud Workshop: HDF5 in the Cloud
The HDF-EOS Tools and Information Center
 
STAC, ZARR, COG, K8S and Data Cubes: The brave new world of satellite EO anal...
STAC, ZARR, COG, K8S and Data Cubes: The brave new world of satellite EO anal...STAC, ZARR, COG, K8S and Data Cubes: The brave new world of satellite EO anal...
STAC, ZARR, COG, K8S and Data Cubes: The brave new world of satellite EO anal...
GEO Analytics Canada
 
Using GDAL In Your GIS Workflow
Using GDAL In Your GIS WorkflowUsing GDAL In Your GIS Workflow
Using GDAL In Your GIS Workflow
Gerry James
 
Improved Methods for Accessing Scientific Data for the Masses
Improved Methods for Accessing Scientific Data for the MassesImproved Methods for Accessing Scientific Data for the Masses
Improved Methods for Accessing Scientific Data for the Masses
The HDF-EOS Tools and Information Center
 
Guided Tour of Pythonian Museum
Guided Tour of Pythonian MuseumGuided Tour of Pythonian Museum
Guided Tour of Pythonian Museum
The HDF-EOS Tools and Information Center
 
CLIM Program: Remote Sensing Workshop, The Earth System Grid Federation as a ...
CLIM Program: Remote Sensing Workshop, The Earth System Grid Federation as a ...CLIM Program: Remote Sensing Workshop, The Earth System Grid Federation as a ...
CLIM Program: Remote Sensing Workshop, The Earth System Grid Federation as a ...
The Statistical and Applied Mathematical Sciences Institute
 

What's hot (20)

HDF Town Hall
HDF Town HallHDF Town Hall
HDF Town Hall
 
Pilot Project for HDF5 Metadata Structures for SWOT
Pilot Project for HDF5 Metadata Structures for SWOTPilot Project for HDF5 Metadata Structures for SWOT
Pilot Project for HDF5 Metadata Structures for SWOT
 
HDF and netCDF Data Support in ArcGIS
HDF and netCDF Data Support in ArcGISHDF and netCDF Data Support in ArcGIS
HDF and netCDF Data Support in ArcGIS
 
Multidimensional Scientific Data in ArcGIS
Multidimensional Scientific Data in ArcGISMultidimensional Scientific Data in ArcGIS
Multidimensional Scientific Data in ArcGIS
 
Earth Science Data and Information System (ESDIS) Project Update
Earth Science Data and Information System (ESDIS) Project UpdateEarth Science Data and Information System (ESDIS) Project Update
Earth Science Data and Information System (ESDIS) Project Update
 
Bridging ICESat and ICESat-2 Standard Data Products
Bridging ICESat and ICESat-2 Standard Data ProductsBridging ICESat and ICESat-2 Standard Data Products
Bridging ICESat and ICESat-2 Standard Data Products
 
The National Oceanographic Data Center’s NetCDF Templates
The National Oceanographic Data Center’s NetCDF TemplatesThe National Oceanographic Data Center’s NetCDF Templates
The National Oceanographic Data Center’s NetCDF Templates
 
Geo Analytics Canada Overview - May 2020
Geo Analytics Canada Overview - May 2020Geo Analytics Canada Overview - May 2020
Geo Analytics Canada Overview - May 2020
 
GDAL Enhancement for ESDIS Project
GDAL Enhancement for ESDIS ProjectGDAL Enhancement for ESDIS Project
GDAL Enhancement for ESDIS Project
 
Data Are from Mars, Tools Are from Venus
Data Are from Mars, Tools Are from VenusData Are from Mars, Tools Are from Venus
Data Are from Mars, Tools Are from Venus
 
HDFCloud Workshop: HDF5 in the Cloud
HDFCloud Workshop: HDF5 in the CloudHDFCloud Workshop: HDF5 in the Cloud
HDFCloud Workshop: HDF5 in the Cloud
 
STAC, ZARR, COG, K8S and Data Cubes: The brave new world of satellite EO anal...
STAC, ZARR, COG, K8S and Data Cubes: The brave new world of satellite EO anal...STAC, ZARR, COG, K8S and Data Cubes: The brave new world of satellite EO anal...
STAC, ZARR, COG, K8S and Data Cubes: The brave new world of satellite EO anal...
 
Using GDAL In Your GIS Workflow
Using GDAL In Your GIS WorkflowUsing GDAL In Your GIS Workflow
Using GDAL In Your GIS Workflow
 
Improved Methods for Accessing Scientific Data for the Masses
Improved Methods for Accessing Scientific Data for the MassesImproved Methods for Accessing Scientific Data for the Masses
Improved Methods for Accessing Scientific Data for the Masses
 
Guided Tour of Pythonian Museum
Guided Tour of Pythonian MuseumGuided Tour of Pythonian Museum
Guided Tour of Pythonian Museum
 
HDF Update
HDF UpdateHDF Update
HDF Update
 
ICESat-2 H5-ES Product Development Strategy
ICESat-2 H5-ES Product Development StrategyICESat-2 H5-ES Product Development Strategy
ICESat-2 H5-ES Product Development Strategy
 
HDF & HDF-EOS Data & Support at NSIDC
HDF & HDF-EOS Data & Support at NSIDCHDF & HDF-EOS Data & Support at NSIDC
HDF & HDF-EOS Data & Support at NSIDC
 
CLIM Program: Remote Sensing Workshop, The Earth System Grid Federation as a ...
CLIM Program: Remote Sensing Workshop, The Earth System Grid Federation as a ...CLIM Program: Remote Sensing Workshop, The Earth System Grid Federation as a ...
CLIM Program: Remote Sensing Workshop, The Earth System Grid Federation as a ...
 
EOS MLS software
EOS MLS softwareEOS MLS software
EOS MLS software
 

Similar to NUIG LOSD tools

Generating Executable Mappings from RDF Data Cube Data Structure Definitions
Generating Executable Mappings from RDF Data Cube Data Structure DefinitionsGenerating Executable Mappings from RDF Data Cube Data Structure Definitions
Generating Executable Mappings from RDF Data Cube Data Structure Definitions
Christophe Debruyne
 
Using the Data Cube vocabulary for Publishing Environmental Linked Data on la...
Using the Data Cube vocabulary for Publishing Environmental Linked Data on la...Using the Data Cube vocabulary for Publishing Environmental Linked Data on la...
Using the Data Cube vocabulary for Publishing Environmental Linked Data on la...
Laurent Lefort
 
Web analytics at scale with Druid at naver.com
Web analytics at scale with Druid at naver.comWeb analytics at scale with Druid at naver.com
Web analytics at scale with Druid at naver.com
Jungsu Heo
 
Ozri 2013 Brisbane, Australia - Geodatabase Efficiencies
Ozri 2013 Brisbane, Australia - Geodatabase EfficienciesOzri 2013 Brisbane, Australia - Geodatabase Efficiencies
Ozri 2013 Brisbane, Australia - Geodatabase Efficiencies
Walter Simonazzi
 
Is there a way that we can build our Azure Data Factory all with parameters b...
Is there a way that we can build our Azure Data Factory all with parameters b...Is there a way that we can build our Azure Data Factory all with parameters b...
Is there a way that we can build our Azure Data Factory all with parameters b...
Erwin de Kreuk
 
Scalable Preservation Workflows
Scalable Preservation WorkflowsScalable Preservation Workflows
Scalable Preservation Workflows
SCAPE Project
 
11. From Hadoop to Spark 1:2
11. From Hadoop to Spark 1:211. From Hadoop to Spark 1:2
11. From Hadoop to Spark 1:2
Fabio Fumarola
 
DSD-INT 2020 BlueEarth Engine - hydroMT - model builder framework
DSD-INT 2020 BlueEarth Engine - hydroMT - model builder frameworkDSD-INT 2020 BlueEarth Engine - hydroMT - model builder framework
DSD-INT 2020 BlueEarth Engine - hydroMT - model builder framework
Deltares
 
Lodstats: The Data Web Census Dataset. Kobe, Japan, 2016
Lodstats: The Data Web Census Dataset. Kobe, Japan, 2016Lodstats: The Data Web Census Dataset. Kobe, Japan, 2016
Lodstats: The Data Web Census Dataset. Kobe, Japan, 2016
Ivan Ermilov
 
Introduction to Google Earth Engine .pptx
Introduction to Google Earth Engine .pptxIntroduction to Google Earth Engine .pptx
Introduction to Google Earth Engine .pptx
Putu Perdana Kusuma Wiguna
 
Geospatial Data Abstraction Library (GDAL) Enhancement for ESDIS (GEE)
Geospatial Data Abstraction Library (GDAL) Enhancement for ESDIS (GEE)Geospatial Data Abstraction Library (GDAL) Enhancement for ESDIS (GEE)
Geospatial Data Abstraction Library (GDAL) Enhancement for ESDIS (GEE)
The HDF-EOS Tools and Information Center
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Edward King SPEDDEXES 2014
Edward King SPEDDEXES 2014Edward King SPEDDEXES 2014
Edward King SPEDDEXES 2014
aceas13tern
 
Towards INSPIRE environmental 5* Open Data
Towards INSPIRE environmental 5* Open Data Towards INSPIRE environmental 5* Open Data
Towards INSPIRE environmental 5* Open Data
Martin Tuchyna
 
October 2014 HUG : Hive On Spark
October 2014 HUG : Hive On SparkOctober 2014 HUG : Hive On Spark
October 2014 HUG : Hive On Spark
Yahoo Developer Network
 
MATLAB and Scientific Data: New Features and Capabilities
MATLAB and Scientific Data: New Features and CapabilitiesMATLAB and Scientific Data: New Features and Capabilities
MATLAB and Scientific Data: New Features and Capabilities
The HDF-EOS Tools and Information Center
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Safe Software
 
Big Data Processing
Big Data ProcessingBig Data Processing
Big Data Processing
Michael Ming Lei
 
RDF-Gen: Generating RDF from streaming and archival data
RDF-Gen: Generating RDF from streaming and archival dataRDF-Gen: Generating RDF from streaming and archival data
RDF-Gen: Generating RDF from streaming and archival data
Giorgos Santipantakis
 
Wed roman tut_open_datapub
Wed roman tut_open_datapubWed roman tut_open_datapub
Wed roman tut_open_datapubeswcsummerschool
 

Similar to NUIG LOSD tools (20)

Generating Executable Mappings from RDF Data Cube Data Structure Definitions
Generating Executable Mappings from RDF Data Cube Data Structure DefinitionsGenerating Executable Mappings from RDF Data Cube Data Structure Definitions
Generating Executable Mappings from RDF Data Cube Data Structure Definitions
 
Using the Data Cube vocabulary for Publishing Environmental Linked Data on la...
Using the Data Cube vocabulary for Publishing Environmental Linked Data on la...Using the Data Cube vocabulary for Publishing Environmental Linked Data on la...
Using the Data Cube vocabulary for Publishing Environmental Linked Data on la...
 
Web analytics at scale with Druid at naver.com
Web analytics at scale with Druid at naver.comWeb analytics at scale with Druid at naver.com
Web analytics at scale with Druid at naver.com
 
Ozri 2013 Brisbane, Australia - Geodatabase Efficiencies
Ozri 2013 Brisbane, Australia - Geodatabase EfficienciesOzri 2013 Brisbane, Australia - Geodatabase Efficiencies
Ozri 2013 Brisbane, Australia - Geodatabase Efficiencies
 
Is there a way that we can build our Azure Data Factory all with parameters b...
Is there a way that we can build our Azure Data Factory all with parameters b...Is there a way that we can build our Azure Data Factory all with parameters b...
Is there a way that we can build our Azure Data Factory all with parameters b...
 
Scalable Preservation Workflows
Scalable Preservation WorkflowsScalable Preservation Workflows
Scalable Preservation Workflows
 
11. From Hadoop to Spark 1:2
11. From Hadoop to Spark 1:211. From Hadoop to Spark 1:2
11. From Hadoop to Spark 1:2
 
DSD-INT 2020 BlueEarth Engine - hydroMT - model builder framework
DSD-INT 2020 BlueEarth Engine - hydroMT - model builder frameworkDSD-INT 2020 BlueEarth Engine - hydroMT - model builder framework
DSD-INT 2020 BlueEarth Engine - hydroMT - model builder framework
 
Lodstats: The Data Web Census Dataset. Kobe, Japan, 2016
Lodstats: The Data Web Census Dataset. Kobe, Japan, 2016Lodstats: The Data Web Census Dataset. Kobe, Japan, 2016
Lodstats: The Data Web Census Dataset. Kobe, Japan, 2016
 
Introduction to Google Earth Engine .pptx
Introduction to Google Earth Engine .pptxIntroduction to Google Earth Engine .pptx
Introduction to Google Earth Engine .pptx
 
Geospatial Data Abstraction Library (GDAL) Enhancement for ESDIS (GEE)
Geospatial Data Abstraction Library (GDAL) Enhancement for ESDIS (GEE)Geospatial Data Abstraction Library (GDAL) Enhancement for ESDIS (GEE)
Geospatial Data Abstraction Library (GDAL) Enhancement for ESDIS (GEE)
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Edward King SPEDDEXES 2014
Edward King SPEDDEXES 2014Edward King SPEDDEXES 2014
Edward King SPEDDEXES 2014
 
Towards INSPIRE environmental 5* Open Data
Towards INSPIRE environmental 5* Open Data Towards INSPIRE environmental 5* Open Data
Towards INSPIRE environmental 5* Open Data
 
October 2014 HUG : Hive On Spark
October 2014 HUG : Hive On SparkOctober 2014 HUG : Hive On Spark
October 2014 HUG : Hive On Spark
 
MATLAB and Scientific Data: New Features and Capabilities
MATLAB and Scientific Data: New Features and CapabilitiesMATLAB and Scientific Data: New Features and Capabilities
MATLAB and Scientific Data: New Features and Capabilities
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
 
Big Data Processing
Big Data ProcessingBig Data Processing
Big Data Processing
 
RDF-Gen: Generating RDF from streaming and archival data
RDF-Gen: Generating RDF from streaming and archival dataRDF-Gen: Generating RDF from streaming and archival data
RDF-Gen: Generating RDF from streaming and archival data
 
Wed roman tut_open_datapub
Wed roman tut_open_datapubWed roman tut_open_datapub
Wed roman tut_open_datapub
 

Recently uploaded

一比一原版(UofS毕业证书)萨省大学毕业证如何办理
一比一原版(UofS毕业证书)萨省大学毕业证如何办理一比一原版(UofS毕业证书)萨省大学毕业证如何办理
一比一原版(UofS毕业证书)萨省大学毕业证如何办理
v3tuleee
 
一比一原版(ArtEZ毕业证)ArtEZ艺术学院毕业证成绩单
一比一原版(ArtEZ毕业证)ArtEZ艺术学院毕业证成绩单一比一原版(ArtEZ毕业证)ArtEZ艺术学院毕业证成绩单
一比一原版(ArtEZ毕业证)ArtEZ艺术学院毕业证成绩单
vcaxypu
 
一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理
一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理
一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理
oz8q3jxlp
 
Data Centers - Striving Within A Narrow Range - Research Report - MCG - May 2...
Data Centers - Striving Within A Narrow Range - Research Report - MCG - May 2...Data Centers - Striving Within A Narrow Range - Research Report - MCG - May 2...
Data Centers - Striving Within A Narrow Range - Research Report - MCG - May 2...
pchutichetpong
 
一比一原版(CU毕业证)卡尔顿大学毕业证成绩单
一比一原版(CU毕业证)卡尔顿大学毕业证成绩单一比一原版(CU毕业证)卡尔顿大学毕业证成绩单
一比一原版(CU毕业证)卡尔顿大学毕业证成绩单
yhkoc
 
Ch03-Managing the Object-Oriented Information Systems Project a.pdf
Ch03-Managing the Object-Oriented Information Systems Project a.pdfCh03-Managing the Object-Oriented Information Systems Project a.pdf
Ch03-Managing the Object-Oriented Information Systems Project a.pdf
haila53
 
一比一原版(BU毕业证)波士顿大学毕业证成绩单
一比一原版(BU毕业证)波士顿大学毕业证成绩单一比一原版(BU毕业证)波士顿大学毕业证成绩单
一比一原版(BU毕业证)波士顿大学毕业证成绩单
ewymefz
 
一比一原版(Adelaide毕业证书)阿德莱德大学毕业证如何办理
一比一原版(Adelaide毕业证书)阿德莱德大学毕业证如何办理一比一原版(Adelaide毕业证书)阿德莱德大学毕业证如何办理
一比一原版(Adelaide毕业证书)阿德莱德大学毕业证如何办理
slg6lamcq
 
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
slg6lamcq
 
Criminal IP - Threat Hunting Webinar.pdf
Criminal IP - Threat Hunting Webinar.pdfCriminal IP - Threat Hunting Webinar.pdf
Criminal IP - Threat Hunting Webinar.pdf
Criminal IP
 
SOCRadar Germany 2024 Threat Landscape Report
SOCRadar Germany 2024 Threat Landscape ReportSOCRadar Germany 2024 Threat Landscape Report
SOCRadar Germany 2024 Threat Landscape Report
SOCRadar
 
一比一原版(YU毕业证)约克大学毕业证成绩单
一比一原版(YU毕业证)约克大学毕业证成绩单一比一原版(YU毕业证)约克大学毕业证成绩单
一比一原版(YU毕业证)约克大学毕业证成绩单
enxupq
 
社内勉強会資料_LLM Agents                              .
社内勉強会資料_LLM Agents                              .社内勉強会資料_LLM Agents                              .
社内勉強会資料_LLM Agents                              .
NABLAS株式会社
 
一比一原版(CBU毕业证)卡普顿大学毕业证成绩单
一比一原版(CBU毕业证)卡普顿大学毕业证成绩单一比一原版(CBU毕业证)卡普顿大学毕业证成绩单
一比一原版(CBU毕业证)卡普顿大学毕业证成绩单
nscud
 
The affect of service quality and online reviews on customer loyalty in the E...
The affect of service quality and online reviews on customer loyalty in the E...The affect of service quality and online reviews on customer loyalty in the E...
The affect of service quality and online reviews on customer loyalty in the E...
jerlynmaetalle
 
一比一原版(NYU毕业证)纽约大学毕业证成绩单
一比一原版(NYU毕业证)纽约大学毕业证成绩单一比一原版(NYU毕业证)纽约大学毕业证成绩单
一比一原版(NYU毕业证)纽约大学毕业证成绩单
ewymefz
 
一比一原版(UVic毕业证)维多利亚大学毕业证成绩单
一比一原版(UVic毕业证)维多利亚大学毕业证成绩单一比一原版(UVic毕业证)维多利亚大学毕业证成绩单
一比一原版(UVic毕业证)维多利亚大学毕业证成绩单
ukgaet
 
一比一原版(Bradford毕业证书)布拉德福德大学毕业证如何办理
一比一原版(Bradford毕业证书)布拉德福德大学毕业证如何办理一比一原版(Bradford毕业证书)布拉德福德大学毕业证如何办理
一比一原版(Bradford毕业证书)布拉德福德大学毕业证如何办理
mbawufebxi
 
Empowering Data Analytics Ecosystem.pptx
Empowering Data Analytics Ecosystem.pptxEmpowering Data Analytics Ecosystem.pptx
Empowering Data Analytics Ecosystem.pptx
benishzehra469
 
一比一原版(UMich毕业证)密歇根大学|安娜堡分校毕业证成绩单
一比一原版(UMich毕业证)密歇根大学|安娜堡分校毕业证成绩单一比一原版(UMich毕业证)密歇根大学|安娜堡分校毕业证成绩单
一比一原版(UMich毕业证)密歇根大学|安娜堡分校毕业证成绩单
ewymefz
 

Recently uploaded (20)

一比一原版(UofS毕业证书)萨省大学毕业证如何办理
一比一原版(UofS毕业证书)萨省大学毕业证如何办理一比一原版(UofS毕业证书)萨省大学毕业证如何办理
一比一原版(UofS毕业证书)萨省大学毕业证如何办理
 
一比一原版(ArtEZ毕业证)ArtEZ艺术学院毕业证成绩单
一比一原版(ArtEZ毕业证)ArtEZ艺术学院毕业证成绩单一比一原版(ArtEZ毕业证)ArtEZ艺术学院毕业证成绩单
一比一原版(ArtEZ毕业证)ArtEZ艺术学院毕业证成绩单
 
一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理
一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理
一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理
 
Data Centers - Striving Within A Narrow Range - Research Report - MCG - May 2...
Data Centers - Striving Within A Narrow Range - Research Report - MCG - May 2...Data Centers - Striving Within A Narrow Range - Research Report - MCG - May 2...
Data Centers - Striving Within A Narrow Range - Research Report - MCG - May 2...
 
一比一原版(CU毕业证)卡尔顿大学毕业证成绩单
一比一原版(CU毕业证)卡尔顿大学毕业证成绩单一比一原版(CU毕业证)卡尔顿大学毕业证成绩单
一比一原版(CU毕业证)卡尔顿大学毕业证成绩单
 
Ch03-Managing the Object-Oriented Information Systems Project a.pdf
Ch03-Managing the Object-Oriented Information Systems Project a.pdfCh03-Managing the Object-Oriented Information Systems Project a.pdf
Ch03-Managing the Object-Oriented Information Systems Project a.pdf
 
一比一原版(BU毕业证)波士顿大学毕业证成绩单
一比一原版(BU毕业证)波士顿大学毕业证成绩单一比一原版(BU毕业证)波士顿大学毕业证成绩单
一比一原版(BU毕业证)波士顿大学毕业证成绩单
 
一比一原版(Adelaide毕业证书)阿德莱德大学毕业证如何办理
一比一原版(Adelaide毕业证书)阿德莱德大学毕业证如何办理一比一原版(Adelaide毕业证书)阿德莱德大学毕业证如何办理
一比一原版(Adelaide毕业证书)阿德莱德大学毕业证如何办理
 
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
 
Criminal IP - Threat Hunting Webinar.pdf
Criminal IP - Threat Hunting Webinar.pdfCriminal IP - Threat Hunting Webinar.pdf
Criminal IP - Threat Hunting Webinar.pdf
 
SOCRadar Germany 2024 Threat Landscape Report
SOCRadar Germany 2024 Threat Landscape ReportSOCRadar Germany 2024 Threat Landscape Report
SOCRadar Germany 2024 Threat Landscape Report
 
一比一原版(YU毕业证)约克大学毕业证成绩单
一比一原版(YU毕业证)约克大学毕业证成绩单一比一原版(YU毕业证)约克大学毕业证成绩单
一比一原版(YU毕业证)约克大学毕业证成绩单
 
社内勉強会資料_LLM Agents                              .
社内勉強会資料_LLM Agents                              .社内勉強会資料_LLM Agents                              .
社内勉強会資料_LLM Agents                              .
 
一比一原版(CBU毕业证)卡普顿大学毕业证成绩单
一比一原版(CBU毕业证)卡普顿大学毕业证成绩单一比一原版(CBU毕业证)卡普顿大学毕业证成绩单
一比一原版(CBU毕业证)卡普顿大学毕业证成绩单
 
The affect of service quality and online reviews on customer loyalty in the E...
The affect of service quality and online reviews on customer loyalty in the E...The affect of service quality and online reviews on customer loyalty in the E...
The affect of service quality and online reviews on customer loyalty in the E...
 
一比一原版(NYU毕业证)纽约大学毕业证成绩单
一比一原版(NYU毕业证)纽约大学毕业证成绩单一比一原版(NYU毕业证)纽约大学毕业证成绩单
一比一原版(NYU毕业证)纽约大学毕业证成绩单
 
一比一原版(UVic毕业证)维多利亚大学毕业证成绩单
一比一原版(UVic毕业证)维多利亚大学毕业证成绩单一比一原版(UVic毕业证)维多利亚大学毕业证成绩单
一比一原版(UVic毕业证)维多利亚大学毕业证成绩单
 
一比一原版(Bradford毕业证书)布拉德福德大学毕业证如何办理
一比一原版(Bradford毕业证书)布拉德福德大学毕业证如何办理一比一原版(Bradford毕业证书)布拉德福德大学毕业证如何办理
一比一原版(Bradford毕业证书)布拉德福德大学毕业证如何办理
 
Empowering Data Analytics Ecosystem.pptx
Empowering Data Analytics Ecosystem.pptxEmpowering Data Analytics Ecosystem.pptx
Empowering Data Analytics Ecosystem.pptx
 
一比一原版(UMich毕业证)密歇根大学|安娜堡分校毕业证成绩单
一比一原版(UMich毕业证)密歇根大学|安娜堡分校毕业证成绩单一比一原版(UMich毕业证)密歇根大学|安娜堡分校毕业证成绩单
一比一原版(UMich毕业证)密歇根大学|安娜堡分校毕业证成绩单
 

NUIG LOSD tools

  • 1. WP3 OpenGovIntelligence LOSD Tools NUIG Mohamed Adel Rezk Arkadiusz Stasiewicz Dr. Adegboyega Ojo
  • 2. Agenda • Achieved / Ongoing Tools • Assisted RDF Data Cube Schema Mapping • Generic RDF Data Cube Builder • RDF Data Cube Explorer • RDF Data Cube geo-data supported Dashboard • Planned Tools • Real-Time LOSD Publishing Pipeline • Marine Institute and Lithuanian Pilots’ Dashboards updates • Collaboration Space and A Feedback Mechanism 2
  • 3. LOSD Publishing pipeline [NUIG Approach] 3 Assisted RDF Data Cube Schema Mapping Generic RDF Data Cube Builder RDF Data Cube Explorer LOSD Dashboards [ET - MI]
  • 4. LOSD Publishing pipeline [flow chart] 4 Schema Mapping Stage Pilot / user dataset to RDF Data Cube Vocabulary Mapping [Spreadsheet] RDF Data Cube schema as rdf refine configuration file [json] [Open refine] and [RDF refine ext.] Pilot / user mapped RDF Data Cube schema [rdf/xml or ttl] Pilot / user dataset [csv] RDF Data Cube Builder Application [Web App – Desktop – CMD - Web service] Pilot / user RDF Data Cube [rdf/xml or ttl] Data Cube Building Stage Data Cube Exploration Stage JSON QB REST API implementation [java web service] RDF Data Store [virtuso/f] Data Cube Explorer [django - wordpress] ET dashboard MI dashboard
  • 5. Assisted RDF Data Cube Schema Mapping • This tool/pipeline is created to ease the mapping process of government statistical datasets into RDF according to the RDF data cube vocabulary. • This tool integrates [spreadsheets, open refine, rdf refine, and rdf data cube] to produce a generic LOSD mapping tool that fits any pilot or use case. • The following demo shows the Schema Mapping pipeline steps/stages. 5
  • 6. Assisted RDF Data Cube Schema Mapping • Step 1 [Mapping Spreadsheet inputs] 6
  • 7. Assisted RDF Data Cube Schema Mapping • Step 1 [Mapping Spreadsheet inputs] 7
  • 8. Assisted RDF Data Cube Schema Mapping • Step 1 [Mapping Spreadsheet inputs] 8
  • 9. Assisted RDF Data Cube Schema Mapping • Step 1 [Mapping Spreadsheet inputs] 9
  • 10. Assisted RDF Data Cube Schema Mapping • Step 1 [Mapping Spreadsheet inputs] 10
  • 11. Assisted RDF Data Cube Schema Mapping • Step 2 [Loading Mapping Spreadsheet to Open Refine “+RDF refine ext.”] 11
  • 12. Assisted RDF Data Cube Schema Mapping • Step 2 [Loading Mapping Spreadsheet to Open Refine “+RDF refine ext.”] 12
  • 13. Assisted RDF Data Cube Schema Mapping • Step 2 [Loading Mapping Spreadsheet to Open Refine “+RDF refine ext.”] 13
  • 14. Assisted RDF Data Cube Schema Mapping • Step 2 [Loading Mapping Spreadsheet to Open Refine “+RDF refine ext.”] 14
  • 15. Assisted RDF Data Cube Schema Mapping • Step 3 [Applying predefined RDF Data Cube Schema to the input Spreadsheet] 15
  • 16. Assisted RDF Data Cube Schema Mapping 16 • Step 3 [Applying predefined RDF Data Cube Schema to the input Spreadsheet]
  • 17. Assisted RDF Data Cube Schema Mapping • Step 3 [Applying predefined RDF Data Cube Schema to the input Spreadsheet] 17
  • 18. Assisted RDF Data Cube Schema Mapping • Step 3 [Applying predefined RDF Data Cube Schema to the input Spreadsheet] 18
  • 19. Assisted RDF Data Cube Schema Mapping • Step 3 [Applying predefined RDF Data Cube Schema to the input Spreadsheet] 19
  • 20. Assisted RDF Data Cube Schema Mapping 20 • Step 3 [Applying predefined RDF Data Cube Schema to the input Spreadsheet]
  • 21. Assisted RDF Data Cube Schema Mapping • Step 3 [Applying predefined RDF Data Cube Schema to the input Spreadsheet] 21
  • 22. Assisted RDF Data Cube Schema Mapping • Step 3 [Extracting Schema mapping of the dataset in rdf/xml or ttl formats] 22
  • 23. Assisted RDF Data Cube Schema Mapping • Step 3 [Extracting Schema mapping of the dataset in rdf/xml or ttl formats] 23
  • 24. Assisted RDF Data Cube Schema Mapping • Step 3 [Extracting Schema mapping of the dataset in rdf/xml or ttl formats] 24
  • 25. Assisted RDF Data Cube Schema Mapping • Step 3 [Extracting Schema mapping of the dataset in rdf/xml or ttl formats] 25
  • 26. Assisted RDF Data Cube Schema Mapping • Git Repo: https://github.com/mohadelrezk/open-gov-RDFDataCube-schema- creator 26
  • 27. LOSD Publishing pipeline [flow chart] 27 Schema Mapping Stage Pilot / user dataset to RDF Data Cube Vocabulary Mapping [Spreadsheet] RDF Data Cube schema as rdf refine configuration file [json] [Open refine] and [RDF refine ext.] Pilot / user mapped RDF Data Cube schema [rdf/xml or ttl] Pilot / user dataset [csv] RDF Data Cube Builder Application [Web App – Desktop – CMD - Web service] Pilot / user RDF Data Cube [rdf/xml or ttl] Data Cube Building Stage Data Cube Exploration Stage JSON QB REST API implementation [java web service] RDF Data Store [virtuso/f] Data Cube Explorer [django - wordpress] ET dashboard MI dashboard
  • 28. Generic RDF Data Cube Builder • RDF Data Cube Builder is used to transform government statistical data [csv] into RDF Data Cubes [rdf] based on the predefined mapping produced in the previous stage. • Cube Builder Application implemented in deferent interfaces Web App., Desktop App., Command Line and Web Service. • The following demo shows the Cube Builder usages. 28
  • 29. Generic RDF Data Cube Builder • Web Application Interface [using the Web Service] 29
  • 30. Generic RDF Data Cube Builder • Web Application Interface [using the Web Service] 30
  • 31. Generic RDF Data Cube Builder • Web Application Interface [using the Web Service] 31
  • 32. Generic RDF Data Cube Builder • Web Application Interface [using the Web Service] 32
  • 33. Generic RDF Data Cube Builder • Web Application Interface [using the Web Service] 33
  • 34. Generic RDF Data Cube Builder • Web Application Interface [using the Web Service] 34
  • 35. Generic RDF Data Cube Builder • Web Application Interface [using the Web Service] 35
  • 36. Generic RDF Data Cube Builder • Web Application Interface [using the Web Service] 36
  • 37. Generic RDF Data Cube Builder • Web Application Interface [using the Web Service] 37
  • 38. Generic RDF Data Cube Builder • Web Application Interface [using the Web Service] 38
  • 39. Generic RDF Data Cube Builder • Web Application Interface [using the Web Service] 39
  • 40. Generic RDF Data Cube Builder • CMD Interface 40
  • 41. Generic RDF Data Cube Builder • Desktop Interface 41
  • 42. Generic RDF Data Cube Builder • Desktop Interface 42
  • 43. Generic RDF Data Cube Builder • Desktop Interface 43
  • 44. Generic RDF Data Cube Builder • Git repo: https://github.com/OpenGovIntelligence/data-cube-builder 44
  • 45. LOSD Publishing pipeline [flow chart] 45 Schema Mapping Stage Pilot / user dataset to RDF Data Cube Vocabulary Mapping [Spreadsheet] RDF Data Cube schema as rdf refine configuration file [json] [Open refine] and [RDF refine ext.] Pilot / user mapped RDF Data Cube schema [rdf/xml or ttl] Pilot / user dataset [csv] RDF Data Cube Builder Application [Web App – Desktop – CMD - Web service] Pilot / user RDF Data Cube [rdf/xml or ttl] Data Cube Building Stage Data Cube Exploration Stage JSON QB REST API implementation [java web service] RDF Data Store [virtuso/f] Data Cube Explorer [django - wordpress] ET dashboard MI dashboard
  • 46. RDF Data Cube Explorer • RDF Data Cube Explorer is a webserver that integrates [json QB API, RDF Data Stores, Pivot table, Maps etc .. ] to produce explorations and visualizations dashboards for LOSD previously created and loaded into RDF Data Stores. • Those Dash Boards are designed and implemented to meet the pilot use cases requirements. • The following demo shows the Cube Explorer abilities. 46
  • 47. RDF Data Cube Explorer [Maritime Search and Rescue]
  • 48. • Considering the sad incident in March 2017, the Cube Explorer would assist the search and rescue efforts by pointing out possible search points for the remains using wind and wave direction readings. RDF Data Cube Explorer [Maritime Search and Rescue]
  • 49. • Average Wind Direction in (+) or (–) 24 hours of the incident date (assuming the incident happened on 12/09/2016) grouped by longitude and latitude points using data sourced from [Irish Weather Buoy Network] data source. • As shown in the next slide using our cube explorer rescue staff can choose the dataset contains the relevant data and choose the visualization and aggregation types, then they can filter out the data to keep only relevant dimensions (e.g. longitude ) and measures (e.g. day) and the relevant values (e.g. 11,12, and 13/09/2016). • From the visualizations the rescue staff can see that, during the incident. RDF Data Cube Explorer [Maritime Search and Rescue]
  • 50. Maritime Search and Rescue, avg. wind direction RDF Data Cube Explorer [Maritime Search and Rescue]
  • 51. Maritime Search and Rescue, mean wave direction RDF Data Cube Explorer [Maritime Search and Rescue]
  • 52. • Using the visualizations the rescue staff can see that, during the relevant time period and geo-coordinates of the incident avg. wind direction was between (181.8 and 355 degrees) and mean wave direction was between (216.75 and 273 degrees). So that they will focus their efforts accordingly. RDF Data Cube Explorer [Maritime Search and Rescue]
  • 53. RDF Data Cube Explorer [Marine Renewable Energy]
  • 54. • The installation process of new generators (wind or wave based) starts with locating the most fitting spot that will guarantee the best operating conditions for the maximum productivity and safety. RDF Data Cube Explorer [Marine Renewable Energy]
  • 55. • Using avg. wind speed around the year [for fixed location generators] or in certain periods [for movable generators] it can be decided where and when to install the generators. RDF Data Cube Explorer [Marine Renewable Energy]
  • 56. • Marine Renewable Energy, avg. wind speed {year based} RDF Data Cube Explorer [Marine Renewable Energy]
  • 57. Marine Renewable Energy, avg. wind speed {monthly based} RDF Data Cube Explorer [Marine Renewable Energy]
  • 58. • Using the visualizations we can see that, • (1) Buoy doesn't get data for wind speed at the geo point (5.42, 58.48), the wind speed sensor could be in need for replacement or maintenance. • (2) Geo points (9.99, 54.99) and (10.54, 51.21) have the highest avg. wind speed around the year. • (3) Goe point (6.70, 51.69) have the highest avg. wind speed in November and geo point (9.99, 54.99) have the highest avg. wind speed in August. • So, “regarding other decision inputs” for fixed wind generators marine staff could place it at the area around geo point (9.99, 54.99) and (10.54, 51.21). • And for movable wind generators, best spot in August is (9.99, 54.99) and in November is (6.70, 51.69). • Same process will fit for the wave generators. RDF Data Cube Explorer [Marine Renewable Energy]
  • 59. RDF Data Cube Explorer [Maritime Tourism and Leisure]
  • 60. • For setting up safe diving and swimming locations, and also avoiding dangerous locations swimmer and surfers can use wave speed and height, wind speed, gust, sea temperature and air temperature readings. RDF Data Cube Explorer [Maritime Tourism and Leisure]
  • 61. Maritime Tourism and Leisure, avg. sea temperature in August RDF Data Cube Explorer [Maritime Tourism and Leisure]
  • 62. Maritime Tourism and Leisure, avg. air temperature in August RDF Data Cube Explorer [Maritime Tourism and Leisure]
  • 63. RDF Data Cube Explorer [Generic Use Cases of the Cube Explorer Tool]
  • 64. • Cube Explorer works by pointing it to a RDF Data Cube server, then it will fetch all available cube data sets and offer the user a verity of RDF data cube visualizations and filtering options. • e.g. choosing the Foreign direct investment per capita RDF Data Cube for explorations as shown below. RDF Data Cube Explorer [Generic Use Cases of the Cube Explorer Tool]
  • 65. • Cube Explorer offers a verity of visualization and exploration options e.g. Bar charts, Heat Maps, Line chart, Table Bar Chart etc. . RDF Data Cube Explorer [Generic Use Cases of the Cube Explorer Tool]
  • 66. • e.g. Highlighting the Foreign direct investment per capita for the country of Lithuania. • Visualizing the Data Cube using country wide, county, and municipality levels with temporal filtering would surely assist decision makers to achieve more with this information in hand. RDF Data Cube Explorer [Generic Use Cases of the Cube Explorer Tool]
  • 67. RDF Data Cube Explorer [Generic Use Cases of the Cube Explorer Tool]
  • 68. RDF Data Cube Explorer [Generic Use Cases of the Cube Explorer Tool]
  • 69. RDF Data Cube Explorer [Generic Use Cases of the Cube Explorer Tool]
  • 70. • e.g. Highlighting the Average Monthly Earnings for the country of Lithuania. • Visualizing the Data Cube using country wide, county, and municipality levels with temporal filtering would surely assist decision makers to achieve more with this information in hand. RDF Data Cube Explorer [Generic Use Cases of the Cube Explorer Tool]
  • 71. RDF Data Cube Explorer [Generic Use Cases of the Cube Explorer Tool]
  • 72. RDF Data Cube Explorer [Generic Use Cases of the Cube Explorer Tool]
  • 73. • e.g. Highlighting the Resident Population Demography for the country of Lithuania. • Visualizing the Data Cube using country wide, county, and municipality levels with temporal filtering would surely assist decision makers to achieve more with this information in hand. RDF Data Cube Explorer [Generic Use Cases of the Cube Explorer Tool]
  • 74. RDF Data Cube Explorer [Generic Use Cases of the Cube Explorer Tool]
  • 75. RDF Data Cube Explorer [Generic Use Cases of the Cube Explorer Tool]
  • 76. RDF Data Cube Explorer [Generic Use Cases of the Cube Explorer Tool] • Git repo: https://github.com/OpenGovIntelligence/data-cube-explorer
  • 77. RDF Data Cube geo-data supported Dashboard • This dashboard mainly designed and implemented to meet the Lithuanian pilot use case requirements. • The following demo shows the Data Cube geo-data supported Dashboard look. 77
  • 78. RDF Data Cube geo-data supported Dashboard • Lithuanian Data Cubes Geo based Visualization screen samples. 78
  • 79. RDF Data Cube geo-data supported Dashboard • Lithuanian Data Cubes Geo based Visualization screen samples. 79
  • 80. RDF Data Cube geo-data supported Dashboard • Lithuanian Data Cubes Geo based Visualization screen samples. 80
  • 81. Planned • Real-Time LOSD Publishing Pipeline • Marine Institute / Lithuanian Pilot Dashboards updates • Collaboration Space / Feed Back Mechanism 81
  • 82. Real-Time LOSD Publishing Pipeline • To target real time marine data use case [ search and rescue], we are planning to design and implement a real time version of the LOSD publishing pipeline. • Following a drafted vision of the above pipeline. 82
  • 83. MI servers APIs OGI RDF store (Data of Curre Week only){Weekly} API calls Cube Annotation Temporary CSV DB Cube Builder Push to RDF Store Cursor Tracking Drop outdated data Real-Time LOSD Publishing Pipeline