SlideShare a Scribd company logo
1 of 24
Download to read offline
Open Data Hub
ARCHITECTURE DESIGN BY ANIMESHON
01 – Open Data Hub
02 – Technologies
03 – A new way to communicate
04 – Reactivity
05 – Plug & Research
00 – Introduction
AGEND
A
06 – Conclusion
00
INTRODUCTIO
N
Like to a post
send notification to
users
weather
forecast
model
improve future
forecasts
notify consumers
recommendation
engine
recommend ads
update feed
update profiling
information
Resolve Captcha
IoT sensor
sending pressure
feedback to
AI models
protect from
unauthorized actors
collect data for
models training
Open Data hub
A modern system which already serves
thousands of citizens while processing
data from dozens of providers
Open Data Hub v2
A system designed to meet future
requirements while reaching more people
with more and faster information
State of the art
technologies.
International standards
enforcement
01
OPEN DATA HUB
A distributed, event-driven,
zero-downtime system
Open Data Hub:
distributed
microservices
Open Data Hub is a cluster of countless
microservices
single responsibility
Each service has only one responsibility which
can be carried out in the most efficient way
event-driven
async communication
The subprocess chain communicates
asynchronously, making each component
independent from the others
service self-healing
Events are thrown in the system, and the
consumer can read them when they are ready.
If something goes down, events are preserved
until it comes back online
agnostic communication
Services do not know who will be the next
process to take control, they just inform the
system about their own results
zero-downtime
no data loss
Once an information enters the system, it will be
preserved and backed up to ensure data integrity
and data durability
update rollout
Service orchestration allows the team to rollout
updates with different strategies zero or minimal
downtime
fault tolerant
Even if multiple services are offline, the system
can still accept data and serve results
02
TECHNOLOGIES
no vendor lock-in
All technologies are provider
agnostic and can be migrated to
any private or public cloud provider
open source
All technologies are open source
easy to contribute
Open Data Hub’s public repository
allows anyone to start a local
cluster of the core architecture
and start contributing
anyone can locally instantiate an Open Data
Hub cluster in just 20 minutes and start
contributing to the project
https://github.com/noi-techpark/odh-infrastructure-v2
02
A NEW WAY TO
COMMUNICATE
Open Data Hub uses a standard communication protocol to both
serve and process incoming data in alignment with internet 2.0
common practices
standard communication protocol
Speaking a common language makes it more appealing for third
parties to integrate with Open Data Hub
easy to integrate
A more structured and standardized communication protocol allows
for a reduction of custom solutions and more precise specifications
open specifications
RestApi
Open Data Hub enforces communication standards
with providers.
MQTT Websocket
04
REACTIVITY
easy to maintain
Changing the order of the
blocks, or substitute some of
them will not affect the rest of
the system
The whole architecture is
composed by small
lego-blocks
modular design
All lego-blocks can be
rearranged in whatever form
to satisfy new requirements
adaptability
The system is designed to
react to future needs and
evolve
evolution by design
05
PLUG & RESEARCH
Open Data Hub allows for
isolated environments to
utilize historical raw data
Thanks to no-code tools
researches could create
data processing pipeline
without IT knowledge
Allowing institutions to
experiment with both
processed and raw data
enables an exponential
grow of the system itself
sandbox
semplicity
grow
06
CONCLUSION
Open Data Hub promotes open source by
adopting only open technologies while
publishing it’s core infrastructure
open
State-of-the-art technologies and modern paradigms
ensure an easy maintenance and integrability
innovative
Designed to meet any present and future requirement
scalable
Thank you

More Related Content

Similar to Open Data Hub - Matteo Roggia - Animeshon - Real-time Open Data Hub.pdf

Microservices: Why and When? - Alon Fliess, CodeValue - Cloud Native Day Tel ...
Microservices: Why and When? - Alon Fliess, CodeValue - Cloud Native Day Tel ...Microservices: Why and When? - Alon Fliess, CodeValue - Cloud Native Day Tel ...
Microservices: Why and When? - Alon Fliess, CodeValue - Cloud Native Day Tel ...Cloud Native Day Tel Aviv
 
Collaborating with OpenDaylight for a Network-Enabled Cloud
Collaborating with OpenDaylight for a Network-Enabled CloudCollaborating with OpenDaylight for a Network-Enabled Cloud
Collaborating with OpenDaylight for a Network-Enabled CloudTesora
 
Mainflux Labs - References (1).pdf
Mainflux Labs - References (1).pdfMainflux Labs - References (1).pdf
Mainflux Labs - References (1).pdfWlamir Molinari
 
why-choose-.net-for-your-next-iot-project.pdf
why-choose-.net-for-your-next-iot-project.pdfwhy-choose-.net-for-your-next-iot-project.pdf
why-choose-.net-for-your-next-iot-project.pdfPixelQA
 
IBM Bluemix: science fiction has been overtaken....now everything is possible
IBM Bluemix: science fiction has been overtaken....now everything is possibleIBM Bluemix: science fiction has been overtaken....now everything is possible
IBM Bluemix: science fiction has been overtaken....now everything is possibleCodemotion
 
VEDLIoT at Stockholm Tech Live 2022
VEDLIoT at Stockholm Tech Live 2022VEDLIoT at Stockholm Tech Live 2022
VEDLIoT at Stockholm Tech Live 2022VEDLIoT Project
 
21 Innovative IoT Products To Watch at Hannover Messe 2017
21 Innovative IoT Products To Watch at Hannover Messe 201721 Innovative IoT Products To Watch at Hannover Messe 2017
21 Innovative IoT Products To Watch at Hannover Messe 2017Kudzai Manditereza
 
OSCON 2013 - The Hitchiker’s Guide to Open Source Cloud Computing
OSCON 2013 - The Hitchiker’s Guide to Open Source Cloud ComputingOSCON 2013 - The Hitchiker’s Guide to Open Source Cloud Computing
OSCON 2013 - The Hitchiker’s Guide to Open Source Cloud ComputingMark Hinkle
 
Io t standard_bis_arpanpal
Io t standard_bis_arpanpalIo t standard_bis_arpanpal
Io t standard_bis_arpanpalArpan Pal
 
The Architecture Of Software Defined Radios Essay
The Architecture Of Software Defined Radios EssayThe Architecture Of Software Defined Radios Essay
The Architecture Of Software Defined Radios EssayDivya Watson
 
The Management of The Future Internet With SDN and NFV
The Management of The Future Internet With SDN and NFV The Management of The Future Internet With SDN and NFV
The Management of The Future Internet With SDN and NFV AmeerAlSadi
 
OpenShift Overview - Red Hat Open School 2017
OpenShift Overview - Red Hat Open School 2017OpenShift Overview - Red Hat Open School 2017
OpenShift Overview - Red Hat Open School 2017Rodolfo Carvalho
 
apidays LIVE Australia - Move to the next generation API ecosystem by looking...
apidays LIVE Australia - Move to the next generation API ecosystem by looking...apidays LIVE Australia - Move to the next generation API ecosystem by looking...
apidays LIVE Australia - Move to the next generation API ecosystem by looking...apidays
 
What Is OpenTelemetry? A Complete Introduction
What Is OpenTelemetry? A Complete IntroductionWhat Is OpenTelemetry? A Complete Introduction
What Is OpenTelemetry? A Complete IntroductionCiente
 
ABC of IoT Consortiums
ABC of IoT ConsortiumsABC of IoT Consortiums
ABC of IoT ConsortiumsIan Skerrett
 
Internet of Things.pptx
Internet of Things.pptxInternet of Things.pptx
Internet of Things.pptxEshwar Prasad
 

Similar to Open Data Hub - Matteo Roggia - Animeshon - Real-time Open Data Hub.pdf (20)

Microservices: Why and When? - Alon Fliess, CodeValue - Cloud Native Day Tel ...
Microservices: Why and When? - Alon Fliess, CodeValue - Cloud Native Day Tel ...Microservices: Why and When? - Alon Fliess, CodeValue - Cloud Native Day Tel ...
Microservices: Why and When? - Alon Fliess, CodeValue - Cloud Native Day Tel ...
 
Collaborating with OpenDaylight for a Network-Enabled Cloud
Collaborating with OpenDaylight for a Network-Enabled CloudCollaborating with OpenDaylight for a Network-Enabled Cloud
Collaborating with OpenDaylight for a Network-Enabled Cloud
 
Mainflux Labs - References (1).pdf
Mainflux Labs - References (1).pdfMainflux Labs - References (1).pdf
Mainflux Labs - References (1).pdf
 
why-choose-.net-for-your-next-iot-project.pdf
why-choose-.net-for-your-next-iot-project.pdfwhy-choose-.net-for-your-next-iot-project.pdf
why-choose-.net-for-your-next-iot-project.pdf
 
iot
iotiot
iot
 
IBM Bluemix: science fiction has been overtaken....now everything is possible
IBM Bluemix: science fiction has been overtaken....now everything is possibleIBM Bluemix: science fiction has been overtaken....now everything is possible
IBM Bluemix: science fiction has been overtaken....now everything is possible
 
IoT-01-00005-v2.pdf
IoT-01-00005-v2.pdfIoT-01-00005-v2.pdf
IoT-01-00005-v2.pdf
 
VEDLIoT at Stockholm Tech Live 2022
VEDLIoT at Stockholm Tech Live 2022VEDLIoT at Stockholm Tech Live 2022
VEDLIoT at Stockholm Tech Live 2022
 
21 Innovative IoT Products To Watch at Hannover Messe 2017
21 Innovative IoT Products To Watch at Hannover Messe 201721 Innovative IoT Products To Watch at Hannover Messe 2017
21 Innovative IoT Products To Watch at Hannover Messe 2017
 
OSCON 2013 - The Hitchiker’s Guide to Open Source Cloud Computing
OSCON 2013 - The Hitchiker’s Guide to Open Source Cloud ComputingOSCON 2013 - The Hitchiker’s Guide to Open Source Cloud Computing
OSCON 2013 - The Hitchiker’s Guide to Open Source Cloud Computing
 
Io t standard_bis_arpanpal
Io t standard_bis_arpanpalIo t standard_bis_arpanpal
Io t standard_bis_arpanpal
 
The Architecture Of Software Defined Radios Essay
The Architecture Of Software Defined Radios EssayThe Architecture Of Software Defined Radios Essay
The Architecture Of Software Defined Radios Essay
 
The Management of The Future Internet With SDN and NFV
The Management of The Future Internet With SDN and NFV The Management of The Future Internet With SDN and NFV
The Management of The Future Internet With SDN and NFV
 
Cloud Manufacturing
Cloud ManufacturingCloud Manufacturing
Cloud Manufacturing
 
Designing Internet of things
Designing Internet of thingsDesigning Internet of things
Designing Internet of things
 
OpenShift Overview - Red Hat Open School 2017
OpenShift Overview - Red Hat Open School 2017OpenShift Overview - Red Hat Open School 2017
OpenShift Overview - Red Hat Open School 2017
 
apidays LIVE Australia - Move to the next generation API ecosystem by looking...
apidays LIVE Australia - Move to the next generation API ecosystem by looking...apidays LIVE Australia - Move to the next generation API ecosystem by looking...
apidays LIVE Australia - Move to the next generation API ecosystem by looking...
 
What Is OpenTelemetry? A Complete Introduction
What Is OpenTelemetry? A Complete IntroductionWhat Is OpenTelemetry? A Complete Introduction
What Is OpenTelemetry? A Complete Introduction
 
ABC of IoT Consortiums
ABC of IoT ConsortiumsABC of IoT Consortiums
ABC of IoT Consortiums
 
Internet of Things.pptx
Internet of Things.pptxInternet of Things.pptx
Internet of Things.pptx
 

More from South Tyrol Free Software Conference

SFSCON23 - Rufai Omowunmi Balogun - SMODEX – a Python package for understandi...
SFSCON23 - Rufai Omowunmi Balogun - SMODEX – a Python package for understandi...SFSCON23 - Rufai Omowunmi Balogun - SMODEX – a Python package for understandi...
SFSCON23 - Rufai Omowunmi Balogun - SMODEX – a Python package for understandi...South Tyrol Free Software Conference
 
SFSCON23 - Roberto Innocenti - From the design to reality is here the Communi...
SFSCON23 - Roberto Innocenti - From the design to reality is here the Communi...SFSCON23 - Roberto Innocenti - From the design to reality is here the Communi...
SFSCON23 - Roberto Innocenti - From the design to reality is here the Communi...South Tyrol Free Software Conference
 
SFSCON23 - Martin Rabanser - Real-time aeroplane tracking and the Open Data Hub
SFSCON23 - Martin Rabanser - Real-time aeroplane tracking and the Open Data HubSFSCON23 - Martin Rabanser - Real-time aeroplane tracking and the Open Data Hub
SFSCON23 - Martin Rabanser - Real-time aeroplane tracking and the Open Data HubSouth Tyrol Free Software Conference
 
SFSCON23 - Marianna d'Atri Enrico Zanardo - How can Blockchain technologies i...
SFSCON23 - Marianna d'Atri Enrico Zanardo - How can Blockchain technologies i...SFSCON23 - Marianna d'Atri Enrico Zanardo - How can Blockchain technologies i...
SFSCON23 - Marianna d'Atri Enrico Zanardo - How can Blockchain technologies i...South Tyrol Free Software Conference
 
SFSCON23 - Lucas Lasota - The Future of Connectivity, Open Internet and Human...
SFSCON23 - Lucas Lasota - The Future of Connectivity, Open Internet and Human...SFSCON23 - Lucas Lasota - The Future of Connectivity, Open Internet and Human...
SFSCON23 - Lucas Lasota - The Future of Connectivity, Open Internet and Human...South Tyrol Free Software Conference
 
SFSCON23 - Giovanni Giannotta - Intelligent Decision Support System for trace...
SFSCON23 - Giovanni Giannotta - Intelligent Decision Support System for trace...SFSCON23 - Giovanni Giannotta - Intelligent Decision Support System for trace...
SFSCON23 - Giovanni Giannotta - Intelligent Decision Support System for trace...South Tyrol Free Software Conference
 
SFSCON23 - Elena Maines - Embracing CI/CD workflows for building ETL pipelines
SFSCON23 - Elena Maines - Embracing CI/CD workflows for building ETL pipelinesSFSCON23 - Elena Maines - Embracing CI/CD workflows for building ETL pipelines
SFSCON23 - Elena Maines - Embracing CI/CD workflows for building ETL pipelinesSouth Tyrol Free Software Conference
 
SFSCON23 - Charles H. Schulz - Why open digital infrastructure matters
SFSCON23 - Charles H. Schulz - Why open digital infrastructure mattersSFSCON23 - Charles H. Schulz - Why open digital infrastructure matters
SFSCON23 - Charles H. Schulz - Why open digital infrastructure mattersSouth Tyrol Free Software Conference
 
SFSCON23 - Thomas Aichner - How IoT and AI are revolutionizing Mass Customiza...
SFSCON23 - Thomas Aichner - How IoT and AI are revolutionizing Mass Customiza...SFSCON23 - Thomas Aichner - How IoT and AI are revolutionizing Mass Customiza...
SFSCON23 - Thomas Aichner - How IoT and AI are revolutionizing Mass Customiza...South Tyrol Free Software Conference
 
SFSCON23 - Mirko Boehm - European regulators cast their eyes on maturing OSS ...
SFSCON23 - Mirko Boehm - European regulators cast their eyes on maturing OSS ...SFSCON23 - Mirko Boehm - European regulators cast their eyes on maturing OSS ...
SFSCON23 - Mirko Boehm - European regulators cast their eyes on maturing OSS ...South Tyrol Free Software Conference
 
SFSCON23 - Marco Pavanelli - Monitoring the fleet of Sasa with free software
SFSCON23 - Marco Pavanelli - Monitoring the fleet of Sasa with free softwareSFSCON23 - Marco Pavanelli - Monitoring the fleet of Sasa with free software
SFSCON23 - Marco Pavanelli - Monitoring the fleet of Sasa with free softwareSouth Tyrol Free Software Conference
 
SFSCON23 - Marco Cortella - KNOWAGE and AICS for 2030 agenda SDG goals monito...
SFSCON23 - Marco Cortella - KNOWAGE and AICS for 2030 agenda SDG goals monito...SFSCON23 - Marco Cortella - KNOWAGE and AICS for 2030 agenda SDG goals monito...
SFSCON23 - Marco Cortella - KNOWAGE and AICS for 2030 agenda SDG goals monito...South Tyrol Free Software Conference
 
SFSCON23 - Lina Ceballos - Interoperable Europe Act - A real game changer
SFSCON23 - Lina Ceballos - Interoperable Europe Act - A real game changerSFSCON23 - Lina Ceballos - Interoperable Europe Act - A real game changer
SFSCON23 - Lina Ceballos - Interoperable Europe Act - A real game changerSouth Tyrol Free Software Conference
 
SFSCON23 - Johannes Näder Linus Sehn - Let’s monitor implementation of Free S...
SFSCON23 - Johannes Näder Linus Sehn - Let’s monitor implementation of Free S...SFSCON23 - Johannes Näder Linus Sehn - Let’s monitor implementation of Free S...
SFSCON23 - Johannes Näder Linus Sehn - Let’s monitor implementation of Free S...South Tyrol Free Software Conference
 
SFSCON23 - Gabriel Ku Wei Bin - Why Do We Need A Next Generation Internet
SFSCON23 - Gabriel Ku Wei Bin - Why Do We Need A Next Generation InternetSFSCON23 - Gabriel Ku Wei Bin - Why Do We Need A Next Generation Internet
SFSCON23 - Gabriel Ku Wei Bin - Why Do We Need A Next Generation InternetSouth Tyrol Free Software Conference
 
SFSCON23 - Davide Vernassa - Empowering Insights Unveiling the latest innova...
SFSCON23 - Davide Vernassa - Empowering Insights  Unveiling the latest innova...SFSCON23 - Davide Vernassa - Empowering Insights  Unveiling the latest innova...
SFSCON23 - Davide Vernassa - Empowering Insights Unveiling the latest innova...South Tyrol Free Software Conference
 

More from South Tyrol Free Software Conference (20)

SFSCON23 - Rufai Omowunmi Balogun - SMODEX – a Python package for understandi...
SFSCON23 - Rufai Omowunmi Balogun - SMODEX – a Python package for understandi...SFSCON23 - Rufai Omowunmi Balogun - SMODEX – a Python package for understandi...
SFSCON23 - Rufai Omowunmi Balogun - SMODEX – a Python package for understandi...
 
SFSCON23 - Roberto Innocenti - From the design to reality is here the Communi...
SFSCON23 - Roberto Innocenti - From the design to reality is here the Communi...SFSCON23 - Roberto Innocenti - From the design to reality is here the Communi...
SFSCON23 - Roberto Innocenti - From the design to reality is here the Communi...
 
SFSCON23 - Martin Rabanser - Real-time aeroplane tracking and the Open Data Hub
SFSCON23 - Martin Rabanser - Real-time aeroplane tracking and the Open Data HubSFSCON23 - Martin Rabanser - Real-time aeroplane tracking and the Open Data Hub
SFSCON23 - Martin Rabanser - Real-time aeroplane tracking and the Open Data Hub
 
SFSCON23 - Marianna d'Atri Enrico Zanardo - How can Blockchain technologies i...
SFSCON23 - Marianna d'Atri Enrico Zanardo - How can Blockchain technologies i...SFSCON23 - Marianna d'Atri Enrico Zanardo - How can Blockchain technologies i...
SFSCON23 - Marianna d'Atri Enrico Zanardo - How can Blockchain technologies i...
 
SFSCON23 - Lucas Lasota - The Future of Connectivity, Open Internet and Human...
SFSCON23 - Lucas Lasota - The Future of Connectivity, Open Internet and Human...SFSCON23 - Lucas Lasota - The Future of Connectivity, Open Internet and Human...
SFSCON23 - Lucas Lasota - The Future of Connectivity, Open Internet and Human...
 
SFSCON23 - Giovanni Giannotta - Intelligent Decision Support System for trace...
SFSCON23 - Giovanni Giannotta - Intelligent Decision Support System for trace...SFSCON23 - Giovanni Giannotta - Intelligent Decision Support System for trace...
SFSCON23 - Giovanni Giannotta - Intelligent Decision Support System for trace...
 
SFSCON23 - Elena Maines - Embracing CI/CD workflows for building ETL pipelines
SFSCON23 - Elena Maines - Embracing CI/CD workflows for building ETL pipelinesSFSCON23 - Elena Maines - Embracing CI/CD workflows for building ETL pipelines
SFSCON23 - Elena Maines - Embracing CI/CD workflows for building ETL pipelines
 
SFSCON23 - Christian Busse - Free Software and Open Science
SFSCON23 - Christian Busse - Free Software and Open ScienceSFSCON23 - Christian Busse - Free Software and Open Science
SFSCON23 - Christian Busse - Free Software and Open Science
 
SFSCON23 - Charles H. Schulz - Why open digital infrastructure matters
SFSCON23 - Charles H. Schulz - Why open digital infrastructure mattersSFSCON23 - Charles H. Schulz - Why open digital infrastructure matters
SFSCON23 - Charles H. Schulz - Why open digital infrastructure matters
 
SFSCON23 - Andrea Vianello - Achieving FAIRness with EDP-portal
SFSCON23 - Andrea Vianello - Achieving FAIRness with EDP-portalSFSCON23 - Andrea Vianello - Achieving FAIRness with EDP-portal
SFSCON23 - Andrea Vianello - Achieving FAIRness with EDP-portal
 
SFSCON23 - Thomas Aichner - How IoT and AI are revolutionizing Mass Customiza...
SFSCON23 - Thomas Aichner - How IoT and AI are revolutionizing Mass Customiza...SFSCON23 - Thomas Aichner - How IoT and AI are revolutionizing Mass Customiza...
SFSCON23 - Thomas Aichner - How IoT and AI are revolutionizing Mass Customiza...
 
SFSCON23 - Stefan Mutschlechner - Smart Werke Meran
SFSCON23 - Stefan Mutschlechner - Smart Werke MeranSFSCON23 - Stefan Mutschlechner - Smart Werke Meran
SFSCON23 - Stefan Mutschlechner - Smart Werke Meran
 
SFSCON23 - Mirko Boehm - European regulators cast their eyes on maturing OSS ...
SFSCON23 - Mirko Boehm - European regulators cast their eyes on maturing OSS ...SFSCON23 - Mirko Boehm - European regulators cast their eyes on maturing OSS ...
SFSCON23 - Mirko Boehm - European regulators cast their eyes on maturing OSS ...
 
SFSCON23 - Marco Pavanelli - Monitoring the fleet of Sasa with free software
SFSCON23 - Marco Pavanelli - Monitoring the fleet of Sasa with free softwareSFSCON23 - Marco Pavanelli - Monitoring the fleet of Sasa with free software
SFSCON23 - Marco Pavanelli - Monitoring the fleet of Sasa with free software
 
SFSCON23 - Marco Cortella - KNOWAGE and AICS for 2030 agenda SDG goals monito...
SFSCON23 - Marco Cortella - KNOWAGE and AICS for 2030 agenda SDG goals monito...SFSCON23 - Marco Cortella - KNOWAGE and AICS for 2030 agenda SDG goals monito...
SFSCON23 - Marco Cortella - KNOWAGE and AICS for 2030 agenda SDG goals monito...
 
SFSCON23 - Lina Ceballos - Interoperable Europe Act - A real game changer
SFSCON23 - Lina Ceballos - Interoperable Europe Act - A real game changerSFSCON23 - Lina Ceballos - Interoperable Europe Act - A real game changer
SFSCON23 - Lina Ceballos - Interoperable Europe Act - A real game changer
 
SFSCON23 - Johannes Näder Linus Sehn - Let’s monitor implementation of Free S...
SFSCON23 - Johannes Näder Linus Sehn - Let’s monitor implementation of Free S...SFSCON23 - Johannes Näder Linus Sehn - Let’s monitor implementation of Free S...
SFSCON23 - Johannes Näder Linus Sehn - Let’s monitor implementation of Free S...
 
SFSCON23 - Gabriel Ku Wei Bin - Why Do We Need A Next Generation Internet
SFSCON23 - Gabriel Ku Wei Bin - Why Do We Need A Next Generation InternetSFSCON23 - Gabriel Ku Wei Bin - Why Do We Need A Next Generation Internet
SFSCON23 - Gabriel Ku Wei Bin - Why Do We Need A Next Generation Internet
 
SFSCON23 - Edoardo Scepi - The Brand-New Version of IGis Maps
SFSCON23 - Edoardo Scepi - The Brand-New Version of IGis MapsSFSCON23 - Edoardo Scepi - The Brand-New Version of IGis Maps
SFSCON23 - Edoardo Scepi - The Brand-New Version of IGis Maps
 
SFSCON23 - Davide Vernassa - Empowering Insights Unveiling the latest innova...
SFSCON23 - Davide Vernassa - Empowering Insights  Unveiling the latest innova...SFSCON23 - Davide Vernassa - Empowering Insights  Unveiling the latest innova...
SFSCON23 - Davide Vernassa - Empowering Insights Unveiling the latest innova...
 

Recently uploaded

Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsNathaniel Shimoni
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxLoriGlavin3
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfNeo4j
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rick Flair
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Hiroshi SHIBATA
 
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...panagenda
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersNicole Novielli
 
Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesKari Kakkonen
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentPim van der Noll
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPathCommunity
 
A Framework for Development in the AI Age
A Framework for Development in the AI AgeA Framework for Development in the AI Age
A Framework for Development in the AI AgeCprime
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity PlanDatabarracks
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc
 
Data governance with Unity Catalog Presentation
Data governance with Unity Catalog PresentationData governance with Unity Catalog Presentation
Data governance with Unity Catalog PresentationKnoldus Inc.
 
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...Wes McKinney
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxLoriGlavin3
 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterMydbops
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxLoriGlavin3
 
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...AliaaTarek5
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxLoriGlavin3
 

Recently uploaded (20)

Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directions
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdf
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024
 
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software Developers
 
Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examples
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to Hero
 
A Framework for Development in the AI Age
A Framework for Development in the AI AgeA Framework for Development in the AI Age
A Framework for Development in the AI Age
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity Plan
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
 
Data governance with Unity Catalog Presentation
Data governance with Unity Catalog PresentationData governance with Unity Catalog Presentation
Data governance with Unity Catalog Presentation
 
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL Router
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
 
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
 

Open Data Hub - Matteo Roggia - Animeshon - Real-time Open Data Hub.pdf

  • 1. Open Data Hub ARCHITECTURE DESIGN BY ANIMESHON
  • 2. 01 – Open Data Hub 02 – Technologies 03 – A new way to communicate 04 – Reactivity 05 – Plug & Research 00 – Introduction AGEND A 06 – Conclusion
  • 4. Like to a post send notification to users weather forecast model improve future forecasts notify consumers recommendation engine recommend ads update feed update profiling information Resolve Captcha IoT sensor sending pressure feedback to AI models protect from unauthorized actors collect data for models training
  • 5. Open Data hub A modern system which already serves thousands of citizens while processing data from dozens of providers Open Data Hub v2 A system designed to meet future requirements while reaching more people with more and faster information State of the art technologies. International standards enforcement
  • 8. distributed microservices Open Data Hub is a cluster of countless microservices single responsibility Each service has only one responsibility which can be carried out in the most efficient way
  • 9. event-driven async communication The subprocess chain communicates asynchronously, making each component independent from the others service self-healing Events are thrown in the system, and the consumer can read them when they are ready. If something goes down, events are preserved until it comes back online agnostic communication Services do not know who will be the next process to take control, they just inform the system about their own results
  • 10. zero-downtime no data loss Once an information enters the system, it will be preserved and backed up to ensure data integrity and data durability update rollout Service orchestration allows the team to rollout updates with different strategies zero or minimal downtime fault tolerant Even if multiple services are offline, the system can still accept data and serve results
  • 12. no vendor lock-in All technologies are provider agnostic and can be migrated to any private or public cloud provider open source All technologies are open source easy to contribute Open Data Hub’s public repository allows anyone to start a local cluster of the core architecture and start contributing
  • 13. anyone can locally instantiate an Open Data Hub cluster in just 20 minutes and start contributing to the project https://github.com/noi-techpark/odh-infrastructure-v2
  • 14. 02 A NEW WAY TO COMMUNICATE
  • 15. Open Data Hub uses a standard communication protocol to both serve and process incoming data in alignment with internet 2.0 common practices standard communication protocol Speaking a common language makes it more appealing for third parties to integrate with Open Data Hub easy to integrate A more structured and standardized communication protocol allows for a reduction of custom solutions and more precise specifications open specifications
  • 16. RestApi Open Data Hub enforces communication standards with providers. MQTT Websocket
  • 18. easy to maintain Changing the order of the blocks, or substitute some of them will not affect the rest of the system The whole architecture is composed by small lego-blocks modular design
  • 19. All lego-blocks can be rearranged in whatever form to satisfy new requirements adaptability The system is designed to react to future needs and evolve evolution by design
  • 21. Open Data Hub allows for isolated environments to utilize historical raw data Thanks to no-code tools researches could create data processing pipeline without IT knowledge Allowing institutions to experiment with both processed and raw data enables an exponential grow of the system itself sandbox semplicity grow
  • 23. Open Data Hub promotes open source by adopting only open technologies while publishing it’s core infrastructure open State-of-the-art technologies and modern paradigms ensure an easy maintenance and integrability innovative Designed to meet any present and future requirement scalable