SlideShare a Scribd company logo
1 of 6
Download to read offline
Francesco Gallo Heiko Maus
EURIX DFKI
Demo of integrated synergetic
preservation workflow
ForgetIT 1st Review Meeting, April 29-30, 2014
Kaiserslautern, Germany
ForgetIT Project GA600826, 1st Review Meeting, Kaiserslautern, April 2014
PoF Framework: Architecture Components
middlewareapplications archive
cloud storage
ForgetIT Project GA600826, 1st Review Meeting, Kaiserslautern, April 2014
PoF Middleware: Message Oriented Middleware
© F. Munz, Middleware and Cloud Computing, 2011
© D.A. Chappel, Enterprise Service Bus, 2004
Preservation event triggered by the Active System
Request sent to PoF Middleware (REST API)
Synergetic Preservation Worklow is triggered
●
Scheduler creates a new job, initial message sent
Resource retrieved from the Active System CMIS server
●
Collector processes message, retrieves resource from CMIS server
●
ID Manager generates a new ID
●
Collector stores retrieved resource on the Middleware server
●
Flow control returned to Scheduler, new messages sent
ForgetIT Project GA600826, 1st Review Meeting, Kaiserslautern, April 2014
Preliminary integrated end-to-end workflow
Content processing
●
Extractor processes messages for image analysis (QA and concept
detection)
●
Contextualizer processes messages for text contextualization (mentions
and context)
●
Flow control returned to Scheduler, results messages sent
SIP preparation
●
Archiver processes message to prepare SIP, with all resources and MD
SIP ingest
●
Archiver processes message to ingest SIP into AIS
●
ID Manager updates IDs adding AIP ID
ForgetIT Project GA600826, 1st Review Meeting, Kaiserslautern, April 2014
Preliminary integrated end-to-end workflow - 2
AIP store
●
AIS Packager stores AIP into PDS
●
ID Manager updates IDs adding PDS ID
●
PDS triggers Storlets according to rules
Resource status update
●
The resource is shown as “preserved” in the Active System
Resource Restore Workflow
●
Active System can send request to PoF Middleware (REST API)
●
Scheduler creates a new job, initial message sent
●
Collector retrieves DIP, containing original resource (or new one after
transformation by PDS), which is returned to Active System
ForgetIT Project GA600826, 1st Review Meeting, Kaiserslautern, April 2014
Preliminary integrated end-to-end workflow - 3

More Related Content

What's hot

NSW Office of Environment and Heritage - CKAN Meetup
NSW Office of Environment and Heritage - CKAN MeetupNSW Office of Environment and Heritage - CKAN Meetup
NSW Office of Environment and Heritage - CKAN MeetupSteven De Costa
 
Monitoring docker, k8s and your applications with the elastic stack
Monitoring docker, k8s and your applications with the elastic stackMonitoring docker, k8s and your applications with the elastic stack
Monitoring docker, k8s and your applications with the elastic stackSmartWave
 
Logging using ELK Stack for Microservices
Logging using ELK Stack for MicroservicesLogging using ELK Stack for Microservices
Logging using ELK Stack for MicroservicesVineet Sabharwal
 
Material property data
Material property dataMaterial property data
Material property datapleasename1
 

What's hot (7)

DBOps
DBOpsDBOps
DBOps
 
NSW Office of Environment and Heritage - CKAN Meetup
NSW Office of Environment and Heritage - CKAN MeetupNSW Office of Environment and Heritage - CKAN Meetup
NSW Office of Environment and Heritage - CKAN Meetup
 
Monitoring docker, k8s and your applications with the elastic stack
Monitoring docker, k8s and your applications with the elastic stackMonitoring docker, k8s and your applications with the elastic stack
Monitoring docker, k8s and your applications with the elastic stack
 
Logging using ELK Stack for Microservices
Logging using ELK Stack for MicroservicesLogging using ELK Stack for Microservices
Logging using ELK Stack for Microservices
 
Arcadia overview nr2
Arcadia overview nr2Arcadia overview nr2
Arcadia overview nr2
 
Sprint 83
Sprint 83Sprint 83
Sprint 83
 
Material property data
Material property dataMaterial property data
Material property data
 

Viewers also liked

Organizational Preservation (WP10 ForgetIT 1st year review)
Organizational Preservation (WP10 ForgetIT 1st year review)Organizational Preservation (WP10 ForgetIT 1st year review)
Organizational Preservation (WP10 ForgetIT 1st year review)ForgetIT Project
 
Personal Preservation (WP9 ForgetIT 1st year review)
Personal Preservation (WP9 ForgetIT 1st year review)Personal Preservation (WP9 ForgetIT 1st year review)
Personal Preservation (WP9 ForgetIT 1st year review)ForgetIT Project
 
Contextualization / Decontextualization (WP6 ForgetIT 1st year review)
Contextualization / Decontextualization (WP6 ForgetIT 1st year review)Contextualization / Decontextualization (WP6 ForgetIT 1st year review)
Contextualization / Decontextualization (WP6 ForgetIT 1st year review)ForgetIT Project
 
Computational Storage Services (WP7 ForgetIT 1st year review)
Computational Storage Services (WP7 ForgetIT 1st year review)Computational Storage Services (WP7 ForgetIT 1st year review)
Computational Storage Services (WP7 ForgetIT 1st year review)ForgetIT Project
 
Managed Forgetting (WP3 - ForgetIT 1st year review)
Managed Forgetting (WP3 - ForgetIT 1st year review)Managed Forgetting (WP3 - ForgetIT 1st year review)
Managed Forgetting (WP3 - ForgetIT 1st year review)ForgetIT Project
 
Information Consolidation and Concentration (WP4 ForgetIT 1st year review)
Information Consolidation and Concentration (WP4 ForgetIT 1st year review)Information Consolidation and Concentration (WP4 ForgetIT 1st year review)
Information Consolidation and Concentration (WP4 ForgetIT 1st year review)ForgetIT Project
 
Joint Information and Preservation Management (WP5 ForgetIT 1st year review)
Joint Information and Preservation Management (WP5 ForgetIT 1st year review)Joint Information and Preservation Management (WP5 ForgetIT 1st year review)
Joint Information and Preservation Management (WP5 ForgetIT 1st year review)ForgetIT Project
 
The Preserve-or-Forget Reference Model and Framework (WP8 ForgetIT 1st year r...
The Preserve-or-Forget Reference Model and Framework (WP8 ForgetIT 1st year r...The Preserve-or-Forget Reference Model and Framework (WP8 ForgetIT 1st year r...
The Preserve-or-Forget Reference Model and Framework (WP8 ForgetIT 1st year r...ForgetIT Project
 
Foundations of Forgetting and Remembering (WP2 - ForgetIT 1st year review)
Foundations of Forgetting and Remembering (WP2 - ForgetIT 1st year review)Foundations of Forgetting and Remembering (WP2 - ForgetIT 1st year review)
Foundations of Forgetting and Remembering (WP2 - ForgetIT 1st year review)ForgetIT Project
 

Viewers also liked (13)

IMS
IMSIMS
IMS
 
Pengetahuan lingkungan
Pengetahuan lingkunganPengetahuan lingkungan
Pengetahuan lingkungan
 
Daily routine (1)
Daily routine (1)Daily routine (1)
Daily routine (1)
 
Organizational Preservation (WP10 ForgetIT 1st year review)
Organizational Preservation (WP10 ForgetIT 1st year review)Organizational Preservation (WP10 ForgetIT 1st year review)
Organizational Preservation (WP10 ForgetIT 1st year review)
 
Personal Preservation (WP9 ForgetIT 1st year review)
Personal Preservation (WP9 ForgetIT 1st year review)Personal Preservation (WP9 ForgetIT 1st year review)
Personal Preservation (WP9 ForgetIT 1st year review)
 
Contextualization / Decontextualization (WP6 ForgetIT 1st year review)
Contextualization / Decontextualization (WP6 ForgetIT 1st year review)Contextualization / Decontextualization (WP6 ForgetIT 1st year review)
Contextualization / Decontextualization (WP6 ForgetIT 1st year review)
 
Computational Storage Services (WP7 ForgetIT 1st year review)
Computational Storage Services (WP7 ForgetIT 1st year review)Computational Storage Services (WP7 ForgetIT 1st year review)
Computational Storage Services (WP7 ForgetIT 1st year review)
 
Managed Forgetting (WP3 - ForgetIT 1st year review)
Managed Forgetting (WP3 - ForgetIT 1st year review)Managed Forgetting (WP3 - ForgetIT 1st year review)
Managed Forgetting (WP3 - ForgetIT 1st year review)
 
Information Consolidation and Concentration (WP4 ForgetIT 1st year review)
Information Consolidation and Concentration (WP4 ForgetIT 1st year review)Information Consolidation and Concentration (WP4 ForgetIT 1st year review)
Information Consolidation and Concentration (WP4 ForgetIT 1st year review)
 
Joint Information and Preservation Management (WP5 ForgetIT 1st year review)
Joint Information and Preservation Management (WP5 ForgetIT 1st year review)Joint Information and Preservation Management (WP5 ForgetIT 1st year review)
Joint Information and Preservation Management (WP5 ForgetIT 1st year review)
 
The Preserve-or-Forget Reference Model and Framework (WP8 ForgetIT 1st year r...
The Preserve-or-Forget Reference Model and Framework (WP8 ForgetIT 1st year r...The Preserve-or-Forget Reference Model and Framework (WP8 ForgetIT 1st year r...
The Preserve-or-Forget Reference Model and Framework (WP8 ForgetIT 1st year r...
 
English activity
English activityEnglish activity
English activity
 
Foundations of Forgetting and Remembering (WP2 - ForgetIT 1st year review)
Foundations of Forgetting and Remembering (WP2 - ForgetIT 1st year review)Foundations of Forgetting and Remembering (WP2 - ForgetIT 1st year review)
Foundations of Forgetting and Remembering (WP2 - ForgetIT 1st year review)
 

Similar to Demo of integrated synergetic preservation workflow (WP8 ForgetIT 1st year review)

Combinação de logs, métricas e rastreamentos para observabilidade unificada
Combinação de logs, métricas e rastreamentos para observabilidade unificadaCombinação de logs, métricas e rastreamentos para observabilidade unificada
Combinação de logs, métricas e rastreamentos para observabilidade unificadaElasticsearch
 
Delivering Cloud Native Batch Solutions - Dodd Pfeffer
Delivering Cloud Native Batch Solutions - Dodd PfefferDelivering Cloud Native Batch Solutions - Dodd Pfeffer
Delivering Cloud Native Batch Solutions - Dodd PfefferVMware Tanzu
 
Pivotal Cloud Foundry 2.0: First Look
Pivotal Cloud Foundry 2.0: First LookPivotal Cloud Foundry 2.0: First Look
Pivotal Cloud Foundry 2.0: First LookVMware Tanzu
 
openstackreferencearchitecturewhitepaper
openstackreferencearchitecturewhitepaperopenstackreferencearchitecturewhitepaper
openstackreferencearchitecturewhitepaperRichard Haigh
 
Pivotal Cloud Foundry 2.6: A First Look
Pivotal Cloud Foundry 2.6: A First LookPivotal Cloud Foundry 2.6: A First Look
Pivotal Cloud Foundry 2.6: A First LookVMware Tanzu
 
PERICLES Process Compiler - ‘Eye of the Storm: Preserving Digital Content in ...
PERICLES Process Compiler - ‘Eye of the Storm: Preserving Digital Content in ...PERICLES Process Compiler - ‘Eye of the Storm: Preserving Digital Content in ...
PERICLES Process Compiler - ‘Eye of the Storm: Preserving Digital Content in ...PERICLES_FP7
 
Building 12-factor Cloud Native Microservices
Building 12-factor Cloud Native MicroservicesBuilding 12-factor Cloud Native Microservices
Building 12-factor Cloud Native MicroservicesJakarta_EE
 
SC'18 BoF Presentation
SC'18 BoF PresentationSC'18 BoF Presentation
SC'18 BoF Presentationrcastain
 
Parallelizing Large Simulations with Apache SparkR with Daniel Jeavons and Wa...
Parallelizing Large Simulations with Apache SparkR with Daniel Jeavons and Wa...Parallelizing Large Simulations with Apache SparkR with Daniel Jeavons and Wa...
Parallelizing Large Simulations with Apache SparkR with Daniel Jeavons and Wa...Spark Summit
 
Combinación de logs, métricas y seguimiento para una visibilidad centralizada
Combinación de logs, métricas y seguimiento para una visibilidad centralizadaCombinación de logs, métricas y seguimiento para una visibilidad centralizada
Combinación de logs, métricas y seguimiento para una visibilidad centralizadaElasticsearch
 
Adxis Product Overview
Adxis Product OverviewAdxis Product Overview
Adxis Product OverviewAndreas Wolf
 
Moving at the speed of startup with Pivotal Cloud Foundry 1.11
Moving at the speed of startup with Pivotal Cloud Foundry 1.11Moving at the speed of startup with Pivotal Cloud Foundry 1.11
Moving at the speed of startup with Pivotal Cloud Foundry 1.11VMware Tanzu
 
[Research] deploying predictive models with the actor framework - Brian Gawalt
[Research] deploying predictive models with the actor framework - Brian Gawalt[Research] deploying predictive models with the actor framework - Brian Gawalt
[Research] deploying predictive models with the actor framework - Brian GawaltPAPIs.io
 
HiPEAC 2019 Tutorial - Maestro RTOS
HiPEAC 2019 Tutorial - Maestro RTOSHiPEAC 2019 Tutorial - Maestro RTOS
HiPEAC 2019 Tutorial - Maestro RTOSTulipp. Eu
 
Ceilometer presentation ods havana final - published
Ceilometer presentation ods havana   final - publishedCeilometer presentation ods havana   final - published
Ceilometer presentation ods havana final - publishedeNovance
 
Fusepool Machine Learning Framework
Fusepool Machine Learning FrameworkFusepool Machine Learning Framework
Fusepool Machine Learning FrameworkFusepool SME project
 
Spring and Pivotal Application Service - SpringOne Tour - Boston
Spring and Pivotal Application Service - SpringOne Tour - BostonSpring and Pivotal Application Service - SpringOne Tour - Boston
Spring and Pivotal Application Service - SpringOne Tour - BostonVMware Tanzu
 

Similar to Demo of integrated synergetic preservation workflow (WP8 ForgetIT 1st year review) (20)

PREMIS in METS in Archivematica
PREMIS in METS in ArchivematicaPREMIS in METS in Archivematica
PREMIS in METS in Archivematica
 
Combinação de logs, métricas e rastreamentos para observabilidade unificada
Combinação de logs, métricas e rastreamentos para observabilidade unificadaCombinação de logs, métricas e rastreamentos para observabilidade unificada
Combinação de logs, métricas e rastreamentos para observabilidade unificada
 
Delivering Cloud Native Batch Solutions - Dodd Pfeffer
Delivering Cloud Native Batch Solutions - Dodd PfefferDelivering Cloud Native Batch Solutions - Dodd Pfeffer
Delivering Cloud Native Batch Solutions - Dodd Pfeffer
 
Pivotal Cloud Foundry 2.0: First Look
Pivotal Cloud Foundry 2.0: First LookPivotal Cloud Foundry 2.0: First Look
Pivotal Cloud Foundry 2.0: First Look
 
MAVRL Workshop 2014 - Python Materials Genomics (pymatgen)
MAVRL Workshop 2014 - Python Materials Genomics (pymatgen)MAVRL Workshop 2014 - Python Materials Genomics (pymatgen)
MAVRL Workshop 2014 - Python Materials Genomics (pymatgen)
 
openstackreferencearchitecturewhitepaper
openstackreferencearchitecturewhitepaperopenstackreferencearchitecturewhitepaper
openstackreferencearchitecturewhitepaper
 
Fg v1r1
Fg v1r1Fg v1r1
Fg v1r1
 
Pivotal Cloud Foundry 2.6: A First Look
Pivotal Cloud Foundry 2.6: A First LookPivotal Cloud Foundry 2.6: A First Look
Pivotal Cloud Foundry 2.6: A First Look
 
PERICLES Process Compiler - ‘Eye of the Storm: Preserving Digital Content in ...
PERICLES Process Compiler - ‘Eye of the Storm: Preserving Digital Content in ...PERICLES Process Compiler - ‘Eye of the Storm: Preserving Digital Content in ...
PERICLES Process Compiler - ‘Eye of the Storm: Preserving Digital Content in ...
 
Building 12-factor Cloud Native Microservices
Building 12-factor Cloud Native MicroservicesBuilding 12-factor Cloud Native Microservices
Building 12-factor Cloud Native Microservices
 
SC'18 BoF Presentation
SC'18 BoF PresentationSC'18 BoF Presentation
SC'18 BoF Presentation
 
Parallelizing Large Simulations with Apache SparkR with Daniel Jeavons and Wa...
Parallelizing Large Simulations with Apache SparkR with Daniel Jeavons and Wa...Parallelizing Large Simulations with Apache SparkR with Daniel Jeavons and Wa...
Parallelizing Large Simulations with Apache SparkR with Daniel Jeavons and Wa...
 
Combinación de logs, métricas y seguimiento para una visibilidad centralizada
Combinación de logs, métricas y seguimiento para una visibilidad centralizadaCombinación de logs, métricas y seguimiento para una visibilidad centralizada
Combinación de logs, métricas y seguimiento para una visibilidad centralizada
 
Adxis Product Overview
Adxis Product OverviewAdxis Product Overview
Adxis Product Overview
 
Moving at the speed of startup with Pivotal Cloud Foundry 1.11
Moving at the speed of startup with Pivotal Cloud Foundry 1.11Moving at the speed of startup with Pivotal Cloud Foundry 1.11
Moving at the speed of startup with Pivotal Cloud Foundry 1.11
 
[Research] deploying predictive models with the actor framework - Brian Gawalt
[Research] deploying predictive models with the actor framework - Brian Gawalt[Research] deploying predictive models with the actor framework - Brian Gawalt
[Research] deploying predictive models with the actor framework - Brian Gawalt
 
HiPEAC 2019 Tutorial - Maestro RTOS
HiPEAC 2019 Tutorial - Maestro RTOSHiPEAC 2019 Tutorial - Maestro RTOS
HiPEAC 2019 Tutorial - Maestro RTOS
 
Ceilometer presentation ods havana final - published
Ceilometer presentation ods havana   final - publishedCeilometer presentation ods havana   final - published
Ceilometer presentation ods havana final - published
 
Fusepool Machine Learning Framework
Fusepool Machine Learning FrameworkFusepool Machine Learning Framework
Fusepool Machine Learning Framework
 
Spring and Pivotal Application Service - SpringOne Tour - Boston
Spring and Pivotal Application Service - SpringOne Tour - BostonSpring and Pivotal Application Service - SpringOne Tour - Boston
Spring and Pivotal Application Service - SpringOne Tour - Boston
 

Demo of integrated synergetic preservation workflow (WP8 ForgetIT 1st year review)

  • 1. Francesco Gallo Heiko Maus EURIX DFKI Demo of integrated synergetic preservation workflow ForgetIT 1st Review Meeting, April 29-30, 2014 Kaiserslautern, Germany
  • 2. ForgetIT Project GA600826, 1st Review Meeting, Kaiserslautern, April 2014 PoF Framework: Architecture Components middlewareapplications archive cloud storage
  • 3. ForgetIT Project GA600826, 1st Review Meeting, Kaiserslautern, April 2014 PoF Middleware: Message Oriented Middleware © F. Munz, Middleware and Cloud Computing, 2011 © D.A. Chappel, Enterprise Service Bus, 2004
  • 4. Preservation event triggered by the Active System Request sent to PoF Middleware (REST API) Synergetic Preservation Worklow is triggered ● Scheduler creates a new job, initial message sent Resource retrieved from the Active System CMIS server ● Collector processes message, retrieves resource from CMIS server ● ID Manager generates a new ID ● Collector stores retrieved resource on the Middleware server ● Flow control returned to Scheduler, new messages sent ForgetIT Project GA600826, 1st Review Meeting, Kaiserslautern, April 2014 Preliminary integrated end-to-end workflow
  • 5. Content processing ● Extractor processes messages for image analysis (QA and concept detection) ● Contextualizer processes messages for text contextualization (mentions and context) ● Flow control returned to Scheduler, results messages sent SIP preparation ● Archiver processes message to prepare SIP, with all resources and MD SIP ingest ● Archiver processes message to ingest SIP into AIS ● ID Manager updates IDs adding AIP ID ForgetIT Project GA600826, 1st Review Meeting, Kaiserslautern, April 2014 Preliminary integrated end-to-end workflow - 2
  • 6. AIP store ● AIS Packager stores AIP into PDS ● ID Manager updates IDs adding PDS ID ● PDS triggers Storlets according to rules Resource status update ● The resource is shown as “preserved” in the Active System Resource Restore Workflow ● Active System can send request to PoF Middleware (REST API) ● Scheduler creates a new job, initial message sent ● Collector retrieves DIP, containing original resource (or new one after transformation by PDS), which is returned to Active System ForgetIT Project GA600826, 1st Review Meeting, Kaiserslautern, April 2014 Preliminary integrated end-to-end workflow - 3