The MICO Broker: An Orchestration Framework for Linked Data Extractors

1. © Fraunhofer IDMT
Patrick Aichroth, Marcel Sieland, Luca Cuccovillo, Thomas Köllmer
The MICO Broker: An Orchestration Framework
for Linked Data Extractors
Fraunhofer IDMT
Thomas Köllmer
thomas.koellmer@idmt.fraunhofer.de

2. © Fraunhofer IDMT 2
The MICO Project
 http://www.mico-project.eu
 3.4 Million Euro Project
 7 Partners: Salzburg Research, Fraunhofer IDMT, Oxford University,
University of Passau, Umeå University, Zaizi, InsideOut10
 Duration: 36 Months (end: 10/2016)
 Goal of MICO is to develop a platform to analyse “media in context
” …
 ...by orchestrating different content extraction tools that can work
simultaneously or in a sequence and …..
 ...provide valuable metadata to third party applications.

3. © Fraunhofer IDMT 3
MICO and the MICO broker
MICO: EU project to develop a platform for distributed analysis of
multimedia content, based on Linked Data:
 media (textual, visual, audio) extractors
 persistence (binary + metadata/annotations)
 extractor orchestration (“broker”)
 common metadata model (MMM, Anno4J)
 cross-media querying (SPARQL/MM)
 cross-media reco (using all data sources, content-based + CF-based)

4. © Fraunhofer IDMT 4
MICO and the MICO broker

5. © Fraunhofer IDMT 5
An example workflow / pipeline

6. © Fraunhofer IDMT 6
An example workflow / pipeline

7. © Fraunhofer IDMT 7
An example workflow / pipeline

8. © Fraunhofer IDMT 8
 Initial broker version
 implemented on top of RabbitMQ
 accessible via MICO Event API (Java and C++)
 content item input queue: receiving new content
 content item reply-to queue: providing extractor results
 registry queue: extractor service registration
 Initial RabbitMQ approach proved to be quite stable and robust, but:
 orchestration via simple mime-types: unintended processing and loops
 Other features desired → new Broker version needed
Broker challenges and requirements (1/4)

9. © Fraunhofer IDMT 9
Broker challenges and requirements (2/4)
 Extractor dependencies and parameters
 support for extractor configuration
 support for extractor “modes”
 support for more than one extractor input and output
 support for extractor versioning
 Workflow creation
 support for extractor dependency checking
 simple workflow creation

10. © Fraunhofer IDMT 10
 Workflow execution
 support for several workflows per MICO instance
 support for error handling
 support for workflow status tracking and logging
 support for Enterprise Integration Pattern (EIP) support, e.g. routing,
aggregation, splitting
 support for dynamic routing for context-aware processing
Broker challenges and requirements (3/4)

11. © Fraunhofer IDMT 11
Broker challenges and requirements (4/4)

12. © Fraunhofer IDMT 12
 Extractor information is provided at different times (during vs. after packaging)
and by different people (dev, other dev, admin)
→ dedicated registration REST service
 Extractor I/O typing knows different layers:
 mime-types, e.g. image/png,
 syntactic types, e.g. image region,
 semantic types, e.g. face region
 All three must match!
New MICO broker: Design Choices (1/2)

13. © Fraunhofer IDMT 13
New MICO broker: Design Choices (2/2)
 Existing Linked Data sources and the MICO metadata model (MMM) to be
reused as far as possible; types should be identified via URIs
 Apache Camel for workflow execution: EIP and broad protocol support
 Support for semi-automatic workflow creation via dedicated GUI/App
 Support for admin feedback
 extractors performance
 pipeline performance
 Support for content sets, and assignment of content sets to workflows

14. © Fraunhofer IDMT 14
New MICO broker: components & overall process

15. © Fraunhofer IDMT 15
New MICO broker: data model

16. © Fraunhofer IDMT 16
New MICO broker: data model

17. © Fraunhofer IDMT 17
New MICO broker: semi-automatic workflow creation
 Manual workflow creation: complicated (depends on extractor
interdependencies and constraints, content, use case etc.)
 Now: Semi-automatic approach uses data model (mime types, syntactic
and semantic types)
 Output: Camel route

18. © Fraunhofer IDMT 18
Example Camel route

19. © Fraunhofer IDMT 19
Example Camel route

20. © Fraunhofer IDMT 20
New MICO broker: status and outlook
 Part of the final MICO release (late 2016)
 Current broker status:
 Mostly completed
 Partially integrated into the platform release
 Basic functionalities released under Apache OSS
 Future functionalities planned as premium features:
 Advanced semi-automatic workflow creation
 Advanced distributed process management
 Advanced security features

21. © Fraunhofer IDMT
Patrick Aichroth, Marcel Sieland, Luca Cuccovillo, Thomas Köllmer
The MICO Broker: An Orchestration Framework
for Linked Data Extractors
Fraunhofer IDMT
Ενχαριδτώ!
Questions?

22. © Fraunhofer IDMT
Patrick Aichroth, Marcel Sieland, Luca Cuccovillo, Thomas Köllmer
The MICO Broker: An Orchestration Framework
for Linked Data Extractors
Fraunhofer IDMT
Thank you!
Questions?

23. © Fraunhofer IDMT 23
New MICO broker: data model

24. © Fraunhofer IDMT 24
New MICO broker: data model

25. © Fraunhofer IDMT 25
New MICO broker: data model

26. © Fraunhofer IDMT 26
New MICO broker: data model