Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
CASSANDRA Framework: A Service Oriented Distributed Multimedia                                   Content Analysis Engine  ...
systems, spanning from music genre analysis to                     easily integrate new pieces of software of unknown     ...
1 it uses just (1) although the services it accesses are                               This service-oriented organization ...
handling and time consuming message interpretation,                  approach. The framework has been tested and shown    ...
Upcoming SlideShare
Loading in …5

Cassandra framework a service oriented distributed multimedia


Published on

J Gabriel Lima -

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Cassandra framework a service oriented distributed multimedia

  1. 1. CASSANDRA Framework: A Service Oriented Distributed Multimedia Content Analysis Engine Fons de Lange, Jan Nesvadba, Johan Lukkien Philips Research Eindhoven, The Netherlands Eindhoven University of Technology Abstract consumer-driven personal content production stir consumers into the dilemma of multimedia retrieval Connected Consumer Electronics (CE) devices face and management, equivalent to the search of the needle an exponential growth in storage, processing in the haystack. capabilities and connectivity bandwidth. This As a matter of fact, petabytes of content distributed increased complexity calls for new software across a network of memory devices, similar to the architectures that naturally admit self-organization, incredible amount of memory scattered across the resource management for efficient workload human brain, are only manageable under the condition distribution, and a transparent cooperation of of (semantic) content- and to some extent also context connected CE-terminals. The interconnected nature awareness. Hence, content-awareness creation is of makes advanced applications feasible such as the utmost importance. Fortunately, the distributed, but smart distribution of complex multimedia content connected processing and memory faculties of CE analysis algorithms, resulting in automatic semantic networks provide sufficient computational resources to content-awareness creation. In this paper we describe perform the required multimedia content analysis and a modular, self-aware, self-organizing, real-time and to memorize the generated content-awareness-creating distributed multimedia content analysis system. Based on a Service Oriented Architecture it is, furthermore, content descriptors. It is, therefore, foreseeable, that capable of efficiently and dynamically using the such networks will efficiently share functionality, available resources in a grid-like manner. resources (memory, processing) and content such as to support distributed analysis applications. Hence, 1. Introduction connected CE devices will attract consumers soon through smartly interoperating with each other and Petabytes of storage, a million MIPS (Million through their integration with e.g. internet services Instructions Per Second) at an 1000-Euro-pricepoint providing intuitive, state-of-the-art capabilities and and several gigabits-per-second Internet-Protocol- applications, respectively. based (IP) connectivity are realistic technical capabilities of next decades Consumer Electronics The underlying required domain-specific audio-, (CE) networks [1]. But, next to the technology also music-, speech-, image- or video content analysis consumers’ behaviors evolve. Today’s consumers, for (expert) systems, each exploiting one sensorial signal example, gradually change from passive absorbers to in isolation, have been developed during the past active and often internet-community-attached decade by various independent expert teams. The multimedia content producers. Unfortunately, this majority, whereof, has been developed in isolation of results in a massive production of individual content- the other expert systems and for one specific unaware multimedia assets, made easy by ubiquitous application domain only, mainly because of capability and pervasive content creation CE devices such as constraints. Unfortunately, human-communication-like camera mobile phones, camcorders, video recorders content-awareness, enabling e.g. human-like search and PC software. Hence, the enormous memory queries, has to be based on smart combinations of resources scattered across distributed, but those orthogonal multimedia content analysis results as interconnected CE devices combined with the massive done by our human brains automatically. In order to acquisition of commercial content and the massive enable smart collaborations of those individual expertEight International Workshop on Image Analysis for Multimedia Interactive Services(WIAMIS07)0-7695-2818-X/07 $20.00 © 2007
  2. 2. systems, spanning from music genre analysis to easily integrate new pieces of software of unknown Automatic Speech Recognition (ASR) and from face quality. Our software model supports this by localization / identification to emotion analysis, they describing applications as the coordination of passive have to be embedded in a flexible, extendable, scalable services running on devices in the network. In order to and upgradeable system [2]. Flexibility, extendibility explain this we first look at more traditional ways to and scalability are guaranteed through a rigid modular look at distributed applications. approach, wherein each individual expert system (e.g. Figure 1 shows a simplified view of a running an individual analysis component) is encapsulated into program, which we call a component. It has four a functional unit, further called Service Unit (SU), with dependencies: the operating system platform (4), the clearly defined and standardized Input / Output (I/O) user interface (3), services provided (2) and services interfaces. The latter guarantee hassle-free obtained from the network (1). upgradeability of individual SU expert systems through Graphics simple replacement of the involved analysis User Interface algorithms, provided that metadata I/O formats do not change. 3: User Interactions 2: Provided The complex nature of such a system, i.e. a 1: Required Services Component (Application) Services (to network) distributed service-oriented analysis engine hosting a (from network) multitude of disciplinary-orthogonal analysis 4: Platform Dependencies algorithms, has to provide ease-of-use, robustness and interoperability. Platform Particularly, the integration of know-how provided (Operating System) by various disciplinary-orthogonal expert teams and the management of such a system should be time and Figure 1. Basic model of network component. effort efficient. Each component has dependencies (4); a typical In this paper we describe a distributed multimedia client application has dependencies (1) and (3); a content analysis prototyping framework, built typical server has dependency (2) and a stand-alone according to service oriented principles [3], which application has just (3). In our approach we use mainly allows algorithm expert teams to efficiently and two types of components. First, the ones with transparently integrate their algorithms into the system, but also to effortlessly upgrade (replace) them and, dependencies (1) and (2), i.e., components that provide finally, to co-develop combined joint solutions. This services to the network while possibly using services framework also serves as a demonstration of themselves. These services are made available on the automatically cooperating devices. The next section network through interfaces comprising function calls, elaborates the service concepts in more detail. eventing and connection points (called pins) for streaming data. In our system, called the Cassandra 2. The service oriented architecture Framework, we call these components Service Units (SU). Each SU can be run on a particular device, called 2.1 Global Structure a Node, while nodes may run several components and component instances at the same time. Our aim is to develop a software architecture that facilitates the composition of applications from Given a set of available passive services there is the distributed components. The platform of our question where the control lies. This control pertains to applications consists of a network of cooperating establishing connections between services as well as devices such as personal computers, consumer the control flow within a set of connected services. In electronics equipments as well as mobile terminals a regular client-server application both types of control such as telephones. Hence, our applications are usually lie at the client. The second type of component naturally distributed over multiple devices and must be in the Cassandra Framework is therefore a coordinator designed such that they can function in a changing engine. The task of the coordinator is to setup a environment. The internal software of devices comes collection of services according to a given description from different manufacturers and we must be able to and to connect them accordingly. This is done a.o. through dynamic service discovery. In terms of FigureEight International Workshop on Image Analysis for Multimedia Interactive Services(WIAMIS07)0-7695-2818-X/07 $20.00 © 2007
  3. 3. 1 it uses just (1) although the services it accesses are This service-oriented organization of the system not for its own use. Instead it instructs services to build introduces significant benefits. connections between them coupling data pins as well First, by separating composition and configuration as control interfaces. The control flow is then entirely entirely from the services themselves we postpone the defined by this connectivity pattern and the behavior of decision how to use the services. In fact, by improving the services. In many cases, a simple data flow is used our coordination software we improve the operation of but more advanced behavior is possible as well. Note the system without having to change the services. This that the coordinator is actually not a part of the concept is used in the robustness. The services have a application; after setting up the network state it might separate interface supporting error monitoring and as well disappear. In the framework we call the error reporting; a monitor in the network may use this coordinator the use-case manager and a composition of interface to improve the robustness [5]. However, this services a use-case. is not required and the system will function without it. Second, the architecture makes it straightforward to For the purpose of setting up a collection of services, take decisions based on resource information and each Node runs an instance of the so-called Component performance indicators. In this way load-balancing Repository service unit. The Component Repository is decisions are made upon startup of a use case. In responsible for maintaining a cache of locally-available principle it is just as easy to do re-balancing when a components and keeping them synchronized with a new use case needs to be instantiated. Master Repository. This is done using UPnP Third, the integration of new functionality is just as networking [4], where the Component Repository of easy as deploying it as a service and exposing it to the each Node functions as a UPnP device, announcing it network. Although in practice a standard middleware presence on the network and where the Master approach will be used to do this, it is not required. The Component Repository Manager is a UPnP control system does not depend on any (OS-)platform or point, capable discovering all UPnP devices such as the language. The only dependence is through the Component Repository of each Node. A possible setup interfaces found on the network. Hence, the of services and service units is given in Figure 2 standardization is on the format and semantics of the below. exchanged messages. This leads to an approach of node component node building new functionality by extension and re- component composition rather than reworking existing software. repository service unit Fourth, the architecture supports the cooperation of component Use-case Master component start/stop devices without this pre-installed knowledge of the component component repository repository manager sync cooperation context, providing application and data browse get / change service unit (SU) SU SU transparency. The applications in our system (the use use-case configuration cases) are in fact scripts that coordinate the services. start/stop component node As such they could be executed anywhere in the service X start/stop Use-case manager component repository component system. component service unit SU SU SU 2.2 Communication styles and performance Services can be used through interfaces. Two styles Figure 2. Overall system architecture. of interfaces are used: control-type interfaces comprising function calls and eventing, and streaming The Component Repository starts or stops a type interfaces enabling only simple data pushing and component upon a request of the use-case manager. pulling. Compared to the simple data push-pull After starting a component, the Component Repository interface, the control-type interface is more expressive service keeps track of its execution lifetime. To this in terms of actions and data types, but has more end, the Master Repository maintains an up-to-date list performance overhead. In the Cassandra Framework, of all nodes and their configuration, i.e. the available the control-type communication between services is components on these nodes and the services that they based on UPnP, while the simple data push/pull implement (Figure 2). Note that the configuration may communication is based on TCP/IP. In both cases change dynamically, when components are added or communication is based on ports and IP addresses, but removed to the system. UPnP communication involves more complex protocolEight International Workshop on Image Analysis for Multimedia Interactive Services(WIAMIS07)0-7695-2818-X/07 $20.00 © 2007
  4. 4. handling and time consuming message interpretation, approach. The framework has been tested and shown which is not done in case of basic TCP/IP streaming effective by integrating components on different communication. operating systems and in different languages. The As an example, Figure 3 shows a commercial block current system works well because of over- detector consisting of several connected services. It dimensioning, particularly of the network. We want to explicitly shows the connection entities that take care study an approach in which Quality of Service of the network communication. In our system, these tradeoffs must be made. On the one hand this may lead can be UPnP device protocol stacks, TCP/IP sockets or to re-balancing from the perspective of the use-case a combination of both. Note that the figure does not manager. As an example, Figure 4 shows the Graphics User show the repository managers and nodes, since these Interface of the Cassandra Use-Case Manager for an are not relevant to the discussion. Components take example network of Service Units (SUs), realizing a care of setting up the UPnP device stack for control complex Content Analysis System. By means of this type communication and TCP/IP sockets for data GUI, networks can be defined, instantiated, executed, push/pull communication. controlled and monitored. component component GUI component Client audio channel bit rate silence select setting detect conn. client client SU connect connect connect conn. conn. conn. conn. conn. conn. AV time black frame Tuner heuristic Codec stamp frame repeat SU decider SU generator detect analyzer SU SU SU SU comp. comp. comp. comp. component Figure 3. Commercial block detect use-case, Figure 4. Use Case Manager GUI, showing showing network connection points. execution of complex network of SUs. The real-time nature of the stream processing poses an 10. References extra challenge for the software architecture. After a use case has been setup, the data that flows between [1] PC technology trends, connected services should not pass up and down a complicated software stack for interpretation. Instead, [2] CASSANDRA Project, after setup the data should flow smoothly between functional processing units making up the services. dra This can be hardware processing as well (e.g. an [3] Sayed Hashimi, Service Oriented Architecture encoder or decoder). One way to look at this is that the Explained, control part of the system is used to configure the hardware plane perpendicular to it. The simple data ined.html push/pull is the best choice for realizing such [4] UPnP framework, . communication. In Figure 3 the AV streams are best [5] C. Fetzer, M. Raynal, and F. Tronel. An adaptive failure communicated via sockets, while the Channel select detection protocol. Proc. of the 8th IEEE Pacific Rim Symp. and bit-rate-setting clients operate best via a (UPnP) on Dependable Computing. Seoul, Korea, 2001. control-type interface. The GUI may use a mix of both. 3. Conclusions We have presented the architecture of a robust distributed media processing framework that supports cooperation by composition through a Service OrientedEight International Workshop on Image Analysis for Multimedia Interactive Services(WIAMIS07)0-7695-2818-X/07 $20.00 © 2007