The document provides an overview of event processing, including its history, current state, and future trends. It discusses event processing concepts like events, patterns, context, and languages. It also outlines some challenges like ordering events in distributed systems and dealing with uncertainty. The future trends highlighted include expanding applications of event processing, diversifying platforms and quality of service needs, standardizing aspects of event processing, and making it more accessible to non-programmers.
Event processing systems have evolved from early reactive systems to now support predictive capabilities using techniques like machine learning. Key challenges for these systems include handling uncertain and incomplete event data, performing predictive event processing through graphical model techniques, leveraging machine learning to automate pattern discovery and modeling, and evolving from purely reactive systems to proactively take actions based on predictions. Addressing these challenges will help make event processing systems more intelligent and adaptive.
Debs 2010 context based computing tutorialOpher Etzion
The document discusses context-aware computing and its utilization in event-based systems. It provides an agenda for a tutorial that covers context in general, context categories including temporal, spatial, state and segmentation contexts, and context implementation in practice and event processing. The document also provides examples of different types of contexts and how context can be composed from multiple contexts.
How Events Are Reshaping Modern SystemsJonas Bonér
The document discusses how event-driven architecture and event-based design principles are reshaping modern systems. Some key points:
- Events increase autonomy, reduce risk, help systems move faster, improve loose coupling, stability, scalability, resilience and traceability.
- Modern architectures like cloud, microservices and distributed systems benefit from event-driven approaches. Events also help with data-centric applications and meeting increasing customer demands.
- High-level abstractions powered by message-driven fabrics are needed to address limitations of low-level event loops and callbacks. Event-driven services publish facts as events and use event streams for integration.
- Thinking in terms of immutable events and promises can help increase certainty
Brief introduction on CEP and Terminology
o Drools Vision
o Drools Fusion: Complex Event Processing extensions
o Event Declaration and Semantics
o Event Cloud, Streams and the Session Clock
o Temporal Reasoning
o Sliding Window Support
o Streams Support
o Memory Management
Complex Event Processing (CEP) involves detecting patterns in streams of event data. CEP tools analyze multiple simple events to identify complex events inferred from simpler ones. Typical applications of CEP include monitoring for business anomalies, detecting fraud or security threats. CEP augments service-oriented architectures by allowing services to trigger from events and generate new event streams. Event processing engines use techniques like filtering, windows, and correlation to detect patterns across events over time.
This document provides an overview of event processing, including:
1) Event processing involves performing operations on events and has roots in areas like active databases, data stream management systems, and network management.
2) Key concepts in event processing include events, event types, event producers, event consumers, and detecting patterns in events. Common applications are in system monitoring, complex event processing, and event-driven architectures.
3) Event processing has standard building blocks like event processing agents, event channels, and event processing networks that connect producers and consumers of events. Common operations include filtering, transforming, correlating, and detecting patterns in streams of events.
Event processing systems have evolved from early reactive systems to now support predictive capabilities using techniques like machine learning. Key challenges for these systems include handling uncertain and incomplete event data, performing predictive event processing through graphical model techniques, leveraging machine learning to automate pattern discovery and modeling, and evolving from purely reactive systems to proactively take actions based on predictions. Addressing these challenges will help make event processing systems more intelligent and adaptive.
Debs 2010 context based computing tutorialOpher Etzion
The document discusses context-aware computing and its utilization in event-based systems. It provides an agenda for a tutorial that covers context in general, context categories including temporal, spatial, state and segmentation contexts, and context implementation in practice and event processing. The document also provides examples of different types of contexts and how context can be composed from multiple contexts.
How Events Are Reshaping Modern SystemsJonas Bonér
The document discusses how event-driven architecture and event-based design principles are reshaping modern systems. Some key points:
- Events increase autonomy, reduce risk, help systems move faster, improve loose coupling, stability, scalability, resilience and traceability.
- Modern architectures like cloud, microservices and distributed systems benefit from event-driven approaches. Events also help with data-centric applications and meeting increasing customer demands.
- High-level abstractions powered by message-driven fabrics are needed to address limitations of low-level event loops and callbacks. Event-driven services publish facts as events and use event streams for integration.
- Thinking in terms of immutable events and promises can help increase certainty
Brief introduction on CEP and Terminology
o Drools Vision
o Drools Fusion: Complex Event Processing extensions
o Event Declaration and Semantics
o Event Cloud, Streams and the Session Clock
o Temporal Reasoning
o Sliding Window Support
o Streams Support
o Memory Management
Complex Event Processing (CEP) involves detecting patterns in streams of event data. CEP tools analyze multiple simple events to identify complex events inferred from simpler ones. Typical applications of CEP include monitoring for business anomalies, detecting fraud or security threats. CEP augments service-oriented architectures by allowing services to trigger from events and generate new event streams. Event processing engines use techniques like filtering, windows, and correlation to detect patterns across events over time.
This document provides an overview of event processing, including:
1) Event processing involves performing operations on events and has roots in areas like active databases, data stream management systems, and network management.
2) Key concepts in event processing include events, event types, event producers, event consumers, and detecting patterns in events. Common applications are in system monitoring, complex event processing, and event-driven architectures.
3) Event processing has standard building blocks like event processing agents, event channels, and event processing networks that connect producers and consumers of events. Common operations include filtering, transforming, correlating, and detecting patterns in streams of events.
The document discusses complex event processing (CEP) technology and the CEP GE instance available in FIWARE. It provides an overview of CEP's event-condition-action paradigm and how it can detect patterns over incoming events. It also describes how to define event types, processing rules, contexts and producers/consumers in the CEP GE's web interface. Finally, it provides an example of detecting a denial of service attack by defining an event processing agent to detect increasing traffic report events over time.
This document discusses complex event processing (CEP) using the FIWARE CEP instance called Proton. It describes how applications can act as event producers and consumers, how patterns are defined over events, and how Proton's web interface is used to define CEP applications at build time which can then process input events and detect patterns at runtime. It also provides examples of using Proton's REST APIs to manage definitions, administrate instances, and send input events.
This document discusses complex event processing (CEP) using the FIWARE CEP instance called Proton. It describes how applications can act as event producers and consumers, how patterns are defined over events, and how Proton's web interface is used to define CEP applications at build time which can then process input events and detect patterns at runtime. It also provides examples of using REST APIs to manage definitions, administrate instances, and send input events to a running CEP engine.
Proposed Event Processing Definitions, DRAFT Work in Progress, September 20, 2006 4th Draft, Tim Bass, CISSP, Principal Global Architect, Director, TIBCO Software Inc.
Open Source Event Processing for Sensor Fusion Applicationsguestc4ce526
This deck is from my Robodev 2008 presentation on sensor fusion using open source technologies. It includes a detailed description of my homebrew sensor fusion Roomba, which I demonstrated at the conference.
This document discusses event-driven business process management using jBPM and Drools. It provides an example of how software development tools and infrastructure can be combined with jBPM 5 and Drools to gain real-time understanding of processes and increase agility. Traditional BPM systems struggle with change, complexity, and flexibility, but event-driven BPM addresses these issues. The document also outlines a logistics company use case and how a rules-based solution could dynamically manage shipments and routing.
A Deep Dive into Spring Application EventsVMware Tanzu
Spring application events provide a lightweight way to implement domain events in a modular Spring application. Events are published from aggregate roots using Spring Data, which allows loosely coupled interaction between bounded contexts. Depending on the integration mechanism, events can be externalized to technologies like JMS, AMQP or Kafka to notify other systems. If an event listener or transactional listener fails, the entire transaction will roll back to maintain strong consistency. A registry can be used to retry publishing events if listeners fail. Events help reduce coupling between components by making integration points explicit.
Debs 2011 tutorial on non functional properties of event processingOpher Etzion
The document discusses various non-functional properties of event processing systems including performance, scalability, availability, usability, and security considerations. It covers topics such as performance benchmarks and indicators, approaches to scaling systems both vertically and horizontally, high availability techniques using redundancy and duplication, usability factors like learnability and satisfaction, and validation methods for ensuring correctness.
Event data may the fastest growing form of data most people have never heard of. At its core, event data is simply any data point that has a timestamp, entity, and attributes of an action. By analyzing event data, we can better understand user behavior.
1) The document discusses how event-driven systems are reshaping modern architectures. It argues that modeling systems around publishing and reacting to events can help increase autonomy, loose coupling, scalability and other advantages.
2) Events represent immutable facts that accumulate over time and allow systems to manage uncertainty through eventual consistency. Well-designed events can help systems balance determinism and availability.
3) Event sourcing is presented as a cure for the "cardinal sin" of in-place updates, since it models the database as a cache of events from an immutable log. This allows for replay, auditing and other benefits like managing time within systems.
Design and Implementation of A Data Stream Management SystemErdi Olmezogullari
This presentation is related to my Master's Thesis at Ozyegin University. We focused on data mining on the real streaming (not binary) data. The most popular data mining algorithm, Association Rule Mining (ARM), was performed during this study from scratch. At the end of the thesis, we published four national/international papers in the different conferences such as Cloud Computing and Big Data.
Semantic CEP with Reaction RuleML, Keynote at 8th International Web Rule Symposium (RuleML 2014) @ ECAI 2014, Prague, Czech Republic, August 18-22, 2014
The document discusses a state-based computing platform called Ergo that uses event streams and state machines to model patient health data over time. Key capabilities include:
- Representing patient data as a sequence of "states" connected by triggering "events"
- Analyzing event/state sequences to identify patterns and predictive pathways
- Applying social modeling approaches like "ingeniums" and "twins" to understand individuals and connect similar patients
- Enabling applications and widgets to access modeled data via syndication interfaces
How can the concepts of event-driven linked with the concepts of serivce-oriented architectures. and what is the added value of such a combination?
What do events mean in the context of Business Process Management (BPM) and Business Activity Monitoring (BAM), and how can such architectures/solutions be enhanced with the concepts of Complex Event Processing?
Overview of the SOPRANO Ambient Middleware including context management and service matchmaking ontop of OSGi. SOPRANO is an Integrating Project funded by the European Commission in the field of Ambient Assisted Living (ICT and Ageing).
Event Driven Architecture (EDA), November 2, 2006Tim Bass
Event Driven Architecture (EDA), SOA Seminar Crystal City, Virginia, November 2nd, 2006, Tim Bass, CISSP, Principal Global Architect, Director. Co-Chair, Event Processing Reference Architecture Working Group (EPRAWG)
This presentation discusses discrete-event simulation software. It begins by defining discrete-event simulation as modeling the operation of a system as a discrete sequence of events in time, where each event occurs at a particular instant and marks a change in the system's state. It then discusses key aspects of discrete-event simulation like event time being discrete, transaction-flow modeling, and ordering simultaneous events. The presentation also covers common discrete-event modeling languages and basic simulation constructs like entities, resources, controls, and operations. It explains how simulation execution works by advancing the simulation clock at each event. Finally, it discusses entity states, management structures in simulations, different tool implementations, and why discrete-event simulation matters.
This presentation introduces discrete-event simulation software. It discusses what discrete-event simulation is, how it models systems as sequences of events over time. It covers the basic constructs like entities, resources, control elements and operations. It explains how simulation execution advances by processing the next event. It discusses entity states like active, ready, time-delayed and conditional-delayed. It also summarizes different implementations in discrete-event modeling languages and tools like Arena and AutoMod.
Event processing allows analyzing streams of event data in real-time to derive conclusions and drive actions. It works by capturing events from sources, parsing, enriching, and processing them using defined scenarios to detect patterns and meet conditions for taking actions. Complex event processing (CEP) is becoming more important for real-time business intelligence, enabling uses cases like real-time marketing, customer experience management, fraud detection, and processing sensor data alarms. As more event data becomes available and latency requirements increase, CEP will continue growing in importance for driving faster, more intelligent business responses and decisions.
DEBS 2019 tutorial : correctness and consistency of event-based systems Opher Etzion
The tutorial includes: temporal correctness, tuning up the semantics of event-based applications, data consistency, and validation and verification of event-based systems
Sw architectures 2018 on microservices and edaOpher Etzion
This document discusses microservices and event-driven architectures. It provides an example of a "smart road" system using these approaches. Events are used to communicate between isolated microservices to handle tasks like vehicle identification, traffic rate calculation, billing, and detecting stolen vehicles. The document also discusses using events to handle consistency requirements when processing salary increases across budget-checking and other microservices. Events allow microservices to remain independent while coping with global consistency needs through approaches like compensating transactions triggered by event notifications.
The document discusses complex event processing (CEP) technology and the CEP GE instance available in FIWARE. It provides an overview of CEP's event-condition-action paradigm and how it can detect patterns over incoming events. It also describes how to define event types, processing rules, contexts and producers/consumers in the CEP GE's web interface. Finally, it provides an example of detecting a denial of service attack by defining an event processing agent to detect increasing traffic report events over time.
This document discusses complex event processing (CEP) using the FIWARE CEP instance called Proton. It describes how applications can act as event producers and consumers, how patterns are defined over events, and how Proton's web interface is used to define CEP applications at build time which can then process input events and detect patterns at runtime. It also provides examples of using Proton's REST APIs to manage definitions, administrate instances, and send input events.
This document discusses complex event processing (CEP) using the FIWARE CEP instance called Proton. It describes how applications can act as event producers and consumers, how patterns are defined over events, and how Proton's web interface is used to define CEP applications at build time which can then process input events and detect patterns at runtime. It also provides examples of using REST APIs to manage definitions, administrate instances, and send input events to a running CEP engine.
Proposed Event Processing Definitions, DRAFT Work in Progress, September 20, 2006 4th Draft, Tim Bass, CISSP, Principal Global Architect, Director, TIBCO Software Inc.
Open Source Event Processing for Sensor Fusion Applicationsguestc4ce526
This deck is from my Robodev 2008 presentation on sensor fusion using open source technologies. It includes a detailed description of my homebrew sensor fusion Roomba, which I demonstrated at the conference.
This document discusses event-driven business process management using jBPM and Drools. It provides an example of how software development tools and infrastructure can be combined with jBPM 5 and Drools to gain real-time understanding of processes and increase agility. Traditional BPM systems struggle with change, complexity, and flexibility, but event-driven BPM addresses these issues. The document also outlines a logistics company use case and how a rules-based solution could dynamically manage shipments and routing.
A Deep Dive into Spring Application EventsVMware Tanzu
Spring application events provide a lightweight way to implement domain events in a modular Spring application. Events are published from aggregate roots using Spring Data, which allows loosely coupled interaction between bounded contexts. Depending on the integration mechanism, events can be externalized to technologies like JMS, AMQP or Kafka to notify other systems. If an event listener or transactional listener fails, the entire transaction will roll back to maintain strong consistency. A registry can be used to retry publishing events if listeners fail. Events help reduce coupling between components by making integration points explicit.
Debs 2011 tutorial on non functional properties of event processingOpher Etzion
The document discusses various non-functional properties of event processing systems including performance, scalability, availability, usability, and security considerations. It covers topics such as performance benchmarks and indicators, approaches to scaling systems both vertically and horizontally, high availability techniques using redundancy and duplication, usability factors like learnability and satisfaction, and validation methods for ensuring correctness.
Event data may the fastest growing form of data most people have never heard of. At its core, event data is simply any data point that has a timestamp, entity, and attributes of an action. By analyzing event data, we can better understand user behavior.
1) The document discusses how event-driven systems are reshaping modern architectures. It argues that modeling systems around publishing and reacting to events can help increase autonomy, loose coupling, scalability and other advantages.
2) Events represent immutable facts that accumulate over time and allow systems to manage uncertainty through eventual consistency. Well-designed events can help systems balance determinism and availability.
3) Event sourcing is presented as a cure for the "cardinal sin" of in-place updates, since it models the database as a cache of events from an immutable log. This allows for replay, auditing and other benefits like managing time within systems.
Design and Implementation of A Data Stream Management SystemErdi Olmezogullari
This presentation is related to my Master's Thesis at Ozyegin University. We focused on data mining on the real streaming (not binary) data. The most popular data mining algorithm, Association Rule Mining (ARM), was performed during this study from scratch. At the end of the thesis, we published four national/international papers in the different conferences such as Cloud Computing and Big Data.
Semantic CEP with Reaction RuleML, Keynote at 8th International Web Rule Symposium (RuleML 2014) @ ECAI 2014, Prague, Czech Republic, August 18-22, 2014
The document discusses a state-based computing platform called Ergo that uses event streams and state machines to model patient health data over time. Key capabilities include:
- Representing patient data as a sequence of "states" connected by triggering "events"
- Analyzing event/state sequences to identify patterns and predictive pathways
- Applying social modeling approaches like "ingeniums" and "twins" to understand individuals and connect similar patients
- Enabling applications and widgets to access modeled data via syndication interfaces
How can the concepts of event-driven linked with the concepts of serivce-oriented architectures. and what is the added value of such a combination?
What do events mean in the context of Business Process Management (BPM) and Business Activity Monitoring (BAM), and how can such architectures/solutions be enhanced with the concepts of Complex Event Processing?
Overview of the SOPRANO Ambient Middleware including context management and service matchmaking ontop of OSGi. SOPRANO is an Integrating Project funded by the European Commission in the field of Ambient Assisted Living (ICT and Ageing).
Event Driven Architecture (EDA), November 2, 2006Tim Bass
Event Driven Architecture (EDA), SOA Seminar Crystal City, Virginia, November 2nd, 2006, Tim Bass, CISSP, Principal Global Architect, Director. Co-Chair, Event Processing Reference Architecture Working Group (EPRAWG)
This presentation discusses discrete-event simulation software. It begins by defining discrete-event simulation as modeling the operation of a system as a discrete sequence of events in time, where each event occurs at a particular instant and marks a change in the system's state. It then discusses key aspects of discrete-event simulation like event time being discrete, transaction-flow modeling, and ordering simultaneous events. The presentation also covers common discrete-event modeling languages and basic simulation constructs like entities, resources, controls, and operations. It explains how simulation execution works by advancing the simulation clock at each event. Finally, it discusses entity states, management structures in simulations, different tool implementations, and why discrete-event simulation matters.
This presentation introduces discrete-event simulation software. It discusses what discrete-event simulation is, how it models systems as sequences of events over time. It covers the basic constructs like entities, resources, control elements and operations. It explains how simulation execution advances by processing the next event. It discusses entity states like active, ready, time-delayed and conditional-delayed. It also summarizes different implementations in discrete-event modeling languages and tools like Arena and AutoMod.
Event processing allows analyzing streams of event data in real-time to derive conclusions and drive actions. It works by capturing events from sources, parsing, enriching, and processing them using defined scenarios to detect patterns and meet conditions for taking actions. Complex event processing (CEP) is becoming more important for real-time business intelligence, enabling uses cases like real-time marketing, customer experience management, fraud detection, and processing sensor data alarms. As more event data becomes available and latency requirements increase, CEP will continue growing in importance for driving faster, more intelligent business responses and decisions.
Similar to Vldb 2010 event processing tutorial (20)
DEBS 2019 tutorial : correctness and consistency of event-based systems Opher Etzion
The tutorial includes: temporal correctness, tuning up the semantics of event-based applications, data consistency, and validation and verification of event-based systems
Sw architectures 2018 on microservices and edaOpher Etzion
This document discusses microservices and event-driven architectures. It provides an example of a "smart road" system using these approaches. Events are used to communicate between isolated microservices to handle tasks like vehicle identification, traffic rate calculation, billing, and detecting stolen vehicles. The document also discusses using events to handle consistency requirements when processing salary increases across budget-checking and other microservices. Events allow microservices to remain independent while coping with global consistency needs through approaches like compensating transactions triggered by event notifications.
ER 2017 tutorial - On Paradoxes, Autonomous Systems and dilemmasOpher Etzion
This document discusses paradoxes, autonomous systems, and the dilemmas they present. It covers paradoxes like Russell's paradox and Zeno's paradox to illustrate logical contradictions. It then outlines how autonomous systems work by sensing their environment, making sense of inputs, decision-making, and acting. Examples of current and future applications of healthcare, industrial, and military robotics are provided. However, it also notes the dangers of causal inference from correlation and sources of uncertainty. Deep learning allows autonomous systems to self-learn but lacks transparency. This could enable remote killing and raises issues around social equality if advanced systems surpass human
This presentation given in the un-conference on technology and art is a first glance into new research on taking event-driven technology to create a new type of literature.
Has Internet of Things really happened? Opher Etzion
The document discusses challenges facing widespread adoption of the Internet of Things (IoT). While sensor technology exists, each sensor operates in isolation and multi-sensor systems are difficult to construct. Other challenges include data quality issues, lack of ubiquitous networks, integration difficulties, need for more sensor innovation, and inadequate security for the status quo. Standardization efforts are underway but democratizing use through sensor integration, personalization, and pervasive applications remains challenging.
On the personalization of event-based systems Opher Etzion
The document discusses personalizing event-based systems to better assist elderly individuals. It proposes sensors around the home that can detect events like an unlocked door, falling, or vocal distress to alert family members. While the technology exists, it needs to be more personalized, affordable, and simple to use. The research requires a multi-disciplinary effort across technology, human factors, economics, and specific domains. Personalizing such systems to individuals and simplifying the technology is seen as key to widespread adoption.
On Internet of Everything and Personalization. Talk in INTEROP 2014Opher Etzion
This document discusses the Internet of Everything (IoE) and personalization. It begins by defining the IoE as connecting people and machines through technologies like sensors. Examples are given of how sensors can detect situations and trigger actions. Challenges to the IoE include data quality issues, lack of integration between sensors, and complexity that limits widespread use. The document argues for standards and simplifying models to make IoE technologies more accessible and customizable to individual needs and situations. Security, privacy and the democratization of IoE technologies are also addressed.
Introduction to the institute of technological empowermentOpher Etzion
This short presentation was presented in July-9-2014 at a meeting with people interested in the Institute of Technology Empowerment and provides brief introduction to the institute's goals and activities.
DEBS 2014 tutorial on the Internet of Everything. Opher Etzion
This document provides an overview of the Internet of Everything (IoE) and event processing. It discusses how IoE is an extension of concepts like the Internet of Things (IoT) and how everything will become connected. The document outlines how IoE will impact different areas like healthcare, agriculture, cities, retail, homes and more. It also presents a futuristic view where driverless cars and automated personal assistants become common due to advances in sensors and connectivity.
The Internet of Things and some introduction to the Technological Empowerment...Opher Etzion
This document outlines the vision for the Technological Empowerment Institute, which aims to empower people and businesses in developing areas through smart sensor-based systems for situation awareness. The institute will have three legs: a multi-disciplinary research leg, an implementation leg through partnerships, and an education leg. Some example target applications mentioned are assistance for elderly independent living through situational awareness alerts, supply chain monitoring, agriculture environmental control, and water quality monitoring. The overall goal is to help realize the power of event-driven systems to create a better world.
ER 2013 tutorial: modeling the event driven world Opher Etzion
This document discusses event-driven thinking and modeling compared to traditional request-driven computing. Some key points made:
1) Event-driven applications follow the "4D" paradigm of detect, derive, decide, do to achieve awareness and reaction, while traditional systems are more request-driven.
2) Event-driven logic is sensitive to the timing and order of events, rather than just the event occurrence. Temporal considerations are important.
3) State handling is more complex with events, as logic may depend on patterns spanning unmatched past events.
Existing event processing languages still require technical skills and do not fully address modeling at the business level in an intuitive way. Purely computational approaches are not enough
Debs 2013 tutorial : Why is event-driven thinking different from traditional ...Opher Etzion
This document discusses the differences between traditional and event-driven thinking in computing. It begins with a brief history of event processing, noting its origins in academic projects in the early 2000s and increased adoption in recent years driven by big data and IoT trends. The document then outlines some key differences in event-driven thinking, such as its focus on reacting immediately to events rather than responding to periodic queries. It provides an example of using events to detect suspicious banking activity in real-time. Finally, it discusses challenges in expressing event-driven logic and temporal considerations using traditional request-driven approaches.
This document proposes using event processing as a key to immortality. It discusses recent medical research using sensors and actuators to monitor and treat diseases at the cellular level. The goal is to create an "ultimate automatic physician" that can detect any sickness, react proactively or reactively, and make treatment decisions using event processing networks as its "brain". The challenge presented is using such techniques to significantly extend the human lifespan.
The presentation introduced a basic proactive model for taking timely actions to optimize outcomes given anticipated unplanned events. It uses event patterns to predict events and their timing with uncertainty, then determines the optimal action over time by considering action costs and impacts. Several application scenarios were discussed. While demonstrating feasibility, further work is needed to address challenges like real-time optimization for other cases, improved forecasting models, usability, and scalability.
This document provides an overview of a tutorial on event processing under uncertainty. It begins with an introduction that discusses how most real-world data contains some level of uncertainty and an illustrative example of using event processing to detect crimes from uncertain video surveillance and citizen reports. It then outlines the topics to be covered, including representing and modeling different types of uncertainty, and extending event processing techniques to handle uncertain event data.
Proactive event-driven computing is a new paradigm that aims to take proactive actions based on forecasted events before problems occur. It involves detecting events, forecasting future events, making real-time decisions, and taking proactive actions. The document discusses challenges in creating proactive solutions and outlines key aspects of proactive computing including uncertainty handling, real-time decision making under uncertainty, and learning patterns and causal relationships.
2. An example to kick-off this tutorial: Act: Passenger has been re-routed to another destination – send the luggage Act: Bag has reached to the wrong aircraft Notify: Bag has been checked but did not reach the ULD within 20 minutes Notify: Bag has been checked but did not reach the connecting flight THE LUGGAGE PERSPECTIVE: Across the 24 largest airlines more than 5.6 million bags went missing in 2006, this is an average of 15.7 bags per 1,000 travelers. 15% of the bags are never found. BBC News, April 4, 2007 Event processing can help here..
3. Outline of this tutorial What is behind event processing ? How is it related to other computing terms? Where are its roots? Event processing – architecture, building blocks The present: State of the practice in event processing – languages, implementation issues, challenges in implementing event processing applications The future: Trends and research challenges
4. What is behind event processing ? How is it related to other computing terms? Where are its roots?
5. What is “event processing” anyway? or Event processing is a form of computing that performs operations on events
6. Event processing and Data stream management? Aliases? One of them subset of the other? Totally unrelated concepts?
7. In computing we processed events since early days Network and System Management
9. What’s new? The analog: moving from files to DBMS In recent years – architectures, abstractions, and dedicated commercial products emerge to support functionality that was traditionally carried out within regular programming. For some applications it is an improvement in TCO; for others is breaking the cost-effectiveness barrier.
10. What is an event There are various definitions of events Event is an occurrence within a particular system or domain; the word event has double meaning: the real-world Occurrence as well as its computerized representation
12. Sometime we even react to the occurrence of multiple events I closed the deal with the Canadians We closed two huge deals in a single day, It is a good opportunity to send all the team to Some fun time in Singapore I closed the deal with the Australians
14. What we actually want to react to are – situations TOLL VILOATOR FRUSTRATED CUSTOMER Sometimes the situation is determined by detecting that some pattern occurred in the Flowing events. Toll violation Frustrated customer Sometimes the events can approximate or indicate with some certainty that the situation has occurred
19. Ancestor: Temporal databases There is a substantial temporal nature to event processing. Recently – also spatial and spatio-temporal functions are being added
24. Fast Flower Delivery Flower Store Van Driver Ranking and Reporting System Bid Request Delivery Bid Assignments, Bid alerts, Assign Alerts Control System GPS Location Location Service Location Driver’s Guild Ranking and reports Delivery confirmation Pick Up confirmation Ranked drivers / automatic assignment Bid System Store Preferences Delivery Request Assignment System Manual Assignment Assignment Assignments, Pick Up Alert Delivery Alert http://www.ep-ts.com/EventProcessingInAction
25. Event Processing Agent Context Event Channel Event Consumer Event Type Event Producer Global State The seven Building blocks
40. Filter EPA A filter EPA is an EPA that performs filtering only, and has no matching or derivation steps, so it does not transform the input event .
43. Pattern detection example Pattern name: Manual Assignment Preparation Pattern Type: relative N highest Context: Bid Interval Relevant event types: Delivery Bid Pattern parameter: N = 5; value = Ranking Cardinality: Single deferred Find the five highest bids within the bid interval Taken from the Fast Flower Delivery use case
44.
45. Pattern policies Evaluation policy —This determines when the matching sets are produced Deferred Cardinality policy —This determines how many matching sets are produced within a single context partition Repeated type policy —This determines what happens if the matching step encounters multiple events of the same type Every - Override – First – Last – with max/min value of…
46. Pattern policies – cont. Consumption policy —This specifies what happens to a participant event after it has been included in a matching set Consume Reuse Bounded reuse Order policy —This specifies how temporal order is defined By occurrence time – by detection time – by stream position – by attribute
47.
48. Context has three distinct roles (which may be combined) Partition the incoming events The events that relate to each customer are processed separately Grouping events together Different processing for Different context partitions Determining the processing Grouping together events that happened in the same hour at the same location
49. Context Definition A context is a named specification of conditions that groups event instances so that they can be processed in a related way. It assigns each event instance to one or more context partitions . A context may have one or more context dimensions. Temporal Spatial State Oriented Segmentation Oriented
50. Context Types Examples Spatial State Oriented Temporal Context “ Every day between 08:00 and 10:00 AM” “ A week after borrowing a disk” “ A time window bounded by TradingDayStart and TradingDayEnd events” “ 3 miles from the traffic accident location” “ Within an authorized zone in a manufactory” “ All Children 2-5 years old” “ All platinum customers” “ Airport security level is red” “ Weather is stormy” Segmentation Oriented
52. The present: State of the practice in event processing – languages, implementation issues, challenges in implementing event processing applications
53.
54. Existing Styles for EP languages (samples) Inference Rules State oriented Agent Oriented Imperative/ Script Based RuleCore Oracle Aleri Streambase Esper EventZero TIBCO WBE Apama Spade AMiT Netcool Impact * - if we add simple and mediated event processing the picture is even more diversified ECA Rules SQL extension Coral8 Agent Logic Starview XChange EQ Prova
59. Microsoft Streaminsights var topfive = (from window in inputStream.Snapshot() from e in window orderby e.f ascending, e.i descending select e).Take(5); var avgCount = from v in inputStream group v by v.i % 4 into eachGroup from window in eachGroup.Snapshot() select new { avgNumber = window.Avg(e => e.number) };
60. Esper EPL – FFD Example /* * Not delivered up after 10 mins (600 secs) of the request target delivery time */ insert into AlertW(requestId, message, driver, timestamp) select a.requestId, "not delivered", a.driver, current_timestamp() from pattern[ every a=Assignment (timer:interval(600 + (a.deliveryTime-current_timestamp)/1000) and not DeliveryConfirmation(requestId = a.requestId) and not NoOneToReceiveMSG(requestId = a.requestId)) ];
70. Optimizations Optimizations related to EPA assignment: partition, parallelism, Distribution, load balancing. Optimization related to the execution process: scheduling, routing optimizations and load shedding Optimizations related to the coding of specific EPAs: code optimization, state management.
78. Trend II: Going from monolithic to diversified Variety of functions Variety of Quality of Service requirements: Variety of platforms “ One size fits all” will not work – Instead a collection of building blocks that can fit together
79.
80. Trend IV: Going from programmer centered to semi-technical person centered Source: ebizQ Event processing market pulse
83. Emerging directions: Four directions to observe Multiple platforms – same look and feel The engineering of constructing EP applications Adding intelligence to Event processing Tailor-made optimizations
86. Emerging direction III: Event processing software engineering Best practices Methodologies Modeling & meta-modeling Design Patterns
87. Emerging direction IV: Intelligent event processing Offline and continuous mining of meaningful patterns in event histories Inexact event processing – handling inexact events and also false positives and false negatives Causality – a key for proactive, but also vital for provenance
88. Summary Event processing has emerged from multiple academic disciplines The state of the Practice is the 1 st generation of products – mainly engineering based Going to the next phase – many challenges that require collaboration of research and industry Already attracted coverage of analysts and all major software vendors The Dagstuhl grand challenge