Complex Event Processing with Esper and WSO2 ESB
•Download as PPT, PDF•
4 likes•3,672 views
The document discusses using Esper, an open source complex event processing (CEP) library, with WSO2 ESB. It provides an overview of Esper and how to configure it for use with Axiom and XML event types. The document also includes an example of using an Esper mediator to analyze ticker events and generate new events that are injected back into the ESB for further processing.
Report
Share
Report
Share
![ESB 1.7 Webinar Series ,[object Object],[object Object],[object Object],[object Object],[object Object]](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Recommended
Securing your Pulsar Cluster with Vault_Chris Kellogg
Learn how to secure a Pulsar cluster with Hashicorp Vault and deploy it on Kubernetes. Vault provides a secure way to generate tokens and store sensitive data and Pulsar has a pluggable architecture for authentication, authorization and secret management. This talk will walk through how to create custom plugins for Vault, integrate them with Pulsar and then deploy a Pulsar cluster on Kubernetes.
Strata London 2018: Multi-everything with Apache Pulsar
Ivan Kelly offers an overview of Apache Pulsar, a durable, distributed messaging system, underpinned by Apache BookKeeper, that provides the enterprise features necessary to guarantee that your data is where is should be and only accessible by those who should have access. Ivan explores the features built into Pulsar that will help your organization stay in compliance with key requirements and regulations, for multi-data center replication, multi-tenancy, role-based access control, and end-to-end encryption. Ivan concludes by explaining why Pulsar’s multi-data center story will alleviate headaches for the operations teams ensuring compliance with GDPR.
Akka Microservices Architecture And Design
This document discusses using Akka and microservices architecture for building distributed applications. It covers key Akka concepts like actors and messaging. It also discusses domain-driven design patterns for modeling application domains and boundaries. The document recommends using asynchronous messaging between microservices. It provides an example of using Apache Camel for enterprise integration. Finally, it discusses using Akka Clustering and Persistence for building highly available stateful services in a distributed fashion.
Building Stateful Microservices With Akka
Although most microservices are stateless - they delegate things like persistence and consistency to a database or external storage. But sometimes you benefit when you keep the state inside the application. In this talk I’m going to discuss why you want to build stateful microservices and design choices to make. I’ll use Akka framework and explain tools like Akka Clustering and Akka Persistence in depth and show a few practical examples.
Building Scalable and Extendable Data Pipeline for Call of Duty Games: Lesson...
What can be easier than building a data pipeline nowadays? You add a few Apache Kafka clusters, some way to ingest data (probably over HTTP), design a way to route your data streams, add a few stream processors and consumers, integrate with a data warehouse... wait, it does start to look like A LOT of things, doesn't it? And you probably want to make it highly scalable and available in the end, correct?
We've been developing a data pipeline in Demonware/Activision for a while. We learned how to scale it not only in terms of messages/sec it can handle, but also in terms of supporting more games and more use-cases.
In this presentation you'll hear about the lessons we learned, including (but not limited to):
- Message schemas
- Apache Kafka organization and tuning
- Topics naming conventions, structure and routing
- Reliable and scalable producers and ingestion layer
- Stream processing
Apache Kafka: New Features That You Might Not Know About
In the last two years Apache Kafka rapidly introduced new versions, going from 0.10.x to 2.x. It can be hard to keep up with all the updates and a lot of companies still run 0.10.x clusters (or even older ones).
Join this session to learn new exciting features in Kafka introduced in 0.11, 1.0, 1.1 and 2.0 versions including, but not limited to, the new protocol and message headers, transactional support and exactly-only delivery semantics, as well as controller changes that make it possible to shutdown even large clusters in seconds.
Complex Event Processing with Esper
This document provides an overview of complex event processing (CEP) and Esper, an open source CEP engine. It defines CEP as analyzing and controlling interrelated events to understand modern distributed systems. Esper makes it easier to build CEP applications by providing an event processing language (EPL) to define event types, continuous queries, and event patterns similar to SQL. It supports features like filtering, aggregation, windows, correlations, and pattern detection on streaming event data. While powerful, Esper has limitations around memory usage, resilience, and distribution that must be considered.
Designing Scalable and Extendable Data Pipeline for Call Of Duty Games
This document discusses scaling and extending the data pipeline for Call of Duty games. Some key points:
1) The data pipeline uses Apache Kafka topics partitioned for scalability, but too many topics creates operational overhead, so topic naming follows conventions expressing data types rather than producers/consumers.
2) A stream processing layer called "Refinery" enables measuring, validating, enriching, filtering and routing messages to make the pipeline flexible for new use cases.
3) A unified message envelope and Schema Registry supporting multiple formats allows easy integration of new games into the pipeline by standardizing the message format.
Recommended
Securing your Pulsar Cluster with Vault_Chris Kellogg
Learn how to secure a Pulsar cluster with Hashicorp Vault and deploy it on Kubernetes. Vault provides a secure way to generate tokens and store sensitive data and Pulsar has a pluggable architecture for authentication, authorization and secret management. This talk will walk through how to create custom plugins for Vault, integrate them with Pulsar and then deploy a Pulsar cluster on Kubernetes.
Strata London 2018: Multi-everything with Apache Pulsar
Ivan Kelly offers an overview of Apache Pulsar, a durable, distributed messaging system, underpinned by Apache BookKeeper, that provides the enterprise features necessary to guarantee that your data is where is should be and only accessible by those who should have access. Ivan explores the features built into Pulsar that will help your organization stay in compliance with key requirements and regulations, for multi-data center replication, multi-tenancy, role-based access control, and end-to-end encryption. Ivan concludes by explaining why Pulsar’s multi-data center story will alleviate headaches for the operations teams ensuring compliance with GDPR.
Akka Microservices Architecture And Design
This document discusses using Akka and microservices architecture for building distributed applications. It covers key Akka concepts like actors and messaging. It also discusses domain-driven design patterns for modeling application domains and boundaries. The document recommends using asynchronous messaging between microservices. It provides an example of using Apache Camel for enterprise integration. Finally, it discusses using Akka Clustering and Persistence for building highly available stateful services in a distributed fashion.
Building Stateful Microservices With Akka
Although most microservices are stateless - they delegate things like persistence and consistency to a database or external storage. But sometimes you benefit when you keep the state inside the application. In this talk I’m going to discuss why you want to build stateful microservices and design choices to make. I’ll use Akka framework and explain tools like Akka Clustering and Akka Persistence in depth and show a few practical examples.
Building Scalable and Extendable Data Pipeline for Call of Duty Games: Lesson...
What can be easier than building a data pipeline nowadays? You add a few Apache Kafka clusters, some way to ingest data (probably over HTTP), design a way to route your data streams, add a few stream processors and consumers, integrate with a data warehouse... wait, it does start to look like A LOT of things, doesn't it? And you probably want to make it highly scalable and available in the end, correct?
We've been developing a data pipeline in Demonware/Activision for a while. We learned how to scale it not only in terms of messages/sec it can handle, but also in terms of supporting more games and more use-cases.
In this presentation you'll hear about the lessons we learned, including (but not limited to):
- Message schemas
- Apache Kafka organization and tuning
- Topics naming conventions, structure and routing
- Reliable and scalable producers and ingestion layer
- Stream processing
Apache Kafka: New Features That You Might Not Know About
In the last two years Apache Kafka rapidly introduced new versions, going from 0.10.x to 2.x. It can be hard to keep up with all the updates and a lot of companies still run 0.10.x clusters (or even older ones).
Join this session to learn new exciting features in Kafka introduced in 0.11, 1.0, 1.1 and 2.0 versions including, but not limited to, the new protocol and message headers, transactional support and exactly-only delivery semantics, as well as controller changes that make it possible to shutdown even large clusters in seconds.
Complex Event Processing with Esper
This document provides an overview of complex event processing (CEP) and Esper, an open source CEP engine. It defines CEP as analyzing and controlling interrelated events to understand modern distributed systems. Esper makes it easier to build CEP applications by providing an event processing language (EPL) to define event types, continuous queries, and event patterns similar to SQL. It supports features like filtering, aggregation, windows, correlations, and pattern detection on streaming event data. While powerful, Esper has limitations around memory usage, resilience, and distribution that must be considered.
Designing Scalable and Extendable Data Pipeline for Call Of Duty Games
This document discusses scaling and extending the data pipeline for Call of Duty games. Some key points:
1) The data pipeline uses Apache Kafka topics partitioned for scalability, but too many topics creates operational overhead, so topic naming follows conventions expressing data types rather than producers/consumers.
2) A stream processing layer called "Refinery" enables measuring, validating, enriching, filtering and routing messages to make the pipeline flexible for new use cases.
3) A unified message envelope and Schema Registry supporting multiple formats allows easy integration of new games into the pipeline by standardizing the message format.
Pulsar Architectural Patterns for CI/CD Automation and Self-Service_Devin Bost
We examine real-world architectural patterns involving Apache Pulsar to automate the creation of function and pub/sub flows for improved operational scalability and ease of management. We’ll cover CI/CD automation patterns and reveal our innovative approach of leveraging streaming data to create a self-service platform that automates the provisioning of new users. We will also demonstrate the innovative approach of creating function flows through patterns and configuration, enabling non-developer users to create entire function flows simply by changing configurations. These patterns enable us to drive the automation of managing Pulsar to a whole new level. We also cover CI/CD for on-prem, GCP, and AWS users.
This is Part 2 of this presentation: https://www.youtube.com/watch?v=pmaCG...
In summary, we will cover:
CI/CD for on-prem, GCP, and AWS users
Automated creation of function flows by configuration
Automated provisioning of pub/sub users and topics
Architectural patterns and best practices that enable automation
Overstock has leveraged Pulsar as the backbone of a self-service data fabric, a unified data platform to enable users to publish and consume data across the company and integrate with other services. We utilized Pulsar to solve a data governance problem, and Pulsar has performed marvelously. To support our real-world production use cases, we have developed message flows, integrations, and architectural patterns to solve common use cases, maximize value, simplify ease-of-use, automate management, and unify company data and services around this new platform.
Kafka Summit NYC 2017 - Easy, Scalable, Fault-tolerant Stream Processing with...
Structured Streaming provides a scalable and fault-tolerant stream processing framework on Spark SQL. It allows users to write streaming jobs using simple batch-like SQL queries that Spark will automatically optimize for efficient streaming execution. This includes handling out-of-order and late data, checkpointing to ensure fault-tolerance, and providing end-to-end exactly-once guarantees. The talk discusses how Structured Streaming represents streaming data as unbounded tables and executes queries incrementally to produce streaming query results.
Building a Messaging Solutions for OVHcloud with Apache Pulsar_Pierre Zemb
OVHcloud is the biggest European cloud provider. From dedicated servers to Managed Kubernetes, from VMware® based Hosted Private Cloud to OpenStack-based Public Cloud, we have over 1.4 million customers worldwide.
Internally, we have been running Apache Kafka for years, and despite all the skills obtained operating multiples clusters with millions of messages per second, we decided to shift and build the foundation of our 'topic-as-a-service' product called ioStream on Apache Pulsar.
In this talk, you will have the insights of why we decided to use Apache Pulsar instead of Apache Kafka as the core of ioStream. We will tell you our journey to use Apache Pulsar, from our deployments to the management, what did work and what did not.
Real Time Streaming Data with Kafka and TensorFlow (Yong Tang, MobileIron) Ka...
This document discusses using Kafka and TensorFlow for real-time streaming machine learning. It introduces TensorFlow 2.0 capabilities for data processing and machine learning. It then discusses challenges in integrating streaming data with machine learning frameworks and formats. It proposes using KafkaDataset, a TensorFlow dataset for reading and writing from Kafka. KafkaDataset allows streaming data in and out of TensorFlow models for real-time inference and prediction.
Transaction Support in Pulsar 2.5.0
In Apache Pulsar Beijing Meetup, Sijie Guo and Yong Zhang gave a preview of transaction support in Pulsar 2.5.0. Sijie Guo started with the current state of messaging semantics in Pulsar and talked about the implementation of message deduplication introduced by PIP-6. Then he went into the details of why do we need transaction and how do we implement transaction in Pulsar. Finally Yong walked through the whole transaction execution flow.
Advanced Streaming Analytics with Apache Flink and Apache Kafka, Stephan Ewen
Flink and Kafka are popular components to build an open source stream processing infrastructure. We present how Flink integrates with Kafka to provide a platform with a unique feature set that matches the challenging requirements of advanced stream processing applications. In particular, we will dive into the following points:
Flink’s support for event-time processing, how it handles out-of-order streams, and how it can perform analytics on historical and real-time streams served from Kafka’s persistent log using the same code. We present Flink’s windowing mechanism that supports time-, count- and session- based windows, and intermixing event and processing time semantics in one program.
How Flink’s checkpointing mechanism integrates with Kafka for fault-tolerance, for consistent stateful applications with exactly-once semantics.
We will discuss “”Savepoints””, which allows users to save the state of the streaming program at any point in time. Together with a durable event log like Kafka, savepoints allow users to pause/resume streaming programs, go back to prior states, or switch to different versions of the program, while preserving exactly-once semantics.
We explain the techniques behind the combination of low-latency and high throughput streaming, and how latency/throughput trade-off can configured.
We will give an outlook on current developments for streaming analytics, such as streaming SQL and complex event processing.
Introduction to Structured Streaming
Structured Streaming is a scalable and fault-tolerant stream processing engine built on the Spark SQL engine.
Mufix Network Programming Lecture
The document provides an overview of network programming in Python. It discusses key Python concepts like lists, dictionaries, tuples and strings. It then covers network programming topics like sockets, TCP/IP, HTTP requests and responses. It introduces the select module for building non-blocking servers that can handle multiple clients simultaneously using a single thread.
Query Pulsar Streams using Apache Flink
This document discusses querying Apache Pulsar streams using Apache Flink. It provides an overview of Pulsar and how it can be used for pub/sub messaging and streaming applications. It then describes how Flink connectors allow querying Pulsar streams through Flink's streaming and table APIs, including support for Pulsar schemas, exactly-once processing, and integration with the Pulsar catalog. The integration provides a unified data processing stack of Pulsar for streaming data storage and Flink for querying and analysis.
Introducing Exactly Once Semantics in Apache Kafka with Matthias J. Sax
Apache Kafka’s rise in popularity as a streaming platform has demanded a revisit of its traditional at least once message delivery semantics. In this talk, we present the recent additions to Apache Kafka to achieve exactly once semantics. We shall discuss the newly introduced transactional APIs and use Kafka Streams as an example to show how these APIs are leveraged for streams tasks.
Introduction to Apache Kafka- Part 2
The document discusses Apache Kafka and provides an introduction to key concepts like consumer groups, producer/consumer APIs, and throughput guarantees. It explains how Kafka can be used for both queuing and publish-subscribe models using consumer groups. Code examples are given for Kafka producers and consumers in Scala and Java along with throughput benchmarks showing Kafka's ability to support millions of messages per second.
Netty from the trenches
Netty is an asynchronous event-driven network application framework for rapid development of maintainable high performance protocol servers & clients. AND IT'S TRUE!
In this talk given at JBCNConf 2015 in Barcelona, we will see how we use Netty at Trovit since 2013, what brought to us and how it opened our minds. We will share tips that helped us to learn more about Netty, some performance tricks and all things that worked for us.
Winter is coming? Not if ZooKeeper is there!
[Download the slide to get the entire talk in the form of presentation note embedded in the ppt] Apache ZooKeeper is the chosen leader in distributed coordination. In this talk, I have explored the atomic elements of Apache ZooKeeper, how it fits everything together and some of its popular use cases. For ZooKeeper simplicity is the key and as a consumer of the API, our imagination enables us to push the limits of the ZooKeeper world.
Apache ZooKeeper
Slides for presentation on ZooKeeper I gave at Near Infinity (www.nearinfinity.com) 2012 spring conference.
The associated sample code is on GitHub at https://github.com/sleberknight/zookeeper-samples
Introduction to Akka-Streams
This document provides an introduction to Akka Streams, which implements the Reactive Streams specification. It discusses the limitations of traditional concurrency models and Actor models in dealing with modern challenges like high availability and large data volumes. Reactive Streams aims to provide a minimalistic asynchronous model with back pressure to prevent resource exhaustion. Akka Streams builds on the Akka framework and Actor model to provide a streaming data flow library that uses Reactive Streams interfaces. It allows defining processing pipelines with sources, flows, and sinks and includes features like graph DSL, back pressure, and integration with other Reactive Streams implementations.
Distributed Real-Time Stream Processing: Why and How 2.0
The demand for stream processing is increasing a lot these day. Immense amounts of data has to be processed fast from a rapidly growing set of disparate data sources. This pushes the limits of traditional data processing infrastructures. These stream-based applications include trading, social networks, Internet of things, system monitoring, and many other examples.
In this talk we are going to discuss various state of the art open-source distributed streaming frameworks, their similarities and differences, implementation trade-offs and their intended use-cases. Apart of that, I’m going to speak about Fast Data, theory of streaming, framework evaluation and so on. My goal is to provide comprehensive overview about modern streaming frameworks and to help fellow developers with picking the best possible for their particular use-case.
Serverless Event Streaming with Pulsar Functions
The last few years have seen the emergence of Serverless as a paradigm for event streaming. Its very simple programming model has attracted developers in droves. At the same time, its ability to elastically scale has simplified operations significantly. Combined together with the ubiquity of their presence across all cloud providers, serverless today has become the leading choice to do event processing at scale for a lot of companies.
In this talk, Sijie Guo from StreamNative will explore how the serverless paradigm is applied to event streaming in Apache Pulsar, a next-generation event streaming system. Pulsar provides native support for serverless functions where the events are processed as soon as they arrive in a streaming manner and that provides flexible deployment options (thread, process, container). He will describe how these serverless functions make data engineering easier and share the real world usage of Pulsar Functions.
Kafka Streams: the easiest way to start with stream processing
Stream processing is getting more & more important in our data-centric systems. In the world of Big Data, batch processing is not enough anymore - everyone needs interactive, real-time analytics for making critical business decisions, as well as providing great features to the customers.
There are many stream processing frameworks available nowadays, but the cost of provisioning infrastructure and maintaining distributed computations is usually very high. Sometimes you just have to satisfy some specific requirements, like using HDFS or YARN.
Apache Kafka is de facto a standard for building data pipelines. Kafka Streams is a lightweight library (available since 0.10) that uses powerful Kafka abstractions internally and doesn't require any complex setup or special infrastructure - you just deploy it like any other regular application.
In this session I want to talk about the goals behind stream processing, basic techniques and some best practices. Then I'm going to explain main fundamental concepts behind Kafka and explore Kafka Streams syntax and streaming features. By the end of the session you'll be able to write stream processing applications in your domain, especially if you already use Kafka as your data pipeline.
Reactor, Reactive streams and MicroServices
This document discusses Reactive Programming and Reactive Streams. It introduces Reactor, a reactive programming framework, and how it addresses issues like latency in microservices architectures. Reactive Streams provide an interoperable way to work with asynchronous data streams in a non-blocking manner. Streams represent sequences of data that can be processed reactively through operators like map and filter.
Developing real-time data pipelines with Spring and Kafka
Talk given at the Apache Kafka NYC Meetup, October 20, 2015.
http://www.meetup.com/Apache-Kafka-NYC/events/225697500/
Kafka has emerged as a clear choice for a high-throughput, low latency messaging system that addresses the needs of high-performance streaming applications. The Spring Framework has been, in the last decade, the de-facto standard for developing enterprise Java applications, providing a simple and powerful programming model that allows developers to focus on the business needs, leaving the boilerplate and middleware integration to the framework itself. In fact, it has evolved into a rich and powerful ecosystem, with projects focusing on specific aspects of enterprise software development - like Spring Boot, Spring Data, Spring Integration, Spring XD, Spring Cloud Stream/Data Flow to name just a few.
In this presentation, Marius Bogoevici from the Spring team will take the perspective of the Kafka user, and show, with live demos, how the various projects in the Spring ecosystem address their needs:
- how to build simple data integration applications using Spring Integration Kafka;
- how to build sophisticated data pipelines with Spring XD and Kafka;
- how to build cloud native message-driven microservices using Spring Cloud Stream and Kafka, and how to orchestrate them using Spring Cloud Data Flow;
Comparative Analysis of Personal Firewalls
This thesis describes the analysis of 18 personal firewalls. It discovers the differences in their behaviour while they are under various techniques of port scanning and Denial of Service (DoS) attacks. With port scanning, the detection ability, time consumption, leaked port states and obfuscation techniques are analysed. With using different DoS attacks, performance measurements of CPU and network adapter are taken. The potential of firewall fingerprinting based on the different behaviour across multiple products is also addressed.
Reactconf 2014 - Event Stream Processing
Presentation from reactconf 2014 in San Francisco.
Covers Event Stream Processing, some of the theory behind it and some implementation details in the context of local and distributed. Also covers some Big Data technologies
More Related Content
What's hot
Pulsar Architectural Patterns for CI/CD Automation and Self-Service_Devin Bost
We examine real-world architectural patterns involving Apache Pulsar to automate the creation of function and pub/sub flows for improved operational scalability and ease of management. We’ll cover CI/CD automation patterns and reveal our innovative approach of leveraging streaming data to create a self-service platform that automates the provisioning of new users. We will also demonstrate the innovative approach of creating function flows through patterns and configuration, enabling non-developer users to create entire function flows simply by changing configurations. These patterns enable us to drive the automation of managing Pulsar to a whole new level. We also cover CI/CD for on-prem, GCP, and AWS users.
This is Part 2 of this presentation: https://www.youtube.com/watch?v=pmaCG...
In summary, we will cover:
CI/CD for on-prem, GCP, and AWS users
Automated creation of function flows by configuration
Automated provisioning of pub/sub users and topics
Architectural patterns and best practices that enable automation
Overstock has leveraged Pulsar as the backbone of a self-service data fabric, a unified data platform to enable users to publish and consume data across the company and integrate with other services. We utilized Pulsar to solve a data governance problem, and Pulsar has performed marvelously. To support our real-world production use cases, we have developed message flows, integrations, and architectural patterns to solve common use cases, maximize value, simplify ease-of-use, automate management, and unify company data and services around this new platform.
Kafka Summit NYC 2017 - Easy, Scalable, Fault-tolerant Stream Processing with...
Structured Streaming provides a scalable and fault-tolerant stream processing framework on Spark SQL. It allows users to write streaming jobs using simple batch-like SQL queries that Spark will automatically optimize for efficient streaming execution. This includes handling out-of-order and late data, checkpointing to ensure fault-tolerance, and providing end-to-end exactly-once guarantees. The talk discusses how Structured Streaming represents streaming data as unbounded tables and executes queries incrementally to produce streaming query results.
Building a Messaging Solutions for OVHcloud with Apache Pulsar_Pierre Zemb
OVHcloud is the biggest European cloud provider. From dedicated servers to Managed Kubernetes, from VMware® based Hosted Private Cloud to OpenStack-based Public Cloud, we have over 1.4 million customers worldwide.
Internally, we have been running Apache Kafka for years, and despite all the skills obtained operating multiples clusters with millions of messages per second, we decided to shift and build the foundation of our 'topic-as-a-service' product called ioStream on Apache Pulsar.
In this talk, you will have the insights of why we decided to use Apache Pulsar instead of Apache Kafka as the core of ioStream. We will tell you our journey to use Apache Pulsar, from our deployments to the management, what did work and what did not.
Real Time Streaming Data with Kafka and TensorFlow (Yong Tang, MobileIron) Ka...
This document discusses using Kafka and TensorFlow for real-time streaming machine learning. It introduces TensorFlow 2.0 capabilities for data processing and machine learning. It then discusses challenges in integrating streaming data with machine learning frameworks and formats. It proposes using KafkaDataset, a TensorFlow dataset for reading and writing from Kafka. KafkaDataset allows streaming data in and out of TensorFlow models for real-time inference and prediction.
Transaction Support in Pulsar 2.5.0
In Apache Pulsar Beijing Meetup, Sijie Guo and Yong Zhang gave a preview of transaction support in Pulsar 2.5.0. Sijie Guo started with the current state of messaging semantics in Pulsar and talked about the implementation of message deduplication introduced by PIP-6. Then he went into the details of why do we need transaction and how do we implement transaction in Pulsar. Finally Yong walked through the whole transaction execution flow.
Advanced Streaming Analytics with Apache Flink and Apache Kafka, Stephan Ewen
Flink and Kafka are popular components to build an open source stream processing infrastructure. We present how Flink integrates with Kafka to provide a platform with a unique feature set that matches the challenging requirements of advanced stream processing applications. In particular, we will dive into the following points:
Flink’s support for event-time processing, how it handles out-of-order streams, and how it can perform analytics on historical and real-time streams served from Kafka’s persistent log using the same code. We present Flink’s windowing mechanism that supports time-, count- and session- based windows, and intermixing event and processing time semantics in one program.
How Flink’s checkpointing mechanism integrates with Kafka for fault-tolerance, for consistent stateful applications with exactly-once semantics.
We will discuss “”Savepoints””, which allows users to save the state of the streaming program at any point in time. Together with a durable event log like Kafka, savepoints allow users to pause/resume streaming programs, go back to prior states, or switch to different versions of the program, while preserving exactly-once semantics.
We explain the techniques behind the combination of low-latency and high throughput streaming, and how latency/throughput trade-off can configured.
We will give an outlook on current developments for streaming analytics, such as streaming SQL and complex event processing.
Introduction to Structured Streaming
Structured Streaming is a scalable and fault-tolerant stream processing engine built on the Spark SQL engine.
Mufix Network Programming Lecture
The document provides an overview of network programming in Python. It discusses key Python concepts like lists, dictionaries, tuples and strings. It then covers network programming topics like sockets, TCP/IP, HTTP requests and responses. It introduces the select module for building non-blocking servers that can handle multiple clients simultaneously using a single thread.
Query Pulsar Streams using Apache Flink
This document discusses querying Apache Pulsar streams using Apache Flink. It provides an overview of Pulsar and how it can be used for pub/sub messaging and streaming applications. It then describes how Flink connectors allow querying Pulsar streams through Flink's streaming and table APIs, including support for Pulsar schemas, exactly-once processing, and integration with the Pulsar catalog. The integration provides a unified data processing stack of Pulsar for streaming data storage and Flink for querying and analysis.
Introducing Exactly Once Semantics in Apache Kafka with Matthias J. Sax
Apache Kafka’s rise in popularity as a streaming platform has demanded a revisit of its traditional at least once message delivery semantics. In this talk, we present the recent additions to Apache Kafka to achieve exactly once semantics. We shall discuss the newly introduced transactional APIs and use Kafka Streams as an example to show how these APIs are leveraged for streams tasks.
Introduction to Apache Kafka- Part 2
The document discusses Apache Kafka and provides an introduction to key concepts like consumer groups, producer/consumer APIs, and throughput guarantees. It explains how Kafka can be used for both queuing and publish-subscribe models using consumer groups. Code examples are given for Kafka producers and consumers in Scala and Java along with throughput benchmarks showing Kafka's ability to support millions of messages per second.
Netty from the trenches
Netty is an asynchronous event-driven network application framework for rapid development of maintainable high performance protocol servers & clients. AND IT'S TRUE!
In this talk given at JBCNConf 2015 in Barcelona, we will see how we use Netty at Trovit since 2013, what brought to us and how it opened our minds. We will share tips that helped us to learn more about Netty, some performance tricks and all things that worked for us.
Winter is coming? Not if ZooKeeper is there!
[Download the slide to get the entire talk in the form of presentation note embedded in the ppt] Apache ZooKeeper is the chosen leader in distributed coordination. In this talk, I have explored the atomic elements of Apache ZooKeeper, how it fits everything together and some of its popular use cases. For ZooKeeper simplicity is the key and as a consumer of the API, our imagination enables us to push the limits of the ZooKeeper world.
Apache ZooKeeper
Slides for presentation on ZooKeeper I gave at Near Infinity (www.nearinfinity.com) 2012 spring conference.
The associated sample code is on GitHub at https://github.com/sleberknight/zookeeper-samples
Introduction to Akka-Streams
This document provides an introduction to Akka Streams, which implements the Reactive Streams specification. It discusses the limitations of traditional concurrency models and Actor models in dealing with modern challenges like high availability and large data volumes. Reactive Streams aims to provide a minimalistic asynchronous model with back pressure to prevent resource exhaustion. Akka Streams builds on the Akka framework and Actor model to provide a streaming data flow library that uses Reactive Streams interfaces. It allows defining processing pipelines with sources, flows, and sinks and includes features like graph DSL, back pressure, and integration with other Reactive Streams implementations.
Distributed Real-Time Stream Processing: Why and How 2.0
The demand for stream processing is increasing a lot these day. Immense amounts of data has to be processed fast from a rapidly growing set of disparate data sources. This pushes the limits of traditional data processing infrastructures. These stream-based applications include trading, social networks, Internet of things, system monitoring, and many other examples.
In this talk we are going to discuss various state of the art open-source distributed streaming frameworks, their similarities and differences, implementation trade-offs and their intended use-cases. Apart of that, I’m going to speak about Fast Data, theory of streaming, framework evaluation and so on. My goal is to provide comprehensive overview about modern streaming frameworks and to help fellow developers with picking the best possible for their particular use-case.
Serverless Event Streaming with Pulsar Functions
The last few years have seen the emergence of Serverless as a paradigm for event streaming. Its very simple programming model has attracted developers in droves. At the same time, its ability to elastically scale has simplified operations significantly. Combined together with the ubiquity of their presence across all cloud providers, serverless today has become the leading choice to do event processing at scale for a lot of companies.
In this talk, Sijie Guo from StreamNative will explore how the serverless paradigm is applied to event streaming in Apache Pulsar, a next-generation event streaming system. Pulsar provides native support for serverless functions where the events are processed as soon as they arrive in a streaming manner and that provides flexible deployment options (thread, process, container). He will describe how these serverless functions make data engineering easier and share the real world usage of Pulsar Functions.
Kafka Streams: the easiest way to start with stream processing
Stream processing is getting more & more important in our data-centric systems. In the world of Big Data, batch processing is not enough anymore - everyone needs interactive, real-time analytics for making critical business decisions, as well as providing great features to the customers.
There are many stream processing frameworks available nowadays, but the cost of provisioning infrastructure and maintaining distributed computations is usually very high. Sometimes you just have to satisfy some specific requirements, like using HDFS or YARN.
Apache Kafka is de facto a standard for building data pipelines. Kafka Streams is a lightweight library (available since 0.10) that uses powerful Kafka abstractions internally and doesn't require any complex setup or special infrastructure - you just deploy it like any other regular application.
In this session I want to talk about the goals behind stream processing, basic techniques and some best practices. Then I'm going to explain main fundamental concepts behind Kafka and explore Kafka Streams syntax and streaming features. By the end of the session you'll be able to write stream processing applications in your domain, especially if you already use Kafka as your data pipeline.
Reactor, Reactive streams and MicroServices
This document discusses Reactive Programming and Reactive Streams. It introduces Reactor, a reactive programming framework, and how it addresses issues like latency in microservices architectures. Reactive Streams provide an interoperable way to work with asynchronous data streams in a non-blocking manner. Streams represent sequences of data that can be processed reactively through operators like map and filter.
Developing real-time data pipelines with Spring and Kafka
Talk given at the Apache Kafka NYC Meetup, October 20, 2015.
http://www.meetup.com/Apache-Kafka-NYC/events/225697500/
Kafka has emerged as a clear choice for a high-throughput, low latency messaging system that addresses the needs of high-performance streaming applications. The Spring Framework has been, in the last decade, the de-facto standard for developing enterprise Java applications, providing a simple and powerful programming model that allows developers to focus on the business needs, leaving the boilerplate and middleware integration to the framework itself. In fact, it has evolved into a rich and powerful ecosystem, with projects focusing on specific aspects of enterprise software development - like Spring Boot, Spring Data, Spring Integration, Spring XD, Spring Cloud Stream/Data Flow to name just a few.
In this presentation, Marius Bogoevici from the Spring team will take the perspective of the Kafka user, and show, with live demos, how the various projects in the Spring ecosystem address their needs:
- how to build simple data integration applications using Spring Integration Kafka;
- how to build sophisticated data pipelines with Spring XD and Kafka;
- how to build cloud native message-driven microservices using Spring Cloud Stream and Kafka, and how to orchestrate them using Spring Cloud Data Flow;
What's hot (20)
Pulsar Architectural Patterns for CI/CD Automation and Self-Service_Devin Bost
Pulsar Architectural Patterns for CI/CD Automation and Self-Service_Devin Bost
Kafka Summit NYC 2017 - Easy, Scalable, Fault-tolerant Stream Processing with...
Kafka Summit NYC 2017 - Easy, Scalable, Fault-tolerant Stream Processing with...
Building a Messaging Solutions for OVHcloud with Apache Pulsar_Pierre Zemb
Building a Messaging Solutions for OVHcloud with Apache Pulsar_Pierre Zemb
Real Time Streaming Data with Kafka and TensorFlow (Yong Tang, MobileIron) Ka...
Real Time Streaming Data with Kafka and TensorFlow (Yong Tang, MobileIron) Ka...
Advanced Streaming Analytics with Apache Flink and Apache Kafka, Stephan Ewen
Advanced Streaming Analytics with Apache Flink and Apache Kafka, Stephan Ewen
Introducing Exactly Once Semantics in Apache Kafka with Matthias J. Sax
Introducing Exactly Once Semantics in Apache Kafka with Matthias J. Sax
Distributed Real-Time Stream Processing: Why and How 2.0
Distributed Real-Time Stream Processing: Why and How 2.0
Kafka Streams: the easiest way to start with stream processing
Kafka Streams: the easiest way to start with stream processing
Developing real-time data pipelines with Spring and Kafka
Developing real-time data pipelines with Spring and Kafka
Viewers also liked
Comparative Analysis of Personal Firewalls
This thesis describes the analysis of 18 personal firewalls. It discovers the differences in their behaviour while they are under various techniques of port scanning and Denial of Service (DoS) attacks. With port scanning, the detection ability, time consumption, leaked port states and obfuscation techniques are analysed. With using different DoS attacks, performance measurements of CPU and network adapter are taken. The potential of firewall fingerprinting based on the different behaviour across multiple products is also addressed.
Reactconf 2014 - Event Stream Processing
Presentation from reactconf 2014 in San Francisco.
Covers Event Stream Processing, some of the theory behind it and some implementation details in the context of local and distributed. Also covers some Big Data technologies
Debs 2011 tutorial on non functional properties of event processing
The document discusses various non-functional properties of event processing systems including performance, scalability, availability, usability, and security considerations. It covers topics such as performance benchmarks and indicators, approaches to scaling systems both vertically and horizontally, high availability techniques using redundancy and duplication, usability factors like learnability and satisfaction, and validation methods for ensuring correctness.
Tutorial in DEBS 2008 - Event Processing Patterns
1. The IBM Haifa Research Lab focuses on event processing.
2. It discusses three major building blocks of event processing systems: event producers, an event processing network, and event consumers.
3. The document provides examples of using event processing to detect patterns in customer requests to identify potentially unhappy customers.
Installing Complex Event Processing On Linux
The document is a 14 page guide for installing Oracle Complex Event Processing on Linux written by Osama Mustafa, an Oracle ACE who is a database specialist and certified ethical hacker. It provides background on the author and states that Oracle Event Processing is a solution for building applications that can filter, correlate and process events in real-time using true real-time intelligence. The document provides step-by-step instructions over 14 pages.
Access control attacks by nor liyana binti azman
This document discusses different types of access control attacks, including backdoors, spoofing attacks, man-in-the-middle attacks, replays, and TCP hijacking. Backdoors involve bypassing authentication to gain illegal access. Spoofing involves pretending to be someone else to access restricted resources. Man-in-the-middle attacks involve intercepting and relaying messages between victims to make them think they are communicating directly. Replays involve resending valid transmissions to exploit the system. TCP hijacking takes over user sessions by obtaining session IDs. Examples and video links are provided for each type of attack.
Session hijacking
This document discusses session hijacking, including the 3-way handshake in TCP, types of session hijacking like predictable tokens and man-in-the-middle attacks, methods for hijacking a session by sniffing packets and predicting sequence numbers, mitigations like HTTPS and VPNs, tools for hijacking sessions including Firesheep, and provides a link to download Firesheep.
Ceh v8 labs module 03 scanning networks
Vulnerability scanning evaluates an organization's systems and network to identify vulnerabilities such as missing patches, unnecessary services, weak authentication, and weak encryption. The document discusses using the Advanced IP Scanner tool to perform a network scan on a target Windows Server 2008 system from a Windows 8 attacker system to check for live systems, open ports, and gather information about computers on the local network. It provides instructions on launching Advanced IP Scanner, entering an IP address range to scan, and viewing the scan results.
Chapter 12
The document discusses network security tools in Linux including port scanners, packet sniffers, and intrusion detection systems. Port scanners like Nmap can identify services on a system by probing open ports, while packet sniffers such as Ethereal examine all network traffic. Intrusion detection software watches for intrusion attempts, with PortSentry monitoring for port scans and LIDS securing the system. System administrators can use these tools along with security audits to test vulnerabilities and improve their network security.
CyberLab CCEH Session - 3 Scanning Networks
This document appears to be the title slide for a presentation on scanning networks. It includes the main title "Title of the Presentation" and subtitle "SUBTITLE OF THE PRESENTATION" as well as information on the session number and topic which is scanning networks. The document also includes a website URL for an organization called CyberLabZone.
Nmap scripting engine
This document provides an overview and agenda for a training on the Nmap Scripting Engine (NSE). It begins with a 10 minute introduction to Nmap, covering what Nmap is used for and some basic scan options. Next, it spends 20 minutes reviewing the existing NSE script categories and how to use available scripts, demonstrating two sample scripts. Finally, it dedicates 20 minutes to explaining how to write your own NSE script, including the basic structure and providing an example of writing a script to find the website title.
Debs2009 Event Processing Languages Tutorial
The document outlines a tutorial on event processing languages. It discusses different styles of event processing languages including stream processing languages, rule oriented languages, and agent oriented languages. For stream processing languages, it covers key concepts like events, state, computational models, and programming models. It provides examples of stream processing languages like Esper, Coral8, and CCL.
Analizadores de Protocolos
Este documento describe varios analizadores de protocolos como NetworkMiner, Wireshark, TCPDump y SSLDUMP. Explica que un analizador de protocolos permite supervisar el tráfico de red capturando la información que circula por la red al poner la interfaz de red en modo promiscuo. También describe algunas características y usos de Wireshark y otros analizadores como Appsniffing, Observer y SuperAgent.
Tutoriel esper
Introduction au traitement des flux d'événements, et mise en oeuvre à travers un scénario, utilisant le système Esper. (http://esper.codehaus.org/)
Why Data Virtualization Is Good For Big Data Analytics?
What Is Data Virtualization ?
Data virtualization, like any virtualization, is an approach that allows you to access, administer, and optimize a heterogeneous infrastructure as if it were a single, logically unified resource. This enables you to abstract the external interface from the internal implementation of some service, functionality, or other resource.
Scanning with nmap
Nmap is an open source tool that scans networks to identify devices, services, and operating systems. It works by crafting custom IP packets with different flags using raw sockets to elicit responses that provide information not otherwise available. Nmap can perform various types of scans, identify hosts and services, detect firewalls and IDS, and determine operating systems through detailed analysis of responses. It provides flexible output options and techniques for advanced scanning, packet alteration, and timing control.
Module 3 Scanning
The document discusses scanning techniques used during penetration testing and hacking. It defines different types of scanning like port scanning, network scanning, and vulnerability scanning. It describes tools like Nmap that can be used to perform these scans and examines techniques like SYN scanning, XMAS scanning, NULL scanning, and IDLE scanning. The document also discusses using proxies and anonymizers to hide one's location while scanning and ways to document results like creating network diagrams of vulnerable systems.
Port Scanning Overview
Port scanning involves attempting to connect to ports on a target system to discover which ports are open and what services they correspond to. It is done by software that scans a range of ports, usually 0 to 65,536, and analyzes responses to determine whether ports are open, closed, or filtered. Common port scanning tools include Nmap and Netcat. While port scanning can be used maliciously for hacking, it is also used by system administrators to diagnose network issues.
Optimizing Your SOA with Event Processing
Optimizing Your SOA with Event Processing, TIBCO, TUCON 2007, Tim Bass, Principal Global Architect, Director Emerging Technologies Group TIBCO Software Inc.
Viewers also liked (20)
Debs 2011 tutorial on non functional properties of event processing
Debs 2011 tutorial on non functional properties of event processing
Why Data Virtualization Is Good For Big Data Analytics?
Why Data Virtualization Is Good For Big Data Analytics?
Building Real-time CEP Application with Open Source Projects
Building Real-time CEP Application with Open Source Projects
Similar to Complex Event Processing with Esper and WSO2 ESB
Wso2 Scenarios Esb Webinar July 1st
The document describes three real-world integration scenarios using WSO2 ESB:
1) XML transformation and message augmentation using an XML transformation and database lookup to enrich messages.
2) A financial services use case of reading legacy flat files and integrating with JMS. It involves splitting, iterating, transforming, and sending messages.
3) The PushMePullYou scenario uses polling to integrate two services, involving task configuration, calling external services, splitting/transforming messages, and aggregating responses.
Aceu2009 Apache Synapse Events
Event driven architectures allow for empowered distributed systems where each component can own its own integration logic and wiring. With an event driven approach, components act as actuators that emit events and sensors that subscribe to events. Apache Synapse supports event driven integration through its concept of event sources that act as event publishers and a subscription manager that routes events to subscribers. Complex event processing technologies like Esper can analyze event streams across multiple sources to detect patterns and correlations.
Janus Mobile e-Seminar
Janus is a tool that allows Flash developers to create and deploy applications for Windows, Pocket PC and Symbian devices from their SWF files. It expands ActionScript functionality and allows embedding files and customizing applications. For Symbian, it can generate signed installation files and extend Flash Lite capabilities. For Pocket PC, it supports Flash Lite, Flash Player 7, and provides classes to access device features and databases. Code samples demonstrate taking screenshots and intercepting SMS messages on devices.
Fast SOA with Apache Synapse
Presented at ApacheCon Europe 2008: this presentation introduces the ApacheCon Synapse ESB and the performance benefits of the design.
FMS Administration Seminar
This document provides an overview of Flash Media Server, including what it is, its versions and types, how to configure applications and streams, and how to install and set up the server. It discusses topics like RTMP, shared objects, NetStream connections, and database integration. Code examples are provided for connecting to the server and playing streams.
Introduction to Actionscript3
The document summarizes the evolution of ActionScript and Flash Player over multiple versions, highlighting some of the key features introduced at each stage:
- Flash Player versions 4-7 introduced timeline control, loading movies, and scripting capabilities like variables and conditions. ActionScript 1 was introduced in Flash Player 5.
- Flash Player 7 introduced ActionScript 2 with inheritance, CSS support, and RTMP. Flash Player 9 introduced ActionScript 3 with a new virtual machine, just-in-time compilation, and ECMAScript for XML support.
- ActionScript 3 brought strict typing, classes, packages and other improvements making it fully object-oriented compared to previous versions. It also introduced significant performance enhancements over
Silicon Valley CodeCamp 2008: High performance Ajax with ExtJS and ASP.NET
This document provides an overview and introduction to using ExtJS with ASP.NET for building high performance AJAX applications. It covers setting up Visual Studio for JavaScript development, debugging techniques, an introduction to ExtJS, examples of ExtJS components like grids and form panels, handling errors, and tips for optimizing AJAX performance.
Java XML Parsing
This document summarizes key aspects of XML including:
- XML is a text-based format for describing data structures that is both human and machine readable.
- XML became a W3C standard in 1998 and is commonly used for exchanging data between disparate systems.
- Java can be used to generate, access, format, parse, validate, and transform XML data.
- XML documents have a root element containing other nested elements and attributes to describe hierarchical data.
- Well-formed XML documents follow syntax rules for proper nesting of start/end tags and quotes around attribute values.
- XML parsers like SAX and DOM are used to read XML documents sequentially or build a navigable tree structure in memory
Open Source XMPP for Cloud Services
Matt Tucker discusses how XMPP (Jabber) can be used for cloud services and architectures. Some key benefits of XMPP over traditional web services include its support for real-time bidirectional communication, presence, and easier firewall traversal. Open source XMPP servers like Openfire and client libraries provide tools to build scalable cloud components and services. Examples like Twitter's use of XMPP for its firehose API demonstrate how XMPP can enable new types of cloud applications.
Sax Dom Tutorial
This document provides an overview of DOM and SAX, two common XML APIs in Java. It describes the key differences between DOM and SAX, including that DOM builds an in-memory tree representation, while SAX parses the XML as a stream of events. The document also provides code examples for using SAX to parse XML and extract data, and examples of how to access and manipulate DOM trees after parsing XML.
Processing XML with Java
The document discusses different XML parsers in Java including DOM, SAX, and StAX. DOM represents the XML document as an in-memory tree which allows flexible processing but uses more memory. SAX is event-driven and reads the XML sequentially using less memory. StAX is similar to SAX but simplified and "pull"-based where the developer manually navigates elements. The document also covers using JAXP for XML processing independence and the key classes involved in DOM and StAX parsing.
A Deep Dive into Structured Streaming: Apache Spark Meetup at Bloomberg 2016
Tathagata 'TD' Das presented at Bay Area Apache Spark Meetup. This talk covers the merits and motivations of Structured Streaming, and how you can start writing end-to-end continuous applications using Structured Streaming APIs.
Rapid JCR Applications Development with Sling
The document discusses the Java Content Repository (JCR) and the Sling framework, which builds on JCR to enable scriptable web applications. It provides an overview of using Sling to develop a simple blog application with only 46 lines of code through JavaScript scripts and RESTful interfaces. The document also demonstrates more advanced features of Sling like content observation and generation of image thumbnails through an OSGi bundle.
Winning the Lottery with Spring: A Microservices Case Study for the Dutch Lot...
SpringOne 2021
Session Title: Winning the Lottery with Spring: A Microservices Case Study for the Dutch Lotteries
Speaker: Joris Kuipers, CTO at Trifork
Let's Peel Mangos
Samidip Basu is a manager at Columbus ADI Practice Solutions and Microsoft mobility lead. The document discusses upcoming features in Windows Phone SDK 7.1 including push notifications, live tiles, execution model, sensors, location services, and integration of Silverlight and XNA. It encourages developers to try out the new features and capabilities in the Windows Phone platform.
Implementing the Genetic Algorithm in XSLT: PoC
The document discusses using a genetic algorithm to automatically compose SOA applications by evolving XSLT programs. It describes setting up an initial population of random XSLT documents, transforming source XML with each XSLT to get result XML, comparing result XML to target XML via XML diff to calculate fitness, selecting individuals for reproduction and crossover to create new offspring, and repeating over generations to evolve higher fitness XSLT programs. The goal is to evolve an XSLT that can transform source XML into target XML.
A Deep Dive into Structured Streaming in Apache Spark
This document provides an overview of Structured Streaming in Apache Spark. It begins with a brief history of streaming in Spark and outlines some of the limitations of the previous DStream API. It then introduces the new Structured Streaming API, which allows for continuous queries to be expressed as standard Spark SQL queries against continuously arriving data. It describes the new processing model and how queries are executed incrementally. It also covers features like event-time processing, windows, joins, and fault-tolerance guarantees through checkpointing and write-ahead logging. Overall, the document presents Structured Streaming as providing a simpler way to perform streaming analytics by allowing streaming queries to be expressed using the same APIs as batch queries.
Scalaxb preso
The document discusses SOAP, describing it as a protocol specification for exchanging structured information in web services using XML format. It outlines the key parts of a SOAP message including the envelope, header, body and optional fault. The document then provides an example SOAP request and discusses how WSDL and XSD describe the structure and data types of a web service. It evaluates different options for working with SOAP in Scala including rolling your own implementation, using JAXB, Apache CXF, and ScalaXB which generates case classes. Finally, it notes some common pitfalls like Sax parsing errors and timeouts when interacting with web services.
5 xml parsing
The document discusses XML parsing and processing using different approaches like SAX, DOM and StAX. It explains that XML parsers allow handling XML content in a desired manner by shielding programmers from manually parsing XML. SAX is an event-based API for sequential XML parsing while DOM represents XML as a tree of nodes. StAX is a pull-based parsing approach controlled by the application. The key aspects of SAX parsing in JAXP like creating parsers, handlers and parsing XML files are also covered.
EclipseCon 2008: Fundamentals of the Eclipse Modeling Framework
EclipseCon 2008 long talk introducing EMF. A slightly different take from the preceding EclipseWorld talk, with a few more details on the EMF core and no discussion of CDO or Teneo.
Similar to Complex Event Processing with Esper and WSO2 ESB (20)
Silicon Valley CodeCamp 2008: High performance Ajax with ExtJS and ASP.NET
Silicon Valley CodeCamp 2008: High performance Ajax with ExtJS and ASP.NET
A Deep Dive into Structured Streaming: Apache Spark Meetup at Bloomberg 2016
A Deep Dive into Structured Streaming: Apache Spark Meetup at Bloomberg 2016
Winning the Lottery with Spring: A Microservices Case Study for the Dutch Lot...
Winning the Lottery with Spring: A Microservices Case Study for the Dutch Lot...
A Deep Dive into Structured Streaming in Apache Spark
A Deep Dive into Structured Streaming in Apache Spark
EclipseCon 2008: Fundamentals of the Eclipse Modeling Framework
EclipseCon 2008: Fundamentals of the Eclipse Modeling Framework
More from Prabath Siriwardena
Microservices Security Landscape
This document discusses security in microservices architectures. It begins by comparing monolithic and microservices approaches. It then covers edge security using API gateways and OAuth 2.0 authorization. Other topics include service-to-service security using TLS mutual authentication, JSON Web Tokens (JWTs), and SPIFFE/SPIRE for identity management. Patterns like JWTs, nested JWTs, and token exchange are presented for secure communication between services. The document also discusses access control and policy evaluation using approaches like embedded policy decision points and the Open Policy Agent (OPA) framework.
Cloud Native Identity with SPIFFE
This document discusses Cloud Native Identity Management using SPIFFE (Secure Production Identity Framework for Everyone) and SPIRE (SPIFFE Runtime Environment). It provides an overview of SPIFFE and SPIRE, including how they address identity management challenges in cloud-native environments. It then summarizes how SPIRE implements the SPIFFE specifications through a node attestation and workload attestation process where a SPIRE agent authenticates to a server, retrieves selectors to verify workloads, and issues signed identity documents when a workload matches the selectors.
API Security Best Practices & Guidelines
This document provides guidelines for securing managed APIs. It discusses defining an API's audience and whether they are direct users or relying parties. It also covers bootstrapping trust either directly through user credentials or brokerd through a third party. The document then discusses various OAuth 2.0 grant types and federated access scenarios. It emphasizes using TLS, strong credentials, short-lived tokens, and access control to secure APIs and their communication.
Identity is Eating the World!
This document discusses trends in identity and access management. It notes that identity standards like OpenID Connect are rising while others like SAML are fading. Authentication methods are shifting from traditional multi-factor authentication to continuous adaptive authentication. Privacy and centralized identity systems are concerns driving interest in self-sovereign identity models using blockchain and decentralized identifiers. Large countries are implementing national digital identity systems and blockchain may help improve transparency. The role of mobile identity is growing as phone numbers become integral identifiers.
Microservices Security Landscape
This document discusses security considerations for microservices architectures. It covers edge security using API gateways, service-to-service authentication using TLS and JWT, access control using centralized and embedded policy decision points, deployment models like Docker and Kubernetes, and the use of sidecars and service meshes like Istio for security. Key challenges with microservices include a broader attack surface, performance issues, and complexity in deployment and observability across services.
OAuth 2.0 Threat Landscape
The document discusses OAuth 2.0 security threats including session injection with CSRF, token leakage, token reuse/misuse, and token export. It provides details on the threats, victims, and best practices for mitigation. These include using short-lived authorization codes, TLS, proof-key-for-code-exchange, limiting grant types by client, white-listing redirect URIs, token binding, and enforcing throttle limits to prevent token misuse. The presentation encourages attendees to review their OAuth 2.0 implementations against these threats and ensure authorization servers follow security best practices.
GDPR for Identity Architects
This document provides an overview of the General Data Protection Regulation (GDPR) for identity architects. It discusses GDPR requirements such as data protection impact assessments, data processing records that must be maintained by controllers and processors, and data subject's rights. It also discusses principles of identity and access management design and best practices related to GDPR compliance. Finally, it includes links to Facebook and Google's cookie and data use policies.
Blockchain-based Solutions for Identity & Access Management
This document discusses self-sovereign identity and decentralized identifiers (DIDs). It provides an overview of identity evolution from centralized to user-centric models. Self-sovereign identity allows individuals to control their digital identities across systems without relying on centralized authorities. DIDs are a new type of identifier that can be registered on a distributed ledger without a centralized registration authority. The document outlines the goals and components of DID specifications and describes how DIDs and verifiable claims work on networks like Sovrin to enable self-sovereign identity.
OAuth 2.0 Threat Landscapes
This document summarizes OAuth 2.0 threat landscapes and best practices for mitigation. It discusses threats such as CSRF, session injection, token leakage, IDP mix-up, and token reuse/misuse. Recommended mitigations include using the state parameter, PKCE, short-lived tokens, TLS, white-listing callback URLs, scoped tokens, audience restriction, OpenID Connect, and throttling. The document provides technical details on various OAuth 2.0 flows and threats as well as references to relevant IETF draft specifications.
OAuth 2.0 for Web and Native (Mobile) App Developers
1. The evolution of identity delegation protocols.
2. Understanding OAuth 2.0 fundamentals
3. OAuth 2.0 profiles and OpenID Connect
4. Securing Single Page Applications (SPAs) with OAuth 2.0
5. OAuth 2.0 for Native Apps
6. OAuth 2.0 threat landscapes
7. User Managed Access (UMA)
Identity Management for Web Application Developers
Prabath Siriwardena is an expert in identity management who has authored books and articles on the topic. This document discusses key concepts in identity management including identity landscapes, federation, provisioning, access control, and governance. It also provides an overview of the speaker and describes various identity management demonstrations that will be shown, such as single sign-on, provisioning, access control, and securing APIs with OAuth.
API Security Best Practices & Guidelines
This document provides API security best practices and guidelines. It discusses defining APIs and who may access them, such as employees, partners, customers or the general public. Authentication can be direct, using credentials, or brokered, using a third party. Best practices include using TLS, strong credentials, short-lived tokens, and throttling access. The guidelines aim to prevent attacks like CSRF, authorization code interception, and brute force attacks through measures like state parameters, PKCE, and long random tokens.
Open Standards in Identity Management
WSO2 produces open source identity and access management software. Through Google Summer of Code, WSO2 has mentored 11 projects implementing key identity standards like UMA, SAML, and OAuth. These standards, developed by organizations like OASIS and IETF, provide frameworks for identity federation, SSO, provisioning, and access control. Formats include SAML for SSO, SCIM for provisioning using REST, and XACML for fine-grained authorization control. WSO2 contributes implementations of these standards to help users manage identity and access securely across domains.
Securing Single-Page Applications with OAuth 2.0
This document discusses securing single-page applications (SPAs) with OAuth 2.0. It describes how SPAs work by loading a single HTML page and dynamically updating content without page reloads. It outlines two issues with using OAuth in SPAs: client authentication and exposing access tokens to users. It proposes using the implicit grant type to address these by enabling single sign-on without exposing credentials to the SPA. It also describes using an OAuth proxy to further improve security by encrypting tokens in cookies and routing all API calls through the proxy.
API Security : Patterns and Practices
The document discusses API security patterns and practices. It covers topics like API gateways, authentication methods like basic authentication and OAuth 2.0, authorization with XACML policies, and securing APIs through measures like TLS, JWTs, and throttling to ensure authentication, authorization, confidentiality, integrity, non-repudiation, and availability. Key points covered include the gateway pattern, direct vs brokered authentication, JSON web tokens for self-contained access tokens, and combining OAuth and XACML for fine-grained access control.
Best Practices in Building an API Security Ecosystem
Enterprise API adoption has gone beyond predictions. It has become the 'coolest' way of exposing business functionalities to the outside world. Both your public and private APIs, need to be protected, monitored and managed.
This session focuses on API Security. There are so many options out there to make someone easily confused. When to select one over the other is always a question - and you need to deal with it quite carefully to identify and isolate the tradeoffs. Security is not an afterthought. It has to be an integral part of any development project - so as for APIs. API security has evolved a lot in last five years. This talk covers best practices in building an API Security Ecosystem with OAuth 2.0, UMA, SCIM, XACML and LDAP.
Connected Identity : The Role of the Identity Bus
The document discusses the role of an identity broker and its key functions. An identity broker acts as a centralized hub that can connect to multiple identity providers and service providers in a protocol-agnostic manner. It allows for identity federation across different protocols and systems. The broker supports important identity management capabilities like claim transformation, home realm discovery, multi-factor authentication, adaptive authentication, identity mapping, attribute aggregation, and just-in-time provisioning in a centralized manner. Fifteen fundamentals of the identity broker pattern are described. The document also discusses the concept of an identity mediation language and seven fundamentals of future identity and access management.
Connected Identity : Benefits, Risks & Challenges
Connected business is a very dynamic and complex environment. Your desire is to reach out to your customers, partners, distributors and suppliers and create more and more business interactions and activities, that will generate more revenue. The goal here is not just integrate technological silos, in your enterprise – but also make your business more accessible and reactive. The ability to propagate identities across borders in a protocol-agnostic manner is a core ingredient in producing a connected business environment.
SAML, OpenID, OpenID Connect, WS-Federation all support identity federation – cross domain authentication. But, can we always expect all the parties in a connected environment to support SAML, OpenID or OpenID Connect ? Most of the federation systems we see today are in silos. It can be a silo of SAML federation, a silo of OpenID Connect federation or a silo of OpenID federation.
Even in a given federation silo how do you scale with increasing number of service providers and identity providers? Each service provider has to trust each identity provider and this leads into the Spaghetti Identity anti-pattern.
Federation Silos and Spaghetti Identity are two anti-patterns that needs to be addressed in a connected environment.
This talk will present benefits, risks and challenges in a connected identity environment
The Evolution of Internet Identity
The document outlines the evolution of internet identity from 1997 to the present, listing key events, technologies, and standards. It starts with early services like Hotmail and Yahoo in 1997 and continues through the development of platforms and standards like OpenID, OAuth, SAML, and newer initiatives for decentralized identity. The timeline shows the ongoing progression of identity management online from the first webmail services to current innovations in user-centric identity and authentication.
Next-Gen Apps with IoT and Cloud
This document discusses next generation applications with Internet of Things (IoT) and cloud technologies. It notes that the number of devices connected to the internet exceeded the world's population in 2008 and will reach 50 billion devices connected by 2020. These devices will generate zetabytes of data. IoT applications are built on tiers including devices, integration, analytics and presentation layers facilitated through APIs. The document outlines WSO2's reference architecture for IoT and application development, as well as an app store and publisher components for a connected enterprise.
More from Prabath Siriwardena (20)
Blockchain-based Solutions for Identity & Access Management
Blockchain-based Solutions for Identity & Access Management
OAuth 2.0 for Web and Native (Mobile) App Developers
OAuth 2.0 for Web and Native (Mobile) App Developers
Identity Management for Web Application Developers
Identity Management for Web Application Developers
Best Practices in Building an API Security Ecosystem
Best Practices in Building an API Security Ecosystem
Recently uploaded
HCL Notes and Domino License Cost Reduction in the World of DLAU
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-and-domino-license-cost-reduction-in-the-world-of-dlau/
The introduction of DLAU and the CCB & CCX licensing model caused quite a stir in the HCL community. As a Notes and Domino customer, you may have faced challenges with unexpected user counts and license costs. You probably have questions on how this new licensing approach works and how to benefit from it. Most importantly, you likely have budget constraints and want to save money where possible. Don’t worry, we can help with all of this!
We’ll show you how to fix common misconfigurations that cause higher-than-expected user counts, and how to identify accounts which you can deactivate to save money. There are also frequent patterns that can cause unnecessary cost, like using a person document instead of a mail-in for shared mailboxes. We’ll provide examples and solutions for those as well. And naturally we’ll explain the new licensing model.
Join HCL Ambassador Marc Thomas in this webinar with a special guest appearance from Franz Walder. It will give you the tools and know-how to stay on top of what is going on with Domino licensing. You will be able lower your cost through an optimized configuration and keep it low going forward.
These topics will be covered
- Reducing license cost by finding and fixing misconfigurations and superfluous accounts
- How do CCB and CCX licenses really work?
- Understanding the DLAU tool and how to best utilize it
- Tips for common problem areas, like team mailboxes, functional/test users, etc
- Practical examples and best practices to implement right away
Best 20 SEO Techniques To Improve Website Visibility In SERP
Boost your website's visibility with proven SEO techniques! Our latest blog dives into essential strategies to enhance your online presence, increase traffic, and rank higher on search engines. From keyword optimization to quality content creation, learn how to make your site stand out in the crowded digital landscape. Discover actionable tips and expert insights to elevate your SEO game.
5th LF Energy Power Grid Model Meet-up Slides
5th Power Grid Model Meet-up
It is with great pleasure that we extend to you an invitation to the 5th Power Grid Model Meet-up, scheduled for 6th June 2024. This event will adopt a hybrid format, allowing participants to join us either through an online Mircosoft Teams session or in person at TU/e located at Den Dolech 2, Eindhoven, Netherlands. The meet-up will be hosted by Eindhoven University of Technology (TU/e), a research university specializing in engineering science & technology.
Power Grid Model
The global energy transition is placing new and unprecedented demands on Distribution System Operators (DSOs). Alongside upgrades to grid capacity, processes such as digitization, capacity optimization, and congestion management are becoming vital for delivering reliable services.
Power Grid Model is an open source project from Linux Foundation Energy and provides a calculation engine that is increasingly essential for DSOs. It offers a standards-based foundation enabling real-time power systems analysis, simulations of electrical power grids, and sophisticated what-if analysis. In addition, it enables in-depth studies and analysis of the electrical power grid’s behavior and performance. This comprehensive model incorporates essential factors such as power generation capacity, electrical losses, voltage levels, power flows, and system stability.
Power Grid Model is currently being applied in a wide variety of use cases, including grid planning, expansion, reliability, and congestion studies. It can also help in analyzing the impact of renewable energy integration, assessing the effects of disturbances or faults, and developing strategies for grid control and optimization.
What to expect
For the upcoming meetup we are organizing, we have an exciting lineup of activities planned:
-Insightful presentations covering two practical applications of the Power Grid Model.
-An update on the latest advancements in Power Grid -Model technology during the first and second quarters of 2024.
-An interactive brainstorming session to discuss and propose new feature requests.
-An opportunity to connect with fellow Power Grid Model enthusiasts and users.
Introduction of Cybersecurity with OSS at Code Europe 2024
I develop the Ruby programming language, RubyGems, and Bundler, which are package managers for Ruby. Today, I will introduce how to enhance the security of your application using open-source software (OSS) examples from Ruby and RubyGems.
The first topic is CVE (Common Vulnerabilities and Exposures). I have published CVEs many times. But what exactly is a CVE? I'll provide a basic understanding of CVEs and explain how to detect and handle vulnerabilities in OSS.
Next, let's discuss package managers. Package managers play a critical role in the OSS ecosystem. I'll explain how to manage library dependencies in your application.
I'll share insights into how the Ruby and RubyGems core team works to keep our ecosystem safe. By the end of this talk, you'll have a better understanding of how to safeguard your code.
Webinar: Designing a schema for a Data Warehouse
Are you new to data warehouses (DWH)? Do you need to check whether your data warehouse follows the best practices for a good design? In both cases, this webinar is for you.
A data warehouse is a central relational database that contains all measurements about a business or an organisation. This data comes from a variety of heterogeneous data sources, which includes databases of any type that back the applications used by the company, data files exported by some applications, or APIs provided by internal or external services.
But designing a data warehouse correctly is a hard task, which requires gathering information about the business processes that need to be analysed in the first place. These processes must be translated into so-called star schemas, which means, denormalised databases where each table represents a dimension or facts.
We will discuss these topics:
- How to gather information about a business;
- Understanding dictionaries and how to identify business entities;
- Dimensions and facts;
- Setting a table granularity;
- Types of facts;
- Types of dimensions;
- Snowflakes and how to avoid them;
- Expanding existing dimensions and facts.
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays, June 2024 with Maria Copot 20
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
A Mix Chart displays historical data of numbers in a graphical or tabular form. The Kalyan Rajdhani Mix Chart specifically shows the results of a sequence of numbers over different periods.
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Discover how MongoDB Atlas and vector search technology can revolutionize your application's search capabilities. This comprehensive presentation covers:
* What is Vector Search?
* Importance and benefits of vector search
* Practical use cases across various industries
* Step-by-step implementation guide
* Live demos with code snippets
* Enhancing LLM capabilities with vector search
* Best practices and optimization strategies
Perfect for developers, AI enthusiasts, and tech leaders. Learn how to leverage MongoDB Atlas to deliver highly relevant, context-aware search results, transforming your data retrieval process. Stay ahead in tech innovation and maximize the potential of your applications.
#MongoDB #VectorSearch #AI #SemanticSearch #TechInnovation #DataScience #LLM #MachineLearning #SearchTechnology
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...Edge AI and Vision Alliance
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/building-and-scaling-ai-applications-with-the-nx-ai-manager-a-presentation-from-network-optix/
Robin van Emden, Senior Director of Data Science at Network Optix, presents the “Building and Scaling AI Applications with the Nx AI Manager,” tutorial at the May 2024 Embedded Vision Summit.
In this presentation, van Emden covers the basics of scaling edge AI solutions using the Nx tool kit. He emphasizes the process of developing AI models and deploying them globally. He also showcases the conversion of AI models and the creation of effective edge AI pipelines, with a focus on pre-processing, model conversion, selecting the appropriate inference engine for the target hardware and post-processing.
van Emden shows how Nx can simplify the developer’s life and facilitate a rapid transition from concept to production-ready applications.He provides valuable insights into developing scalable and efficient edge AI solutions, with a strong focus on practical implementation.How to Get CNIC Information System with Paksim Ga.pptx
Pakdata Cf is a groundbreaking system designed to streamline and facilitate access to CNIC information. This innovative platform leverages advanced technology to provide users with efficient and secure access to their CNIC details.
Artificial Intelligence for XMLDevelopment
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
Building Production Ready Search Pipelines with Spark and Milvus
Spark is the widely used ETL tool for processing, indexing and ingesting data to serving stack for search. Milvus is the production-ready open-source vector database. In this talk we will show how to use Spark to process unstructured data to extract vector representations, and push the vectors to Milvus vector database for search serving.
Presentation of the OECD Artificial Intelligence Review of Germany
Consult the full report at https://www.oecd.org/digital/oecd-artificial-intelligence-review-of-germany-609808d6-en.htm
OpenID AuthZEN Interop Read Out - Authorization
During Identiverse 2024 and EIC 2024, members of the OpenID AuthZEN WG got together and demoed their authorization endpoints conforming to the AuthZEN API
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on integration of Salesforce with Bonterra Impact Management.
Interested in deploying an integration with Salesforce for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Recently uploaded (20)
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Complex Event Processing with Esper and WSO2 ESB
- 1. Complex Event Processing with Esper and WSO2 ESB Paul Fremantle, CTO, WSO2 29 th July 2008
- 4. WSO2 ESB Configured through the simple UI
- 11. Esper config for Axiom <esper-configuration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.espertech.com/schema/esper" xsi:noNamespaceSchemaLocation="esper-configuration-2-0.xsd"> <plugin-event-representation uri="type://xml/apacheaxiom/OMNode" class-name="com.espertech.esperio.representation.axiom.AxiomEventRepresentation"/> <plugin-event-type alias=“Ticker"> <resolution-uri value="type://xml/apacheaxiom/OMNode"/> <initializer> <xml-axiom root-element-name="getQuote" default-namespace="http://mycom.com/ns/tick"> <namespace-prefix prefix=“t" namespace= " http://mycom.com/ns/tick"/> <xpath-property property-name="symbol“ xpath="//t:tick/t:s" type="string"/> <xpath-property property-name=“price“ xpath="//t:tick/t:p" type=“number"/> </xml-axiom> </initializer> </plugin-event-type> </esper-configuration>
- 16. Simple example <task class="org.apache.synapse.startup.tasks.MessageInjector" name="Tick"> <property name="to" value="urn:tick"/> <property name="soapAction" value="urn:tick"/> <property name="message"> <tick xmlns="http://mycom.com/ns/tick"> <s>IBM</s> <p>89.6</p> </tick> </property> <trigger interval="1"/> </task>
- 17. In the console
- 18. Simple example continued <filter source="get-property('To')" regex="urn:tick"> <class name="org.sciflex.plugins.synapse.esper.mediators.AxiomMediator"> <property name="Configuration"> <esper-configuration> <!-- AS ON PREVIOUS SLIDE --> </esper-configuration> </property> <property name="statement" value="select * from Ticker.win:length_batch(10)"/> <property name="EventToAddress" value="urn:newevent"/> </class> </filter> <filter source="get-property('To')" regex="urn:newevent"> <log level="full"/> </filter> RESULT: Every ten ticks one event gets logged
- 20. Example: Looking for system monitoring events http://ganglia.info Synapse Task Ganglia gmond telnet 8649 <GANGLIA_XML> GMondPoller Job Esper Mediator SELECT AVG(CPU_USER) FROM GANGLIA urn:gmond Log Mediator urn:cpu
- 22. FIX example FIX Endpoint (Order generator) WSO2 ESB FIX Endpoint (Order acceptor) Fix Proxy <class AxiomEsperMediator..> <send/> Esper New Event JMS Topic
- 24. Questions