This document provides an overview of Sybase BAM (Business Activity Monitoring). It discusses the technology background of BAM, CEP, and RTBI. It then describes Sybase BAM's analytic model, architecture, main features, and a demo. The analytic model uses fields, rules, actions and timers to process events. The architecture includes components like the BAM engine and tools. Main features include support for complex event processing, real-time BI, alerts, visualization, metadata-driven design, and high volume processing.
During Kylin OLAP development, we setup many engineering principles in the team. These principles are very important to delivery Kylin with high quality and on schedule.
Experimentation plays a vital role in business growth at eBay by providing valuable insights and prediction on how users will reach to changes made to the eBay website and applications. On a given day, eBay has several hundred experiments running at the same time. Our experimentation data processing pipeline handles billions of rows user behavioral and transactional data per day to generate detailed reports covering 100+ metrics over 50 dimensions.
In this session, we will share our journey of how we moved this complex process from Data warehouse to Hadoop. We will give an overview of the experimentation platform and data processing pipeline. We will highlight the challenges and learnings we faced implementing this platform in Hadoop and how this transformation led us to build a scalable, flexible and reliable data processing workflow in Hadoop. We will cover our work done on performance optimizations, methods to establish resilience and configurability, efficient storage formats and choices of different frameworks used in the pipeline.
HBaseCon 2015: Industrial Internet Case Study using HBase and TSDBHBaseCon
This case study involves analysis of high-volume, continuous time-series aviation data from jet engines that consist of temperature, pressure, vibration and related parameters from the on-board sensors, joined with well-characterized slowly changing engine asset configuration data and other enterprise data for continuous engine diagnostics and analytics. This data is ingested via distributed fabric comprising transient containers, message queues and a columnar, compressed storage leveraging OpenTSDB over Apache HBase.
In Search of Database Nirvana: Challenges of Delivering HTAPHBaseCon
Rohit Jain (Esgyn)
Customers are looking for one database engine to address all their varied needs--from transactional to analytical workloads--against structured, semi-structured, and unstructured data (Gartner’s term Hybrid Transactional/Analytical Processing, or HTAP, perhaps comes closest to describing this nirvana.) But can it be achieved? The motivation of this talk is to establish a framework for assessing the maturity and capabilities of query engines on Apache Hadoop ecosystem storage engines such as HBase in meeting these diverse needs.
Apache kylin 2.0: from classic olap to real-time data warehouseYang Li
Apache Kylin, which started as a big data OLAP engine, is reaching its v2.0. Yang Li explains how, armed with snowflake schema support, a full SQL interface, spark cubing, and the ability to consume real-time streaming data, Apache Kylin is closing the gap to becoming a real-time data warehouse.
During Kylin OLAP development, we setup many engineering principles in the team. These principles are very important to delivery Kylin with high quality and on schedule.
Experimentation plays a vital role in business growth at eBay by providing valuable insights and prediction on how users will reach to changes made to the eBay website and applications. On a given day, eBay has several hundred experiments running at the same time. Our experimentation data processing pipeline handles billions of rows user behavioral and transactional data per day to generate detailed reports covering 100+ metrics over 50 dimensions.
In this session, we will share our journey of how we moved this complex process from Data warehouse to Hadoop. We will give an overview of the experimentation platform and data processing pipeline. We will highlight the challenges and learnings we faced implementing this platform in Hadoop and how this transformation led us to build a scalable, flexible and reliable data processing workflow in Hadoop. We will cover our work done on performance optimizations, methods to establish resilience and configurability, efficient storage formats and choices of different frameworks used in the pipeline.
HBaseCon 2015: Industrial Internet Case Study using HBase and TSDBHBaseCon
This case study involves analysis of high-volume, continuous time-series aviation data from jet engines that consist of temperature, pressure, vibration and related parameters from the on-board sensors, joined with well-characterized slowly changing engine asset configuration data and other enterprise data for continuous engine diagnostics and analytics. This data is ingested via distributed fabric comprising transient containers, message queues and a columnar, compressed storage leveraging OpenTSDB over Apache HBase.
In Search of Database Nirvana: Challenges of Delivering HTAPHBaseCon
Rohit Jain (Esgyn)
Customers are looking for one database engine to address all their varied needs--from transactional to analytical workloads--against structured, semi-structured, and unstructured data (Gartner’s term Hybrid Transactional/Analytical Processing, or HTAP, perhaps comes closest to describing this nirvana.) But can it be achieved? The motivation of this talk is to establish a framework for assessing the maturity and capabilities of query engines on Apache Hadoop ecosystem storage engines such as HBase in meeting these diverse needs.
Apache kylin 2.0: from classic olap to real-time data warehouseYang Li
Apache Kylin, which started as a big data OLAP engine, is reaching its v2.0. Yang Li explains how, armed with snowflake schema support, a full SQL interface, spark cubing, and the ability to consume real-time streaming data, Apache Kylin is closing the gap to becoming a real-time data warehouse.
Apache Hive is a rapidly evolving project which continues to enjoy great adoption in big data ecosystem. Although, Hive started primarily as batch ingestion and reporting tool, community is hard at work in improving it along many different dimensions and use cases. This talk will provide an overview of latest and greatest features and optimizations which have landed in project over last year. Materialized view, micro managed tables and workload management are some noteworthy features.
I will deep dive into some optimizations which promise to provide major performance gains. Support for ACID tables has also improved considerably. Although some of these features and enhancements are not novel but have existed for years in other DB systems, implementing them on Hive poses some unique challenges and results in lessons which are generally applicable in many other contexts. I will also provide a glimpse of what is expected to come in near future.
Speaker: Ashutosh Chauhan, Engineering Manager, Hortonworks
Building Apps with Distributed In-Memory Computing Using Apache GeodePivotalOpenSourceHub
Slides from the Meetup Monday March 7, 2016 just before the beginning of #GeodeSummit, where we cover an introduction of the technology and community that is Apache Geode, the in-memory data grid.
Drivetribe is the world’s digital hub for motoring, as envisioned by Jeremy Clarkson, Richard Hammond, and James May. The Drivetribe platform was designed ground up with high scalability in mind. Built on top of the Event Sourcing/CQRS pattern, the platform uses Apache Kafka as its source of truth and Apache Flink as its processing backbone. This talk aims to introduce the architecture, and elaborate on how common problems in social media, such as counting big numbers and dealing with outliers, can be resolved by a healthy mix of Flink and functional programming.
More Data, More Problems: Scaling Kafka-Mirroring Pipelines at LinkedIn confluent
(Celia Kung, LinkedIn) Kafka Summit SF 2018
For several years, LinkedIn has been using Kafka MirrorMaker as the mirroring solution for copying data between Kafka clusters across data centers. However, as LinkedIn data continued to grow, mirroring trillions of Kafka messages per day across data centers uncovered the scale limitations and operability challenges of Kafka MirrorMaker. To address such issues, we have developed a new mirroring solution, built on top our stream ingestion service, Brooklin. Brooklin MirrorMaker aims to provide improved performance and stability, while facilitating better management through finer control of data pipelines. Through flushless Kafka produce, dynamic management of data pipelines, per-partition error handling and flow control, we are able to increase throughput, better withstand consume and produce failures and reduce overall operating costs. As a result, we have eliminated the major pain points of Kafka MirrorMaker. In this talk, we will dive deeper into the challenges LinkedIn has faced with Kafka MirrorMaker, how we tackled them with Brooklin MirrorMaker and our plans for iterating further on this new mirroring solution.
Change data capture with MongoDB and Kafka.Dan Harvey
In any modern web platform you end up with a need to store different views of your data in many different datastores. I will cover how we have coped with doing this in a reliable way at State.com across a range of different languages, tools and datastores.
Data Warehouse Modernization - Big Data in the Cloud Success with Qubole on O...Qubole
The effective use of big data is the key to gaining a competitive advantage and outperforming the competition. This change demands that companies consume and blend enormous amount of data created from divergent and inherently mismatched sources, which represents a paradigm shift to the traditional data warehouse.
Companies need to modernize their data warehouse, augmenting it with a platform that allows storage, processing, exploration and analysis of large and diverse datasets without limiting the ability to deliver the data access, and flexibility responding to the needs of the business. That’s where Oracle Cloud and Qubole work together delivering a new breed of data platform —capable of storing and processing the overwhelming amount of data that on-premises big data deployments cannot handle.
Watch this on-demand webinar to understand:
- Why deploying big data on-premises is expensive, complex to maintain and limits your ability to scale across new use cases and data sources
- How Oracle Bare Metal Cloud's predictable and fast performance compute and network services deliver the foundation of a cost-effective, high-performance big data platform
- How Qubole leverages Oracle Bare Metal Cloud to provide a turnkey big data service that optimizes cost, performance, and scale, enabling self-service data exploration.
Qubole delivers a cloud-based, turnkey, self-service big data service that removes the complexity and reduces the cost of doing big data. It leverages Oracle Bare Metal Cloud’s next generation of scalable, inexpensive and performant compute, network and storage public cloud infrastructure to provide a solution that accelerates time to market and reduces the risk of your big data initiatives.
Flink in Zalando's world of Microservices ZalandoHayley
Apache Flink Meetup at Zalando Technology, May 2016
By Javier Lopez & Mihail Vieru, Zalando
In this talk we present Zalando's microservices architecture and introduce Saiki – our next generation data integration and distribution platform on AWS. We show why we chose Apache Flink to serve as our stream processing framework and describe how we employ it for our current use cases: business process monitoring and continuous ETL. We then have an outlook on future use cases.
Spark summit 2017- Transforming B2B sales with Spark powered sales intelligenceWei Di
B2B sales intelligence has become an integral part of LinkedIn’s business to help companies optimize resource allocation and design effective sales and marketing strategies. This new trend of data-driven approaches has “sparked” a new wave of AI and ML needs in companies large and small. Given the tremendous complexity that arises from the multitude of business needs across different verticals and product lines, Apache Spark, with its rich machine learning libraries, scalable data processing engine and developer-friendly APIs, has been proven to be a great fit for delivering such intelligence at scale.
See how Linkedin is utilizing Spark for building sales intelligence products. This session will introduce a comprehensive B2B intelligence system built on top of various open source stacks. The system puts advanced data science to work in a dynamic and complex scenario, in an easily controllable and interpretable way. Balancing flexibility and complexity, the system can deal with various problems in a unified manner and yield actionable insights to empower successful business. You will also learn about some impactful Spark-ML powered applications such as prospect prediction and prioritization, churn prediction, model interpretation, as well as challenges and lessons learned at LinkedIn while building such platform.
NoSQL and Spatial Database Capabilities using PostgreSQLEDB
PostgreSQL is an object-relational database system. NoSQL on the other hand is a non-relational database and is document-oriented. Learn how the PostgreSQL database gives one the flexible options to combine NoSQL workloads with the relational query power by offering JSON data types. With PostgreSQL, new capabilities can be developed and plugged into the database as required.
Attend this webinar to learn:
- The new features and capabilities in PostgreSQL for new workloads, requiring greater flexibility in the data model
- NoSQL with JSON, Hstore and its performance and features for enterprises
- Spatial SQL - advanced features in PostGIS application with PostGIS extension
Hive LLAP: A High Performance, Cost-effective Alternative to Traditional MPP ...DataWorks Summit
At Walmart Labs, we get close to 200 million customers every week across our 11000+ stores & online all over the world. As part of our data lake initiatives, we started a full-fledged migration to Hadoop based solutions for all our data needs at lower cost than traditional RDBMS/MPP solutions. While we have seen significant success in migrating to Hadoop based Data Lake solutions from traditional RDBMS based data warehouses, one challenge that we have faced is around migrating end users to Hadoop due to query latency issues. To solve this problem and to reduce the cost of the solution, Walmart Labs started using Hive LLAP.
In this session, we will introduce you to Hive LLAP, its architecture, best practices for deployment to achieve sub-second query performance and its cost comparison with traditional RDBMS systems for the same use case.
Talks about best practices and patterns on how to design an efficient cube in Kylin. Covers concepts like mandatory dimension, hierarchy dimension, derived dimension, incremental build, aggregation group etc.
Microsoft has embraced OSS by placing a big bet on Apache YARN to govern the resources of our computing clusters, and we did so by working with the community and adding many new capabilities in YARN. We now look to undertake a similar journey and build the next generation of our job execution engine on top of Apache Tez. We will be building a common platform for executing batch, interactive, ML, and streaming queries at exabyte scale for Microsoft's BigData system called Cosmos. This requires us to push the limits of Tez API to support new graph models, change the executing DAG by dynamically adding new vertices, scheduling for interactive and streaming workloads, squeeze out all the computing power in the cluster by integrating Tez with opportunistic containers in YARN, and scaling a DAG across tens of thousands of machines. We have started out on this journey and want to share our progress, lessons learned, seek help from the community to add these new capabilities, and push Apache Tez to new levels.
SPEAKERS
Hitesh Sharma, Principal Software Engineering Manager, Microsoft Engineering manager in the Big Data team at Microsoft.
Anupam, Senior Software Engineer, Microsoft
Apache Kylin: OLAP Engine on Hadoop - Tech Deep DiveXu Jiang
Kylin is an open source Distributed Analytics Engine from eBay Inc. that provides SQL interface and multi-dimensional analysis (OLAP) on Hadoop supporting extremely large datasets.
If you want to do multi-dimension analysis on large data sets (billion+ rows) with low query latency (sub-seconds), Kylin is a good option. Kylin also provides seamless integration with existing BI tools (e.g Tableau).
Apache Hive is a rapidly evolving project which continues to enjoy great adoption in big data ecosystem. Although, Hive started primarily as batch ingestion and reporting tool, community is hard at work in improving it along many different dimensions and use cases. This talk will provide an overview of latest and greatest features and optimizations which have landed in project over last year. Materialized view, micro managed tables and workload management are some noteworthy features.
I will deep dive into some optimizations which promise to provide major performance gains. Support for ACID tables has also improved considerably. Although some of these features and enhancements are not novel but have existed for years in other DB systems, implementing them on Hive poses some unique challenges and results in lessons which are generally applicable in many other contexts. I will also provide a glimpse of what is expected to come in near future.
Speaker: Ashutosh Chauhan, Engineering Manager, Hortonworks
Building Apps with Distributed In-Memory Computing Using Apache GeodePivotalOpenSourceHub
Slides from the Meetup Monday March 7, 2016 just before the beginning of #GeodeSummit, where we cover an introduction of the technology and community that is Apache Geode, the in-memory data grid.
Drivetribe is the world’s digital hub for motoring, as envisioned by Jeremy Clarkson, Richard Hammond, and James May. The Drivetribe platform was designed ground up with high scalability in mind. Built on top of the Event Sourcing/CQRS pattern, the platform uses Apache Kafka as its source of truth and Apache Flink as its processing backbone. This talk aims to introduce the architecture, and elaborate on how common problems in social media, such as counting big numbers and dealing with outliers, can be resolved by a healthy mix of Flink and functional programming.
More Data, More Problems: Scaling Kafka-Mirroring Pipelines at LinkedIn confluent
(Celia Kung, LinkedIn) Kafka Summit SF 2018
For several years, LinkedIn has been using Kafka MirrorMaker as the mirroring solution for copying data between Kafka clusters across data centers. However, as LinkedIn data continued to grow, mirroring trillions of Kafka messages per day across data centers uncovered the scale limitations and operability challenges of Kafka MirrorMaker. To address such issues, we have developed a new mirroring solution, built on top our stream ingestion service, Brooklin. Brooklin MirrorMaker aims to provide improved performance and stability, while facilitating better management through finer control of data pipelines. Through flushless Kafka produce, dynamic management of data pipelines, per-partition error handling and flow control, we are able to increase throughput, better withstand consume and produce failures and reduce overall operating costs. As a result, we have eliminated the major pain points of Kafka MirrorMaker. In this talk, we will dive deeper into the challenges LinkedIn has faced with Kafka MirrorMaker, how we tackled them with Brooklin MirrorMaker and our plans for iterating further on this new mirroring solution.
Change data capture with MongoDB and Kafka.Dan Harvey
In any modern web platform you end up with a need to store different views of your data in many different datastores. I will cover how we have coped with doing this in a reliable way at State.com across a range of different languages, tools and datastores.
Data Warehouse Modernization - Big Data in the Cloud Success with Qubole on O...Qubole
The effective use of big data is the key to gaining a competitive advantage and outperforming the competition. This change demands that companies consume and blend enormous amount of data created from divergent and inherently mismatched sources, which represents a paradigm shift to the traditional data warehouse.
Companies need to modernize their data warehouse, augmenting it with a platform that allows storage, processing, exploration and analysis of large and diverse datasets without limiting the ability to deliver the data access, and flexibility responding to the needs of the business. That’s where Oracle Cloud and Qubole work together delivering a new breed of data platform —capable of storing and processing the overwhelming amount of data that on-premises big data deployments cannot handle.
Watch this on-demand webinar to understand:
- Why deploying big data on-premises is expensive, complex to maintain and limits your ability to scale across new use cases and data sources
- How Oracle Bare Metal Cloud's predictable and fast performance compute and network services deliver the foundation of a cost-effective, high-performance big data platform
- How Qubole leverages Oracle Bare Metal Cloud to provide a turnkey big data service that optimizes cost, performance, and scale, enabling self-service data exploration.
Qubole delivers a cloud-based, turnkey, self-service big data service that removes the complexity and reduces the cost of doing big data. It leverages Oracle Bare Metal Cloud’s next generation of scalable, inexpensive and performant compute, network and storage public cloud infrastructure to provide a solution that accelerates time to market and reduces the risk of your big data initiatives.
Flink in Zalando's world of Microservices ZalandoHayley
Apache Flink Meetup at Zalando Technology, May 2016
By Javier Lopez & Mihail Vieru, Zalando
In this talk we present Zalando's microservices architecture and introduce Saiki – our next generation data integration and distribution platform on AWS. We show why we chose Apache Flink to serve as our stream processing framework and describe how we employ it for our current use cases: business process monitoring and continuous ETL. We then have an outlook on future use cases.
Spark summit 2017- Transforming B2B sales with Spark powered sales intelligenceWei Di
B2B sales intelligence has become an integral part of LinkedIn’s business to help companies optimize resource allocation and design effective sales and marketing strategies. This new trend of data-driven approaches has “sparked” a new wave of AI and ML needs in companies large and small. Given the tremendous complexity that arises from the multitude of business needs across different verticals and product lines, Apache Spark, with its rich machine learning libraries, scalable data processing engine and developer-friendly APIs, has been proven to be a great fit for delivering such intelligence at scale.
See how Linkedin is utilizing Spark for building sales intelligence products. This session will introduce a comprehensive B2B intelligence system built on top of various open source stacks. The system puts advanced data science to work in a dynamic and complex scenario, in an easily controllable and interpretable way. Balancing flexibility and complexity, the system can deal with various problems in a unified manner and yield actionable insights to empower successful business. You will also learn about some impactful Spark-ML powered applications such as prospect prediction and prioritization, churn prediction, model interpretation, as well as challenges and lessons learned at LinkedIn while building such platform.
NoSQL and Spatial Database Capabilities using PostgreSQLEDB
PostgreSQL is an object-relational database system. NoSQL on the other hand is a non-relational database and is document-oriented. Learn how the PostgreSQL database gives one the flexible options to combine NoSQL workloads with the relational query power by offering JSON data types. With PostgreSQL, new capabilities can be developed and plugged into the database as required.
Attend this webinar to learn:
- The new features and capabilities in PostgreSQL for new workloads, requiring greater flexibility in the data model
- NoSQL with JSON, Hstore and its performance and features for enterprises
- Spatial SQL - advanced features in PostGIS application with PostGIS extension
Hive LLAP: A High Performance, Cost-effective Alternative to Traditional MPP ...DataWorks Summit
At Walmart Labs, we get close to 200 million customers every week across our 11000+ stores & online all over the world. As part of our data lake initiatives, we started a full-fledged migration to Hadoop based solutions for all our data needs at lower cost than traditional RDBMS/MPP solutions. While we have seen significant success in migrating to Hadoop based Data Lake solutions from traditional RDBMS based data warehouses, one challenge that we have faced is around migrating end users to Hadoop due to query latency issues. To solve this problem and to reduce the cost of the solution, Walmart Labs started using Hive LLAP.
In this session, we will introduce you to Hive LLAP, its architecture, best practices for deployment to achieve sub-second query performance and its cost comparison with traditional RDBMS systems for the same use case.
Talks about best practices and patterns on how to design an efficient cube in Kylin. Covers concepts like mandatory dimension, hierarchy dimension, derived dimension, incremental build, aggregation group etc.
Microsoft has embraced OSS by placing a big bet on Apache YARN to govern the resources of our computing clusters, and we did so by working with the community and adding many new capabilities in YARN. We now look to undertake a similar journey and build the next generation of our job execution engine on top of Apache Tez. We will be building a common platform for executing batch, interactive, ML, and streaming queries at exabyte scale for Microsoft's BigData system called Cosmos. This requires us to push the limits of Tez API to support new graph models, change the executing DAG by dynamically adding new vertices, scheduling for interactive and streaming workloads, squeeze out all the computing power in the cluster by integrating Tez with opportunistic containers in YARN, and scaling a DAG across tens of thousands of machines. We have started out on this journey and want to share our progress, lessons learned, seek help from the community to add these new capabilities, and push Apache Tez to new levels.
SPEAKERS
Hitesh Sharma, Principal Software Engineering Manager, Microsoft Engineering manager in the Big Data team at Microsoft.
Anupam, Senior Software Engineer, Microsoft
Apache Kylin: OLAP Engine on Hadoop - Tech Deep DiveXu Jiang
Kylin is an open source Distributed Analytics Engine from eBay Inc. that provides SQL interface and multi-dimensional analysis (OLAP) on Hadoop supporting extremely large datasets.
If you want to do multi-dimension analysis on large data sets (billion+ rows) with low query latency (sub-seconds), Kylin is a good option. Kylin also provides seamless integration with existing BI tools (e.g Tableau).
Apache Kylin is an open source Distributed Analytics Engine from eBay Inc. that provides SQL interface and multi-dimensional analysis (OLAP) on Hadoop supporting extremely large datasets and subsecond query latency.
Kylin is an open source Distributed Analytics Engine from eBay Inc. that provides SQL interface and multi-dimensional analysis (OLAP) on Hadoop supporting extremely large datasets
Kylin Open Source Web Site: http://kylin.io
Apache Kylin general introduction, including background, business needs and technical challenges, theory and architecture, features and some tech detail. Following with performance and benchmark, finally, ecosystem and roadmap.
More detail, please visit http://kylin.io or follow @ApacheKylin.
We use "SaasBase Analytics" to incrementally process large heterogeneous data sets into pre-aggregated, indexed views, stored in HBase to be queried in realtime. The requirement we started from was to get large amounts of data available in near realtime (minutes) to large amounts of users for large amounts of (different) queries that take milliseconds to execute. This set our problem apart from classical solutions such as Hive and PIG. In this talk I`ll go through the design of the solution and the strategies (and hacks) to achieve low latency and scalability from theoretical model to the entire process of ETL to warehousing and queries.
As eBay is moving to OpenStack, we need to find capacity conversion ratio between ESX and KVM. Moreover, we hope to tunning KVM performance that make KVM to be same as or better than ESX
HBaseCon 2015: Apache Kylin - Extreme OLAP Engine for HadoopHBaseCon
Kylin is an open source distributed analytics engine contributed by eBay that provides a SQL interface and OLAP on Hadoop supporting extremely large datasets. Kylin's pre-built MOLAP cubes (stored in HBase), distributed architecture, and high concurrency helps users analyze multidimensional queries via SQL and other BI tools. During this session, you'll learn how Kylin uses HBase's key-value store to serve SQL queries with relational schema.
Apache Kylin’s Performance Boost from Apache HBaseHBaseCon
Hongbin Ma and Luke Han (Kyligence)
Apache Kylin is an open source distributed analytics engine that provides a SQL interface and multi-dimensional analysis on Hadoop supporting extremely large datasets. In the forthcoming Kylin release, we optimized query performance by exploring the potentials of parallel storage on top of HBase. This talk explains how that work was done.
Apache Kylin - OLAP Cubes for SQL on HadoopTed Dunning
Apache Kylin (incubating) is a new project to bring OLAP cubes to Hadoop. I walk through the project and describe how it works and how users see the project.
New usage model for real-time analytics by Dr. WILLIAM L. BAIN at Big Data S...Big Data Spain
Operational systems manage our finances, shopping, devices and much more. Adding real-time analytics to these systems enables them to instantly respond to changing conditions and provide immediate, targeted feedback. This use of analytics is called “operational intelligence,” and the need for it is widespread.
Operational systems manage our finances, shopping, devices and much more. Adding real-time analytics to these systems enables them to instantly respond to changing conditions and provide immediate, targeted feedback. This use of analytics is called "operational intelligence," and the need for it is widespread.
This talk will explain how in-memory computing techniques can be used to implement operational intelligence. It will show how an in-memory data grid integrated with a data-parallel compute engine can track events generated by a live system, analyze them in real time, and create alerts that help steer the system’s behavior. Code samples will demonstrate how an in-memory data grid employs object-oriented techniques to simplify the correlation and analysis of incoming events by maintaining an in-memory model of a live system.
The talk also will examine simplifications offered by this approach over directly analyzing incoming event streams from a live system using complex event processing or Storm. Lastly, it will explain key requirements of the in-memory computing platform for operational intelligence, in particular real-time updating of individual objects and high availability using data replication, and contrast these requirements to the design goals for stream processing in Spark.
Hbb 2852 gain insights into your business operations with bpm and kibanaAllen Chan
At IBM InterConnect 2017, we discussed the ability for IBM BPM to send business events into analytics + visualization framework such as Elasticsearch + Kibana.
Making Hadoop Realtime by Dr. William Bain of Scaleout SoftwareData Con LA
Hadoop has been widely embraced for its ability to economically store and analyze large data sets. Using parallel computing techniques like MapReduce, Hadoop can reduce long computation times to hours or minutes. This works well for mining large volumes of historical data stored on disk, but it is not suitable for gaining real-time insights from live operational data. Still, the idea of using Hadoop for real-time data analytics on live data is appealing because it leverages existing programming skills and infrastructure – and the parallel architecture of Hadoop itself. This presentation will describe how real-time analytics using Hadoop can be performed by combining an in-memory data grid (IMDG) with an integrated, stand-alone Hadoop MapReduce execution engine. This new technology delivers fast results for live data and also accelerates the analysis of large, static data sets.
[db tech showcase Tokyo 2017] C37: MariaDB ColumnStore analytics engine : use...Insight Technology, Inc.
MariaDB ColumnStore is the analytics engine for MariaDB. This talk will introduce the product, use cases, and also introduce the new features coming in the next major release 1.1.
Introducing Amazon Kinesis: Real-time Processing of Streaming Big Data (BDT10...Amazon Web Services
"This presentation will introduce Kinesis, the new AWS service for real-time streaming big data ingestion and processing.
We’ll provide an overview of the key scenarios and business use cases suitable for real-time processing, and discuss how AWS designed Amazon Kinesis to help customers shift from a traditional batch-oriented processing of data to a continual real-time processing model. We’ll provide an overview of the key concepts, attributes, APIs and features of the service, and discuss building a Kinesis-enabled application for real-time processing. We’ll also contrast with other approaches for streaming data ingestion and processing. Finally, we’ll also discuss how Kinesis fits as part of a larger big data infrastructure on AWS, including S3, DynamoDB, EMR, and Redshift."
Process Analytics with Oracle BPM Suite 12c and BAM - OGh SIG SOA & BPM, 1st ...Lucas Jellema
Business Processes implemented in BPEL and BPM(N) and running on Oracle BPM Suite 12c or SOA Suite 12c have to fulfill a business purpose and as such must meet business requirements - both functionally and non-functionally. SLAs for throughput, response time, quality are usually associated with these processes and we typically also would like insight in the number of process executions (per group) and the paths taken through our processes.
This presentation introduces process analytics in both BPEL and BPM processes in Oracle SOA Suite and BPM Suite 12c. It explains how to configure out of the box generic analytics and process specific business indicators. The presentation than introduces BAM 12c. It demonstrates the out of the box process analytics reports and dashboards. Then it explains how to create custom reports on the unified process analytics star schema or on custom tables. Finally the presentation goes into real-time monitoring in BAM using JMS and enterprise message resources in combination with the event processing templates in BAM.
Introducing Ironstream Support for ServiceNow Event Management Precisely
Your IT infrastructure is the foundation for everything your organization does – customer engagement, transaction processing, business decision-making, and much more. When your IT services go down, so does your business.
ServiceNow Event Management is a powerful tool to keep your business up and running, 24x7. It consolidates disconnected monitoring tools into a single view, and uses AIOps and machine learning to transform infrastructure events into actionable alerts so you can act fast.
However, there’s been no easy way to integrate your critical mainframe and IBM i systems with ServiceNow Event Management – until now.
View our webcast to learn about Ironstream’s new support for ServiceNow Event Management. It is the first and only solution to seamlessly integrate IBM mainframe and IBM i data into ServiceNow – giving you a complete view of service availability across your entire infrastructure.
Our product experts will cover how:
Ironstream for ServiceNow works
Deploying Ironstream with ServiceNow Event Management benefits your business
Combining Event Management and the ServiceNow CMDB takes your insights one step further
AWS re:Invent 2016: How Fulfillment by Amazon (FBA) and Scopely Improved Resu...Amazon Web Services
We’ll share an overview of leveraging serverless architectures to support high performance data intensive applications. Fulfillment by Amazon (FBA) built the Seller Inventory Authority Platform (IAP) using Amazon DynamoDB Streams, AWS Lambda functions, Amazon Elasticsearch Service, and Amazon Redshift to improve results and reduce costs. Scopely will share how they used a flexible logging system built on Kinesis, Lambda, and Amazon Elasticsearch to provide high-fidelity reporting on hotkeys in Memcached and DynamoDB, and drastically reduce the incidence of hotkeys. Both of these customers are using managed services and serverless architecture to build scalable systems that can meet the projected business growth without a corresponding increase in operational costs.
Comment transformer vos données en informations exploitablesElasticsearch
Découvrez des fonctionnalités stratégiques de la Suite Elastic, notamment Elasticsearch, un moteur de données incomparable, et Kibana, véritable fenêtre ouverte sur la Suite Elastic.
Dans cette session, vous apprendrez à :
injecter des données dans la Suite Elastic ;
stocker des données ;
analyser des données ;
exploiter des données.
Open Blueprint for Real-Time Analytics in Retail: Strata Hadoop World 2017 S...Grid Dynamics
This presentation outlines key business drivers for real-time analytics applications in retail and describes the emerging architectures based on In-Stream Processing (ISP) technologies. The slides present a complete open blueprint for an ISP platform - including a demo application for real-time Twitter Sentiment Analytics - designed with 100% open source components and deployable to any cloud.
To learn more, read an adjoining blog series on this topic here : https://blog.griddynamics.com/in-stream-processing-service-blueprint
(SEC310) Keeping Developers and Auditors Happy in the CloudAmazon Web Services
Often times, developers and auditors can be at odds. The agile, fast-moving environments that developers enjoy will typically give auditors heartburn. The more controlled and stable environments that auditors prefer to demonstrate and maintain compliance are traditionally not friendly to developers or innovation. We'll walk through how Netflix moved its PCI and SOX environments to the cloud and how we were able to leverage the benefits of the cloud and agile development to satisfy both auditors and developers. Topics covered will include shared responsibility, using compartmentalization and microservices for scope control, immutable infrastructure, and continuous security testing.
Amazon Kinesis is a fully managed service for real-time processing of streaming data at massive scale. Amazon Kinesis can collect and process hundreds of terabytes of data per hour from hundreds of thousands of sources, allowing you to easily write applications that process information in real-time, from sources such as web site click-streams, marketing and financial information, manufacturing instrumentation and social media, and operational logs and metering data.
This introductory webinar, presented by Adi Krishnan, Senior Product Manager for Amazon Kinesis, will provide you with an overview of the service, sample use cases, and some examples of customer experiences with the service so you can better understand its capabilities and see how it might be integrated into your own applications.
Stream Processing and Complex Event Processing together with Kafka, Flink and...HostedbyConfluent
"Getting actionable insights from events and addressing common business scenarios usually require bridging together several technologies and programming models to achieve the expected business outcome and bring the flexibility modern applications need.
This talks is about how we can concretely wire together Apache Kafka to collect and distribute real-time events, Apache Flink and stream processing to reduce the noise and transform raw events into business events and a rule engine to analyse complex patterns of events, detect business-meaningful situations in context and leveraging business rules to implement the detection logic. Many Apache Flink users in various industries are already combining these technologies to make their event solutions more powerful and more flexible. Built on a fictitious but realistic airline scenario, this talk shows how to technically integrate a rule engine technology, like Redhat Drools, with Kafka and Apache Flink and pinpoint on how stream processing and complex event processing are complementary and different at the same time, and where one need to pay attention to scale the solution to production environment."
Assessing New Databases– Translytical Use CasesDATAVERSITY
Organizations run their day-in-and-day-out businesses with transactional applications and databases. On the other hand, organizations glean insights and make critical decisions using analytical databases and business intelligence tools.
The transactional workloads are relegated to database engines designed and tuned for transactional high throughput. Meanwhile, the big data generated by all the transactions require analytics platforms to load, store, and analyze volumes of data at high speed, providing timely insights to businesses.
Thus, in conventional information architectures, this requires two different database architectures and platforms: online transactional processing (OLTP) platforms to handle transactional workloads and online analytical processing (OLAP) engines to perform analytics and reporting.
Today, a particular focus and interest of operational analytics includes streaming data ingest and analysis in real time. Some refer to operational analytics as hybrid transaction/analytical processing (HTAP), translytical, or hybrid operational analytic processing (HOAP). We’ll address if this model is a way to create efficiencies in our environments.
How to Create Observable Integration Solutions Using WSO2 Enterprise IntegratorWSO2
This slide deck introduces the WSO2 Enterprise Integrator analytics profile and explore its observability features.
Watch the webinar here: https://wso2.com/library/webinars/2018/09/how-to-create-observable-integration-solutions-using-wso2-enterprise-integrator
Listen to the keynote address and hear about the latest developments from Rachana Ananthakrishnan and Ian Foster who review the updates to the Globus Platform and Service, and the relevance of Globus to the scientific community as an automation platform to accelerate scientific discovery.
Globus Compute wth IRI Workflows - GlobusWorld 2024Globus
As part of the DOE Integrated Research Infrastructure (IRI) program, NERSC at Lawrence Berkeley National Lab and ALCF at Argonne National Lab are working closely with General Atomics on accelerating the computing requirements of the DIII-D experiment. As part of the work the team is investigating ways to speedup the time to solution for many different parts of the DIII-D workflow including how they run jobs on HPC systems. One of these routes is looking at Globus Compute as a way to replace the current method for managing tasks and we describe a brief proof of concept showing how Globus Compute could help to schedule jobs and be a tool to connect compute at different facilities.
AI Pilot Review: The World’s First Virtual Assistant Marketing SuiteGoogle
AI Pilot Review: The World’s First Virtual Assistant Marketing Suite
👉👉 Click Here To Get More Info 👇👇
https://sumonreview.com/ai-pilot-review/
AI Pilot Review: Key Features
✅Deploy AI expert bots in Any Niche With Just A Click
✅With one keyword, generate complete funnels, websites, landing pages, and more.
✅More than 85 AI features are included in the AI pilot.
✅No setup or configuration; use your voice (like Siri) to do whatever you want.
✅You Can Use AI Pilot To Create your version of AI Pilot And Charge People For It…
✅ZERO Manual Work With AI Pilot. Never write, Design, Or Code Again.
✅ZERO Limits On Features Or Usages
✅Use Our AI-powered Traffic To Get Hundreds Of Customers
✅No Complicated Setup: Get Up And Running In 2 Minutes
✅99.99% Up-Time Guaranteed
✅30 Days Money-Back Guarantee
✅ZERO Upfront Cost
See My Other Reviews Article:
(1) TubeTrivia AI Review: https://sumonreview.com/tubetrivia-ai-review
(2) SocioWave Review: https://sumonreview.com/sociowave-review
(3) AI Partner & Profit Review: https://sumonreview.com/ai-partner-profit-review
(4) AI Ebook Suite Review: https://sumonreview.com/ai-ebook-suite-review
Developing Distributed High-performance Computing Capabilities of an Open Sci...Globus
COVID-19 had an unprecedented impact on scientific collaboration. The pandemic and its broad response from the scientific community has forged new relationships among public health practitioners, mathematical modelers, and scientific computing specialists, while revealing critical gaps in exploiting advanced computing systems to support urgent decision making. Informed by our team’s work in applying high-performance computing in support of public health decision makers during the COVID-19 pandemic, we present how Globus technologies are enabling the development of an open science platform for robust epidemic analysis, with the goal of collaborative, secure, distributed, on-demand, and fast time-to-solution analyses to support public health.
Enhancing Research Orchestration Capabilities at ORNL.pdfGlobus
Cross-facility research orchestration comes with ever-changing constraints regarding the availability and suitability of various compute and data resources. In short, a flexible data and processing fabric is needed to enable the dynamic redirection of data and compute tasks throughout the lifecycle of an experiment. In this talk, we illustrate how we easily leveraged Globus services to instrument the ACE research testbed at the Oak Ridge Leadership Computing Facility with flexible data and task orchestration capabilities.
Into the Box Keynote Day 2: Unveiling amazing updates and announcements for modern CFML developers! Get ready for exciting releases and updates on Ortus tools and products. Stay tuned for cutting-edge innovations designed to boost your productivity.
Prosigns: Transforming Business with Tailored Technology SolutionsProsigns
Unlocking Business Potential: Tailored Technology Solutions by Prosigns
Discover how Prosigns, a leading technology solutions provider, partners with businesses to drive innovation and success. Our presentation showcases our comprehensive range of services, including custom software development, web and mobile app development, AI & ML solutions, blockchain integration, DevOps services, and Microsoft Dynamics 365 support.
Custom Software Development: Prosigns specializes in creating bespoke software solutions that cater to your unique business needs. Our team of experts works closely with you to understand your requirements and deliver tailor-made software that enhances efficiency and drives growth.
Web and Mobile App Development: From responsive websites to intuitive mobile applications, Prosigns develops cutting-edge solutions that engage users and deliver seamless experiences across devices.
AI & ML Solutions: Harnessing the power of Artificial Intelligence and Machine Learning, Prosigns provides smart solutions that automate processes, provide valuable insights, and drive informed decision-making.
Blockchain Integration: Prosigns offers comprehensive blockchain solutions, including development, integration, and consulting services, enabling businesses to leverage blockchain technology for enhanced security, transparency, and efficiency.
DevOps Services: Prosigns' DevOps services streamline development and operations processes, ensuring faster and more reliable software delivery through automation and continuous integration.
Microsoft Dynamics 365 Support: Prosigns provides comprehensive support and maintenance services for Microsoft Dynamics 365, ensuring your system is always up-to-date, secure, and running smoothly.
Learn how our collaborative approach and dedication to excellence help businesses achieve their goals and stay ahead in today's digital landscape. From concept to deployment, Prosigns is your trusted partner for transforming ideas into reality and unlocking the full potential of your business.
Join us on a journey of innovation and growth. Let's partner for success with Prosigns.
May Marketo Masterclass, London MUG May 22 2024.pdfAdele Miller
Can't make Adobe Summit in Vegas? No sweat because the EMEA Marketo Engage Champions are coming to London to share their Summit sessions, insights and more!
This is a MUG with a twist you don't want to miss.
Large Language Models and the End of ProgrammingMatt Welsh
Talk by Matt Welsh at Craft Conference 2024 on the impact that Large Language Models will have on the future of software development. In this talk, I discuss the ways in which LLMs will impact the software industry, from replacing human software developers with AI, to replacing conventional software with models that perform reasoning, computation, and problem-solving.
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...Globus
Large Language Models (LLMs) are currently the center of attention in the tech world, particularly for their potential to advance research. In this presentation, we'll explore a straightforward and effective method for quickly initiating inference runs on supercomputers using the vLLM tool with Globus Compute, specifically on the Polaris system at ALCF. We'll begin by briefly discussing the popularity and applications of LLMs in various fields. Following this, we will introduce the vLLM tool, and explain how it integrates with Globus Compute to efficiently manage LLM operations on Polaris. Attendees will learn the practical aspects of setting up and remotely triggering LLMs from local machines, focusing on ease of use and efficiency. This talk is ideal for researchers and practitioners looking to leverage the power of LLMs in their work, offering a clear guide to harnessing supercomputing resources for quick and effective LLM inference.
top nidhi software solution freedownloadvrstrong314
This presentation emphasizes the importance of data security and legal compliance for Nidhi companies in India. It highlights how online Nidhi software solutions, like Vector Nidhi Software, offer advanced features tailored to these needs. Key aspects include encryption, access controls, and audit trails to ensure data security. The software complies with regulatory guidelines from the MCA and RBI and adheres to Nidhi Rules, 2014. With customizable, user-friendly interfaces and real-time features, these Nidhi software solutions enhance efficiency, support growth, and provide exceptional member services. The presentation concludes with contact information for further inquiries.
Enhancing Project Management Efficiency_ Leveraging AI Tools like ChatGPT.pdfJay Das
With the advent of artificial intelligence or AI tools, project management processes are undergoing a transformative shift. By using tools like ChatGPT, and Bard organizations can empower their leaders and managers to plan, execute, and monitor projects more effectively.
Cyaniclab : Software Development Agency Portfolio.pdfCyanic lab
CyanicLab, an offshore custom software development company based in Sweden,India, Finland, is your go-to partner for startup development and innovative web design solutions. Our expert team specializes in crafting cutting-edge software tailored to meet the unique needs of startups and established enterprises alike. From conceptualization to execution, we offer comprehensive services including web and mobile app development, UI/UX design, and ongoing software maintenance. Ready to elevate your business? Contact CyanicLab today and let us propel your vision to success with our top-notch IT solutions.
First Steps with Globus Compute Multi-User EndpointsGlobus
In this presentation we will share our experiences around getting started with the Globus Compute multi-user endpoint. Working with the Pharmacology group at the University of Auckland, we have previously written an application using Globus Compute that can offload computationally expensive steps in the researcher's workflows, which they wish to manage from their familiar Windows environments, onto the NeSI (New Zealand eScience Infrastructure) cluster. Some of the challenges we have encountered were that each researcher had to set up and manage their own single-user globus compute endpoint and that the workloads had varying resource requirements (CPUs, memory and wall time) between different runs. We hope that the multi-user endpoint will help to address these challenges and share an update on our progress here.
How Recreation Management Software Can Streamline Your Operations.pptxwottaspaceseo
Recreation management software streamlines operations by automating key tasks such as scheduling, registration, and payment processing, reducing manual workload and errors. It provides centralized management of facilities, classes, and events, ensuring efficient resource allocation and facility usage. The software offers user-friendly online portals for easy access to bookings and program information, enhancing customer experience. Real-time reporting and data analytics deliver insights into attendance and preferences, aiding in strategic decision-making. Additionally, effective communication tools keep participants and staff informed with timely updates. Overall, recreation management software enhances efficiency, improves service delivery, and boosts customer satisfaction.
Providing Globus Services to Users of JASMIN for Environmental Data AnalysisGlobus
JASMIN is the UK’s high-performance data analysis platform for environmental science, operated by STFC on behalf of the UK Natural Environment Research Council (NERC). In addition to its role in hosting the CEDA Archive (NERC’s long-term repository for climate, atmospheric science & Earth observation data in the UK), JASMIN provides a collaborative platform to a community of around 2,000 scientists in the UK and beyond, providing nearly 400 environmental science projects with working space, compute resources and tools to facilitate their work. High-performance data transfer into and out of JASMIN has always been a key feature, with many scientists bringing model outputs from supercomputers elsewhere in the UK, to analyse against observational or other model data in the CEDA Archive. A growing number of JASMIN users are now realising the benefits of using the Globus service to provide reliable and efficient data movement and other tasks in this and other contexts. Further use cases involve long-distance (intercontinental) transfers to and from JASMIN, and collecting results from a mobile atmospheric radar system, pushing data to JASMIN via a lightweight Globus deployment. We provide details of how Globus fits into our current infrastructure, our experience of the recent migration to GCSv5.4, and of our interest in developing use of the wider ecosystem of Globus services for the benefit of our user community.
3. Sybase Confidential 3
Background - Overview
• Business Activity Monitoring (BAM)
• Complex Event Processing / Event Stream Processing
• Two approach of CEP/ESP
• Real Time Business Intelligence
• Two approach of RTBI
4. Sybase Confidential 4
Background - BAM
"Business activity monitoring" (BAM) is Gartner's
term defining how we can provide real-time access
to critical business performance indicators to
improve the speed and effectiveness of business
operations.
Unlike traditional real-time monitoring, BAM draws
its information from multiple application systems
and other internal and external sources, enabling a
broader and richer view of business activities.
5. Sybase Confidential 5
Background – ESP/CEP
“Event Stream Processing” (ESP) is software technology
that allows applications to monitor streams of event data,
analyze those events, and act upon opportunities and
threats in real time. ESP systems often utilize, or include,
event databases and event visualization tools, event-driven
middleware, and event processing languages
“Complex Event Processing” (CEP) is a key element of
ESP that provides language elements that allows
applications to express the complex patterns among events
it's looking for. CEP provides constructs that include event
correlation, event abstraction, event hierarchies, and the
ability to express relationships between events such as
causality, membership, and timing.
6. Sybase Confidential 6
Two approach of CEP/ESP – SQL
Based Approach I
Some people coming from RDBMS development
have extended SQL to provide CEP/ESP.
• The SQL processing in traditional RDBMS is
“data is static and query is dynamic”.
• The SQL processing in CEP/ESP is “data is
dynamic and query is static”.
• Because the event data may be overflow, it is
necessary to introduce “time window” to SQL
7. Sybase Confidential 7
Two approach of CEP/ESP – SQL
Based Approach II
SELECT I1.SourceIP As SourceIP, I1.AttackKind As AttackKind, V1.Virus As Virus
FROM InIDSAlerts As I1 KEEP 30 SECONDS, InVirusAlerts As V1 KEEP 30
SECONDS
WHERE I1.SourceIP=V1.SourceIP
Join
Projection
.
.
.
.
.
.
Scan Scan
.
.
.
I1 KEEP 30 SEC I2 KEEP 30 SEC
I1 I2
8. Sybase Confidential 8
Two approach of CEP/ESP – Rule
Based Approach I
Some people come from integration development
have extend rule engine to provide CEP/ESP.
Sybase BAM chooses this approach.
The key of this approach is to add complex state
management and corresponding operator to the
traditional rule engine that can support complex
event pattern and event correlation.
10. Sybase Confidential 10
Background – RT BI
“Real time business intelligence” (RT BI) is the
process of delivering information about business
operations without any latency.
While traditional business intelligence presents
historical information to users for analysis, real
time business intelligence compares current
business events with historical patterns to detect
problems or opportunities automatically.
11. Sybase Confidential 11
Two approach of RT BI
• Event driven, Real time Business Intelligence
Real time Business Intelligence systems are event driven,
and use ESP/CEP techniques to enable events to be
analyzed without being first transformed and stored in a date
warehouse.
This approach is better for BAM.
• Real time Data warehouse
An alternative approach to event driven architectures is to
increase the refresh cycle of an existing data warehouse to
update the data more frequently. These real time data
warehouse systems can achieve near real time update of
data, where the data latency typically is in the rage from
minutes to hours out of date.
This approach is better for ETL.
12. Sybase Confidential 12
Analytic Model - Overview
Fields: Abstract states definition.
Key, Unbound, Bound, Aggregation
Rules: Intelligence
If condition Then action
Actions: Behavior
Update, Aggregation, Alert, Timer, SQL, Java
Script, Purge
Timers: Scheduler
If timer arrive Then action
Binder: Concrete states storage
BAMDB, UserDB, RefAM
13. Sybase Confidential 13
Analytic Model – Processing
Fields
Key
Bound
Bound
Bound
Aggregate
Unbound
Rules Actions
• Update
• Aggregate
• SQL
• Alert
• Java Script
• Timer control
• Purge
if…
if…
1. Keys, (some) other field passed into Analytic Model
2. Historical values found based on keys
3. Rules applied to data
4. Actions performed, update data
5. Repeat 3, 4 as needed
6. New values stored
15. Sybase Confidential 15
Analytic Model - Functionality
• Monitor services interact with multiple Analytic Models,
setting key fields to define specific object instance.
• Within Analytic Object, multiple rule calls trigger actions
that further update object and perform other activities.
• Any field set in one Analytic Object is then available to
subsequent objects, as determined by the Monitor Service.
• If there is implicate or explicate key fields setting between
different Analytic Objects, record the cross correlation of
Analytic Objects.
• Service output fields may be return result of any field from
any Analytic Object.
16. Sybase Confidential 16
Architecture - Overview
Monitor
Service Editor Monitor Service
WSDL
Monitor
Command and
Control
Monitor
Analytic
Model Editor
Dashboard
Business
Process
External
Client
SOAP,
JMS, etc
Monitor
Service
BAM-Defined
Database
Binding
User Defined
Database
Binding
SCS Container
Analytic
Object
Access
Library
Rules
Timed Event
Daemon
18. Sybase Confidential 18
Runtime Processing of BAM
Queu
e
SCS
JMS
WSHF
Provider
CSB
Monitor
Service
WSIF
Provider
Optimus
Analytic
Object
Access
Library
DB
Timed Event
Daemon
19. Sybase Confidential 19
Main Features - Overview
• Complex Event Processing Support
• Real Time Business Intelligence Support
• Comprehensive Alert Capability
• Intuitive Visualization for Monitoring and Analysis
• Metadata-Driven Design Tooling
• Service Oriented Architecture Support
• High Volume
20. Sybase Confidential 20
Main Features - Complex Event
Processing Support I
• Event-Condition-Action (ECA) model
§ Event Triggering, Rule Evaluation, Execute Action
• Event Transport/Triggering
§ JMS, HTTP, Email, File, Timer
• Event Parsing/Transformation
§ XML, CWF, SOAP
• Event Routing
§ Body-based, Header-based, Endpoint-based
21. Sybase Confidential 21
Main Features - Complex Event
Processing Support II
• Event States
§ Stateless, Stateful, Historical
• Event Correlation
§ Correlate low-level events to high-level event
§ Key correlation, Cross correlation, History correlation
• Event Reprocess
§ Take corrective action for closed loop integration
• Complex Event Pattern Support
§ Based on ECA model + Event States + Event Correlation.
22. Sybase Confidential 22
Main Features - Real Time Business
Intelligence Support I
• Rule-based intelligence
§ Light-weight BAM Rule Engine (BRE)
§ Patent-pending Boolean Network Rule Engine (BNRE)
• Analyzing real-time data in the context of historic
information
§ Reference contextual data from ASE, IQ, EII
23. Sybase Confidential 23
Main Features - Real Time Business
Intelligence Support II
• Time windowed aggregation / computation
§ User-defined computation expression
§ Extensible Aggregator: Average, Rate, Standard Deviation
§ Sliding Time Window / Fixed Time Window
• Multi-dimensional analysis support
§ Based on Event Correlation + Aggregation + Computation
24. Sybase Confidential 24
Main Features – Comprehensive Alert
Capability
• Publish-subscribe model
§ XML Messages Publish via JMS
§ Customized Subscription
• Multiple Delivery Target
§ JMS, JMX, Email
• Alert escalation
§ Timer, On-demand
• Alert lifecycle
§ Active, Canceled, Completed, Escalated, Suppressed
25. Sybase Confidential 25
Main Features - Intuitive Visualization
for Monitoring and Analysis
• Dashboard
§ Visual objects of Key Performance Indicator (KPI) is changed
dynamically as events occur in real time
• Monitoring
§ Real time event is displayed in tabular forms
§ Drill-down from high-level event to low-level events
• Alerting
§ View and resolve alerts
26. Sybase Confidential 26
Main Features - Metadata-Driven
Design Tooling
• Based on Eclipse and EMF (Eclipse Modeling
Framework)
• Fully integrated and conformed to Sybase
WorkSpace
27. Sybase Confidential 27
Main Features – SOA Support
• BAM is exposed as “Monitoring Service” in
Sybase Service Container
28. Sybase Confidential 28
Main Features - High Volume
• High Performance
§ BAM engine can process about 2000 messages/sec on a 2
CPU machine
• Linear Scalability
§ BAM engine is linear scalability
§ Single BAM DB is linear scalability with CPU number
§ Multiple BAM DB are linear scalability with machine number
29. Sybase Confidential 29
Reference
Business Activity Monitoring
http://en.wikipedia.org/wiki/Business_activity_monitoring
Complex Event Processing
http://en.wikipedia.org/wiki/Complex_event_processing
Event Stream Processing
http://en.wikipedia.org/wiki/Event_Stream_Processing
Real-time Business Intelligence
http://en.wikipedia.org/wiki/Real_time_business_intelligence
BI 2.0: The Next Generation
http://www.dmreview.com/article_sub.cfm?articleId=1066763
BAM: Event-Driven Business Intelligence for the Real-Time Enterprise
http://www.dmreview.com/article_sub.cfm?articleId=8177
Data Integration—the Foundation of a Robust Enterprise Architecture
http://www.informatica.com/company/featured_articles/data_integration_foundation_082004.htm