How to manage one million messages per second using Azure, Radu Vunvulea, ITDays 2015, Cluj-Napoca
•Download as PPTX, PDF•
0 likes•827 views
At the beginning of a project it is simple to promise to clients different things, but when you need to prove them you might have discover that is impossible. Living in the IoT era we need to be able to process large amounts of content per second. This is why in this session we will see how we can construct a solution around Azure that can handle very easy 1M messages per second. We will start the session with a real time demo and we will continue to describe how we can construct such a system using Azure Services in less than 8h. Each Azure component that was used for the demo will be describes in detail and will see what are the pros and cons of it.
Report
Share
Report
Share
Recommended
How to crunch 1 m messages per second in 4 easy steps - Radu Vunvulea
This document discusses Radu Vunvulea's background and experience in areas like banking, automotive, pharma, e-commerce, Azure, JavaScript, .NET, and event streaming technologies. It also provides an example of how event data can be ingested from IoT devices or applications via Event Hubs, processed in real-time using Stream Analytics, and stored in SQL databases, blob storage, or used for notifications and reporting. Finally, it includes a short bio of Radu Vunvulea and examples of messages he may post.
Real Time Twitter sentiment analysis
Utilized the Twitter WPF client to extract data based on Hashtags and fed to sentiment140 for sentiment analysis.
Loaded sentiment analyzed tweets in Azure using event hubs and performed analysis using SQL in stream analytics
Stored the analyzed data in Azure Blob Storage and visualized the outcomes of analysis in real time using Power BI
Building Software to Scale
Building scalable software requires designing it so that adding more hardware allows the software to utilize that hardware. Key considerations include avoiding contention over shared resources like CPU, disk, memory and network. Examples of scalable architectures include lock-free skiplist indexes, sharding or partitioning data across multiple machines, distributed query execution, and columnar data stores. Building for scale changes how software features are developed, requiring simple initial designs, leveraging existing resources, ensuring the right technical decisions through code reviews and technical leadership.
Building an intelligent big data application in 30 minutes
Strata Barcelona presentation slides, a live demo of building an intelligent big data application from a web console. The tools and APIs behind are built on top of Spark, Spark SQL/Shark, Tachyon, Mesos, Cassandra, SolrCloud, iPython and include: ELT pipeline (ingestion and transformation), data warehouse explorer, export to NoSql and generated APIs, export to SolrCloud and generated APIs, predictive model building, training and publishing, dashboard UI, monitoring and instrumentation.
PowerStream Demo
At Spark Summit East in New York, we unveil PowerStream, an Internet of Things (IoT) simulation with visualizations and alerts based on real-time data from 2 million sensors across global wind farms.
Big data for dot net Devs with Spark
The document discusses Apache Spark, a unified analytics engine for large-scale data processing. It introduces .Net for Apache Spark, which provides .Net language bindings for Spark. It also mentions using the MovieLens dataset with Spark on Azure Synapse Analytics. Key components of Spark include RDDs, DataFrames, SparkSession, and transformations/actions. The document provides an overview of Spark and demonstrates it through a movie recommendation example on Azure Synapse Analytics.
Resilient Microservices Architecture with kubernetes
Slides from the VoxxedDays Singapore 2019 talk about Resilient Microservices Architecture with Kubernetes
Serverless real use cases and best practices: Asavari Tayal, Microsoft, Serve...
Let's take a look at real cases from Microsoft customers worldwide to learn how they solved their problems with the Azure serverless platform, as well as best practices (and some caveats) from the architectures used developing these solutions.
Recommended
How to crunch 1 m messages per second in 4 easy steps - Radu Vunvulea
This document discusses Radu Vunvulea's background and experience in areas like banking, automotive, pharma, e-commerce, Azure, JavaScript, .NET, and event streaming technologies. It also provides an example of how event data can be ingested from IoT devices or applications via Event Hubs, processed in real-time using Stream Analytics, and stored in SQL databases, blob storage, or used for notifications and reporting. Finally, it includes a short bio of Radu Vunvulea and examples of messages he may post.
Real Time Twitter sentiment analysis
Utilized the Twitter WPF client to extract data based on Hashtags and fed to sentiment140 for sentiment analysis.
Loaded sentiment analyzed tweets in Azure using event hubs and performed analysis using SQL in stream analytics
Stored the analyzed data in Azure Blob Storage and visualized the outcomes of analysis in real time using Power BI
Building Software to Scale
Building scalable software requires designing it so that adding more hardware allows the software to utilize that hardware. Key considerations include avoiding contention over shared resources like CPU, disk, memory and network. Examples of scalable architectures include lock-free skiplist indexes, sharding or partitioning data across multiple machines, distributed query execution, and columnar data stores. Building for scale changes how software features are developed, requiring simple initial designs, leveraging existing resources, ensuring the right technical decisions through code reviews and technical leadership.
Building an intelligent big data application in 30 minutes
Strata Barcelona presentation slides, a live demo of building an intelligent big data application from a web console. The tools and APIs behind are built on top of Spark, Spark SQL/Shark, Tachyon, Mesos, Cassandra, SolrCloud, iPython and include: ELT pipeline (ingestion and transformation), data warehouse explorer, export to NoSql and generated APIs, export to SolrCloud and generated APIs, predictive model building, training and publishing, dashboard UI, monitoring and instrumentation.
PowerStream Demo
At Spark Summit East in New York, we unveil PowerStream, an Internet of Things (IoT) simulation with visualizations and alerts based on real-time data from 2 million sensors across global wind farms.
Big data for dot net Devs with Spark
The document discusses Apache Spark, a unified analytics engine for large-scale data processing. It introduces .Net for Apache Spark, which provides .Net language bindings for Spark. It also mentions using the MovieLens dataset with Spark on Azure Synapse Analytics. Key components of Spark include RDDs, DataFrames, SparkSession, and transformations/actions. The document provides an overview of Spark and demonstrates it through a movie recommendation example on Azure Synapse Analytics.
Resilient Microservices Architecture with kubernetes
Slides from the VoxxedDays Singapore 2019 talk about Resilient Microservices Architecture with Kubernetes
Serverless real use cases and best practices: Asavari Tayal, Microsoft, Serve...
Let's take a look at real cases from Microsoft customers worldwide to learn how they solved their problems with the Azure serverless platform, as well as best practices (and some caveats) from the architectures used developing these solutions.
Azure Container Services
The document summarizes a DevOps Porto Community meeting. It includes the name and contact information of a speaker, Pedro Sousa, as well as links to some of his past presentations. It also lists several Microsoft employees who were present at the meeting including Taylor Brown, Stefan Scherer, Ross Gardler, and Donovan Brown. The document poses a question and includes a diagram outlining the Azure container platform.
Analytics in the Cloud
The document discusses building an end-to-end analytic solution in the cloud using Microsoft Azure tools, including ingesting data from various sources into Azure Data Factory, storing it in Azure Data Lake, transforming the data using U-SQL scripts in Azure Data Lake Analytics, developing predictive models with Azure Machine Learning Studio, and visualizing insights with Power BI. It provides examples of how each tool in the analytic lifecycle can be leveraged as part of an overall cloud-based analytics solution handling large volumes of data.
Azure Container Services
"Azure Container Services" public presentation at IV Docker Porto Community Meeting, Anniversary Edition
Event driven workloads on Kubernetes with KEDA
Slide deck of the presentation done at the Pune User Group on 27th February 2021. Demonstrate how Kubernetes based event driven autoscaling (KEDA) can be used with RabbitMQ as the event source.
Azure containers fundamentals
Slides of the session presented at Dear Azure user group on 25th October. The session covered Docker basics, working with SQL Server 2017 & 2019 in Linux containers, containerizing the .Net Core application. Also demonstrated the options for deploying containers to Azure using Web Apps, Serverless Azure Container Instances and Azure Kubernetes Service (AKS).
Scaling containers with KEDA
SLide deck from the presentation done at the PD Tech Fest event in Manila on 10th August 2019. The talk demonstrated the use of Kubernates-based Event Driven Autoscaling (KEDA) features. The demo shows how to scale RabbitMQ consumer based on the number of messages in a queue.
Building near real-time HTAP solutions using Synapse Link for Azure Cosmos DB
This document discusses using Azure Cosmos DB and Azure Synapse Analytics together to enable near real-time analytics on operational data. It notes that running analytics directly on an operational database can impact performance. Instead, it recommends using Azure Synapse Link for Cosmos DB to automatically sync operational data to Azure Synapse Analytics for analytics and reporting. This allows separating OLTP and OLAP workloads while still providing near real-time insights. It then demonstrates this solution for a retailer doing supply chain management and sales forecasting across thousands of locations.
BTUG - Dec 2014 - Hybrid Connectivity Options
This document discusses various integration patterns and architectures that involve Microsoft Azure and BizTalk Server. It presents questions that customers may ask about integration solutions. It also provides examples of hybrid integration architectures that leverage Azure services like Service Bus along with on-premises BizTalk Server. The document aims to help customers analyze requirements and evaluate different architectural options for their integration needs.
Azuresatpn19 - An Introduction To Azure Data Factory
Slide deck of my session during Azure Saturday 2019 in Pordenone.
We spoke about Azure Data Factory v2.
Serverless Event Driven Containers with KEDA
Slides from the OSSDays conference (https://ossdays.konfhub.com/). The session demonstrated the power of Kubernetes based Event Driven Autoscaling (KEDA) to scale RabbitMQ consumer into serverless Azure Container Instances (ACI) .
Scaling .net containers with event driven workloads
Slide deck related to the talk presented at Surati Tech Talks 2022 event on 10th January 2022. Demonstrates the power of Kubernetes along with Kubernetes-based Event Driven Autoscaing (KEDA) project to support event driven workloads. Microsft Azure is used as the cloud platform with managed Azure Kubernetes Service (AKS) as target environment
StackEngine Demo - Docker Austin
StackEngine describes the problems of running containerized applications in production and the solutions it is offering.
RedisConf18 - Probabilistic Data Structures in Redis
This document discusses using probabilistic data structures in Redis to process large volumes of event data in real-time. It introduces HyperLogLog for cardinality counting, TopK for top elements, CountMinSketch for frequency counting, and Bloom Filters for set membership testing. Code examples are provided for adding, querying and merging data in each structure using Redis modules.
Autoscaling containers with event driven workloads
Slides from the session conducted for Azure Developer Community on 29th January 2022 (https://reskilll.com/event/autoscalecontainers). The live dem showcased capabilities of Kubernetes-based Event Driven Autoscaling (KEDA) project to scale .net core containers based on messages in the RabbitMQ.
Deploy Microsoft Azure Data Solutions
Quali vantaggi ci da Azure? Dal punto di vista dello sviluppo software, uno di questi è certamente la varietà dei servizi di gestione dei dati. Questo ci permette di cominciare a non essere SQL centrici ma utilizzare il servizio giusto per il problema giusto fino ad applicare una strategia di Polyglot Persistence (e vedremo cosa significa) nel rispetto di una corretta gestione delle risorse IT e delle pratiche di DevOps.
Monitor Cloud Resources using Alerts & Insights
Azure Monitor provides centralized monitoring of Azure resources and applications. It collects metrics, logs, and application performance monitoring data from Azure resources, the Azure platform, and on-premises sources. It provides visibility into resource performance and usage, enables alerting and automation of responses to issues. Azure Monitor features include dashboards for visualizing data, log analytics for querying and analyzing logs, and integration with other Azure services for additional monitoring capabilities like Application Insights.
Community day ppt_kinesisv1.0
Kinesis is an AWS streaming data service that allows users to ingest, process, and analyze real-time streaming data. It offers three main services: Kinesis Data Streams to collect and process streaming data in real-time; Kinesis Data Firehose to load streaming data into AWS data stores; and Kinesis Data Analytics to process and analyze streaming data using SQL or Java for real-time analytics and alerts. Kinesis provides scalability, is fully managed, and enables users to build real-time applications on streaming data.
Why would I store my data in more than one database?
This document discusses data storage solutions and introduces MemSQL, an in-memory distributed relational database. It provides a brief history of data storage from files to centralized online services. New data types and volumes require new tools like NoSQL databases but existing relational databases can also address these needs. MemSQL is presented as an in-memory distributed SQL database that is ACID compliant, horizontally scalable, compatible with MySQL, and designed for real-time analytics. A demo shows MemSQL outperforming SQL Server for a slow query by syncing data into MemSQL from SQL Server using change data capture. The key lessons are that mistakes will happen with new tools and knowing limitations is important, and when to abandon an approach.
Cncf event driven autoscaling with keda
This document discusses Kubernetes event-driven autoscaling (KEDA) which allows deployments to scale based on external events rather than resource metrics. KEDA monitors event sources like queues and scales the workload by modifying the horizontal pod autoscaler. It supports scaling deployments from zero replicas and scaling batch jobs. Real-world examples of using KEDA include scaling game workload for events and processing messages from queues in batches.
Introduction to Azure PaaS services (Nick Trogh at Codit Azure PaaS Event)
This document provides an introduction to Azure PaaS services and how they can help with application development and hosting. It describes how Azure App Service can be used to host web and mobile backends and APIs. It also discusses using Azure SQL Database for relational data storage and Azure DocumentDB for NoSQL storage. The document then covers using Azure Service Fabric for microservices architectures and Azure IoT Suite and services like IoT Hub, Stream Analytics and Machine Learning for IoT scenarios. It concludes by listing the various platform services available in Azure.
Building a Real-Time IoT monitoring application with Azure
Being able to analyze data in real-time is a very hot topic already and it will be more and more in. From product recommendations to fraud detection alarms a lot of stuff would be perfect if it could happen in real time. In this session a sample solution using the serverless capabilities of Azure will be developed, right from the ingestion of sensor data to their analysis and recommendation using AI in real time. Come to see how you could do the same in your environment, moving your application capabilities to the next level.
Azure Data Explorer deep dive - review 04.2020
Modern Data Science Lifecycle with ADX & Azure
This document discusses using Azure Data Explorer (ADX) for data science workflows. ADX is a fully managed analytics service for real-time analysis of streaming data. It allows for ad-hoc querying of data using Kusto Query Language (KQL) and integrates with various Azure data ingestion sources. The document provides an overview of the ADX architecture and compares it to other time series databases. It also covers best practices for ingesting data, visualizing results, and automating workflows using tools like Azure Data Factory.
More Related Content
What's hot
Azure Container Services
The document summarizes a DevOps Porto Community meeting. It includes the name and contact information of a speaker, Pedro Sousa, as well as links to some of his past presentations. It also lists several Microsoft employees who were present at the meeting including Taylor Brown, Stefan Scherer, Ross Gardler, and Donovan Brown. The document poses a question and includes a diagram outlining the Azure container platform.
Analytics in the Cloud
The document discusses building an end-to-end analytic solution in the cloud using Microsoft Azure tools, including ingesting data from various sources into Azure Data Factory, storing it in Azure Data Lake, transforming the data using U-SQL scripts in Azure Data Lake Analytics, developing predictive models with Azure Machine Learning Studio, and visualizing insights with Power BI. It provides examples of how each tool in the analytic lifecycle can be leveraged as part of an overall cloud-based analytics solution handling large volumes of data.
Azure Container Services
"Azure Container Services" public presentation at IV Docker Porto Community Meeting, Anniversary Edition
Event driven workloads on Kubernetes with KEDA
Slide deck of the presentation done at the Pune User Group on 27th February 2021. Demonstrate how Kubernetes based event driven autoscaling (KEDA) can be used with RabbitMQ as the event source.
Azure containers fundamentals
Slides of the session presented at Dear Azure user group on 25th October. The session covered Docker basics, working with SQL Server 2017 & 2019 in Linux containers, containerizing the .Net Core application. Also demonstrated the options for deploying containers to Azure using Web Apps, Serverless Azure Container Instances and Azure Kubernetes Service (AKS).
Scaling containers with KEDA
SLide deck from the presentation done at the PD Tech Fest event in Manila on 10th August 2019. The talk demonstrated the use of Kubernates-based Event Driven Autoscaling (KEDA) features. The demo shows how to scale RabbitMQ consumer based on the number of messages in a queue.
Building near real-time HTAP solutions using Synapse Link for Azure Cosmos DB
This document discusses using Azure Cosmos DB and Azure Synapse Analytics together to enable near real-time analytics on operational data. It notes that running analytics directly on an operational database can impact performance. Instead, it recommends using Azure Synapse Link for Cosmos DB to automatically sync operational data to Azure Synapse Analytics for analytics and reporting. This allows separating OLTP and OLAP workloads while still providing near real-time insights. It then demonstrates this solution for a retailer doing supply chain management and sales forecasting across thousands of locations.
BTUG - Dec 2014 - Hybrid Connectivity Options
This document discusses various integration patterns and architectures that involve Microsoft Azure and BizTalk Server. It presents questions that customers may ask about integration solutions. It also provides examples of hybrid integration architectures that leverage Azure services like Service Bus along with on-premises BizTalk Server. The document aims to help customers analyze requirements and evaluate different architectural options for their integration needs.
Azuresatpn19 - An Introduction To Azure Data Factory
Slide deck of my session during Azure Saturday 2019 in Pordenone.
We spoke about Azure Data Factory v2.
Serverless Event Driven Containers with KEDA
Slides from the OSSDays conference (https://ossdays.konfhub.com/). The session demonstrated the power of Kubernetes based Event Driven Autoscaling (KEDA) to scale RabbitMQ consumer into serverless Azure Container Instances (ACI) .
Scaling .net containers with event driven workloads
Slide deck related to the talk presented at Surati Tech Talks 2022 event on 10th January 2022. Demonstrates the power of Kubernetes along with Kubernetes-based Event Driven Autoscaing (KEDA) project to support event driven workloads. Microsft Azure is used as the cloud platform with managed Azure Kubernetes Service (AKS) as target environment
StackEngine Demo - Docker Austin
StackEngine describes the problems of running containerized applications in production and the solutions it is offering.
RedisConf18 - Probabilistic Data Structures in Redis
This document discusses using probabilistic data structures in Redis to process large volumes of event data in real-time. It introduces HyperLogLog for cardinality counting, TopK for top elements, CountMinSketch for frequency counting, and Bloom Filters for set membership testing. Code examples are provided for adding, querying and merging data in each structure using Redis modules.
Autoscaling containers with event driven workloads
Slides from the session conducted for Azure Developer Community on 29th January 2022 (https://reskilll.com/event/autoscalecontainers). The live dem showcased capabilities of Kubernetes-based Event Driven Autoscaling (KEDA) project to scale .net core containers based on messages in the RabbitMQ.
Deploy Microsoft Azure Data Solutions
Quali vantaggi ci da Azure? Dal punto di vista dello sviluppo software, uno di questi è certamente la varietà dei servizi di gestione dei dati. Questo ci permette di cominciare a non essere SQL centrici ma utilizzare il servizio giusto per il problema giusto fino ad applicare una strategia di Polyglot Persistence (e vedremo cosa significa) nel rispetto di una corretta gestione delle risorse IT e delle pratiche di DevOps.
Monitor Cloud Resources using Alerts & Insights
Azure Monitor provides centralized monitoring of Azure resources and applications. It collects metrics, logs, and application performance monitoring data from Azure resources, the Azure platform, and on-premises sources. It provides visibility into resource performance and usage, enables alerting and automation of responses to issues. Azure Monitor features include dashboards for visualizing data, log analytics for querying and analyzing logs, and integration with other Azure services for additional monitoring capabilities like Application Insights.
Community day ppt_kinesisv1.0
Kinesis is an AWS streaming data service that allows users to ingest, process, and analyze real-time streaming data. It offers three main services: Kinesis Data Streams to collect and process streaming data in real-time; Kinesis Data Firehose to load streaming data into AWS data stores; and Kinesis Data Analytics to process and analyze streaming data using SQL or Java for real-time analytics and alerts. Kinesis provides scalability, is fully managed, and enables users to build real-time applications on streaming data.
Why would I store my data in more than one database?
This document discusses data storage solutions and introduces MemSQL, an in-memory distributed relational database. It provides a brief history of data storage from files to centralized online services. New data types and volumes require new tools like NoSQL databases but existing relational databases can also address these needs. MemSQL is presented as an in-memory distributed SQL database that is ACID compliant, horizontally scalable, compatible with MySQL, and designed for real-time analytics. A demo shows MemSQL outperforming SQL Server for a slow query by syncing data into MemSQL from SQL Server using change data capture. The key lessons are that mistakes will happen with new tools and knowing limitations is important, and when to abandon an approach.
Cncf event driven autoscaling with keda
This document discusses Kubernetes event-driven autoscaling (KEDA) which allows deployments to scale based on external events rather than resource metrics. KEDA monitors event sources like queues and scales the workload by modifying the horizontal pod autoscaler. It supports scaling deployments from zero replicas and scaling batch jobs. Real-world examples of using KEDA include scaling game workload for events and processing messages from queues in batches.
Introduction to Azure PaaS services (Nick Trogh at Codit Azure PaaS Event)
This document provides an introduction to Azure PaaS services and how they can help with application development and hosting. It describes how Azure App Service can be used to host web and mobile backends and APIs. It also discusses using Azure SQL Database for relational data storage and Azure DocumentDB for NoSQL storage. The document then covers using Azure Service Fabric for microservices architectures and Azure IoT Suite and services like IoT Hub, Stream Analytics and Machine Learning for IoT scenarios. It concludes by listing the various platform services available in Azure.
What's hot (20)
Building near real-time HTAP solutions using Synapse Link for Azure Cosmos DB
Building near real-time HTAP solutions using Synapse Link for Azure Cosmos DB
Azuresatpn19 - An Introduction To Azure Data Factory
Azuresatpn19 - An Introduction To Azure Data Factory
Scaling .net containers with event driven workloads
Scaling .net containers with event driven workloads
RedisConf18 - Probabilistic Data Structures in Redis
RedisConf18 - Probabilistic Data Structures in Redis
Autoscaling containers with event driven workloads
Autoscaling containers with event driven workloads
Why would I store my data in more than one database?
Why would I store my data in more than one database?
Introduction to Azure PaaS services (Nick Trogh at Codit Azure PaaS Event)
Introduction to Azure PaaS services (Nick Trogh at Codit Azure PaaS Event)
Similar to How to manage one million messages per second using Azure, Radu Vunvulea, ITDays 2015, Cluj-Napoca
Building a Real-Time IoT monitoring application with Azure
Being able to analyze data in real-time is a very hot topic already and it will be more and more in. From product recommendations to fraud detection alarms a lot of stuff would be perfect if it could happen in real time. In this session a sample solution using the serverless capabilities of Azure will be developed, right from the ingestion of sensor data to their analysis and recommendation using AI in real time. Come to see how you could do the same in your environment, moving your application capabilities to the next level.
Azure Data Explorer deep dive - review 04.2020
Modern Data Science Lifecycle with ADX & Azure
This document discusses using Azure Data Explorer (ADX) for data science workflows. ADX is a fully managed analytics service for real-time analysis of streaming data. It allows for ad-hoc querying of data using Kusto Query Language (KQL) and integrates with various Azure data ingestion sources. The document provides an overview of the ADX architecture and compares it to other time series databases. It also covers best practices for ingesting data, visualizing results, and automating workflows using tools like Azure Data Factory.
Building Cloud-Native Applications with Microsoft Windows Azure
Cloud computing is here to stay, and it is never too soon to begin understanding the impact it will have on application architecture. In this talk we will discuss the two most significant architectural mind-shifts, discussing the key patterns changes generally and seeing how these new cloud patterns map naturally into specific programming practices in Windows Azure. Specifically this relates to (a) Azure Roles and Queues and how to combine them using cloud-friendly design
patterns, and (b) the combination of relational data and non-relational data, how to decide among them, and how to combine them. The goal is for mere mortals to build highly reliable applications that scale economically. The concepts discussed in this talk are relevant for developers and architects building systems for the cloud today, or who want to be prepared to move to the cloud in the future.
This talk was delivered by Bill Wilder at the Vermont Code Camp 2 on 11-Sept-2010.
Internet of Things from an Azure Developer Perspective
The document discusses Azure components that can be used to manage and control data streams. It mentions several Azure services including Event Hubs, DocumentDB, Stream Analytics, Notification Hubs, Machine Learning, and HDInsight that can ingest, process, store and analyze streaming data. It provides an overview of how Event Hubs can be used to collect data from various sources and partitions and how Stream Analytics can be used to process and analyze the streaming data in real-time using windowing and jobs. It also discusses how machine learning algorithms can be easily trained and deployed on streaming data to improve efficiency and enable innovation.
1 Introduction to Microsoft data platform analytics for release
Part 1 of a conference workshop. This forms the morning session, which looks at moving from Business Intelligence to Analytics.
Topics Covered: Azure Data Explorer, Azure Data Factory, Azure Synapse Analytics, Event Hubs, HDInsight, Big Data
Why Scale Matters and How the Cloud is Really Different (at scale)
This document discusses how various companies scale their services and applications on AWS to handle large user loads and data volumes. It provides examples of Animoto handling over 1 billion files saved per day and Airbnb having over 9 million guests. It then outlines an approach for scaling an application from 1 user to millions by starting with EC2 instances, adding services like S3, DynamoDB, ElastiCache and auto-scaling groups. The document emphasizes using AWS managed services to avoid re-inventing solutions for tasks like queuing, storage and databases.
Azure Synapse Analytics Overview (r1)
Azure Synapse Analytics is Azure SQL Data Warehouse evolved: a limitless analytics service, that brings together enterprise data warehousing and Big Data analytics into a single service. It gives you the freedom to query data on your terms, using either serverless on-demand or provisioned resources, at scale. Azure Synapse brings these two worlds together with a unified experience to ingest, prepare, manage, and serve data for immediate business intelligence and machine learning needs. This is a huge deck with lots of screenshots so you can see exactly how it works.
Azure Functions Hands-on lab | Global Azure Bootcamp | Radu Vunvulea
The document discusses how Azure Functions work and their uses. Azure Functions can be triggered by events on a schedule, HTTP requests, Azure services like storage, queues, and event hubs. Functions support inputs from these services and can write outputs to storage, queues, databases. Popular uses of Functions include timer-based processing, service event processing, serverless applications, backends, and real-time processing.
Serverless and AI: Orit Nissan-Messing, Iguazio, Serverless NYC 2018
This document discusses serverless computing and introduces Nuclio, an open source serverless platform. Some key points:
- Serverless platforms provide easy deployment of functions but lack performance and number of event sources. Nuclio aims to improve on this with high concurrency and low latency.
- Nuclio's architecture allows extreme performance of up to 400,000 events/second per process with sub-second latency. It supports various event sources and data bindings.
- Nuclio works with Kubernetes, providing portability across clouds, on-premises, and hybrid environments while automating infrastructure management and scaling.
DEVNET-1140 InterCloud Mapreduce and Spark Workload Migration and Sharing: Fi...
Data gravity is a reality when dealing with massive amounts and globally distributed systems. Processing this data requires distributed analytics processing across InterCloud. In this presentation we will share our real world experience with storing, routing, and processing big data workloads on Cisco Cloud Services and Amazon Web Services clouds.
Nosql why and how on Microsoft Azure
This document discusses NoSQL databases and how they can be used on Microsoft Azure. It introduces the presenters, Eva Gjeci and Vito Flavio Lorusso, as technology evangelists for Microsoft focused on Azure. It then provides an overview of different types of NoSQL databases and examples of how they can be implemented on Azure, including using virtual machines, Azure services like DocumentDB, Redis Cache, and Table Storage, as well as hosted database services. The document demonstrates setting up MongoDB and Cassandra on Azure virtual machines and using DocumentDB and its querying capabilities. It also discusses optimizing NoSQL database storage and high availability options on Azure.
An Overview of AWS IoT (November 2016)
This document discusses IoT projects running on Amazon Web Services (AWS). It provides an overview of the AWS IoT platform, which allows connection of IoT devices, collection of data using MQTT protocol, and storage/analysis of data in AWS services. It also demonstrates connecting an Arduino device to AWS IoT and subscribing/publishing data to topics.
Cloud computing and the Windows Azure Services Platform (KU Leuven)
Maarten Balliauw gave a presentation on cloud computing and the Windows Azure Services Platform. He began with an overview of cloud computing and defined infrastructure as a service (IAAS), platform as a service (PAAS), and software as a service (SASS). He then discussed Windows Azure and its core services - Blob storage, Table storage, Queue storage, and SQL Azure. Balliauw also covered the Windows Azure AppFabric and its Service Bus and Access Control capabilities. He concluded with information on getting started with Azure and resources for learning more.
Streaming Visualization
Most data visualisation solutions today still work on data sources which are stored persistently in a data store, using the so called “data at rest” paradigms. More and more data sources today provide a constant stream of data, from IoT devices to Social Media streams. These data stream publish with high velocity and messages often have to be processed as quick as possible. For the processing and analytics on the data, so called stream processing solutions are available. But these only provide minimal or no visualisation capabilities. One was is to first persist the data into a data store and then use a traditional data visualisation solution to present the data.
If latency is not an issue, such a solution might be good enough. An other question is which data store solution is necessary to keep up with the high load on write and read. If it is not an RDBMS but an NoSQL database, then not all traditional visualisation tools might already integrate with the specific data store. An other option is to use a Streaming Visualisation solution. They are specially built for streaming data and often do not support batch data. A much better solution would be to have one tool capable of handling both, batch and streaming data. This talk presents different architecture blueprints for integrating data visualisation into a fast data solution and highlights some of the products available to implement these blueprints.
Samedi SQL Québec - La plateforme data de Azure
6 juin 2015
Samedi SQL à Québec
Session 3 - Data (SQL Azure, Table et Blob Storage) (Eric Moreau)
SQL Azure est une base de données relationnelle en tant que service, Azure Storage permet de stocker et d'extraire de gros volumes de données non structurées (par exemple, des documents et fichiers multimédias) avec les objets blob Azure ; de données NoSql structurées avec les tables Azure ; de messages fiables avec les files d'attente Azure.
Differentiate Big Data vs Data Warehouse use cases for a cloud solution
It can be quite challenging keeping up with the frequent updates to the Microsoft products and understanding all their use cases and how all the products fit together. In this session we will differentiate the use cases for each of the Microsoft services, explaining and demonstrating what is good and what isn't, in order for you to position, design and deliver the proper adoption use cases for each with your customers. We will cover a wide range of products such as Databricks, SQL Data Warehouse, HDInsight, Azure Data Lake Analytics, Azure Data Lake Store, Blob storage, and AAS as well as high-level concepts such as when to use a data lake. We will also review the most common reference architectures (“patterns”) witnessed in customer adoption.
Real-time big data analytics based on product recommendations case study
We started as an ad network. The challenge was to recommend the best product (out of millions) to the right person in a given moment (thousands of users within a second). We have delivered 5 billion ad views since 24 months. To put it in the scale context: If we would serve 1 ad per second it will take 160 years to serve 5 billion ads.
So we needed a solution. SQL databases did not work. Popular NoSQL databases did not work. Standard data warehouse approaches (pre-aggregations, creating schemas) - did not work too.
Re-thinking all the problems with huge data streams flowing to us every second we have built a complete solution based on open-source technologies and fresh, smart ideas from our engineering team. It is called deep.bi and now we make it available to other companies.
deep.bi lets high-growth companies solve fast data problems by providing scalable, flexible and real-time data collection, enrichment and analytics.
It was built using:
- Node.js - API
- Kafka - collecting and distributing data
- Spark Streaming - ETL, data enrichments
- Druid - real-time analytics
- Cassandra - user events store
- Hadoop + Parquet + Spark - raw data store + ad-hoc queries
AWS re:Invent 2016: Life Without SSH: Immutable Infrastructure in Production ...
This session covers what a real-world production deployment of a fully automated deployment pipeline looks like with instances that are deployed without SSH keys. By leveraging AWS CloudFormation along with Docker and AWS CodeDeploy, we show how we achieved semi-immutable and fully immutable infrastructures, and what the challenges and remediations were.
UnConference for Georgia Southern Computer Science March 31, 2015
I presented to the Georgia Southern Computer Science ACM group. Rather than one topic for 90 minutes, I decided to do an UnConference. I presented them a list of 8-9 topics, let them vote on what to talk about, then repeated.
Each presentation was ~8 minutes, (Except Career) and was by no means an attempt to explain the full concept or technology. Only to wake up their interest.
Build 2015 – Azure overview
azure overview, impressions from build 2015, cloud storage, compute, data lake, analytics, service fabric, containerization, etc.
Similar to How to manage one million messages per second using Azure, Radu Vunvulea, ITDays 2015, Cluj-Napoca (20)
Building a Real-Time IoT monitoring application with Azure
Building a Real-Time IoT monitoring application with Azure
Building Cloud-Native Applications with Microsoft Windows Azure
Building Cloud-Native Applications with Microsoft Windows Azure
Internet of Things from an Azure Developer Perspective
Internet of Things from an Azure Developer Perspective
1 Introduction to Microsoft data platform analytics for release
1 Introduction to Microsoft data platform analytics for release
Why Scale Matters and How the Cloud is Really Different (at scale)
Why Scale Matters and How the Cloud is Really Different (at scale)
Azure Functions Hands-on lab | Global Azure Bootcamp | Radu Vunvulea
Azure Functions Hands-on lab | Global Azure Bootcamp | Radu Vunvulea
Serverless and AI: Orit Nissan-Messing, Iguazio, Serverless NYC 2018
Serverless and AI: Orit Nissan-Messing, Iguazio, Serverless NYC 2018
DEVNET-1140 InterCloud Mapreduce and Spark Workload Migration and Sharing: Fi...
DEVNET-1140 InterCloud Mapreduce and Spark Workload Migration and Sharing: Fi...
Cloud computing and the Windows Azure Services Platform (KU Leuven)
Cloud computing and the Windows Azure Services Platform (KU Leuven)
Differentiate Big Data vs Data Warehouse use cases for a cloud solution
Differentiate Big Data vs Data Warehouse use cases for a cloud solution
Real-time big data analytics based on product recommendations case study
Real-time big data analytics based on product recommendations case study
AWS re:Invent 2016: Life Without SSH: Immutable Infrastructure in Production ...
AWS re:Invent 2016: Life Without SSH: Immutable Infrastructure in Production ...
UnConference for Georgia Southern Computer Science March 31, 2015
UnConference for Georgia Southern Computer Science March 31, 2015
Recently uploaded
5th LF Energy Power Grid Model Meet-up Slides
5th Power Grid Model Meet-up
It is with great pleasure that we extend to you an invitation to the 5th Power Grid Model Meet-up, scheduled for 6th June 2024. This event will adopt a hybrid format, allowing participants to join us either through an online Mircosoft Teams session or in person at TU/e located at Den Dolech 2, Eindhoven, Netherlands. The meet-up will be hosted by Eindhoven University of Technology (TU/e), a research university specializing in engineering science & technology.
Power Grid Model
The global energy transition is placing new and unprecedented demands on Distribution System Operators (DSOs). Alongside upgrades to grid capacity, processes such as digitization, capacity optimization, and congestion management are becoming vital for delivering reliable services.
Power Grid Model is an open source project from Linux Foundation Energy and provides a calculation engine that is increasingly essential for DSOs. It offers a standards-based foundation enabling real-time power systems analysis, simulations of electrical power grids, and sophisticated what-if analysis. In addition, it enables in-depth studies and analysis of the electrical power grid’s behavior and performance. This comprehensive model incorporates essential factors such as power generation capacity, electrical losses, voltage levels, power flows, and system stability.
Power Grid Model is currently being applied in a wide variety of use cases, including grid planning, expansion, reliability, and congestion studies. It can also help in analyzing the impact of renewable energy integration, assessing the effects of disturbances or faults, and developing strategies for grid control and optimization.
What to expect
For the upcoming meetup we are organizing, we have an exciting lineup of activities planned:
-Insightful presentations covering two practical applications of the Power Grid Model.
-An update on the latest advancements in Power Grid -Model technology during the first and second quarters of 2024.
-An interactive brainstorming session to discuss and propose new feature requests.
-An opportunity to connect with fellow Power Grid Model enthusiasts and users.
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-und-domino-lizenzkostenreduzierung-in-der-welt-von-dlau/
DLAU und die Lizenzen nach dem CCB- und CCX-Modell sind für viele in der HCL-Community seit letztem Jahr ein heißes Thema. Als Notes- oder Domino-Kunde haben Sie vielleicht mit unerwartet hohen Benutzerzahlen und Lizenzgebühren zu kämpfen. Sie fragen sich vielleicht, wie diese neue Art der Lizenzierung funktioniert und welchen Nutzen sie Ihnen bringt. Vor allem wollen Sie sicherlich Ihr Budget einhalten und Kosten sparen, wo immer möglich. Das verstehen wir und wir möchten Ihnen dabei helfen!
Wir erklären Ihnen, wie Sie häufige Konfigurationsprobleme lösen können, die dazu führen können, dass mehr Benutzer gezählt werden als nötig, und wie Sie überflüssige oder ungenutzte Konten identifizieren und entfernen können, um Geld zu sparen. Es gibt auch einige Ansätze, die zu unnötigen Ausgaben führen können, z. B. wenn ein Personendokument anstelle eines Mail-Ins für geteilte Mailboxen verwendet wird. Wir zeigen Ihnen solche Fälle und deren Lösungen. Und natürlich erklären wir Ihnen das neue Lizenzmodell.
Nehmen Sie an diesem Webinar teil, bei dem HCL-Ambassador Marc Thomas und Gastredner Franz Walder Ihnen diese neue Welt näherbringen. Es vermittelt Ihnen die Tools und das Know-how, um den Überblick zu bewahren. Sie werden in der Lage sein, Ihre Kosten durch eine optimierte Domino-Konfiguration zu reduzieren und auch in Zukunft gering zu halten.
Diese Themen werden behandelt
- Reduzierung der Lizenzkosten durch Auffinden und Beheben von Fehlkonfigurationen und überflüssigen Konten
- Wie funktionieren CCB- und CCX-Lizenzen wirklich?
- Verstehen des DLAU-Tools und wie man es am besten nutzt
- Tipps für häufige Problembereiche, wie z. B. Team-Postfächer, Funktions-/Testbenutzer usw.
- Praxisbeispiele und Best Practices zum sofortigen Umsetzen
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Inconsistent user experience and siloed data, high costs, and changing customer expectations – Citizens Bank was experiencing these challenges while it was attempting to deliver a superior digital banking experience for its clients. Its core banking applications run on the mainframe and Citizens was using legacy utilities to get the critical mainframe data to feed customer-facing channels, like call centers, web, and mobile. Ultimately, this led to higher operating costs (MIPS), delayed response times, and longer time to market.
Ever-changing customer expectations demand more modern digital experiences, and the bank needed to find a solution that could provide real-time data to its customer channels with low latency and operating costs. Join this session to learn how Citizens is leveraging Precisely to replicate mainframe data to its customer channels and deliver on their “modern digital bank” experiences.
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
At this talk we will discuss DDoS protection tools and best practices, discuss network architectures and what AWS has to offer. Also, we will look into one of the largest DDoS attacks on Ukrainian infrastructure that happened in February 2022. We'll see, what techniques helped to keep the web resources available for Ukrainians and how AWS improved DDoS protection for all customers based on Ukraine experience
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...Edge AI and Vision Alliance
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/how-axelera-ai-uses-digital-compute-in-memory-to-deliver-fast-and-energy-efficient-computer-vision-a-presentation-from-axelera-ai/
Bram Verhoef, Head of Machine Learning at Axelera AI, presents the “How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-efficient Computer Vision” tutorial at the May 2024 Embedded Vision Summit.
As artificial intelligence inference transitions from cloud environments to edge locations, computer vision applications achieve heightened responsiveness, reliability and privacy. This migration, however, introduces the challenge of operating within the stringent confines of resource constraints typical at the edge, including small form factors, low energy budgets and diminished memory and computational capacities. Axelera AI addresses these challenges through an innovative approach of performing digital computations within memory itself. This technique facilitates the realization of high-performance, energy-efficient and cost-effective computer vision capabilities at the thin and thick edge, extending the frontier of what is achievable with current technologies.
In this presentation, Verhoef unveils his company’s pioneering chip technology and demonstrates its capacity to deliver exceptional frames-per-second performance across a range of standard computer vision networks typical of applications in security, surveillance and the industrial sector. This shows that advanced computer vision can be accessible and efficient, even at the very edge of our technological ecosystem.zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
Folding is a recent technique for building efficient recursive SNARKs. Several elegant folding protocols have been proposed, such as Nova, Supernova, Hypernova, Protostar, and others. However, all of them rely on an additively homomorphic commitment scheme based on discrete log, and are therefore not post-quantum secure. In this work we present LatticeFold, the first lattice-based folding protocol based on the Module SIS problem. This folding protocol naturally leads to an efficient recursive lattice-based SNARK and an efficient PCD scheme. LatticeFold supports folding low-degree relations, such as R1CS, as well as high-degree relations, such as CCS. The key challenge is to construct a secure folding protocol that works with the Ajtai commitment scheme. The difficulty, is ensuring that extracted witnesses are low norm through many rounds of folding. We present a novel technique using the sumcheck protocol to ensure that extracted witnesses are always low norm no matter how many rounds of folding are used. Our evaluation of the final proof system suggests that it is as performant as Hypernova, while providing post-quantum security.
Paper Link: https://eprint.iacr.org/2024/257
Generating privacy-protected synthetic data using Secludy and Milvus
During this demo, the founders of Secludy will demonstrate how their system utilizes Milvus to store and manipulate embeddings for generating privacy-protected synthetic data. Their approach not only maintains the confidentiality of the original data but also enhances the utility and scalability of LLMs under privacy constraints. Attendees, including machine learning engineers, data scientists, and data managers, will witness first-hand how Secludy's integration with Milvus empowers organizations to harness the power of LLMs securely and efficiently.
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...Edge AI and Vision Alliance
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/temporal-event-neural-networks-a-more-efficient-alternative-to-the-transformer-a-presentation-from-brainchip/
Chris Jones, Director of Product Management at BrainChip , presents the “Temporal Event Neural Networks: A More Efficient Alternative to the Transformer” tutorial at the May 2024 Embedded Vision Summit.
The expansion of AI services necessitates enhanced computational capabilities on edge devices. Temporal Event Neural Networks (TENNs), developed by BrainChip, represent a novel and highly efficient state-space network. TENNs demonstrate exceptional proficiency in handling multi-dimensional streaming data, facilitating advancements in object detection, action recognition, speech enhancement and language model/sequence generation. Through the utilization of polynomial-based continuous convolutions, TENNs streamline models, expedite training processes and significantly diminish memory requirements, achieving notable reductions of up to 50x in parameters and 5,000x in energy consumption compared to prevailing methodologies like transformers.
Integration with BrainChip’s Akida neuromorphic hardware IP further enhances TENNs’ capabilities, enabling the realization of highly capable, portable and passively cooled edge devices. This presentation delves into the technical innovations underlying TENNs, presents real-world benchmarks, and elucidates how this cutting-edge approach is positioned to revolutionize edge AI across diverse applications.Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks creates AI-boosted business software that helps employees work more efficiently and effectively. Managing data across multiple RDBMS and NoSQL databases was already a challenge at their current scale. To prepare for 10X growth, they knew it was time to rethink their database strategy. Learn how they architected a solution that would simplify scaling while keeping costs under control.
Skybuffer SAM4U tool for SAP license adoption
Manage and optimize your license adoption and consumption with SAM4U, an SAP free customer software asset management tool.
SAM4U, an SAP complimentary software asset management tool for customers, delivers a detailed and well-structured overview of license inventory and usage with a user-friendly interface. We offer a hosted, cost-effective, and performance-optimized SAM4U setup in the Skybuffer Cloud environment. You retain ownership of the system and data, while we manage the ABAP 7.58 infrastructure, ensuring fixed Total Cost of Ownership (TCO) and exceptional services through the SAP Fiori interface.
Artificial Intelligence and Electronic Warfare
Artificial Intelligence and Electronic WarfarePapadakis K.-Cyber-Information Warfare Analyst & Cyber Defense/Security Consultant-Hellenic MoD
AI & Electronic WarfareGraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
Tomaz Bratanic
Graph ML and GenAI Expert - Neo4j
What is an RPA CoE? Session 1 – CoE Vision
In the first session, we will review the organization's vision and how this has an impact on the COE Structure.
Topics covered:
• The role of a steering committee
• How do the organization’s priorities determine CoE Structure?
Speaker:
Chris Bolin, Senior Intelligent Automation Architect Anika Systems
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
The typical problem in product engineering is not bad strategy, so much as “no strategy”. This leads to confusion, lack of motivation, and incoherent action. The next time you look for a strategy and find an empty space, instead of waiting for it to be filled, I will show you how to fill it in yourself. If you’re wrong, it forces a correction. If you’re right, it helps create focus. I’ll share how I’ve approached this in the past, both what works and lessons for what didn’t work so well.
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Simplify your search for a reliable Python development partner! This list presents the top 10 trusted US providers offering comprehensive Python development services, ensuring your project's success from conception to completion.
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Dmitrii Kamaev, PhD
Senior Product Owner - QIAGEN
Introduction of Cybersecurity with OSS at Code Europe 2024
I develop the Ruby programming language, RubyGems, and Bundler, which are package managers for Ruby. Today, I will introduce how to enhance the security of your application using open-source software (OSS) examples from Ruby and RubyGems.
The first topic is CVE (Common Vulnerabilities and Exposures). I have published CVEs many times. But what exactly is a CVE? I'll provide a basic understanding of CVEs and explain how to detect and handle vulnerabilities in OSS.
Next, let's discuss package managers. Package managers play a critical role in the OSS ecosystem. I'll explain how to manage library dependencies in your application.
I'll share insights into how the Ruby and RubyGems core team works to keep our ecosystem safe. By the end of this talk, you'll have a better understanding of how to safeguard your code.
Fueling AI with Great Data with Airbyte Webinar
This talk will focus on how to collect data from a variety of sources, leveraging this data for RAG and other GenAI use cases, and finally charting your course to productionalization.
Recently uploaded (20)
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
How to manage one million messages per second using Azure, Radu Vunvulea, ITDays 2015, Cluj-Napoca
- 1. Radu Vunvulea Software Architect, iQuest Group How to manage one million messages per second using Azure
- 2. How it all started?
- 11. Discover how we can create a system that can crunch 1M messages per second
- 12. What we need?
- 14. Ingest Analyze React Store
- 15. Ingest
- 17. HTTPS AMPQ HTTP Consumer Group Consumer Group Consumer Group Event Hub Partitions
- 18. Analyze
- 20. Stream Analytics Event Hubs Storage Blobs (Files) Event Hubs Storage Blobs (Files) SQL SQL Based Syntax | Windowing | Real Time | Jobs
- 21. React
- 23. Real Time | Simple | Cheap | Flexible Event Hubs Event Hub (alerts) Stream Analytics
- 24. Store
- 26. Long Time Persistence | Queryable Stream Analytics Storage Blob SQL Database Consumer
- 29. Overview
- 30. Event Hub | Consumer Groups | Jobs | Persist | Consume Event Hubs Event Hub (alerts) Stream Analytics Storage blob SQL database Real Time Notifications Report Hadoop
- 31. Conclusion
- 32. NOT
- 35. Question Answers
- 36. { “name” : “Radu Vunvulea, “blog” : “vunvulearadu.blogspot.com”, “email” : ”vunvulear@outlook.com”, “socialMedia” : { “twitter” : “@RaduVunvulea”, “fb” : “radu.vunvulea” } }
- 37. Take a look on Azure Event Hub and Stream Analytics base concepts Next Step
- 38. { “name” : “Radu Vunvulea, “blog” : “vunvulearadu.blogspot.com”, “email” : ”vunvulear@outlook.com”, “socialMedia” : { “twitter” : “@RaduVunvulea”, “fb” : “radu.vunvulea” } }