This document summarizes a presentation about Microsoft Azure IoT services. It discusses how Event Hubs can be used to stream millions of events per second from IoT devices into Azure for processing. Event Hubs offers high throughput and partitions events for scale. Stream Analytics can then analyze the event stream in real-time. The presentation provides an overview of Event Hubs, partitioning, throughput units, and reading/writing events. It also compares Event Hubs to Service Bus queues and topics.
Getting started with azure event hubs and stream analytics servicesEastBanc Tachnologies
Author: Vladimir Bychkov, www.eastbanctech.com
The total amount of data in the world almost doubles every 2 years. Storing data for offline processing is no longer a viable business model. In the past few years, new technologies for real-time data processing emerged. Microsoft Azure offers a comprehensive set of tools to ingest and process data in motion. In this presentation we will go over and learn how to collect data from devices, how to process data in real time using Azure Stream Analytic jobs, and how to produce and handle actionable insights.
Being able to analyze data in real-time will be a very hot topic for sure in near future. Not only for IoT-related tasks but as a general approach to user-to-machine or machine-to-machine interaction. From product recommendations to fraud detection alarms, a lot of stuff would be perfect if it could happen in real time. Now, with Azure Event Hubs and Stream Analytics, it’s possible. In this session, Davide will demonstrate how to use Event Hubs to quickly ingest new real-time data and Stream Analytics to query on-the-fly data, in order to do a real-time analysis of what’s happening right now.
by Gavin Adams, Sr. IoT Specialist SA AWS
Join us for AWS IoT day at the AWS San Francisco Loft. AWS IoT enables you to easily connect and manage millions of devices securely. You can gather data from, run sophisticated analytics on, and take actions in real-time on your diverse fleet of IoT devices from edge to the cloud. You will build IoT applications with AWS IoT experts. AWS IoT provides edge-based software and cloud-based services so you can easily build IoT applications. Edge-based software, including AWS Greengrass, enables you to securely connect devices, gather data and take intelligent actions locally even when Internet connectivity is down. Cloud-based services, including AWS IoT Core, allow you to quickly onboard large and diverse fleets, maintain fleet health, and keep fleets secure.
Azure Synapse Analytics is Azure SQL Data Warehouse evolved: a limitless analytics service, that brings together enterprise data warehousing and Big Data analytics into a single service. It gives you the freedom to query data on your terms, using either serverless on-demand or provisioned resources, at scale. Azure Synapse brings these two worlds together with a unified experience to ingest, prepare, manage, and serve data for immediate business intelligence and machine learning needs. This is a huge deck with lots of screenshots so you can see exactly how it works.
Getting started with azure event hubs and stream analytics servicesEastBanc Tachnologies
Author: Vladimir Bychkov, www.eastbanctech.com
The total amount of data in the world almost doubles every 2 years. Storing data for offline processing is no longer a viable business model. In the past few years, new technologies for real-time data processing emerged. Microsoft Azure offers a comprehensive set of tools to ingest and process data in motion. In this presentation we will go over and learn how to collect data from devices, how to process data in real time using Azure Stream Analytic jobs, and how to produce and handle actionable insights.
Being able to analyze data in real-time will be a very hot topic for sure in near future. Not only for IoT-related tasks but as a general approach to user-to-machine or machine-to-machine interaction. From product recommendations to fraud detection alarms, a lot of stuff would be perfect if it could happen in real time. Now, with Azure Event Hubs and Stream Analytics, it’s possible. In this session, Davide will demonstrate how to use Event Hubs to quickly ingest new real-time data and Stream Analytics to query on-the-fly data, in order to do a real-time analysis of what’s happening right now.
by Gavin Adams, Sr. IoT Specialist SA AWS
Join us for AWS IoT day at the AWS San Francisco Loft. AWS IoT enables you to easily connect and manage millions of devices securely. You can gather data from, run sophisticated analytics on, and take actions in real-time on your diverse fleet of IoT devices from edge to the cloud. You will build IoT applications with AWS IoT experts. AWS IoT provides edge-based software and cloud-based services so you can easily build IoT applications. Edge-based software, including AWS Greengrass, enables you to securely connect devices, gather data and take intelligent actions locally even when Internet connectivity is down. Cloud-based services, including AWS IoT Core, allow you to quickly onboard large and diverse fleets, maintain fleet health, and keep fleets secure.
Azure Synapse Analytics is Azure SQL Data Warehouse evolved: a limitless analytics service, that brings together enterprise data warehousing and Big Data analytics into a single service. It gives you the freedom to query data on your terms, using either serverless on-demand or provisioned resources, at scale. Azure Synapse brings these two worlds together with a unified experience to ingest, prepare, manage, and serve data for immediate business intelligence and machine learning needs. This is a huge deck with lots of screenshots so you can see exactly how it works.
Exposing and Controlling Kafka Event Streaming with Kong Konnect Enterprise |...HostedbyConfluent
Event streaming allows companies to build more scalable and loosely coupled real-time applications supporting massive concurrency demands and simplifying the construction of services.
At the same time, API management provides capabilities to securely control the upstream services consumption, including the event processing infrastructure.
This session shows how Kong Konnect Enterprise can complement Kafka Event Streaming, exposing it to new and external consumers while applying specific and critical policies to control its consumption, including API key, OAuth/OIDC and others for authentication, rate limiting, caching, log processing, etc.
Azure Arc offers simplified management, faster app development, and consistent Azure services. Easily organize, govern, and secure Windows, Linux, SQL Server, and Kubernetes clusters across data centers, the edge, and multicloud environments right from Azure. Architect, design, and build cloud-native apps anywhere without sacrificing central visibility and control. Get Azure innovation and cloud benefits by deploying consistent Azure data, application, and machine learning services on any infrastructure.
Gain central visibility, operations, and compliance
Centrally manage a wide range of resources including Windows and Linux servers, SQL server, Kubernetes clusters, and Azure services.
Establish central visibility in the Azure portal and enable multi-environment search with Azure Resource Graph.
Meet governance and compliance standards for apps, infrastructure, and data with Azure Policy.
Delegate access and manage security policies for resources using role-based access control (RBAC) and Azure Lighthouse.
Organize and inventory assets through a variety of Azure scopes, such as management groups, subscriptions, resource groups, and tags.
Learn more about hybrid and multicloud management in the Microsoft Cloud Adoption Framework for Azure.
Introduction to Azure Event Grid and how it can be used for an event-driven architecture in the cloud that provides high availability, consistent performance, and dynamic scale.
Protect your business with a universal identity platform
The Azure Active Directory (Azure AD) enterprise identity service provides single sign-on and multi-factor authentication to help protect your users from 99.9 percent of cybersecurity attacks.
Gartner named Microsoft a leader in Magic Quadrant 2020 for Access Management
Single sign-on simplifies access to your apps from anywhere
Conditional Access and multi-factor authentication help protect and govern access
A single identity platform lets you engage with internal and external users more securely
Developer tools make it easy to integrate identity into your apps and services
Connect your workforce
Whether people are on-site or remote, give them seamless access to all their apps so they can stay productive from anywhere. Automate workflows for user lifecycle and provisioning. Save time and resources with self-service management.
Choose from thousands of SaaS apps
Simplify single sign-on. Azure AD supports thousands of pre-integrated software as a service (SaaS) applications.
Protect and govern access
Safeguard user credentials by enforcing strong authentication and conditional access policies. Efficiently manage your identities by ensuring that the right people have the right access to the right resources.
Engage with your customers and partners
Secure and manage customers and partners beyond your organizational boundaries, with one identity solution. Customize user journeys and simplify authentication with social identity and more.
Integrate identity into your apps
Accelerate adoption of your application in the enterprise by supporting single sign-on and user provisioning. Reduce sign-in friction and automate the creation, removal, and maintenance of user accounts.
Async API and Solace: Enabling the Event-Driven FutureSolace
Fran Méndez, founder of AsyncAPI, and Jonathan Schabowsky, senior architect at Solace, explain how the two companies are working together in this presentation from Gartner AADI.
Apache Kafka vs. Integration Middleware (MQ, ETL, ESB) - Friends, Enemies or ...confluent
MQ, ETL and ESB middleware are often used as integration backbone between legacy applications, modern microservices and cloud services. This introduces several challenges and complexities like point-to-point integration or non-scalable architectures. This session discusses how to build a completely event-driven streaming platform leveraging Apache Kafka’s open source messaging, integration and streaming components to leverage distributed processing, fault-tolerance, rolling upgrades and the ability to reprocess events. Learn the differences between a event-driven streaming platform leveraging Apache Kafka and middleware like MQ, ETL and ESBs – including best practices and anti-patterns, but also how these concepts and tools complement each other in an enterprise architecture.
Azure Synapse Analytics is Azure SQL Data Warehouse evolved: a limitless analytics service, that brings together enterprise data warehousing and Big Data analytics into a single service. It gives you the freedom to query data on your terms, using either serverless on-demand or provisioned resources, at scale. Azure Synapse brings these two worlds together with a unified experience to ingest, prepare, manage, and serve data for immediate business intelligence and machine learning needs. This is a huge deck with lots of screenshots so you can see exactly how it works.
The Top 5 Apache Kafka Use Cases and Architectures in 2022Kai Wähner
I see the following topics coming up more regularly in conversations with customers, prospects, and the broader Kafka community across the globe:
Kappa Architecture: Kappa goes mainstream to replace Lambda and Batch pipelines (that does not mean that there is no batch processing anymore). Examples: Kafka-powered Kappa architectures from Uber, Disney, Shopify, and Twitter.
Hyper-personalized Omnichannel: Retail and customer communication across online and offline channels becomes the new black, including context-specific upselling, recommendations, and location-based services. Examples: Omnichannel Retail and Customer 360 in Real-Time with Apache Kafka.
Multi-Cloud Deployments: Business units and IT infrastructures span across regions, continents, and cloud providers. Linking clusters for bi-directional replication of data in real-time becomes crucial for many business models. Examples: Global Kafka deployments.
Edge Analytics: Low latency requirements, cost efficiency, or security requirements enforce the deployment of (some) event streaming use cases at the far edge (i.e., outside a data center), for instance, for predictive maintenance and quality assurance on the shop floor level in smart factories. Examples: Edge analytics with Kafka.
Real-time Cybersecurity: Situational awareness and threat intelligence need to process massive data in real-time to defend against cyberattacks successfully. The many successful ransomware attacks across the globe in 2021 were a warning for most CIOs. Examples: Cybersecurity for situational awareness and threat intelligence in real-time.
An Introduction to Confluent Cloud: Apache Kafka as a Serviceconfluent
Business breakout during Confluent’s streaming event in Munich, presented by Hans Jespersen, VP WW Systems Engineering at Confluent. This three-day hands-on course focused on how to build, manage, and monitor clusters using industry best-practices developed by the world’s foremost Apache Kafka™ experts. The sessions focused on how Kafka and the Confluent Platform work, how their main subsystems interact, and how to set up, manage, monitor, and tune your cluster.
With Azure Arc customers can manage virtual machines, Kubernetes clusters and data services on-premises, at service providers and in other clouds from a single view in Azure using the innovative security, monitoring and policy tools from Azure they appreciate. In this session you will get an introduction to Azure Arc and we will dive deeper into Azure Arc for Kubernetes specifically.
Blogging can be great fun all the same and it's an ideal learning experience for you if you're a budding writer, a knowledge-nik who has a lot to share on a topic or hobby, or you're just keen to encourage people to see things your way! This article is focused on helping you to avoid the common mistakes many learner bloggers make; while you may still stumble into them, at least you'll be more aware and know how to correct your future approaches.
OMA Seminar/Webinar, October 27, 2016, "How Developers Can Get the Most Out of IoT Standards and Tools" - Presentation #8 from Juan Perez, Principal Program Manager, Microsoft
"Developing Services with Microsoft’s Azure Platform"
Exposing and Controlling Kafka Event Streaming with Kong Konnect Enterprise |...HostedbyConfluent
Event streaming allows companies to build more scalable and loosely coupled real-time applications supporting massive concurrency demands and simplifying the construction of services.
At the same time, API management provides capabilities to securely control the upstream services consumption, including the event processing infrastructure.
This session shows how Kong Konnect Enterprise can complement Kafka Event Streaming, exposing it to new and external consumers while applying specific and critical policies to control its consumption, including API key, OAuth/OIDC and others for authentication, rate limiting, caching, log processing, etc.
Azure Arc offers simplified management, faster app development, and consistent Azure services. Easily organize, govern, and secure Windows, Linux, SQL Server, and Kubernetes clusters across data centers, the edge, and multicloud environments right from Azure. Architect, design, and build cloud-native apps anywhere without sacrificing central visibility and control. Get Azure innovation and cloud benefits by deploying consistent Azure data, application, and machine learning services on any infrastructure.
Gain central visibility, operations, and compliance
Centrally manage a wide range of resources including Windows and Linux servers, SQL server, Kubernetes clusters, and Azure services.
Establish central visibility in the Azure portal and enable multi-environment search with Azure Resource Graph.
Meet governance and compliance standards for apps, infrastructure, and data with Azure Policy.
Delegate access and manage security policies for resources using role-based access control (RBAC) and Azure Lighthouse.
Organize and inventory assets through a variety of Azure scopes, such as management groups, subscriptions, resource groups, and tags.
Learn more about hybrid and multicloud management in the Microsoft Cloud Adoption Framework for Azure.
Introduction to Azure Event Grid and how it can be used for an event-driven architecture in the cloud that provides high availability, consistent performance, and dynamic scale.
Protect your business with a universal identity platform
The Azure Active Directory (Azure AD) enterprise identity service provides single sign-on and multi-factor authentication to help protect your users from 99.9 percent of cybersecurity attacks.
Gartner named Microsoft a leader in Magic Quadrant 2020 for Access Management
Single sign-on simplifies access to your apps from anywhere
Conditional Access and multi-factor authentication help protect and govern access
A single identity platform lets you engage with internal and external users more securely
Developer tools make it easy to integrate identity into your apps and services
Connect your workforce
Whether people are on-site or remote, give them seamless access to all their apps so they can stay productive from anywhere. Automate workflows for user lifecycle and provisioning. Save time and resources with self-service management.
Choose from thousands of SaaS apps
Simplify single sign-on. Azure AD supports thousands of pre-integrated software as a service (SaaS) applications.
Protect and govern access
Safeguard user credentials by enforcing strong authentication and conditional access policies. Efficiently manage your identities by ensuring that the right people have the right access to the right resources.
Engage with your customers and partners
Secure and manage customers and partners beyond your organizational boundaries, with one identity solution. Customize user journeys and simplify authentication with social identity and more.
Integrate identity into your apps
Accelerate adoption of your application in the enterprise by supporting single sign-on and user provisioning. Reduce sign-in friction and automate the creation, removal, and maintenance of user accounts.
Async API and Solace: Enabling the Event-Driven FutureSolace
Fran Méndez, founder of AsyncAPI, and Jonathan Schabowsky, senior architect at Solace, explain how the two companies are working together in this presentation from Gartner AADI.
Apache Kafka vs. Integration Middleware (MQ, ETL, ESB) - Friends, Enemies or ...confluent
MQ, ETL and ESB middleware are often used as integration backbone between legacy applications, modern microservices and cloud services. This introduces several challenges and complexities like point-to-point integration or non-scalable architectures. This session discusses how to build a completely event-driven streaming platform leveraging Apache Kafka’s open source messaging, integration and streaming components to leverage distributed processing, fault-tolerance, rolling upgrades and the ability to reprocess events. Learn the differences between a event-driven streaming platform leveraging Apache Kafka and middleware like MQ, ETL and ESBs – including best practices and anti-patterns, but also how these concepts and tools complement each other in an enterprise architecture.
Azure Synapse Analytics is Azure SQL Data Warehouse evolved: a limitless analytics service, that brings together enterprise data warehousing and Big Data analytics into a single service. It gives you the freedom to query data on your terms, using either serverless on-demand or provisioned resources, at scale. Azure Synapse brings these two worlds together with a unified experience to ingest, prepare, manage, and serve data for immediate business intelligence and machine learning needs. This is a huge deck with lots of screenshots so you can see exactly how it works.
The Top 5 Apache Kafka Use Cases and Architectures in 2022Kai Wähner
I see the following topics coming up more regularly in conversations with customers, prospects, and the broader Kafka community across the globe:
Kappa Architecture: Kappa goes mainstream to replace Lambda and Batch pipelines (that does not mean that there is no batch processing anymore). Examples: Kafka-powered Kappa architectures from Uber, Disney, Shopify, and Twitter.
Hyper-personalized Omnichannel: Retail and customer communication across online and offline channels becomes the new black, including context-specific upselling, recommendations, and location-based services. Examples: Omnichannel Retail and Customer 360 in Real-Time with Apache Kafka.
Multi-Cloud Deployments: Business units and IT infrastructures span across regions, continents, and cloud providers. Linking clusters for bi-directional replication of data in real-time becomes crucial for many business models. Examples: Global Kafka deployments.
Edge Analytics: Low latency requirements, cost efficiency, or security requirements enforce the deployment of (some) event streaming use cases at the far edge (i.e., outside a data center), for instance, for predictive maintenance and quality assurance on the shop floor level in smart factories. Examples: Edge analytics with Kafka.
Real-time Cybersecurity: Situational awareness and threat intelligence need to process massive data in real-time to defend against cyberattacks successfully. The many successful ransomware attacks across the globe in 2021 were a warning for most CIOs. Examples: Cybersecurity for situational awareness and threat intelligence in real-time.
An Introduction to Confluent Cloud: Apache Kafka as a Serviceconfluent
Business breakout during Confluent’s streaming event in Munich, presented by Hans Jespersen, VP WW Systems Engineering at Confluent. This three-day hands-on course focused on how to build, manage, and monitor clusters using industry best-practices developed by the world’s foremost Apache Kafka™ experts. The sessions focused on how Kafka and the Confluent Platform work, how their main subsystems interact, and how to set up, manage, monitor, and tune your cluster.
With Azure Arc customers can manage virtual machines, Kubernetes clusters and data services on-premises, at service providers and in other clouds from a single view in Azure using the innovative security, monitoring and policy tools from Azure they appreciate. In this session you will get an introduction to Azure Arc and we will dive deeper into Azure Arc for Kubernetes specifically.
Blogging can be great fun all the same and it's an ideal learning experience for you if you're a budding writer, a knowledge-nik who has a lot to share on a topic or hobby, or you're just keen to encourage people to see things your way! This article is focused on helping you to avoid the common mistakes many learner bloggers make; while you may still stumble into them, at least you'll be more aware and know how to correct your future approaches.
OMA Seminar/Webinar, October 27, 2016, "How Developers Can Get the Most Out of IoT Standards and Tools" - Presentation #8 from Juan Perez, Principal Program Manager, Microsoft
"Developing Services with Microsoft’s Azure Platform"
Hybrid is normal! Be more like AWS! Decompose DevOps tasks!
Short presentation addressing the challenges of building and operating hybrid infrastructure.
Narration! https://www.youtube.com/watch?v=uorHrvMgwc0
Achieving Multi-tenanted Business Processes in SaaS Applications Malinda Kapuruge
With the emergence of Cloud Computing and maturity of Service Oriented Architecture (SOA), the Software-as-a-Service (SaaS) delivery model has gained popularity, due to advantages such as lower startup cost and reduced time to market. A SaaS vendor owns and takes the responsibility of maintaining a single application for multiple clients/tenants who may have similar but also varying requirements. Business process modeling (BPM) approaches can be used to package service offerings to meet these varying requirements on a shared basis. However the customizations in those business processes can be challenging. In this paper we discuss the challenges arising from single-instance multi-tenancy, and present our approach to defining customizable business processes in SaaS applications to address those challenges.
Inevitability of Multi-Tenancy & SAAS in Product EngineeringPrashanth Panduranga
Inevitability of Multi-Tenancy & SAAS in Product Engineering
an in-depth overview of the SAAS requirements and introduction multi-tenancy by Prashanth B Panduranga
OpenStack Preso: DevOps on Hybrid Infrastructurerhirschfeld
Discusses the approach for making hybrid DevOps workable including what obstacles must be overcome. Includes demo of multiple OpenStack clouds & Kubernetes deploy on AWS, Google and OpenStack
Leverage the highly scalable Windows Azure platform and deploy your existing ASP.NET application to a new home in the clouds. This demo filled session will guide you in how to make successful use of Windows Azure’s hosting and storage platform as well as SQL Azure, the relational database in the cloud, by moving an existing ASP.NET application to a higher level.
Leverage Azure and SQL Azure to build SaaS applications Common Sense
• How to make the most of Azure elasticity storage and scalability
on a global SaaS app
• How to use storage Caching to render web pages, efficiently
Intended for:CIOs, CTOs, IT Managers, IT Developers, Lead Developers
In questa sessione verranno analizzate e discusse le problematiche legate alla pubblicazione dei dati da devices in un tipico scenario IoT. Vedremmo come il servizio Event Hub di Microsoft Azure gestisce l'inserimento per pubblicazione e sottoscrizione offrendo una scalabilità flessibile, adattabile a profili di carico variabile e ai picchi provocati dalla connettività intermittente.
Dopo una breve introduzione dei concetti di base legati all'Internet of Things, durante questa sessione si fornirà una panoramica degli strumenti che Microsoft mette a diposizione degli sviluppatori per creare le proprie soluzioni IoT: Windows 10 for IoT e alcuni servizi di Azure quali Event Hubs e Stream Analytics. Si utilizzerà un semplice esempio di telemetria per mostrare la realizzazione pratica di uno scenario end-to-end per la trasformazione dei dati provenienti da un sensore in informazioni utili per effettuare analisi e/o prendere decisioni.
A guide through the Azure Messaging services - Update ConferenceEldert Grootenboer
https://www.updateconference.net/en/2019/session/a-guide-through-the-azure-messaging-services
A guide through the Azure Messaging services - Update Conference
IoT Suite is an enterprise grade solution that allows you to get started quickly through a set of extensible pre-configured solutions for common IoT scenarios
Real-Time Event & Stream Processing on MS AzureKhalid Salama
These slides discuss the main concepts of event & stream processing, as well as the related technologies on Microsoft Azure. We start by giving and overview of what Event & Stream Processing is. Then we describe the canonical architecture of a Stream Processing solution. We will delve into Message Queuing part of the solution. After that, we Introduce Apache Storm on HDInsight, as well as Azure Stream Analytics. We compare Apache Storm to Azure Stream Analytics, and finally conclude with useful resources
IoT Architectures for Apache Kafka and Event Streaming - Industry 4.0, Digita...Kai Wähner
The Internet of Things (IoT) is getting more and more traction as valuable use cases come to light. Whether you are in Healthcare, Telecommunications, Manufacturing, Banking or Retail to name a few industries, there is one key challenge and that's the integration of backend IoT data logs and applications, business services and cloud services to process the data in real time and at scale.
In this talk, we will be sharing how Kafka has become the leading technology used throughout the business to provide Real Time Event Streaming. Explore real life use cases of Kafka Connect, Kafka Streams and KSQL independent of the data deployment be it on a private or public Cloud, On Premise or at the Edge.
Audi - Connected car infrastructure
Robert Bosch Power Tools - Track and Trace of devices and people at construction areas
Deutsche Bahn - Customer 360 for train timetable updates
E.ON - IoT Streaming Platform to integrate and build smart home, smart building and smart grid infrastructures
Azure IoT Client SDK can be used to connect many different types of devices. At the lowest end, you can use it to connect a less than 3$ WiFi capable system on a chip microcontroller, such as the NodeMcu and WeMOS D1.
In this lecture we will see how to build a WiFI capable, Arduino based, cloud controlled IoT smart switch. We will then use a cross platform Xamarin based application to activate the smart switch. Using this application installed on a mobile phone we will open a car gate on stage as well as 7500 Miles (12,000 KM) away.
You will learn:
• An Internet of Thing system overview
• How to create and use the Azure IoT Hub
• Implementing an Azure IoT client SDK based solution
• Provisioning IoT devices, sending information to the cloud and receiving commands
• Arduino development using Visual Studio with Visual Micro
Apache Kafka as Event Streaming Platform for Microservice ArchitecturesKai Wähner
This session introduces Apache Kafka, an event-driven open source streaming platform. Apache Kafka goes far beyond scalable, high volume messaging. In addition, you can leverage Kafka Connect for integration and the Kafka Streams API for building lightweight stream processing microservices in autonomous teams. The Confluent Platform adds further components such as a Schema Registry, REST Proxy, KSQL, Clients for different programming languages and Connectors for different technologies.
The session discusses how tech giants like LinkedIn, Ebay or Airbnb leverage Apache Kafka as event streaming platform to solve various different business problems and how to create a scalable, flexible microservice architecture. A live demo shows how you can easily process and analyze streams of events using Apache Kafka and KSQL.
apidays LIVE Jakarta - Building an Event-Driven Architecture by Harin Honesty...apidays
apidays LIVE Jakarta 2021 - Accelerating Digitisation
February 24, 2021
Building an Event-Driven Architecture
Harin Honestyandi Parandika, Microservice and Middleware Designer at XL Axiata
Google Cloud infrastructure in Conrad Connect by Google & waylayVeselin Pizurica
Conrad Connect lets users interconnect smart devices from different ecosystems with online services. It provides customized dashboards to visualise data from different vendors. It also allows users to build advanced automation rules or to control devices and services using voice and smart bots.
Conrad Connect application is built on top of the waylay platform and it is managed and deployed in the Google cloud.
With close to 100K connected devices, 20 million API calls a day and few billion metrics per week stored, many challenges need to be addressed: How to constantly scale up the platform with exponential growth of the users? How to manage deployments, new releases and upgrades?
In this talk you will learn more how waylay leverages some of the latest Google technologies to address these challenges
Most data visualisation solutions today still work on data sources which are stored persistently in a data store, using the so called “data at rest” paradigms. More and more data sources today provide a constant stream of data, from IoT devices to Social Media streams. These data stream publish with high velocity and messages often have to be processed as quick as possible. For the processing and analytics on the data, so called stream processing solutions are available. But these only provide minimal or no visualisation capabilities. One was is to first persist the data into a data store and then use a traditional data visualisation solution to present the data.
If latency is not an issue, such a solution might be good enough. An other question is which data store solution is necessary to keep up with the high load on write and read. If it is not an RDBMS but an NoSQL database, then not all traditional visualisation tools might already integrate with the specific data store. An other option is to use a Streaming Visualisation solution. They are specially built for streaming data and often do not support batch data. A much better solution would be to have one tool capable of handling both, batch and streaming data. This talk presents different architecture blueprints for integrating data visualisation into a fast data solution and highlights some of the products available to implement these blueprints.
Azure event hubs, Stream Analytics & Power BI (by Sam Vanhoutte)Codit
In this presentation Sam gives an overview of how the various Azure IoT Services are used to ingest data (Event Hubs), process and analyze data (Stream Analytics) and visualize data (PowerBI).
Async Debugging A Practical Guide to survive !Mirco Vanini
The speech talk about the specialised tools inside visual studio to survive from async code bugs with special look about how to write a right async code
Async Debugging - A Practical Guide to survive !Mirco Vanini
The speech talk about the specialised tools inside visual studio to survive from async code bugs with special look about how to write a right async code
An introduction about how to migration from .NET Full FW to .NET Core 3.0 for your WPF & Windows Forms applications and the incrementally modernizing with Windows 10 features
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Let's dive deeper into the world of ODC! Ricardo Alves (OutSystems) will join us to tell all about the new Data Fabric. After that, Sezen de Bruijn (OutSystems) will get into the details on how to best design a sturdy architecture within ODC.
"Impact of front-end architecture on development cost", Viktor TurskyiFwdays
I have heard many times that architecture is not important for the front-end. Also, many times I have seen how developers implement features on the front-end just following the standard rules for a framework and think that this is enough to successfully launch the project, and then the project fails. How to prevent this and what approach to choose? I have launched dozens of complex projects and during the talk we will analyze which approaches have worked for me and which have not.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
Search and Society: Reimagining Information Access for Radical FuturesBhaskar Mitra
The field of Information retrieval (IR) is currently undergoing a transformative shift, at least partly due to the emerging applications of generative AI to information access. In this talk, we will deliberate on the sociotechnical implications of generative AI for information access. We will argue that there is both a critical necessity and an exciting opportunity for the IR community to re-center our research agendas on societal needs while dismantling the artificial separation between the work on fairness, accountability, transparency, and ethics in IR and the rest of IR research. Instead of adopting a reactionary strategy of trying to mitigate potential social harms from emerging technologies, the community should aim to proactively set the research agenda for the kinds of systems we should build inspired by diverse explicitly stated sociotechnical imaginaries. The sociotechnical imaginaries that underpin the design and development of information access technologies needs to be explicitly articulated, and we need to develop theories of change in context of these diverse perspectives. Our guiding future imaginaries must be informed by other academic fields, such as democratic theory and critical theory, and should be co-developed with social science scholars, legal scholars, civil rights and social justice activists, and artists, among others.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
3. IoT day 2015
Microsoft Azure IoT Services
Azure Service Bus
Azure Event Hubs
Azure Stream Analitycs
Demo
Recap
Agenda
4. IoT day 2015
Microsoft Azure IoT services
Producers Connect Devices Storage Analytics Take Action
Event Hubs SQL Database
Machine
Learning
Azure Websites
Service Bus
Table/Blob
Storage
Stream
Analytics
Power BI
External Data
Sources
DocumentDB HDInsight
Notification
Hubs
External Data
Sources
Data Factory Mobile Services
BizTalk Services
{ }
Microsoft Azure IoT Services
5. IoT day 2015
Azure Service Bus
Relay
Queue
Topic
Notification Hub
Event Hub
NAT and Firewall Traversal Service
Request/Response Services
Unbuffered with TCP Throttling
Transactional Cloud AMQP/HTTP Broker
High-Scale, High-Reliability Messaging
Sessions, Scheduled Delivery, etc.
Transactional Message Distribution
Up to 2000 subscriptions per Topic
Up to 2K/100K filter rules per subscription
High-scale notification distribution
Most mobile push notification services
Millions of notification targets
…
7. IoT day 2015
Telemetry ... the problem
Information flowing from a device to other systems for
conveying status of device and environment
Unidirectional
Data frequency can be different based on
applications/conditions
8. IoT day 2015
Message are durably stored but with TTL
Receive & Delete or Peek Lock
Sessions (for FIFO feature)
Request/Reply pattern (based on correlation)
Transaction for batch send/receive
Dead-letter queue (TTL or “poisoned” messages)
Service Bus - Queus & Topics
9. IoT day 2015
Introducing Azure Event Hubs
Event
Producers
> 1M Producers
> 1GB/sec
Aggregate
Throughput
Direct
PartitionKey
Hash
Receivers
AMQP 1.0
Credit-based flow control
Client-side cursors
Offset by Id or Timestamp
Throughput Units:
• 1 ≤ TUs ≤ Partition Count
• TU: 1 MB/s writes, 2 MB/s
reads
10. IoT day 2015
Event Hubs - Overview
Event Hubs is a highly scalable ingestion system that can process
millions of events per second, enabling your application to
process and analyze the massive amounts of data produced by
your connected devices and applications.
The Azure Event Hubs capability differs from Service Bus topics
in that it is strongly biased towards high throughput and event
processing scenarios.
11. IoT day 2015
Event Hubs - Overview
An Event Hub is created at the namespace level in Service
Bus, similar to queues and topics. Event Hubs uses AMQP
and HTTP as its primary API interfaces.
12. IoT day 2015
Event Hubs - Partitions
Event Hubs provides message streaming through a partitioned
consumer pattern.
A partition is an ordered sequence of events that is held in an
Event Hub. As newer events arrive, they are added to the end of
this sequence. A partition can be thought of as a “commit log.”
The number of partitions is specified at the Event Hub creation
time and must be between 8 and 32.
Segmentation of event stream for scale-out.
13. IoT day 2015
In the Event Hubs context, messages are referred to as event data.
Event data contains the body of the event, a user defined property
bag, and various metadata about the event such as its offset in the
partition and its number in the stream sequence. Partitions are filled
with a sequence of event data (max 256kb).
You can publish an event via AMQP 1.0 or HTTP. Service Bus provides
an EventHubClient class that you can use for publishing events to an
Event Hub from .NET clients. For other runtimes and platforms, you
can use any AMQP 1.0 client such as Apache Qpid.
Event Hubs - Event Data
14. IoT day 2015
Advanced Message Queuing Protocol (AMQP) is an efficient,
reliable, wire-level messaging protocol (OASIS standard) that
you can use to build robust, cross-platform, messaging
applications. The protocol has a simple goal: to define the
mechanics of the secure, reliable, and efficient transfer of
messages between two parties.
Event Hubs - What is AMQP 1.0?
15. IoT day 2015
The choice to use AMQP or HTTP is specific to the usage
scenario. AMQP requires the establishment of a persistent
bidirectional socket in addition to transport level security (TLS)
or SSL/TLS. This can be a costly operation in terms of network
traffic, but only happens at the beginning of an AMQP session.
HTTP has a lower initial overhead, but requires additional SSL
overhead for every request. For publishers who frequently
publish events, AMQP offers significant performance, latency,
and throughput savings
Event Hubs - AMQP or HTTP ?
16. IoT day 2015
A partition key is a value that is used to map incoming event data into
specific partitions for the purposes of data organization. The partition key
is a sender-supplied value passed into an Event Hub. It is processed
through a static hashing function, the result of which creates the partition
assignment.
Event Hubs ensures that any and all events sharing the same partition key
value are delivered in order, and to the same partition.
Event Hubs - Partition key
17. IoT day 2015
The publish/subscribe mechanism of Event Hubs is enabled
through consumer groups. A consumer group is a view (state,
position, or offset) of an entire Event Hub. Consumer groups
enable multiple consuming applications to each have a separate
view of the event stream, and to read the stream independently
at its own pace and with its own offsets.
Event Hubs – Consumer Groups
18. IoT day 2015
An offset is the position of an event within a partition. You can
think of an offset as a client-side cursor. The offset is a byte
numbering of the event. This enables an event consumer
(reader) to specify a point in the event stream from which they
want to begin reading events. You can specify the offset as a
timestamp or as an offset value.
Event Hubs - Stream Offset
19. IoT day 2015
In order to consume events from an Event Hub, a consumer
must connect to a partition. Only a single reader should be
active on a partition at any one time within a consumer group.
Each event data instance contains important metadata such as
the offset and sequence number that are used to facilitate
checkpointing on the event sequence.
Event Hubs - Reading Events
20. IoT day 2015
The throughput capacity of Event Hubs is controlled by throughput units. Throughput units
are pre-purchased units of capacity. A single throughput unit includes the following:
Write: Up to 1MB per second or 1000 events per second.
Read: Up to 2MB per second.
Included retention: 84GByte/day (24h at full ingress rate)
Sending data above this amount results in a quota exceeded exception !
Throughput units are billed per hour and are purchased ahead of time
Up to 20 throughput units can be purchased for a Service Bus namespace
Event Hubs - Throughput Units
21. IoT day 2015
Patterns
Q&T : useful for Command Message and Request/Replay Message (response queue)
EH : useful for Event Messages
Cursor
Q&T : on server side. Message consumed and deleted from queue, cursor to next available
message
EH : on client side. Client can rewind on the stream and re-read same events (during their
retention). Access partition by offset or timestamp
Retention
Q&T : TTL at queue/topic level or message level
EH : max 7 days
Event Hubs vs Queues & Topics
22. IoT day 2015
Security & Authentication
Q&T and EH
• SSL/TLS via HTTP(S) or AMQP(S)
• SAS (Secure Access Signature) for sending/receiving
EH
• Publisher policy (SAS Token)
• Fine grained per device
• Revoke/Restore publisher
Other
EH doesn't have dead lettering, transaction, ... to have higher throughput
Event Hubs vs Queues & Topics
23. IoT day 2015
Real-time business insights
Stream Analytics is an event processing engine that helps uncover real-time insights from
devices, sensors, infrastructure, applications and data.
Stream millions of events per second
Stream Analytics provides out-of-the-box integration with Event Hubs to ingest millions of
events per second. Together, Event Hubs and Stream Analytics let you process massive
amounts of real-time data so you can make business decisions in real-time.
Real-time processing
Stream Analytics will process ingested events in real-time, comparing multiple real-time
streams or comparing real-time streams together with historical values and models
Stream Analitycs - Overview
24. IoT day 2015
Event Hubs - Connect the Things
Intel Galileo
Intel® Quark™ X1000
Win32 / C++
Qpid Proton - Apache
Library
GHI Fez Spider
72 Mhz 32-bit ARM7
.NET MF & Gadgeteer / C#
Amqp.Net Lite
Toradex Colibri T30
NVIDIA® Tegra™ 3
.NET CF / WEC2013 / C#
Amqp.Net Lite
http://qpid.apache.org/proton/
https://amqpnetlite.codeplex.com/
26. IoT day 2015
Event Hubs - pricing
BASIC Standard
Ingress events €0.0209 per million events €0.0209 per million events
Throughput unit (1 MB/s ingress,
2MB/s egress)
€0.0112/hr (~€9/mo) €0.0224/hr (~€17/mo)
Publisher policies √
Consumer groups 1 - Default 20
Maximum throughput units 20 20
Service Bus brokered connections 100 included 1000 included
Additional Service Bus brokered
connections
√
Message Retention 1 day included 1 day included
Additional storage (up to 7 days) √
http://azure.microsoft.com/en-us/pricing/details/event-hubs/
27. IoT day 2015
Stream millions of events per second into multiple applications
Enable applications to process events with variable load profiles
Connect millions of devices across platforms
Available NOW !
Recap