Intelligent Cloud Conference 2018 - Automatically scaling Kubernetes pods based on Azure metrics

•

1 like•412 views

This document discusses automatically scaling Kubernetes pods based on Azure metrics. It introduces Promitor, an open-source tool that scrapes Azure Monitor metrics and provides a scraping endpoint for Prometheus. Promitor makes it easy to declare which Azure metrics to scrape via YAML and APIs. It supports automatically scaling pods using the Horizontal Pod Autoscaler and custom Azure metrics from services like Azure Service Bus queues.

Hi!
Tom Kerkhove
• Azure Consultant at Codit
• Microsoft Azure MVP & Advisor
• Belgian Azure User Group (AZUG)
blog.tomkerkhove.be
@TomKerkhove
tomkerkhove

Orders API
Pod
Orders Processor
Pod
Azure Service Bus

Autoscaling pods in Kubernetes
• Horizontal Pod Autoscaler (HPA)
• Allows you to automatically scale out/in pods in a deployment
• Built-in metrics such as CPU
• Support for custom metrics is coming
• GitHub issue #177
• Requires a metrics aggregator adapter (list)
• Prometheus
• Google StackDriver

What is Prometheus?
• Open-source monitoring & alerting
• Part of Cloud Native Computing Foundation since 2016

Orders API
Pod
Orders Processor
Pod
Azure Service Bus Azure Monitor
Horizontal Pod Autoscaler
Prometheus

Introducing Promitor Alpha
• Automatically scrapes Azure Monitor metrics
• Provides scraping endpoint for Prometheus
• Easy to declare metrics to scrape via YAML & APIs
• Easily deployable via Docker & Kubernetes
• Built-in support for a variety of Azure services
• Currently supports Azure Service Bus Queues, more on the way
• Open-Source on GitHub: https://github.com/tomkerkhove/promitor
• Documentation on promitor.io

Orders API
Pod
Orders Processor
Pod
Azure Service Bus Azure Monitor
Horizontal Pod Autoscaler
Prometheus Promitor

Getting data between systems, particularly at scale, is a common challenge faced by data engineers. Pipelines need to be reliable, flexible, and scalable, and without requiring us to write the same boilerplate code each time. Kafka Connect is a framework that provides scalable & fault-tolerant integration between Apache Kafka and other systems. It can be deployed on containers making it easy to scale for increased capacity, throughput, and resilience. We will give a short intro to Kafka Connect and container technologies before proceeding to a deep dive into practical applications. Attendees will learn about: * Real-world Kafka Connect pipelines. * How to build custom connector container images * Configuration, and orchestration of Kafka Connect pipelines with Kubernetes using GitOps.

Serverless Code Deployments in AWS

Marko Tomic

GAB 2017 - Logic Apps and Azure Functions

Wagner Silveira

Global Azure Virtual - Application Autoscaling with KEDA

Tom Kerkhove

This document discusses Kubernetes Event-driven Autoscaling (KEDA), which allows applications running on Kubernetes to automatically scale based on external events. KEDA manages workloads to provide autoscaling to zero, registers as a custom metrics adapter, and provides metrics for the Horizontal Pod Autoscaler to use for scaling. It supports a variety of event sources and scalers out of the box. KEDA is cloud-agnostic, vendor-neutral, and easy to install via Helm charts or the Operator Framework. It has over 1,800 stars on GitHub and an active community. The document demonstrates how KEDA works and discusses its roadmap and integration with technologies like Knative and Azure Functions.

CICD in the World of Serverless

Srushith Repakula

In the ever fast paced software development, Serverless came in as a boon, enabling developers concentrate primarily on the business logic and nothing more! This only calls for a stringent process flow, making Continuous Development, Countinous Integration and Continuous Delivery highly advisable. This talk will focus on local testing of the serverless functions and their associated services locally.

My slides from SharePoint Saturday Oslo

Rick Van Rousselt

Azure Functions Real World Examples

Yochay Kiriaty

The city of Langford has a homegrown timesheet entry system. This system has been integrated to SharePoint 2010 for submission and approval. This presentation will show how the submission (via code) and approval (code – less) are implemented. Kevin Dube has been working with SharePoint since 2008 for the City of Langford wearing many hats (support, administration & development). Kevin’s blog site is: http://www.greenroomsolutions.ca/blog/

Power Friday - Introduction to Azure Integration services

Wagner Silveira

This document introduces Azure Integration Services, which is Microsoft's integration platform as a service (iPaaS). It discusses why integration is important, the key components of Azure Integration Services including API Management, Logic Apps, Service Bus, and Event Grid. It provides an overview of how these components work together to build an integration platform and enable workflows and messaging. The document concludes by suggesting ways to learn more about Azure Integration Services through Microsoft documentation, whitepapers, and blogs.

Tracing Java Applications on Azure

CodeOps Technologies LLP

CREATING REAL TIME DASHBOARD WITH BLAZOR, AZURE FUNCTION COSMOS DB AN AZURE S...

CodeOps Technologies LLP

Using Terraform for AWS as the IaC tool

Lay How

Microsoft Ignite 2019 - API management for microservices in a hybrid and mult...

Tom Kerkhove

Microservices are on the cusp of becoming the dominant style of software architecture in the enterprise. The benefits that are realized—increased developer velocity, improved organizational agility, and reduced time-to-market of new services—are a powerful catalyst that is driving this transformation. As practitioners, how do we successfully fit microservices into the models and processes we already have in place? Join Tom Kerkhove, an Azure Architect with many years of experience helping enterprises make this exact transition, for a hands-on experience demonstrating how he helps enterprises make the transition to API-first architectures and microservices in a hybrid, multi-cloud world.

IglooConf 2020 - API management for microservices in a hybrid and multi-cloud...

Tom Kerkhove

This document summarizes the journey of Codito, a company that started with a monolithic application and migrated to a microservices architecture managed through API management. It describes how Codito initially used Azure API Management to expose a single API for their monolith. They then split the monolith into logical microservices APIs while maintaining the same public API surface. As traffic grew, they fully migrated internal services to microservices behind the API gateway. Finally, they implemented Azure Arc-enabled API Management to run APIs on-premises to meet a customer need. The document emphasizes that building platforms is an iterative journey and companies should start with a monolith and evolve incrementally to microservices as needs dictate.

App Services - Connecting the dots of Web Mobile and Integration_published

Wagner Silveira

This document discusses Azure App Services, which provides an end-to-end platform for building web, mobile, and API applications. It includes Web Apps for hosting web applications, API Apps for creating and consuming APIs, Mobile Apps for building mobile apps, and Logic Apps for automating workflows and integrating apps and services. These services simplify development, deployment, security, availability, and integration across platforms and devices. The document provides an overview and demonstrations of each service.

European SharePoint Conference - TH3

Rick Van Rousselt

This document discusses using containers and the Azure Container Service to extend Office Add-ins. It describes how containers provide a lightweight platform to simplify building, shipping, and running apps. Containers use a shipping container system for code, allowing apps to run everywhere without conflicts. The Azure Container Service is optimized for hosting containers at large scale and makes it easy to manage containers. It includes Docker swarm or DC/OS for container orchestration and is open source.

Fission Introduction

Ta Ching Chen

Sysctr Track: Integrating the components of the System Center suite… The next...

ITProceed

by Mike Resseler So you have a few or all of the components of the System Center suite installed… How about bringing them all together and integrating them? How about making sure that your investment in this suite pays off and you get some automation work done out of the box? In this session you will see (with lots of demo’s) how you can easily integrate the components, get 80% of your work automated and much more…

Iot meets Serverless

Narendran R

Speed and agility are the most expected in today’s analytics tools. The quicker you get from idea to insights, the more you can innovate & perform ad-hoc data analysis. I will be talking about how we can use AWS serverless architecture to stream IoT data, managed by python. We can be up and running in minutes―starting small, but able to easily grow to millions of devices and billions of messages.

My session slides from unityConnect 2016 in Haarlem

Rick Van Rousselt

Serverless Logging Architecture

Narendran R

This document describes building a serverless log analytics platform. It discusses the challenges with conventional logging architectures that require managing servers and have scalability issues. The document then introduces a serverless approach using AWS services like Kinesis, S3, Elasticsearch, and Kibana that allows logging infrastructure to scale infinitely with no server management. Code examples show how to set up logging pipelines to stream logs in real time to storage and analytics using this serverless architecture.

Session Slides from DEVintersection Europe

Rick Van Rousselt

Introduction to Azure Functions - Tutorial

BizTalk360

CREATE RELIABLE AND LOW-CODE APPLICATION IN SERVERLESS MANNER

CodeOps Technologies LLP

In this session, we will discuss a use case where we need to quickly develop web and mobile front end applications which are using several different frameworks, hosting options, and complex integrations between systems under the hood. Let’s see how we can leverage serverless technologies (Azure Functions and logic apps) and Low Code/No code platform to achieve the goal. During the session we will go though the code followed by a demonstration.

Azure Functions for Python developers

Hansamali Gamage

Using Azure Functions for Integration

BizTalk360

This presentation is from the Integration Day event, a TechMeet360 Community Initiative, held on September 10, 2016 at Microsoft GSTC in Bangalore. In this slide, Microsoft's Escalation Engineers Tulika Chaudharie and Harikharan Krishnaraju explain using Azure Functions for Integration. The presentation starts with a general overview of Azure Functions and then it moves on to some of the common Integration Patterns and how Azure Functions fit into the scenarios.

AWS Community Day Bangkok 2019 - Dev Ops Philosophy Increase Productivity

AWS User Group - Thailand

This document discusses the DevOps philosophy and how it can increase producibility. It defines DevOps as combining cultural philosophies, practices, and tools to increase an organization's ability to deliver applications and services at high velocity. Key aspects of DevOps include breaking down silos between development, QA, security and operations teams; continuous integration and delivery pipelines; automation; and real-time feedback to enable rapid, reliable, and secure delivery of updates. Many DevOps tools are available as managed services on AWS, including CodeCommit, CodeBuild, CodeDeploy, CodePipeline, CloudFormation, and CodeStar, which can help implement DevOps practices.

Azure Functions 101

Martin Abbott

Azure Functions allow developers to write code that runs in response to events, enabling event-driven architectures. Functions can be triggered by common data sources and services and support multiple programming languages. Functions provide automatic scaling and only run code when triggered, avoiding the need to manage servers. They integrate with other Azure services and can be developed, tested, and deployed using common tools like Visual Studio.

Microsoft Partners - Application Autoscaling Made Easy With Kubernetes Event-...

Tom Kerkhove

What's hot

Timesheet Approval Process

Regroove

Power Friday - Introduction to Azure Integration services

Wagner Silveira

Tracing Java Applications on Azure

CodeOps Technologies LLP

CREATING REAL TIME DASHBOARD WITH BLAZOR, AZURE FUNCTION COSMOS DB AN AZURE S...

CodeOps Technologies LLP

Using Terraform for AWS as the IaC tool

Lay How

Microsoft Ignite 2019 - API management for microservices in a hybrid and mult...

Tom Kerkhove

IglooConf 2020 - API management for microservices in a hybrid and multi-cloud...

Tom Kerkhove

App Services - Connecting the dots of Web Mobile and Integration_published

Wagner Silveira

European SharePoint Conference - TH3

Rick Van Rousselt

Fission Introduction

Ta Ching Chen

Sysctr Track: Integrating the components of the System Center suite… The next...

ITProceed

Iot meets Serverless

Narendran R

My session slides from unityConnect 2016 in Haarlem

Rick Van Rousselt

Serverless Logging Architecture

Narendran R

Session Slides from DEVintersection Europe

Rick Van Rousselt

Introduction to Azure Functions - Tutorial

BizTalk360

CREATE RELIABLE AND LOW-CODE APPLICATION IN SERVERLESS MANNER

CodeOps Technologies LLP

Azure Functions for Python developers

Hansamali Gamage

Using Azure Functions for Integration

BizTalk360

AWS Community Day Bangkok 2019 - Dev Ops Philosophy Increase Productivity

AWS User Group - Thailand

What's hot (20)

Timesheet Approval Process

Power Friday - Introduction to Azure Integration services

Tracing Java Applications on Azure

CREATING REAL TIME DASHBOARD WITH BLAZOR, AZURE FUNCTION COSMOS DB AN AZURE S...

Using Terraform for AWS as the IaC tool

Microsoft Ignite 2019 - API management for microservices in a hybrid and mult...

IglooConf 2020 - API management for microservices in a hybrid and multi-cloud...

App Services - Connecting the dots of Web Mobile and Integration_published

European SharePoint Conference - TH3

Fission Introduction

Sysctr Track: Integrating the components of the System Center suite… The next...

Iot meets Serverless

My session slides from unityConnect 2016 in Haarlem

Serverless Logging Architecture

Session Slides from DEVintersection Europe

Introduction to Azure Functions - Tutorial

CREATE RELIABLE AND LOW-CODE APPLICATION IN SERVERLESS MANNER

Azure Functions for Python developers

Using Azure Functions for Integration

AWS Community Day Bangkok 2019 - Dev Ops Philosophy Increase Productivity

Similar to Intelligent Cloud Conference 2018 - Automatically scaling Kubernetes pods based on Azure metrics

Azure Functions 101

Martin Abbott

Microsoft Partners - Application Autoscaling Made Easy With Kubernetes Event-...

Tom Kerkhove

NDC London 2021 - Application Autoscaling Made Easy With Kubernetes Event-Dri...

Tom Kerkhove

Kubernetes Event-driven Autoscaling (KEDA) provides application autoscaling on Kubernetes using a variety of metric sources. It automatically scales deployments, jobs, and other resources. KEDA supports over 30 built-in scalers for sources like Azure, AWS, Google Cloud, and more. It is cloud-agnostic and focuses on scaling applications without managing the scaling internals. The Azure Functions CLI makes it easy to deploy functions to Kubernetes and automatically configure KEDA for autoscaling. KEDA is an open source project with over 2,800 stars on GitHub and contributions from Microsoft, Red Hat, and other companies.

Zure Azure PaaS Zero to Hero - DevOps training day

Okko Oulasvirta

This document provides an overview of Azure DevOps and related Azure services for continuous integration, delivery, and monitoring. It discusses DevOps practices including source control with Azure Repos, work tracking with Azure Boards, continuous integration and deployment pipelines with Azure Pipelines, infrastructure as code with ARM templates, and application monitoring with Application Insights. It also covers security practices like role-based access control and use of Azure Key Vault for secrets management. Live demos are provided for many of the Azure DevOps features and services discussed.

Tokyo Azure Meetup #7 - Introduction to Serverless Architectures with Azure F...

Tokyo Azure Meetup

Serverless architecture is the next big shift in computing - completely abstracting the underlying infrastructure and focusing 100% on the business logic. Today we can create applications directly in our browser and leave the decision how they are hosted and scaled to the cloud provider. Moreover, this approach give us incredible control over the granularity of our applications since most of the time we are dealing with single function at a time. In this presentation we will cover: • Introduce Serverless Architectures • Talk about the advantages of Serverless Architectures • Discuss in details in event-driven computing • Cover common Serverless approaches • See practical applications with Azure Functions • Compare AWS Lambda and Azure Functions • Talk about open source alternatives • Explore the relation between Microservices and Serverless Architectures

Introduction to Promitor

Tom Kerkhove

Promitor is an open source tool that scrapes metrics from Azure Monitor and other sources and makes them available to systems like Prometheus and Graphite. It works by declaring which metrics to collect from which Azure resources. It can also automatically discover resources using criteria like resource tags. The scraper agent queries Azure Monitor and resource discovery to collect metrics from both static and dynamic resources. Promitor supports scraping many Azure services and has over 140 stars on GitHub with growing adoption and downloads. Future plans include new authentication options and adding more Azure service scrapers.

Open service broker API with Azure Kubernetes Services

Jorge Arteiro

Azure DevOps Overview [Arabic]

ahmadezzeir

شرح عن منتج Azure DevOps و اهم مكوناته و كيفيه الحصول على الشهادة Azure DevOps Expert Azure DevOps link: https://dev.azure.com Exam AZ-103: Microsoft Azure Administrator https://docs.microsoft.com/en-us/learn/certifications/exams/az-103 Exam AZ-203: Developing Solutions for Microsoft Azure https://docs.microsoft.com/en-us/learn/certifications/exams/az-203 Exam AZ-400: Microsoft Azure DevOps Solutions https://docs.microsoft.com/en-us/learn/certifications/exams/az-400 مصادر تعليم الخاصه بالشهادات https://cloudsociety.fastlane.live/courses My Twitter account https://twitter.com/ahmad_ezzeir My Facebook page: https://www.facebook.com/Arabic-DevOps-Ahmad-Ezzeir-100543094861932

D2 - Automate Custom Solutions Deployment on Office 365 and Azure - Paolo Pia...

SPS Paris

Modern solutions are based on complex architectures, which are made of multiple layers and services. Often, to run your solutions, you need to host services on Azure, configure services in Office 365, and provision content in SharePoint Online. Deploying manually such kind of solutions in target environments can be really challenging, time consuming, and error-prone. In this session, you will learn how to automate the provisioning of such kind of solutions using the Microsoft Graph, the Azure Management API, and the REST API more in general. During the session you will see a real case study, based on the open source PnP Partner Pack Setup Wizard.

Tokyo azure meetup #8 azure update, august

Tokyo Azure Meetup

- Azure updates include new features for machine learning, operations management, cognitive services, virtual machines, SQL, data warehouse, mobile apps, Active Directory, security, and streaming. - Key updates include improved web services management, OMS security capabilities, new cognitive services APIs, faster GPU virtual machines, increased SQL and data warehouse performance and scale, and single sign-on across apps with Active Directory. - Updates aim to provide more analytics, security, and automation capabilities across the Azure platform.

Tokyo azure meetup #8 - Azure Update, August

Kanio Dimitrov

AKS Azure Kubernetes Services - Azure Nights melbourne feb 2018

Jorge Arteiro

This document outlines a presentation on Azure Kubernetes Services (AKS) and the steps to set up a development environment and deploy applications to AKS. It includes enabling Windows features for containers and the Windows Subsystem for Linux (WSL), installing client tools like Docker, Helm, Azure CLI and Visual Studio Code, creating an AKS cluster with Azure CLI commands, and deploying applications from source code to Kubernetes using Helm. It also discusses integrating AKS with other Azure services and includes demo use cases and relevant links.

How to build a serverless helmet detection system

Mandar Dharmadhikari

This document discusses building a serverless helmet detection system using various Azure services. It provides an overview of Custom Vision Service, Computer Vision Service, Logic Apps, Azure Functions, and Event Grid. It then discusses the system architecture, demonstrates the code, and provides best practices and lessons learned. These include tips for training models in Custom Vision Service and structuring logging in Azure Functions. The document encourages connecting on LinkedIn and GitHub for further discussion.

PuppetConf 2017: Unlocking Azure with Puppet Enterprise- Keiran Sweet, Source...

Puppet

For the last year Sourced has been assisting a large Canadian based financial organization migrate workloads to Microsoft's Azure public cloud platform. As part of this deployment, Puppet is leveraged to ensure high levels of automation and compliance across the environment. In this updated session we will walk through our approach to integrating Puppet in Azure environments to ensure that automation, security, compliance and infrastructure as code is at the forefront.

SPS calgary 2017 introduction to azure functions microsoft flow

Vincent Biret

Azure full

Mohit Chhabra

The document provides an overview of containerization with Microsoft Azure. It defines containers as lightweight alternatives to virtual machines that package an application and its dependencies in a standardized unit. Docker is described as the leading containerization platform, with containers wrapping software in a complete filesystem. Key aspects of Docker architecture and common Docker CLI commands are outlined. Azure Container Service is introduced as providing a robust hosting environment for containers using orchestration tools like Kubernetes, DC/OS and Docker Swarm. Container orchestration facilitates deployment and management at scale through container clusters managed by master nodes.

Serverless Summit 22 - micro-frontends aws archiecture

lagrotteri

Welcome to Azure Devops

Alessandro Scardova

DevOps is an approach that combines people, processes, and products to enable continuous delivery of value to end users. It brings together development and operations teams, automates the software delivery process, and provides continuous software updates and monitoring. Key technologies for DevOps include continuous integration (CI), continuous deployment (CD), and continuous monitoring. Azure DevOps provides tools like Azure Pipelines, Azure Boards, Azure Repos, and Azure Test Plans to support DevOps practices and workflows in the cloud.

Azure DevOps for QA Automation

Evgeny Popovich

To run automated tests, the task is often to create an isolated environment for various platforms, but building such environments for testing is far from the easiest task. We also want to have a monitoring and control system for autotests and test code, analyze the stability of tests and the quality of the tested software - the task becomes even more difficult. Microsoft's Azure DevOps Cloud Services helped me with these challenges, and for free. Let's get acquainted with service services (Azure Pipelines, Azure Test Plans, Azure Boards, Azure Repos, Azure Artifacts), which will help us complement workflows, improve planning accuracy, quality, productivity and speed of delivery of our Open Source solutions both for the user and for our own development ... Let's see how easy it is to set up CI / CD pipelines and flexibly parameterize configurations, manage packages using Azure DevOps on MacOS, Unix, Windows, perform analysis and get results.

Connector API Apps

BizTalk360

Similar to Intelligent Cloud Conference 2018 - Automatically scaling Kubernetes pods based on Azure metrics (20)

Azure Functions 101

Microsoft Partners - Application Autoscaling Made Easy With Kubernetes Event-...

NDC London 2021 - Application Autoscaling Made Easy With Kubernetes Event-Dri...

Zure Azure PaaS Zero to Hero - DevOps training day

Tokyo Azure Meetup #7 - Introduction to Serverless Architectures with Azure F...

Introduction to Promitor

Open service broker API with Azure Kubernetes Services

Azure DevOps Overview [Arabic]

D2 - Automate Custom Solutions Deployment on Office 365 and Azure - Paolo Pia...

Tokyo azure meetup #8 azure update, august

Tokyo azure meetup #8 - Azure Update, August

AKS Azure Kubernetes Services - Azure Nights melbourne feb 2018

How to build a serverless helmet detection system

PuppetConf 2017: Unlocking Azure with Puppet Enterprise- Keiran Sweet, Source...

SPS calgary 2017 introduction to azure functions microsoft flow

Azure full

Serverless Summit 22 - micro-frontends aws archiecture

Welcome to Azure Devops

Azure DevOps for QA Automation

Connector API Apps

More from Tom Kerkhove

Techorama 2022 - Adventures of building Promitor, an open-source product

Tom Kerkhove

Azure Lowlands 2020 - API management for microservices in a hybrid and multi-...

Tom Kerkhove

This document summarizes Codito's journey from a monolithic application to a microservices architecture managed by Azure API Management. It discusses how Codito initially used API Management for a single monolithic backend. It then migrated to use API Management to route to logical microservices for Products, Orders, and Shipments. Finally, Codito explored running API Management's self-hosted gateway on-premises to support customers unable to use cloud services. The document emphasizes starting with platform-as-a-service and evolving architectures over time based on needs rather than prematurely adopting microservices.

Building Bruges 2020 - Adventures of building a multi-tenant PaaS on Microsof...

Tom Kerkhove

The document discusses various challenges and strategies for building a multi-tenant Platform as a Service (PaaS) on Microsoft Azure. It covers topics like scaling applications, choosing a tenancy model, efficient data sharding strategies, monitoring, and consuming webhooks. The key strategies discussed are designing applications to scale horizontally, using elastic SQL pools for cost-efficient sharding, enriching telemetry with context, and routing webhooks through an API gateway for security and decoupling.

AZUG Lightning Talk - Application autoscaling on Kubernetes with Kubernetes E...

Tom Kerkhove

IglooConf 2020 - Adventures of building a multi-tenant PaaS on Microsoft Azure

Tom Kerkhove

Tom Kerkhove discusses his experiences building a multi-tenant Platform as a Service (PaaS) on Microsoft Azure. He covers key considerations for scaling applications, choosing a tenancy model, efficient data sharding strategies, monitoring, and consuming webhooks. The document provides advice on designing applications for scale, using autoscaling features, and creating awareness of autoscaling behavior to avoid issues.

Integrate UK 2019 - Adventures of building a (multi-tenant) PaaS on Microsoft...

Tom Kerkhove

- The document discusses various approaches to scaling applications on Microsoft Azure, including scaling up/out, choosing the right compute infrastructure, and designing applications for scale using serverless, PaaS, and container-based approaches. It provides tips on creating awareness around autoscaling, avoiding infinite loops, and using tools like Azure Monitor Autoscale. - Multi-tenancy is discussed in the context of choices around deployment isolation, data sharding strategies, using shard managers and elastic SQL pools to optimize costs, and determining tenants through authentication. - Monitoring best practices include enriching telemetry with correlations, writing root cause analyses, and handling alerts through centralized processes like Logic Apps. The document also covers consuming and providing webhooks at scale.

Techdays Finland 2019 - Adventures of building a (multi-tenant) PaaS on Micro...

Tom Kerkhove

Building a multi-tenant PaaS is not a walk in the part, certainly if the platform you are building on is constantly changing. In this session I'll walk you through the adventure we've been on where you'll learn about the challenges we've had and how we approached them and whether or not our decisions worked out or not. – How to design for scale – How to operate the platform – How to grow a platform mindset and force ownership – How to run tests for your whole platform – How to design for multi-tenancy – How to approach constant change – etc Cloud projects are never finished so you'd better come prepared.

Azure Low Lands 2019 - Building secure cloud applications with Azure Key Vault

Tom Kerkhove

Next Generation Data Integration with Azure Data Factory

Tom Kerkhove

Azure Data Factory is a managed data integration service that allows users to create data pipelines to move and transform data. It provides triggers to initiate pipelines, activities to perform tasks like data movement and transformation, and integration runtimes to execute pipelines across cloud and on-premises environments. The presentation demonstrated how to use Azure serverless services like Data Factory and Logic Apps to build a pipeline for fulfilling GDPR data requests.

Intelligent Cloud Conference 2018 - Building secure cloud applications with A...

Tom Kerkhove

Intelligent Cloud Conference 2018 - Next Generation of Data Integration with ...

Tom Kerkhove

Azure Data Factory is a hybrid data integration service in Azure that allows you to create, manage & operate data pipelines in Azure. It is a serverless orchestrator that allows you to create data pipelines to either move, transform, load data; a fully managed Extract, Transform, Load (ETL) & Extract, Load, Transform (ELT) service if you will. In this talk I'll cover the basics of Azure Data Factory and show you how you can create, manage & operate data pipelines.

NDC Minnesota - Analyzing StackExchange data with Azure Data Lake

Tom Kerkhove

This document discusses analyzing StackExchange data with Azure Data Lake. It introduces Azure Data Lake Store as an enterprise-grade data lake and Azure Data Lake Analytics as a serverless big data analytics service. It demonstrates using these services to acquire, aggregate, analyze and visualize StackExchange data. Key features of Data Lake Store and Analytics are explained, including security, pricing, and extensibility options.

Techdays Finland 2018 - Building secure cloud applications with Azure Key Vault

Tom Kerkhove

NDC Sydney - Analyzing StackExchange with Azure Data Lake

Tom Kerkhove

Big data is the new big thing where storing the data is the easy part. Gaining insights in your pile of data is something different. Based on a data dump of the well-known StackExchange websites, we will store & analyse 150+ GB of data with Azure Data Lake Store & Analytics to gain some insights about their users. After that we will use Power BI to give an at a glance overview of our learnings. If you are a developer that is interested in big data, this is your time to shine! We will use our existing SQL & C# skills to analyse everything without having to worry about running clusters.

TechDays NL 2016 - Building your scalable secure IoT Solution on Azure

Tom Kerkhove

The Internet-of-Things was one of the big hypes in 2015 but it’s more than that – Customers want to build out their own infrastructures and act on their data. Today we’ll look at how Microsoft Azure helps us to build scalable solutions to process events from thousands of devices in a secure manner and the challenges it has. Once the data is in the cloud we’ll also take a look at ways we can learn from our measurements.

Integration Monday - Analysing StackExchange data with Azure Data Lake

Tom Kerkhove

ITProceed 2015 - Securing Sensitive Data with Azure Key Vault

Tom Kerkhove

Security has become more and more important as we move to the cloud and countries & companies are being hacked – remember the Sony hack? But how do we securely store sensitive data such as connection strings to our databases? Where do we store our encryption keys? Can I share them with my customers? How do I prevent abuse of my secrets and block them from doing so? That’s what this session is all about – I will introduce you to the concepts of Microsoft Azure Key Vault where you can use this as it allows you to securely store keys, credentials and other secrets in the cloud. We will also have a look at how it enables us to store encryption keys for SQL Server TDE and how it can help you safeguard your cloud solutions even more.

Securing sensitive data with Azure Key Vault

Tom Kerkhove

As a developer you often have to use & store a lot of sensitive data going from service credentials to connection strings or even encryption keys. But how do I store these in a secure way? How do I know who has access to them and how do I prevent people from copying them and abusing them? On the other hand, SaaS customers have no clue how you store their sensitive data and how they use it. How can they monitor that? How can they revoke your access easily? Watch the recording here - http://azug.be/2015-05-05---securing-sensitive-data-with-azure-key-vault

GWAB - Hop on the Service Bus!

Tom Kerkhove

GWAB - Kinecting the Cloud.

Tom Kerkhove

The document provides an agenda for a workshop on using Kinect for Windows. It includes an introduction to the speaker, an overview of Kinect hardware features like its camera, depth sensor, and microphone array. It also summarizes the skeletal tracking, speech recognition, and interaction capabilities. Example scenarios are given like home security and virtual try-on mirrors in stores. The document concludes by discussing how Kinect applications can be developed for and scaled in the cloud.

More from Tom Kerkhove (20)