The site was slow. CPU and memory usage everywhere! Some dead objects in the corner. Something terrible must have happened! We have some IIS logs. Some traces from a witness. But not enough to find out what was wrong. In this session, we’ll see how effective telemetry, a profiler or two as well as a refresher of how IIS runs our ASP.NET web applications can help solve this server murder mystery.
It’s all about REST in today’s world. Every service you look at, people are providing a REST API for it. Surprisingly, very few of these API’s are even REST, if we’re talking about REST as in a system that abides by a series of constraints making it RESTful. But the positive note is that it has brought to light the power of HTTP as an application protocol, something very often undervalued and misunderstood, even today.
This talk is going to focus on the core values of HTTP for what it is. It will cover all aspects including
Headers
Error Management
Verbs
Content Negotiation
Caching
Security
Messages
Request and Responses
In essence you'll understand HTTP much better, learn to differentiate an HTTP API from a system with RESTful constraints and also understand benefits of providing REST API - if such a thing were to exist of course!
Building your first android app using xamarin (Gill Cleeren)Visug
Do you have a great knowledge about C#? Maybe you’ve already built a great mobile app on Windows Phone or Windows 8 or even Windows 10? Would you like to be able to use your knowledge to build an Android app without having to resort to other languages or IDEs? This is possible using Xamarin! In this session, Gill Cleeren will take you through the creation of your first Android application: Joe’s Coffee Store. Along the way, you will learn the basic concepts of Android such as Activities and see how we can use these from Xamarin.Android. We’ll need to connect with services to get access to data and we’ll learn how to navigate in the app. You’ll see we can do this all of this directly from Visual Studio. We will finish the session by looking at how we can deploy the application to the Google Play Store so it’s ready for our users! You’ll walk away wanting to port your code to Android!
10 years ago I presented for the first time at Visug on the topic of Visual Studio Team System, the first iteration of a product family that allowed us to automate the long road from requirement to software in production, and everything in between. At the time software development was mainly a manual process. The software itself was either monolithic or composed of large 'SOA Services' and it was a real challenge to get them into production every few months in a so called 'big bang’ deployment. Since then our profession has gone through some major changes, software development looks a lot different now. Today, many applications consist of small parts called 'microservices'. These microservices make their way into the cloud or datacenter automatically, through API driven Continuous Deployment systems, every time anyone on the team commits a small change. While deployments are now happening continuously, they do have an impact on the system: taking down part of it for maintenance all the time. But at the same time, our customers expect the overall system to stay up 24/7. In this talk I will introduce you to an upcoming technology, called Service Fabric, that can help you maintain your development agility in this new world, but still live up to your customer’s expectations.
After ten years, not only Visug has changed the world while staying young and relevant. Let’s revisit our old friend the TFS/VSTS build service and see how it works nowadays, why it has become what it is today and how it can be used and customized to your heart’s content!
Windows 10 is finally here. Now is the time to learn about the awesome new features that the platform offers. Do you want to know what Windows 10 enables for developers? Or do you want to see how you can enhance your existing Windows 8.1 app? In this session, we will give you a full overview of the most important new features of Windows 10.
It is almost there, the RTM version of ASP.NET 5 and MVC6, with a new project system in Visual Studio 2015 and a new cross-platform runtime! In this session we will dig into each one of those. We will look at how you can deploy your website without it being affected, or affecting other runtimes. We will look at how to build your own middleware for this new runtime (and maybe package it as a NuGet package), building POCO controllers for MVC6, how to do unit testing, … At the end of this session you should have a better insight in this exciting new framework and runtime!
Introduction to Azure Functions.
An event-based serverless compute experience to accelerate your development. Scale based on demand and pay only for the resources you consume.
It’s all about REST in today’s world. Every service you look at, people are providing a REST API for it. Surprisingly, very few of these API’s are even REST, if we’re talking about REST as in a system that abides by a series of constraints making it RESTful. But the positive note is that it has brought to light the power of HTTP as an application protocol, something very often undervalued and misunderstood, even today.
This talk is going to focus on the core values of HTTP for what it is. It will cover all aspects including
Headers
Error Management
Verbs
Content Negotiation
Caching
Security
Messages
Request and Responses
In essence you'll understand HTTP much better, learn to differentiate an HTTP API from a system with RESTful constraints and also understand benefits of providing REST API - if such a thing were to exist of course!
Building your first android app using xamarin (Gill Cleeren)Visug
Do you have a great knowledge about C#? Maybe you’ve already built a great mobile app on Windows Phone or Windows 8 or even Windows 10? Would you like to be able to use your knowledge to build an Android app without having to resort to other languages or IDEs? This is possible using Xamarin! In this session, Gill Cleeren will take you through the creation of your first Android application: Joe’s Coffee Store. Along the way, you will learn the basic concepts of Android such as Activities and see how we can use these from Xamarin.Android. We’ll need to connect with services to get access to data and we’ll learn how to navigate in the app. You’ll see we can do this all of this directly from Visual Studio. We will finish the session by looking at how we can deploy the application to the Google Play Store so it’s ready for our users! You’ll walk away wanting to port your code to Android!
10 years ago I presented for the first time at Visug on the topic of Visual Studio Team System, the first iteration of a product family that allowed us to automate the long road from requirement to software in production, and everything in between. At the time software development was mainly a manual process. The software itself was either monolithic or composed of large 'SOA Services' and it was a real challenge to get them into production every few months in a so called 'big bang’ deployment. Since then our profession has gone through some major changes, software development looks a lot different now. Today, many applications consist of small parts called 'microservices'. These microservices make their way into the cloud or datacenter automatically, through API driven Continuous Deployment systems, every time anyone on the team commits a small change. While deployments are now happening continuously, they do have an impact on the system: taking down part of it for maintenance all the time. But at the same time, our customers expect the overall system to stay up 24/7. In this talk I will introduce you to an upcoming technology, called Service Fabric, that can help you maintain your development agility in this new world, but still live up to your customer’s expectations.
After ten years, not only Visug has changed the world while staying young and relevant. Let’s revisit our old friend the TFS/VSTS build service and see how it works nowadays, why it has become what it is today and how it can be used and customized to your heart’s content!
Windows 10 is finally here. Now is the time to learn about the awesome new features that the platform offers. Do you want to know what Windows 10 enables for developers? Or do you want to see how you can enhance your existing Windows 8.1 app? In this session, we will give you a full overview of the most important new features of Windows 10.
It is almost there, the RTM version of ASP.NET 5 and MVC6, with a new project system in Visual Studio 2015 and a new cross-platform runtime! In this session we will dig into each one of those. We will look at how you can deploy your website without it being affected, or affecting other runtimes. We will look at how to build your own middleware for this new runtime (and maybe package it as a NuGet package), building POCO controllers for MVC6, how to do unit testing, … At the end of this session you should have a better insight in this exciting new framework and runtime!
Introduction to Azure Functions.
An event-based serverless compute experience to accelerate your development. Scale based on demand and pay only for the resources you consume.
Best Practices for couchDB developers on Microsoft AzureBrian Benz
This presentation covers best practices for collecting, storing, analyzing and distributing data across a scalable data layer on Windows Azure using CouchDB, JSON, and MapReduce. Highlights include best practices for Windows Azure security, performance, accessibility and reliability.
Rik Hepworth - ARM Yourself for Effective Azure ProvisioningWinOps Conf
Azure Resource Manager templates are a crucial part of your journey to the cloud. Learn the essentials of template creation and maintenance, with some examples of how to deal with complex deployments and manage the PaaS services that born in the cloud apps need.
Azure Functions creates a “serverless” event-driven experience, meaning that they run based on associated and configure events, or “triggers”. For example, an Azure Function could be triggered by a simple timer, such as running a process in a certain interval or triggered by an event in an external system. Azure Functions can also respond to Azure-specific events, such as an image added to a Storage Blob or a notification arriving in a Message Queue.
Azure cloud for the web frontend developersMaxim Salnikov
It’s impossible to reason about modern web application without mentioning its front-end part – extensive, feature-rich, performant, sometimes with quite sophisticated structure and build process, but after all not requiring server-side code to construct it in the runtime, i.e. static. In the Azure cloud, we now have a special service for this kind of apps (including both classic SPAs and statically generated ones), empowering developers to focus on the business logic rather than the infrastructure that builds and hosts them.
In my technical session & demo, let's go through the full development flow: we scaffold a web front-end app, push it to the GitHub, set up Azure Static Web App service, enjoy the fully automatic build, and deploy to global high-availability hosting - all in a few minutes!
SPEAKER: Alisa Petivotova, Quality Architect @EPAM.
TOPIC DESCRIPTION:
We'll talk about the approach not only in terms of how to build contract tests, but when and why we should use them. This will also be a discussion on how to build testing process with close collaboration between development and testing teams.
Windows Server AppFabric Caching - What it is & when you should use it?Robert MacLean
This is from my Tech-Ed Africa 2010 talk. For more information see: http://www.sadev.co.za/content/teched-africa-2010-slides-scripts-and-demos-my-talks
This session looks at what AppFabric Caching is from start to deep dive.
Windows Azure Web Sites- Things they don’t teach kids in school - Comunity D...Maarten Balliauw
Microsoft has a cloud platform which runs .NET, NodeJS and PHP. All 101 talks out there will show you the same: it’s easy to deploy, it scales out on demand and it runs WordPress. Great! But what about doing real things? In this session, we’ll explore the things they don’t teach kids in school. How about trying to find out the architecture of this platform? What about the different deployment options for Windows Azure Web Sites, the development flow and some awesome things you can do with the command line tools? Did you know you can modify the automated build process? Join me in this exploration of some lesser known techniques of the platform.
Sherlock Homepage - A detective story about running large web services - NDC ...Maarten Balliauw
The site was slow. CPU and memory usage everywhere! Some dead objects in the corner. Something terrible must have happened! We have some IIS logs. Some traces from a witness. But not enough to find out what was wrong. In this session, we’ll see how effective telemetry, a profiler or two as well as a refresher of how IIS runs our ASP.NET web applications can help solve this server murder mystery.
Sherlock Homepage - A detective story about running large web services - WebN...Maarten Balliauw
The site was slow. CPU and memory usage everywhere! Some dead objects in the corner. Something terrible must have happened! We have some IIS logs. Some traces from a witness. But not enough to find out what was wrong. In this session, we’ll see how effective telemetry, a profiler or two as well as a refresher of how IIS runs our ASP.NET web applications can help solve this server murder mystery.
Best Practices for couchDB developers on Microsoft AzureBrian Benz
This presentation covers best practices for collecting, storing, analyzing and distributing data across a scalable data layer on Windows Azure using CouchDB, JSON, and MapReduce. Highlights include best practices for Windows Azure security, performance, accessibility and reliability.
Rik Hepworth - ARM Yourself for Effective Azure ProvisioningWinOps Conf
Azure Resource Manager templates are a crucial part of your journey to the cloud. Learn the essentials of template creation and maintenance, with some examples of how to deal with complex deployments and manage the PaaS services that born in the cloud apps need.
Azure Functions creates a “serverless” event-driven experience, meaning that they run based on associated and configure events, or “triggers”. For example, an Azure Function could be triggered by a simple timer, such as running a process in a certain interval or triggered by an event in an external system. Azure Functions can also respond to Azure-specific events, such as an image added to a Storage Blob or a notification arriving in a Message Queue.
Azure cloud for the web frontend developersMaxim Salnikov
It’s impossible to reason about modern web application without mentioning its front-end part – extensive, feature-rich, performant, sometimes with quite sophisticated structure and build process, but after all not requiring server-side code to construct it in the runtime, i.e. static. In the Azure cloud, we now have a special service for this kind of apps (including both classic SPAs and statically generated ones), empowering developers to focus on the business logic rather than the infrastructure that builds and hosts them.
In my technical session & demo, let's go through the full development flow: we scaffold a web front-end app, push it to the GitHub, set up Azure Static Web App service, enjoy the fully automatic build, and deploy to global high-availability hosting - all in a few minutes!
SPEAKER: Alisa Petivotova, Quality Architect @EPAM.
TOPIC DESCRIPTION:
We'll talk about the approach not only in terms of how to build contract tests, but when and why we should use them. This will also be a discussion on how to build testing process with close collaboration between development and testing teams.
Windows Server AppFabric Caching - What it is & when you should use it?Robert MacLean
This is from my Tech-Ed Africa 2010 talk. For more information see: http://www.sadev.co.za/content/teched-africa-2010-slides-scripts-and-demos-my-talks
This session looks at what AppFabric Caching is from start to deep dive.
Windows Azure Web Sites- Things they don’t teach kids in school - Comunity D...Maarten Balliauw
Microsoft has a cloud platform which runs .NET, NodeJS and PHP. All 101 talks out there will show you the same: it’s easy to deploy, it scales out on demand and it runs WordPress. Great! But what about doing real things? In this session, we’ll explore the things they don’t teach kids in school. How about trying to find out the architecture of this platform? What about the different deployment options for Windows Azure Web Sites, the development flow and some awesome things you can do with the command line tools? Did you know you can modify the automated build process? Join me in this exploration of some lesser known techniques of the platform.
Sherlock Homepage - A detective story about running large web services - NDC ...Maarten Balliauw
The site was slow. CPU and memory usage everywhere! Some dead objects in the corner. Something terrible must have happened! We have some IIS logs. Some traces from a witness. But not enough to find out what was wrong. In this session, we’ll see how effective telemetry, a profiler or two as well as a refresher of how IIS runs our ASP.NET web applications can help solve this server murder mystery.
Sherlock Homepage - A detective story about running large web services - WebN...Maarten Balliauw
The site was slow. CPU and memory usage everywhere! Some dead objects in the corner. Something terrible must have happened! We have some IIS logs. Some traces from a witness. But not enough to find out what was wrong. In this session, we’ll see how effective telemetry, a profiler or two as well as a refresher of how IIS runs our ASP.NET web applications can help solve this server murder mystery.
Building Continuous Application with Structured Streaming and Real-Time Data ...Databricks
One of the biggest challenges in data science is to build a continuous data application which delivers results rapidly and reliably. Spark Streaming offers a powerful solution for real-time data processing. However, the challenge remains in how to connect them with various continuous and real-time data sources, guaranteeing the responsiveness and reliability of data applications.
In this talk, Nan and Arijit will summarize their experiences learned from serving the real-time Spark-based data analytic solutions on Azure HDInsight. Their solution seamlessly integrates Spark and Azure EventHubs which is a hyper-scale telemetry ingestion service enabling users to ingress massive amounts of telemetry into the cloud and read the data from multiple applications using publish-subscribe semantics.
They’ll will cover three topics: bridging the gap of data communication model in Spark and data source, accommodating Spark to rate control and message addressing of data source, and the co-design of fault tolerance Mechanisms. This talk will share the insights on how to build continuous data applications with Spark and boost more availabilities of connectors for Spark and different real-time data sources.
How and why we evolved a legacy Java web application to Scala... and we are s...Katia Aresti
Applications get old, and technology moves fast. Overtime, adding or modifying functionalities might become as expensive as re-coding everything all from scratch. But rewriting a complete website and its functionalities it’s hard if we want to minimize the risks of breaking existing functionalities and specially when this application fits in a ecosystem and interacts with other pieces of software and teams.
In this session, you will learn how we moved from a legacy java monolithic website using scala PlayFramework, AngularJS, Elasticsearch and MongoDB, how we built a multi service and REST oriented architecture, which were the technical and human problems we encountered and how we managed to solved them.
(WEB301) Operational Web Log Analysis | AWS re:Invent 2014Amazon Web Services
Log data contains some of the most valuable raw information you can gather and analyze about your infrastructure and applications. Amid the mess of confusing lines of seemingly random text can be hints about performance, security, flaws in code, user access patterns, and other operational data. Without the proper tools, finding insights in these logs can be like searching for a hay-colored needle in a haystack. In this session you learn what practices and patterns you can easily implement that can help you better understand your log files. You see how you can customize web logs to add more information to them, how to digest logs from around your infrastructure, and how to analyze your log files in near real time.
What is going on - Application diagnostics on Azure - TechDays FinlandMaarten Balliauw
We all like building and deploying cloud applications. But what happens once that’s done? How do we know if our application behaves like we expect it to behave? Of course, logging! But how do we get that data off of our machines? How do we sift through a bunch of seemingly meaningless diagnostics? In this session, we’ll look at how we can keep track of our Azure application using structured logging, AppInsights and AppInsights analytics to make all that data more meaningful.
No, this session is not about greener IT. Learn about using the RoleEnvironment and diagnostics provided by Windows Azure. Communication between roles, logging and automatic upscaling of your application are just some of the possibilities of what you can do if you know about how the Windows Azure environment works.
Breaking Parser Logic: Take Your Path Normalization Off and Pop 0days Out!Priyanka Aash
"We propose a new exploit technique that brings a whole-new attack surface to defeat path normalization, which is complicated in implementation due to many implicit properties and edge cases. This complication, being under-estimated or ignored by developers for a long time, has made our proposed attack vector possible, lethal, and general. Therefore, many 0days have been discovered via this approach in popular web frameworks written in trending programming languages, including Python, Ruby, Java, and JavaScript.
Being a very fundamental problem that exists in path normalization logic, sophisticated web frameworks can also suffer. For example, we've found various 0days on Java Spring Framework, Ruby on Rails, Next.js, and Python aiohttp, just to name a few. This general technique can also adapt to multi-layered web architecture, such as using Nginx or Apache as a proxy for Tomcat. In that case, reverse proxy protections can be bypassed. To make things worse, we're able to chain path normalization bugs to bypass authentication and achieve RCE in real world Bug Bounty Programs. Several scenarios will be demonstrated to illustrate how path normalization can be exploited to achieve sensitive information disclosure, SMB-Relay and RCE.
Understanding the basics of this technique, the audience won't be surprised to know that more than 10 vulnerabilities have been found in sophisticated frameworks and multi-layered web architectures aforementioned via this technique."
Making apps for people with a disability with Xamarin tells you about making apps for people with an intellectual disability. Former, people thought people with an intellectual disability cannot read, thus a smartphone isn’t really realistic for them. But nonprofits and startups started to explore this userbase and found out a way of apps people with an intellectual disability can use.
Why is this important? Well there are two big reasons: first of all: If people with an intellectual disability can use your app, they can get info about their life-situation and that empowers quality of life. Smartphones by example uses Text-To-Speech and principal the need to “can read” isn’t there anymore.
For you as developer or startup there’s also a big benefit of apps for these people: nearly every market is discovered for mobile apps. Every market has big concurrency. But the market of apps for people with a disability is quite less discovered, so if you are good, you can rule a hole market!
How many iot technologies do you need to turn on a lightbulb (Kurt Claeys)Visug
IOT is no hype no more, a broad range of technologies are now supporting .NET developers to build great solutions. In this demo-only presentation I’ll show you such an IOT solution distributed over 3 different platforms (Android, Cloud and Raspberry) using 3 different messaging protocols (iBeacons/BLE, Azure Storage REST API and Zwave protocol). The solution shows how a tablet device can be aware of its distance to iBeacons, can send triggers on an Azure queue to be received on a Raspberry PI to remote control a lightbulb through Zwave. Talking cross platform! All with .NET and C#
DDD, CQRS & ES lessons learned (Gitte Vermeiren)Visug
ES, CQRS & DDD: Lessons learned and walls bumped into DDD, CQRS and Event Sourcing are all based on simple principles. Yet, using them on a project brings with it big challenges and new ways of thinking. In this session Gitte will give you the real life story of how DDD, CQRS and ES was used on one of her latest projects and how she bumped into quite some walls while doing it. You will get an overview of the basic buildup of the application and of the points of most friction. Gitte will also try and give some solutions to the problems the project team went through.
DevOps with Visual studio Release Management (Pieter Gheysens)Visug
Are you looking for an efficient way to dev/test your applications in Windows Azure and you want to track your release process by automating your deployments for repeatable success? Are you struggling with a manual and error-prone deployment process which frustrates you every day? Are you looking for automation that is the same across different environments (Dev-Test-Acceptance-Production)? Do you need to build/package your application only once and deploy it with the exact same bits to any provisioned environment? Do you want to setup a formal approval workflow to promote a release to the next stage? If so, come and learn about the new release management features in Visual Studio Team Services.
In this session we will build a mobile application (iOS/Android) using Xamarin Forms.
We will make use of the latest version of XLabs (also known as Xamarin Forms Labs) to build an app that is very maintainable, scalable and extensible.
The built-in MVVM framework allows you to build future proof apps which can be expanded to multiple platforms with minimal effort.
This session covers how to set this up from scratch, apply bindings, use IOC and make use of a simple WebAPI.
Hello windows 10: An overview of the new features for developers in WIndows 10Visug
Windows 10 is finally here. Now is the time to learn about the awesome new features that the platform offers. Do you want to know what Windows 10 enables for developers? Or do you want to see how you can enhance your existing Windows 8.1 app? In this session, we will give you a full overview of the most important new features of Windows 10.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
Generating a custom Ruby SDK for your web service or Rails API using Smithyg2nightmarescribd
Have you ever wanted a Ruby client API to communicate with your web service? Smithy is a protocol-agnostic language for defining services and SDKs. Smithy Ruby is an implementation of Smithy that generates a Ruby SDK using a Smithy model. In this talk, we will explore Smithy and Smithy Ruby to learn how to generate custom feature-rich SDKs that can communicate with any web service, such as a Rails JSON API.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
4. Site unavailable!
Primary website location unavailable!
No problem: traffic manager in front – pfew!
Secondary location unavailable!
Website down…
Initial investigation & monitoring showed:
Primary & secondary website location instances all up
Machines available individually
Not through traffic manager and Azure load balancer
6. How did we find the issue?
Quote from “Mind Hunter” (written by an FBI profiler):
You have to be able to re-create the crime scene in your head. You need
to know as much as you can about the victim so that you can imagine
how she might have reacted. You have to be able to put yourself in her
place as the attacker threatens her with a gun or a knife, a rock, his fists,
or whatever.
You have to be able to feel her fear as he approaches her. You have to be
able to feel her pain. You have to try to imagine what she was going
through when he tortured her. You have to understand what it’s like to
scream in terror and agony, realizing that it won’t help, that it won’t get
him to stop. You have to know what it was like.
http://highscalability.com/blog/2015/7/30/how-debugging-is-like-hunting-serial-killers.html
7. How did we find the issue?
Debugging requires a particular sympathy for the machine. You must be able to run
the machine and networks of machines in your mind while simulating what-ifs based
on mere wisps of insight.
In reality: it was a hunch
Knowing the system you are working on – even by similarity
Empathy for what is going on in that system
Prior experience / insights
9. Who am I?
Maarten Balliauw
Antwerp, Belgium
Software Engineer, Microsoft
Founder, MyGet
AZUG
Focus on web
ASP.NET MVC, Azure, SignalR, ...
Former MVP Azure & ASPInsider
Big passion: Azure
http://blog.maartenballiauw.be
@maartenballiauw
Shameless self promotion: Pro NuGet - http://amzn.to/pronuget2
11. A bit of history
Site serves dependencies for .NET developers worldwide
On average good for ~8.000.000 to ~10.000.000 request per day
(333.333/hr or 5.555/min or 92/sec)
Built 4 years ago on top of a SQL database and OData services
Monolithic – site + OData service are the same app
Minor improvements over the years
Some rough times Q2 2015
12. Architecture overview (Q2 2015)
Front end servers (2 regions)
MVC + WCF OData
Search Service
Lucene based Web API
Search Service
(secondary region)
Lucene based Web API
Azure Storage
Lucene index
Download counts
Azure SQL Database
Metadata
Download counts
Jobs VMs
Create index from database
Create stats
Create download count reports
14. One of the services caused this…
SQL database?
Storage?
Search?
Metrics service?
return new HttpStatusCodeWithBodyResult(AvailabilityStatusCode(galleryServiceAvailable),
String.Format(CultureInfo.InvariantCulture,
StatusMessageFormat,
AvailabilityMessage(galleryServiceAvailable),
AvailabilityMessage(sqlAzureAvailable),
AvailabilityMessage(storageAvailable),
AvailabilityMessage(searchServiceAvailable),
AvailabilityMessage(metricsServiceAvailable),
HostMachine.Name));
15. Log spelunking
Check SQL database logs – we found we had none (fixed now)
Storage – storage statistics seemed stable
Search – no real pointers to issues there
Metrics service – very lightweight and has been stable for months
Start looking around at the crime scene!
IIS logs, event viewer on web servers
Profiling on web servers
19. Turns out it was search!
Profiling the search service revealed some things!
SearcherManager.cs checks Lucene index freshness on Get() – MaybeReopen()
StartReopen() blocks access to the index until finished
Part of HTTP request pipeline – blocking request handling
This was fixed by getting these calls out of the HTTP request path.
The suspect no longer had code available – used our informants
www.jetbrains.com/dotpeek
20. Search had some other flaws…
Actually also visible in the dotTrace snapshot we just saw: high GC!
Memory profiling the search service revealed some things! (I lost the actual traces )
21. Search had some other flaws…
High memory traffic on reading download count # for search ranking
The source: DownloadLookup.cs#L18
Fixed by:
Reusing objects (instead of new)
JsonStreamReader instead of JObject.Parse(theWorld)
22. In the meanwhile…
Added additional monitoring
Added additional tracing
Started looking into using AppInsights for better insights into application behavior
Events happening on the website
Requests
Exceptions
Execution and dependency times (basic but continuous profiling)
24. What we were seeing…
On the V2-based feeds:
500 Internal Server Error
during package restore
Response time goes up
while # of requests goes down
EventVwr on servers: lots of IIS crashes
Lots of crash dumps on web servers
25. So IIS crashes… Could it be?
HTTP.SYS tells us when things go wrong
D:WindowsSystem32LogFilesHTTPERR
2015-07-31 01:46:34 - 60810 - 80 HTTP/1.1 GET /api/v2/FindPackagesById()?id='...' - 1273337584
Connection_Abandoned_By_ReqQueue 86cd3cb1-729c-425c-898f-b15b0330bc38
Connection_Abandoned_By_ReqQueue
“A worker process from the application pool has quit unexpectedly or orphaned a pending
request by closing its handle. Specific to Windows Vista and Windows Server 2008.”
Gift from the gods: crash dumps!
C:ResourcesDirectory31edcaa5186f…...DiagnosticStoreWAD0104CrashDumps
27. An Exception crashes IIS?
Time to crank up the search engine queries!
Found a similar issue: unobserved task exceptions causing IIS to crash
// If metrics service is specified we post the data to it asynchronously.
if (_config != null && _config.MetricsServiceUri != null)
{
// Disable warning about not awaiting async calls
// because we are _intentionally_ not awaiting this.
#pragma warning disable 4014
Task.Run(() => PostDownloadStatistics(id, version, …));
#pragma warning restore 4014
}
28. TaskScheduler.UnobservedTaskException += (object sender, UnobservedTaskExceptionEventArgs excArgs) =>
{
// ... log it ...
excArgs.SetObserved();
};
Tasks and fire-and-forget are evil!
Unobserved task can cause the entire process to give up on Exception
Handle unobserved task Exceptions!
31. What we were seeing…
High response times on the servers
Resulting in higher than normal CPU usage on the servers
Azure would often auto-scale additional instances
Profiling the web application still showed wait times with no obvious cause…
33. Research
Reading and searching on what could be the cause of these issues
http://stackoverflow.com/questions/12304691/why-are-iis-threads-so-precious-as-compared-to-regular-clr-threads
http://www.monitis.com/blog/2012/06/11/improving-asp-net-performance-part3-threading/
https://msdn.microsoft.com/en-us/library/ms998549.aspx
http://blogs.msdn.com/b/tmarq/archive/2007/07/21/asp-net-thread-usage-on-iis-7-0-and-6-0.aspx
https://support.microsoft.com/en-us/kb/821268
Consider minIoThreads and minWorkerThreads for Burst Load
If your application experiences burst loads where there are prolonged periods of inactivity between the burst
loads, the thread pool may not have enough time to reach the optimal level of threads. A burst load occurs
when a large number of users connect to your application suddenly and at the same time. The
minIoThreads and minWorkerThreads settings enable you to configure a minimum number of worker
threads and I/O threads for load conditions.
35. Making it permanent
Part of the cloud service startup script
# Increase the number of available IIS threads for high performance applications
# Uses the recommended values from http://msdn.microsoft.com/en-us/library/ms998549.aspx#scalenetchapt06_topic8
# Assumes running on two cores (medium instance on Azure)
&$appcmd set config /commit:MACHINE -section:processModel -maxWorkerThreads:100
&$appcmd set config /commit:MACHINE -section:processModel -minWorkerThreads:50
&$appcmd set config /commit:MACHINE -section:processModel -minIoThreads:50
&$appcmd set config /commit:MACHINE -section:processModel -maxIoThreads:100
# Adjust the maximum number of connections per core for all IP addresses
&$appcmd set config /commit:MACHINE -section:connectionManagement /+["address='*',maxconnection='240'"]
37. What we were seeing…
On the V2-based feeds:
Package restore timeouts coming from the WCF OData service
Occurs every 7-15 hours, fixes itself ~15 minutes later
Extreme load times on Get(Id=,Version=) – probably the cause of these timeouts
38. No easy way to reproduce…
Happening only on production
Observation after RDP-ing in: 100% CPU when it happens
No way to profile continuously – AppInsights did show us the entry point
Donut time again
The thing we recently changed was minIOThreads and throughput
The slow code path is FindPackagesById()
Makes HTTP calls to search service
What could this setting and HTTP calls have in common…
39. HttpClient, Async and multithreading
Interesting article benchmarking HttpClient in async and multithreading scenarios
Async + HttpClient are not limited in terms of concurrency
Many CPU’s and threads? Many HttpClients and requests
Many HttpClients and requests? Many TCP ports used on machine
Many TCP ports used? TCP port depletion
But aren’t ports reclaimed?
240 seconds TIME_WAIT (4 minutes)
Users also use up TCP ports
“As far as HTTP requests are concerned, a limit should always be set to
ServicePointManager.DefaultConnectionLimit. The limit should be large enough to allow a
good level of parallelism, but low enough to prevent performance and reliability problems (from the
exhaustion of ephemeral ports). “
40. Limiting HttpClient async concurrency
Set ServicePointManager properties on startup
Nagling – “bundle traffic in properly stuffed TCP packets”
Expect100Continue – “only send out traffic if server says 100 Continue”
Both optimizations also disabled
// Tune ServicePointManager
// (based on http://social.technet.microsoft.com/Forums/en-US/windowsazuredata/thread/d84ba34b-b0e0-
4961-a167-bbe7618beb83 and https://msdn.microsoft.com/en-
us/library/system.net.servicepointmanager.aspx)
ServicePointManager.DefaultConnectionLimit = 500;
ServicePointManager.UseNagleAlgorithm = false;
ServicePointManager.Expect100Continue = false;
43. What we were seeing…
Massive memory usage! Even when changing VM sizes.
100% of memory on a Medium Azure instance
100% of memory on a Large Azure instance
100% of memory on a X-Large Azure instance
44. What is eating this memory?
Memory profiling!
On the server?
Try to reproduce it?
Decided on the latter
46. .NET Memory Management 101
Memory Allocation
.NET runtime reserves region of address space for every new process
managed heap
Objects are allocated in the heap
Allocating memory is fast, it’s just adding a pointer
Some unmanaged memory is also consumed (not GC-ed)
.NET CLR, Dynamic libraries, Graphics buffer, …
Memory Release or “Garbage Collection” (GC)
Generations
Large Object Heap
47. .NET Memory Management 101
Memory Allocation
Memory Release or “Garbage Collection” (GC)
GC releases objects no longer in use by examining application roots
GC builds a graph that contains all the objects that are reachable from these
roots.
Object unreachable? GC removes the object from the heap, releasing memory
After the object is removed, GC compacts reachable objects in memory.
Generations
Large Object Heap
48. .NET Memory Management 101
Memory Allocation
Memory Release or “Garbage Collection” (GC)
Generations
Managed heap divided in segments: generation 0, 1 and 2
New objects go into Gen 0
Gen 0 full? Perform GC and promote all reachable objects to Gen 1. This is typically pretty fast.
Gen 1 full? Perform GC on Gen 1 and Gen 0. Promote all reachable objects to Gen 2.
Gen 2 full? Perform full GC (2, 1, 0). If not enough memory for new allocations, throws
OutOfMemoryException
Full GC has performance impact since all objects in managed heap are verified.
Large Object Heap
49. .NET Memory Management 101
Memory Allocation
Memory Release or “Garbage Collection” (GC)
Generations
Large Object Heap
Generation 0 Generation 1 Generation 2
Short-lived objects (e.g. Local
variables)
In-between objects Long-lived objects (e.g. App’s
main form)
50. .NET Memory Management 101
Memory Allocation
Memory Release or “Garbage Collection” (GC)
Generations
Large Object Heap
Large objects (>85KB) stored in separate segment of managed heap: Large
Object Heap (LOH)
Objects in LOH collected only during full garbage collection
Survived objects in LOH are not compacted (by default). This means that LOH
becomes fragmented over time.
Fragmentation can cause OutOfMemoryException
51. The .NET garbage collector
Simulates “infinite memory” by removing objects no longer needed
When does it run? Vague… But usually:
Out of memory condition – when the system fails to allocate or re-allocate memory
After some significant allocation – if X memory is allocated since previous GC
Failure of allocating some native resources – internal to .NET
Profiler – when triggered from profiler API
Forced – when calling methods on System.GC
Application moves to background
GC is not guaranteed to run
http://blogs.msdn.com/b/oldnewthing/archive/2010/08/09/10047586.aspx
http://blogs.msdn.com/b/abhinaba/archive/2008/04/29/when-does-the-net-compact-framework-garbage-collector-run.aspx
53. So our DI container? NInject?
Our memory profiling confirms it. The retained EntitiesContext also
retains entities and SQL connections.
Spelunking the NInject source code, we found the
GarbageCollectionCachePruner responsible for releasing objects.
Runs every 30 seconds (timer)
Releases objects only if GC happened in that time
GC is not guaranteed to run, so NInject potentially never releases objects
Known, old bug.
https://groups.google.com/forum/#!topic/ninject/PQNMIsQhCvE
http://stackoverflow.com/questions/16775362/ninject-caching-object-that-should-be-disposed-memoryleak
54. Replacing our DI container (Autofac)
Perform replacement
Run same analysis on new codebase
and verify objects are freed
Once deployed:
Immediate drop in response times
Memory usage now stable at ~4 GB
56. Conclusion
Debugging requires a particular sympathy for the machine. You must be
able to run the machine and networks of machines in your mind while
simulating what-ifs based on mere wisps of insight.
Bugs hide. They blend in. They can pass for "normal" which makes them tough to find.
One bug off the streets doesn’t mean all of them are gone. Sometimes one gone exposes
another.
Know your system, know your tools, know your options. Look for evidence.
Profilers (performance and memory), dump files, AppInsights and others
Dive in. It builds experience and makes solving the next crime scene easier.
https://msdn.microsoft.com/en-us/library/ee817663.aspx
One of these services was failing – resulting in status 500 – resulting in LB removing the instance from pool – all were removed from pool…
Serial killers are like bugs in the societal machine. They hide. They blend in. They can pass for "normal" which makes them tough to find. They attack weakness causing untold damage until caught. And they will keep causing damage until caught. They are always hunting for opportunity.
Yes it’s fun to work on new things. But try figuring out this stuff. Builds experience.
Profiling the search service issue - search service issue
* Open capture1.dtt
* Notice lots of lock contention
* Disable system methods - notice SearcherManager.Get() is top suspect
* Unfortunately, this code was deployed without symbols and we no longer had the sources around
* Decided to decompile using dotPeek and walk through code - which revealed something (next slide)
Profiling the search service issue - search service issue
* Open capture1.dtt
* Notice lots of lock contention
* Disable system methods - notice SearcherManager.Get() is top suspect
* Unfortunately, this code was deployed without symbols and we no longer had the sources around
* Decided to decompile using dotPeek and walk through code - which revealed something (next slide)
Perhaps do a quick tour of AppInsights depending on time left.
Analyzing a crash dump
* Open w3wp.exe.3344.dmp - explain this can be done with hardcore debugging tools but VS2015 works just as fine
* Explain dump summary - what we can see (threads, modules, last state when it all broke down)
* Becomes more interesting when we debug it - "Debug with managed only"
* Unhandled Exception, it says! That's probably the cause for IIS erroring out. But where does it come from...
* We can try the various windows for inspecting modules, threads, call stacks, ... but not al ot to see in there.
* We need DEBUGGER SYMBOLS! Unfortunately in this case the gallery was deployed without symbols. No symbols on teh build server either, for this deployment.
* DotPeek! Load the actual DLL, enable symbol server, start debugging again.
* See the exception now shows us the (decompiled) code. Not the real code but it does give us an idea.
* Unhandled exception in a task. Which crashes out IIS.
Reproducing production traffic
* We'll use jMeter - a tool that is perfect for replaying web server logs against other servers
* Explain IIS logs come from production
* Explain rconvlog tool to translate them to NCSA format
* Open up jMeter and explain how it is all linked to each other
Application roots: Typically, these are global and static object pointers, local variables, and CPU registers.
Application roots: Typically, these are global and static object pointers, local variables, and CPU registers.
Application roots: Typically, these are global and static object pointers, local variables, and CPU registers.
Application roots: Typically, these are global and static object pointers, local variables, and CPU registers.
Application roots: Typically, these are global and static object pointers, local variables, and CPU registers.
Analyzing memory usage
* Open the Workspace [2015-08-05] [13-24].dmw workspace
* From the snapshots overview we already see a few interesting things: memory keeps going up, when GC'ed (using the profiler API) memory stays around
* Let's open one of the snapshots - snapshot #6 - why? As it says objects have been collected so we want to see what remained in memory
* From the overview: lots of objects on gen2 - eaning they survived many collections - dive in
* Seems NInject.Activation.Caching.Cache is keeping a lot of bytes in memory - dive in
* Lots of objects seem to be cached by NInjet. Check outgoing references to see what they are.
* The first ones look normal, e.g. the controller activator and so on is needed the entire time by ASP.NET MVC
* At [2] we see an EntitiesContext retained. That's weird: it should be disposed of after each request. Let's see if there are more of these still in memory.
* Snapshot - largest size - search entitiescontext - A LOT! Also in size relative to the snapshot size - dive in
* We can see 991 instances are kept around - who's holding on to them?
* Group by similar retention shows NInject for 596 of them. Wow.