A talk delivered at the Oct 1st Sydney AWS Meetup, on how Domain uses CloudFormation, DSC and Octopus Deploy to support our microservices arcitecture with continuous delivery
Mit Docker ist es einfach geworden, Applikationen lokal zu starten, ohne zusätzliche Abhängigkeiten installieren zu müssen. Einen Cluster auf seinem eigenen Rechner laufen zu lassen ist kein großes Ding mehr. Mit ECS bietet AWS einen Container-Management-Service für die Cloud an, der verspricht, Container entsprechend ihrem Ressourcenbedarf und Verfügbarkeitserfordernissen automatisch im Cluster zu platzieren.
Aber was passiert dazwischen? Und ist es wirklich so einfach?
In diesem Talk werden wir betrachten, welche existierenden Services von AWS verwendet werden können, um Container automatisch zu deployen, und was zusätzlich alles benötigt wird, um sie im Betrieb laufen zu lassen.
This document discusses the benefits of implementing infrastructure, pipelines, images, servers, orchestration, and chat operations as code. Key benefits include being reproducible, saving time, avoiding human error, and not becoming outdated. It outlines tools like Terraform, Jenkins, Packer, Kubernetes, Docker, configuration management, and Hubot. Code samples and demos are provided to define infrastructure with Terraform, transform jobs into pipelines with Jenkins, build images with Packer, define containers in Kubernetes, build servers with Docker, and interact with services through Hubot chat. A link is provided to access additional sample code.
This document provides an overview of clusterless serverless container deployment with AWS Fargate. It introduces concepts like containers, microservices, and container orchestration. It then discusses different AWS container services like ECS and EKS. The main topic is AWS Fargate, which is described as a fully managed container service that eliminates the need to manage clusters or nodes. It explains how Fargate works and its pricing model based on vCPU and memory resources used. A sample Fibonacci application deployed on Fargate is also briefly presented.
With Docker it became easy to start applications locally without installing any dependencies. Even running a local cluster is not a big thing anymore. AWS on the other side offers with ECS a managed container service that states to schedule containers based on resource needs, isolation policies and availability requirements. But what happens between? Is it really that easy? In this talk you’ll see which existing services can already be used to deploy your containers automatically and what still needs to be done to get them running on AWS.
Developers like to run services locally to support the development process. With cloud based services that is a challenge. In this talk I will show you a good option to run a number of core AWS services locally.
See the talk here:
https://youtu.be/CeAK9hoRvhk
ASP.NET vNext represents the future of ASP.NET with a new modular structure based on NuGet packages. It uses OWIN and is optimized for the cloud with lighter frameworks. Key features include dependency injection support, EF7, Identity 3, and tools for frontend automation like Grunt and Bower. The new runtime is powered by packages like KRE, KVM and KPM which allow running apps cross-platform using .NET Core.
From NDC Oslo, 2017.
Linux containers run on Linux. Windows containers run on Windows. You can't mix them on a single host, but you can build a cluster of hosts into a single Docker Swarm, using a mixture of Windows and Linux servers. That swarm can run both Windows and Linux containers, and within the swarm all the containers can talk to each other.
This session will show you how to make that happen, but more importantly you'll see why it's a such an important capability - one that will change the way you design, build and deliver software. With a hybrid Docker Swarm you can build a distributed solution where you pick the right technology stack for each component, and leverage high-quality open-source applications to minimize the amount of custom software you need to write and maintain.
Mit Docker ist es einfach geworden, Applikationen lokal zu starten, ohne zusätzliche Abhängigkeiten installieren zu müssen. Einen Cluster auf seinem eigenen Rechner laufen zu lassen ist kein großes Ding mehr. Mit ECS bietet AWS einen Container-Management-Service für die Cloud an, der verspricht, Container entsprechend ihrem Ressourcenbedarf und Verfügbarkeitserfordernissen automatisch im Cluster zu platzieren.
Aber was passiert dazwischen? Und ist es wirklich so einfach?
In diesem Talk werden wir betrachten, welche existierenden Services von AWS verwendet werden können, um Container automatisch zu deployen, und was zusätzlich alles benötigt wird, um sie im Betrieb laufen zu lassen.
This document discusses the benefits of implementing infrastructure, pipelines, images, servers, orchestration, and chat operations as code. Key benefits include being reproducible, saving time, avoiding human error, and not becoming outdated. It outlines tools like Terraform, Jenkins, Packer, Kubernetes, Docker, configuration management, and Hubot. Code samples and demos are provided to define infrastructure with Terraform, transform jobs into pipelines with Jenkins, build images with Packer, define containers in Kubernetes, build servers with Docker, and interact with services through Hubot chat. A link is provided to access additional sample code.
This document provides an overview of clusterless serverless container deployment with AWS Fargate. It introduces concepts like containers, microservices, and container orchestration. It then discusses different AWS container services like ECS and EKS. The main topic is AWS Fargate, which is described as a fully managed container service that eliminates the need to manage clusters or nodes. It explains how Fargate works and its pricing model based on vCPU and memory resources used. A sample Fibonacci application deployed on Fargate is also briefly presented.
With Docker it became easy to start applications locally without installing any dependencies. Even running a local cluster is not a big thing anymore. AWS on the other side offers with ECS a managed container service that states to schedule containers based on resource needs, isolation policies and availability requirements. But what happens between? Is it really that easy? In this talk you’ll see which existing services can already be used to deploy your containers automatically and what still needs to be done to get them running on AWS.
Developers like to run services locally to support the development process. With cloud based services that is a challenge. In this talk I will show you a good option to run a number of core AWS services locally.
See the talk here:
https://youtu.be/CeAK9hoRvhk
ASP.NET vNext represents the future of ASP.NET with a new modular structure based on NuGet packages. It uses OWIN and is optimized for the cloud with lighter frameworks. Key features include dependency injection support, EF7, Identity 3, and tools for frontend automation like Grunt and Bower. The new runtime is powered by packages like KRE, KVM and KPM which allow running apps cross-platform using .NET Core.
From NDC Oslo, 2017.
Linux containers run on Linux. Windows containers run on Windows. You can't mix them on a single host, but you can build a cluster of hosts into a single Docker Swarm, using a mixture of Windows and Linux servers. That swarm can run both Windows and Linux containers, and within the swarm all the containers can talk to each other.
This session will show you how to make that happen, but more importantly you'll see why it's a such an important capability - one that will change the way you design, build and deliver software. With a hybrid Docker Swarm you can build a distributed solution where you pick the right technology stack for each component, and leverage high-quality open-source applications to minimize the amount of custom software you need to write and maintain.
The document discusses configuration management and how Chef can be used to manage numerous environments, services, and servers across platforms by defining roles, environments, and cookbooks; it also discusses how Chef can be integrated with Jenkins for continuous integration and used to automatically scale application servers in AWS through tools that bridge CloudFormation and Chef.
1) The document discusses introducing Elastic Beanstalk and deploying a Python/Flask application to it. It demonstrates importing an existing Flask project and deploying it on Beanstalk.
2) It also demonstrates importing a Flask project contained in a Docker container and deploying it to Beanstalk.
3) Additionally, it covers some new features of Git 2.0 and tips for using Git. It promotes the book Pro Git and mentions the team working on translations.
Self Created Load Balancer for MTA on AWSsharu1204
This document summarizes the creation of a self-managed load balancer on AWS to distribute mail traffic across multiple mail gateway servers. It describes the existing mail system architecture, the need for a load balancer due to traffic volume limitations, and the technical implementation using Linux Virtual Server (LVS) and keepalived for load balancing and iptables for network address translation (SNAT) to support load balancing of SMTP traffic. The results were an increased ability to scale mail gateway servers elastically and observe traffic patterns from email services like Google Apps. A note of caution is provided about network bandwidth limitations based on the EC2 instance type used for the load balancer.
Container Orchestration using KubernetesHesham Amin
This document provides an overview of Kubernetes concepts including:
- Docker containers are individual processes that make up applications, not complete applications themselves.
- Kubernetes manages container deployments through services that define relationships between containers and provide load balancing, discovery, scaling, updates and rollbacks.
- Kubernetes was created by Google and is supported on platforms like Google Kubernetes Engine, Azure Container Service, and others.
- Kubernetes manages clusters of nodes that host containers through masters nodes that control workers.
- The basic building block is the Pod, which contains one or more containers that share resources and networking.
Deliver Docker Containers Continuously on AWS - QCon 2017Philipp Garbe
With Docker it became easy to start applications locally without installing any dependencies. Even running a local cluster is not a big thing anymore.
AWS on the other side offers with ECS a managed container service that starts to schedule containers based on resource needs, isolation policies, and availability requirements.
Sounds good, but is it really that easy? In this talk, you'll get an overview of ECS and all other services that are needed to run your containers in production. Philipp shows how an ECS cluster and your containerized applications can automatically be deployed and scaled. He also shares his experiences and discusses what features are still missing.
Deploy High Availability Kubernetes with KubesprayChe-Chia Chang
This document discusses how to deploy a Kubernetes cluster locally using Kubespray and Ansible. It begins by introducing Kubespray and Ansible and explaining how they can be used together to deploy Kubernetes. Next, it provides instructions for installing the necessary tools like Vagrant and Ansible and cloning the Kubespray repository. It then walks through running vagrant up to deploy the Kubernetes cluster locally. The document also discusses how VirtualBox and Vagrant are used to automate and manage the virtual machines, and how Ansible deploys Kubernetes components to the machines. It concludes by demonstrating how to test the local Kubernetes cluster using kubectl.
AWS December 2015 Webinar Series - Continuous Delivery to Amazon EC2 Containe...Amazon Web Services
Keeping consistent environments across your development, test, and production systems can be a complex task. Docker containers offer a way to develop and test your application in the same environment in which it runs in production. You can use tools such as Docker Compose for local testing of applications; Jenkins and AWS CodePipeline for building and workflow orchestration; and Amazon EC2 Container Service to manage and scale containers.
In this session, you will learn how to build containers into your continuous deployment workflow and orchestrate container deployments using Amazon ECS.
Learning Objectives:
Learn to use the Amazon ECS CLI to test applications locally
Orchestrate continuous delivery workflows using AWS CodePipeline
Schedule containers on production clusters using Amazon ECS
Who Should Attend:
Developers, DevOps Engineers
[Js hcm] Deploying node.js with Forever.js and nginxNicolas Embleton
You have a project in Node.js and you wonder how to make it run on your server? A real "production" server?
You have some application and you want to ensure that it will run - without downtime, that you can easily update - without downtime, and that you can scale it over multiple webservices as a load Balancing?
We will cover that by using:
- Nginx
- Forever.js
- Node.js
Being able to rapidly iterate on, build, and test your code is key to being a productive developer. Without local automation, working with the numerous platforms and technologies in your stack can become very frustrating. In this webinar, Ben Bromhead CTO of Instaclustr will explore best practices to easily integrate Apache CassandraTM into your development workflow, so you spend more time writing good code and less time fighting your environment.
This document provides an introduction to serverless computing and discusses various serverless technologies and concepts. It summarizes Apache OpenWhisk, a serverless platform, and Knative, which provides serverless primitives for container-based applications. It also outlines some benefits and challenges of serverless, including automatic scalability but also deployment complexity. Example use cases like action chaining and event-driven capabilities are demonstrated.
The document discusses using Golang for serverless applications. It introduces serverless computing and why Golang is a good choice. Benefits of Golang include its simplicity with only requiring a single binary output and no dependencies. The document demonstrates building a serverless REST API with Golang and AWS Lambda, and addresses enhancing Lambda with provisioned concurrency. It concludes with references for further information.
This document discusses scaling WordPress on AWS. It introduces AWS services like EC2, EFS, S3, RDS that can be used to deploy and scale a WordPress site. It explains different scaling approaches like vertical scaling by adding more resources, horizontal scaling by adding more machines, and global scaling across availability zones. An enhanced WordPress architecture using EC2 for the frontend and logic layers, RDS for the database, and S3 for file storage is proposed. Core and additional AWS services for development, operations, security and more that can help scale WordPress are also outlined.
Communication tool & Environment for Remote WorkerShotaro Sakamaki
Shotaro Sakamaki is a front-end engineer at PixelGrid.Inc, a company that develops JavaScript applications. He discusses the communication tools and development environment used by PixelGrid's remote workers. Key tools mentioned include Slack for chat, esa.io for documentation sharing, GitHub for source control, and ZenHub as a GitHub extension. Costs for these paid services range from $3.99 to $6.67 per user per month. While costs may seem high, the speaker argues they replace expenses from maintaining multiple free tools and reduce invisible maintenance costs.
This document summarizes MongoDB deployment on CloudFoundry. It discusses how CloudFoundry is a PaaS that allows for easy provisioning and deprovisioning of MongoDB. It then demonstrates deploying a MongoDB app to CloudFoundry and discusses how MongoDB is packaged to run on CloudFoundry.
Setup Kubernetes Cluster On AWS Using KOPSSivaprakash
This document provides steps to set up a Kubernetes cluster on AWS using KOPS. It describes installing required tools like kubectl and KOPS, creating an SSH key and Route53 host on AWS, generating a Kubernetes cluster across multiple Availability Zones, and exposing sample applications. Finally, it covers deleting the cluster.
This document outlines a multi-host container networking solution called Weaveworks. It describes Weave as an overlay container network that allows containers running on different VMs or hosts to communicate. The document demonstrates Weave by showing a data producer container sending messages to an "HTTP" server container running on different hosts and connected through the Weave network. It also briefly describes Weave Run for IPAM, load balancing and service discovery and Weave Scope for visualizing containers.
WordCamp IL 2016 - WordPress Scale on AWSBoaz Ziniman
One of the most popular CMS, WordPress, was not designed for the cloud era. Running a high demand and highly available WordPress in a cluster can be a complex task.
There are several solutions to this problem and this session will focus on one of them. Using several AWS services (EC2, RDS, S3, EFS, CloudFormation and others) and Zend Server, we will create a complex setup running in minutes and scale it up and down by demand.
This document summarizes an OpenShift meetup discussing using OpenShift on IBM Cloud Bare Metal Server. It provides an overview of the speaker's background and experience with OpenShift, Kubernetes, and IBM Cloud. It then outlines the setup of OpenShift on IBM Cloud Bare Metal Server, including using NFS storage, installing OpenShift nodes via Ansible playbooks, and configuring the infrastructure for the OpenShift master, etcd, router, Docker registry, and nodes.
Robotics in army presentation is just an another term used in this presentation as to lead all the way how robots have became main and constructive parts in army and wars
I gotta dependency on dependency injectionmhenroid
Satisfy your desire to be a better software developer by learning how to implement Dependency Injection (DI) in your application. Dependency Injection is one of the more popular implementations of Inversion of Control (IoC) that helps to increase modularity and extensibility of software. In this lecture, we'll discuss the benefits of DI, methods for implementing and refactoring existing code to use DI, adding unit tests using MSTest and Moq, popular DI frameworks like Unity, Ninject, and Castle Windsor, as well as alternative IoC patterns and anti-patterns.
The document discusses configuration management and how Chef can be used to manage numerous environments, services, and servers across platforms by defining roles, environments, and cookbooks; it also discusses how Chef can be integrated with Jenkins for continuous integration and used to automatically scale application servers in AWS through tools that bridge CloudFormation and Chef.
1) The document discusses introducing Elastic Beanstalk and deploying a Python/Flask application to it. It demonstrates importing an existing Flask project and deploying it on Beanstalk.
2) It also demonstrates importing a Flask project contained in a Docker container and deploying it to Beanstalk.
3) Additionally, it covers some new features of Git 2.0 and tips for using Git. It promotes the book Pro Git and mentions the team working on translations.
Self Created Load Balancer for MTA on AWSsharu1204
This document summarizes the creation of a self-managed load balancer on AWS to distribute mail traffic across multiple mail gateway servers. It describes the existing mail system architecture, the need for a load balancer due to traffic volume limitations, and the technical implementation using Linux Virtual Server (LVS) and keepalived for load balancing and iptables for network address translation (SNAT) to support load balancing of SMTP traffic. The results were an increased ability to scale mail gateway servers elastically and observe traffic patterns from email services like Google Apps. A note of caution is provided about network bandwidth limitations based on the EC2 instance type used for the load balancer.
Container Orchestration using KubernetesHesham Amin
This document provides an overview of Kubernetes concepts including:
- Docker containers are individual processes that make up applications, not complete applications themselves.
- Kubernetes manages container deployments through services that define relationships between containers and provide load balancing, discovery, scaling, updates and rollbacks.
- Kubernetes was created by Google and is supported on platforms like Google Kubernetes Engine, Azure Container Service, and others.
- Kubernetes manages clusters of nodes that host containers through masters nodes that control workers.
- The basic building block is the Pod, which contains one or more containers that share resources and networking.
Deliver Docker Containers Continuously on AWS - QCon 2017Philipp Garbe
With Docker it became easy to start applications locally without installing any dependencies. Even running a local cluster is not a big thing anymore.
AWS on the other side offers with ECS a managed container service that starts to schedule containers based on resource needs, isolation policies, and availability requirements.
Sounds good, but is it really that easy? In this talk, you'll get an overview of ECS and all other services that are needed to run your containers in production. Philipp shows how an ECS cluster and your containerized applications can automatically be deployed and scaled. He also shares his experiences and discusses what features are still missing.
Deploy High Availability Kubernetes with KubesprayChe-Chia Chang
This document discusses how to deploy a Kubernetes cluster locally using Kubespray and Ansible. It begins by introducing Kubespray and Ansible and explaining how they can be used together to deploy Kubernetes. Next, it provides instructions for installing the necessary tools like Vagrant and Ansible and cloning the Kubespray repository. It then walks through running vagrant up to deploy the Kubernetes cluster locally. The document also discusses how VirtualBox and Vagrant are used to automate and manage the virtual machines, and how Ansible deploys Kubernetes components to the machines. It concludes by demonstrating how to test the local Kubernetes cluster using kubectl.
AWS December 2015 Webinar Series - Continuous Delivery to Amazon EC2 Containe...Amazon Web Services
Keeping consistent environments across your development, test, and production systems can be a complex task. Docker containers offer a way to develop and test your application in the same environment in which it runs in production. You can use tools such as Docker Compose for local testing of applications; Jenkins and AWS CodePipeline for building and workflow orchestration; and Amazon EC2 Container Service to manage and scale containers.
In this session, you will learn how to build containers into your continuous deployment workflow and orchestrate container deployments using Amazon ECS.
Learning Objectives:
Learn to use the Amazon ECS CLI to test applications locally
Orchestrate continuous delivery workflows using AWS CodePipeline
Schedule containers on production clusters using Amazon ECS
Who Should Attend:
Developers, DevOps Engineers
[Js hcm] Deploying node.js with Forever.js and nginxNicolas Embleton
You have a project in Node.js and you wonder how to make it run on your server? A real "production" server?
You have some application and you want to ensure that it will run - without downtime, that you can easily update - without downtime, and that you can scale it over multiple webservices as a load Balancing?
We will cover that by using:
- Nginx
- Forever.js
- Node.js
Being able to rapidly iterate on, build, and test your code is key to being a productive developer. Without local automation, working with the numerous platforms and technologies in your stack can become very frustrating. In this webinar, Ben Bromhead CTO of Instaclustr will explore best practices to easily integrate Apache CassandraTM into your development workflow, so you spend more time writing good code and less time fighting your environment.
This document provides an introduction to serverless computing and discusses various serverless technologies and concepts. It summarizes Apache OpenWhisk, a serverless platform, and Knative, which provides serverless primitives for container-based applications. It also outlines some benefits and challenges of serverless, including automatic scalability but also deployment complexity. Example use cases like action chaining and event-driven capabilities are demonstrated.
The document discusses using Golang for serverless applications. It introduces serverless computing and why Golang is a good choice. Benefits of Golang include its simplicity with only requiring a single binary output and no dependencies. The document demonstrates building a serverless REST API with Golang and AWS Lambda, and addresses enhancing Lambda with provisioned concurrency. It concludes with references for further information.
This document discusses scaling WordPress on AWS. It introduces AWS services like EC2, EFS, S3, RDS that can be used to deploy and scale a WordPress site. It explains different scaling approaches like vertical scaling by adding more resources, horizontal scaling by adding more machines, and global scaling across availability zones. An enhanced WordPress architecture using EC2 for the frontend and logic layers, RDS for the database, and S3 for file storage is proposed. Core and additional AWS services for development, operations, security and more that can help scale WordPress are also outlined.
Communication tool & Environment for Remote WorkerShotaro Sakamaki
Shotaro Sakamaki is a front-end engineer at PixelGrid.Inc, a company that develops JavaScript applications. He discusses the communication tools and development environment used by PixelGrid's remote workers. Key tools mentioned include Slack for chat, esa.io for documentation sharing, GitHub for source control, and ZenHub as a GitHub extension. Costs for these paid services range from $3.99 to $6.67 per user per month. While costs may seem high, the speaker argues they replace expenses from maintaining multiple free tools and reduce invisible maintenance costs.
This document summarizes MongoDB deployment on CloudFoundry. It discusses how CloudFoundry is a PaaS that allows for easy provisioning and deprovisioning of MongoDB. It then demonstrates deploying a MongoDB app to CloudFoundry and discusses how MongoDB is packaged to run on CloudFoundry.
Setup Kubernetes Cluster On AWS Using KOPSSivaprakash
This document provides steps to set up a Kubernetes cluster on AWS using KOPS. It describes installing required tools like kubectl and KOPS, creating an SSH key and Route53 host on AWS, generating a Kubernetes cluster across multiple Availability Zones, and exposing sample applications. Finally, it covers deleting the cluster.
This document outlines a multi-host container networking solution called Weaveworks. It describes Weave as an overlay container network that allows containers running on different VMs or hosts to communicate. The document demonstrates Weave by showing a data producer container sending messages to an "HTTP" server container running on different hosts and connected through the Weave network. It also briefly describes Weave Run for IPAM, load balancing and service discovery and Weave Scope for visualizing containers.
WordCamp IL 2016 - WordPress Scale on AWSBoaz Ziniman
One of the most popular CMS, WordPress, was not designed for the cloud era. Running a high demand and highly available WordPress in a cluster can be a complex task.
There are several solutions to this problem and this session will focus on one of them. Using several AWS services (EC2, RDS, S3, EFS, CloudFormation and others) and Zend Server, we will create a complex setup running in minutes and scale it up and down by demand.
This document summarizes an OpenShift meetup discussing using OpenShift on IBM Cloud Bare Metal Server. It provides an overview of the speaker's background and experience with OpenShift, Kubernetes, and IBM Cloud. It then outlines the setup of OpenShift on IBM Cloud Bare Metal Server, including using NFS storage, installing OpenShift nodes via Ansible playbooks, and configuring the infrastructure for the OpenShift master, etcd, router, Docker registry, and nodes.
Robotics in army presentation is just an another term used in this presentation as to lead all the way how robots have became main and constructive parts in army and wars
I gotta dependency on dependency injectionmhenroid
Satisfy your desire to be a better software developer by learning how to implement Dependency Injection (DI) in your application. Dependency Injection is one of the more popular implementations of Inversion of Control (IoC) that helps to increase modularity and extensibility of software. In this lecture, we'll discuss the benefits of DI, methods for implementing and refactoring existing code to use DI, adding unit tests using MSTest and Moq, popular DI frameworks like Unity, Ninject, and Castle Windsor, as well as alternative IoC patterns and anti-patterns.
This document discusses Agile systems administration using tools like Puppet, Rake, Ruby, Subversion, and CruiseControl. It summarizes how these tools can be used together in an iterative and collaborative manner to continuously test and deploy system configurations from a versioned code repository. Key benefits include always deploying tested configurations, versioning infrastructure configurations, visibility of project status, and enabling continuous integration practices for system administration work.
This document discusses deployment automation for .NET applications using RapidDeploy. It outlines the benefits of automation such as increased productivity, scalability, and reduced errors. It also discusses challenges with .NET deployment like migrating from older IIS versions. The demonstration will configure the RapidDeploy plugin for IIS, install IIS 7.5 on two servers, deploy a test website, synchronize configurations between servers, make changes to replicate, and restore from a snapshot.
Implementing Continuous Integration in .NET for Cheapskatesmhenroid
Using Continuous Integration in your project is a great way to significantly reduce integration problems, identify and resolve issues quickly, and develop cohesive software more rapidly. In this session, you'll learn how to implement Continuous Integration in your .NET project using a combination of team practices, free or open source tools, and scripts. In particular, we'll discuss various best practices, environment setup, package management (NuGet), source control (Git), automated build tools (NAnt, MSBuild, PowerShell), build servers (Cruise Control .NET), automated testing tools (MSTest, Moq, Selenium, Specflow), and code coverage (Open Cover).
This document discusses automating the deployment of WebSphere Message Broker environments using RapidDeploy. Some key benefits of automation include reducing effort, improving productivity, accelerating growth by releasing additional scalability, eliminating errors to minimize downtime, and expediting time to value. The demonstration shows RapidDeploy running on Windows deploying a Message Broker binary to a Linux VM, creating and deploying environments, taking snapshots, cloning an environment, deploying changes, and comparing snapshots. Automation helps address challenges of manual deployments, composite deployments, upgrades, and providing a consistent framework.
The document discusses test-driven development (TDD). It describes TDD as writing tests before code in order to think about how to use components before implementing them. This results in components that are easy to test and enhance. The TDD process involves red-green-refactor cycles of writing a failing test, making it pass with minimal code, then refactoring. TDD is done at both the unit test and acceptance test levels during iterative software development. Tools like JUnit and IDEs are recommended to automate and facilitate the TDD process.
Agile .NET Development with BDD and Continuous IntegrationTung Nguyen Thanh
Agile .NET Development with BDD and Continuous Integration
Áp dụng CI là một phần không thể thiếu nếu bạn mong muốn nâng cao năng suất và giảm thiểu lỗi lầm khi phát triển cũng như deploy sản phẩm qua việc sử dụng các practices và tools để tự động hóa các khâu
Bài trình bày của anh Trương Anh Quân tại Meetup của Ha Noi .NET Group.
Chi tiết vui lòng xem tại: http://tungnt.net
BuildBot is a continuous integration system written in Python that automates the compile/test cycle. It helps developers practice continuous integration by committing code changes frequently and having the code automatically built and tested on a separate integration machine. This allows bugs to be detected quickly before they become larger issues. The presentation recommends maintaining a single source code repository, automating testing and builds, and keeping the build process fast in order to gain the benefits of continuous integration like reduced risk and easier bug fixing.
Improving code quality with continuous integration (PHPBenelux Conference 2011)Martin de Keijzer
Continuous Integration is the combination of any number of the available tools used to improve quality of both code and process. In this session we will look at these available tools and how these can be used. We will also look at Continuous Integration in general and CruiseControl in particular, at how to set it up and built a platform for robust and high-quality code. This session is a must-see for development leaders and technical managers alike, to understand the available options and the advantages offered by this approach.
Core Principles of CI was a presentation that covered:
1) The definition of continuous integration as automated builds that run many times a day.
2) The key aspects of CI including source code compilation, database integration, testing, inspection, deployment, feedback, and producing deployable software.
3) The benefits of CI like reduced human error, rapid feedback cycles, increased project visibility and confidence, and facilitating communication.
This document discusses automating software deployments in .NET. It introduces the benefits of automation over manual deployments, including consistency and reducing human errors. It describes common automation techniques like using build scripts, configuration management, and database migrations. Specific tools mentioned include Psake for PowerShell builds, Migrator.NET for database schema changes, and services like Heroku, AppHarbor, BuildMaster and Octopus Deploy for release management and deployment. The document advocates setting up an automated deployment process to simplify releasing new code, configurations and database changes across multiple environments.
Technological Evolution of Robotics - Dr Sanjoy SanyalSanjoy Sanyal
• Contrary to what Sci-Fi thrillers would have us believe, Robots are not self-aware machines
• They will not enslave us into bonded labors in their version of ‘Silicon Mines’!
• They will not wage war against Humans, like ‘Skynet’!
• They will not send one of their own from the Future to the Present to kill a boy, who they have determined is destined to lead future Humans against the Robots!!
Read on to see the Technological Evolution of Robotics over the last half-century.
I have created a unique Classification of Robotics that some may find very handy in understanding the complexity and versatility of the current specialized field of Robotics.
Next in this series will see descriptions of Surgical, Space and Military Robots.
Tags
Robots, UNIMATE, IRB 6, CONSIGHT 1, Humanoid Robots, Elektro, WABOT 1, WABOT 2, ASIMO, BEAR, BAXTER
The document provides an overview of the benefits of becoming a Premier Agent member on Zillow. It discusses how Zillow generates over 12.4 million monthly visitors and partners with Yahoo! Real Estate. As a Premier Agent, one would get customer-initiated contacts, exposure to Zillow traffic, and appear as the expert on listings. Premier Agents gain a competitive advantage and market share by following up quickly with contacts. The presentation encourages agents to call a number to check zip code availability and page views before deciding which areas to focus on as a Premier Agent.
1. The document discusses software development methodologies Scrum and Test-Driven Development (TDD). Scrum is an agile framework that uses short iterations called sprints to incrementally develop features. TDD involves writing automated tests before code and refactoring to ensure quality.
2. Continuous integration (CI) is also covered, which is the practice of frequently integrating code changes to reduce integration problems. Tools discussed include CruiseControl for CI and SymbianOSUnit for testing on Symbian platforms.
3. Contact information is provided for Gábor Török from Agil Eight for further questions.
Docker is an open source project which developed in the Linux world, but that doesn't mean Windows and .NET projects can't make use of it. With the Core CLR you can run .NET inside a container running Linux, and with Windows Server 2016 you'll be able to run Docker containers on Windows.
In this session we'll start by looking at how Docker works, how containers are defined and run, and what's already out there in public repositories for you to use. We'll see where Docker can fit in integration projects - running core components like message queues and caches, and hosting .NET projects too.
We've all seen the big "macro" features in .NET, this presentation is to give praise to the "Little Wonders" of .NET -- those little items in the framework that make life as a developer that much easier!
Jenkins is an open-source tool for continuous integration that allows developers to integrate code changes frequently from a main branch using an automated build process. It detects errors early, measures code quality, and improves delivery speed. Jenkins supports various source control, build tools, and plugins to customize notifications and reporting. Security features allow restricting access and privileges based on user roles and projects.
Tracxn Research: Real Estate Tech India Landscape, October 2016Tracxn
This report analyzes the real estate technology sector in India. It covers 231 companies founded in 2015 focusing on consumer and enterprise solutions like online rental listings, property management, co-working spaces, brokerage services, and AR/VR. Total funding in the sector reached $60M in 2016, up from $11M in 2014. Major subsectors are co-working spaces, managed rentals, and property buying platforms. Notable deals include NestAway raising $30M and PropTiger acquiring 4 companies. The report provides statistics on the Indian real estate market and breakdowns of companies and funding by stage and subsector.
This presentation by Andrew Aslinger discusses best practices and pitfalls of integrating Docker into Continuous Delivery Pipelines. Learn how Andrew and his team used Docker to replace Chef to simplify their development and migration processes.
IBM Think Session 8598 Domino and JavaScript Development MasterClassPaul Withers
Session from IBM Think 2018. Note: the architecture used is an extreme case of what's possible (and it could go further), rather than a real-world expectation
Cost is often the conversation starter when customers think about moving to the cloud. AWS helps lower costs for customers through its “pay only for what you use” pricing model, frequent price drops, and pricing model choice to support variable & stable workloads. In this session, you will learn about the financial considerations of owning and operating a traditional data center or managed hosting provider versus utilizing AWS. We will detail our TCO methodology and showcase cost comparisons for some common customer use-cases. We’ll also cover a few AWS cost optimization areas, including Spot and Reserved Instances, EC2 Auto Scaling, and consolidated billing.
Presenter:
Amit Sharma, Solution Architect, Amazon Internet Services
Krishnenjit Roy, Director IT Operations, Freshdesk
OSDC.no 2015 introduction to node.js workshopleffen
This document provides a short introduction to Node.js, Express, and MQTT for IoT applications. It discusses using Node.js and its non-blocking I/O model on devices like the Raspberry Pi. It then demonstrates setting up a basic Express app, adding static files and templates. Finally, it introduces MQTT as a lightweight protocol for IoT with publish/subscribe messaging and shows a simple example of connecting and publishing with the MQTT Node.js client library.
Continuous Integration e Delivery per (r)innovare lo sviluppo software e la g...Amazon Web Services
This document discusses continuous integration and delivery practices using AWS services like CodeCommit, CodeBuild, CodeDeploy, and CodePipeline. It summarizes how these services can be used together in a software development pipeline to automate building, testing, and deploying code changes. It also discusses how infrastructure as code with CloudFormation templates allows infrastructure to be provisioned and managed like code. The document provides an example of how a company implemented continuous integration of their infrastructure stacks using CloudFormation across different environments.
ContainerDayVietnam2016: Dockerize a small businessDocker-Hanoi
This document discusses how Docker can transform development and deployment processes for modern applications. It outlines some of the challenges of developing and deploying applications across different environments, and how Docker addresses these challenges through containerization. The document then provides examples of how to dockerize a Rails and Python application, set up an Nginx reverse proxy with Let's Encrypt, and configure a Docker cluster for continuous integration testing.
Netflix uses containers to run both batch jobs and services. For batch jobs, containers simplify resource management and allow jobs like model training and media encoding to easily share resources. Services are more complex to run in containers due to challenges like constant resizing, statefulness, and networking. Netflix addresses these challenges through solutions like a VPC networking driver and reusing existing infrastructure services for containers. Looking ahead, Netflix aims to run more containers at larger scale for areas like developer experience, continuous integration, and internal resource optimization.
Developer Experience at the Guardian, Equal Experts Sept 2021Akash Askoolum
The Guardian Developer Experience team’s mission statement is to “Enable teams to focus on delivering value at lightning speed by streamlining infrastructure management”.
In this session, the Guardian will describe how we’re working to fulfil our mission; we’ll give a brief history and a glimpse into the future.
DevOps, Continuous Integration & Deployment on AWS discusses practices for software development on AWS including DevOps, continuous integration, continuous delivery, and continuous deployment. It provides an overview of AWS services that can be used at different stages of the software development lifecycle such as CodeCommit for source control, CodePipeline for release automation, and CodeDeploy for deployment. National Novel Writing Month (NaNoWriMo) maintains its websites and services on AWS to support its annual writing challenge. It migrated to AWS to improve uptime and scalability. Its future goals include porting older sites to Rails, using Amazon SES for email, load balancing with ELB, implementing auto scaling, and using services like CodeDeploy, SNS
DevOps, Continuous Integration and Deployment on AWS: Putting Money Back into...Amazon Web Services
Organizations around the globe are leveraging the cloud to accomplish world-changing missions. This session will address how AWS can help organizations put more money toward their mission and scale outreach and operations to achieve more with less. Hear some of AWS’s most advanced customers on how their organizations handle DevOps, continuous integration and deployment. Learn how these practices allow them to rapidly develop, iterate, test and deploy highly-scalable web applications and core operational systems on AWS. The discussion will focus on best practices, lessons learned, and the specific technologies and services they use.
This document provides an overview of Docker and cloud native training presented by Brian Christner of 56K.Cloud. It includes an agenda for Docker labs, common IT struggles Docker can address, and 56K.Cloud's consulting and training services. It discusses concepts like containers, microservices, DevOps, infrastructure as code, and cloud migration. It also includes sections on Docker architecture, networking, volumes, logging, and monitoring tools. Case studies and examples are provided to demonstrate how Docker delivers speed, agility, and cost savings for application development.
This document provides an overview and summary of OpenShift v3 and containers. It discusses how OpenShift v3 uses Docker containers and Kubernetes for orchestration instead of the previous "Gears" system. It also summarizes the key architectural changes in OpenShift v3, including using immutable Docker images, separating development and operations, and abstracting operational complexity.
Dark launching with Consul at Hootsuite - Bill MonkmanAmbassador Labs
Dark Launching (A.K.A. Feature Flagging) is a technique and mindset that has truly shaped the way we write, test, and deploy code at Hootsuite. It gives our team realtime, fine-grained control over our production systems which helps to prevent issues from reaching users, and build developer confidence in a culture of pushing code many times per day.
In this presentation I will go over how the system helps us both in the context of microservices and monoliths, and how we made use of Consul, Hashicorp's HA service discovery / KV store, to make it more resilient and performant at scale.
Workshop; Deploy a Deep Learning Framework on Amazon ECS and Spot InstancesAmazon Web Services
This document provides an overview of a workshop on deploying a deep learning framework on Amazon ECS and Spot Instances. The workshop will introduce MXNet, containers, Amazon ECS, Amazon ECR, AWS CloudFormation, Amazon EC2 Spot Fleet and Spot Instances. It will include hands-on labs to build an MXNet Docker image, deploy an MXNet container with ECS, and run an image classification demo using a Spot Fleet on ECS. The overall goal is to learn how to cost-effectively run deep learning workloads on AWS.
OpenShift is a Platform-as-a-Service that provides development environments on demand using containers. It automates application lifecycles including build, deploy, and retirement. OpenShift uses containers to package applications and dependencies in a portable way. Red Hat addresses concerns around adopting containers at scale through OpenShift, which provides security, scalability, integration, management and certification capabilities. OpenShift runs on a user's choice of infrastructure and orchestrates applications across nodes using Kubernetes.
Test cloud application deployments locally and in CI without staging environm...Thomas Rausch
In this talk we explore the challenges of testing cloud applications, and what strategies there are to overcome the challenges. I present LocalStack, a tool for testing serverless cloud applications locally and in CI.
“An Outlook of the Ongoing and Future Relationship between Blockchain Technologies and Process-aware Information Systems.” Invited talk at the joint workshop on Blockchain for Information Systems (BC4IS) and Blockchain for Trusted Data Sharing (B4TDS), co-located with with the 36th International Conference on Advanced Information Systems Engineering (CAiSE), 3 June 2024, Limassol, Cyprus.
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
Removing Uninteresting Bytes in Software FuzzingAftab Hussain
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024Neo4j
Neha Bajwa, Vice President of Product Marketing, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/building-and-scaling-ai-applications-with-the-nx-ai-manager-a-presentation-from-network-optix/
Robin van Emden, Senior Director of Data Science at Network Optix, presents the “Building and Scaling AI Applications with the Nx AI Manager,” tutorial at the May 2024 Embedded Vision Summit.
In this presentation, van Emden covers the basics of scaling edge AI solutions using the Nx tool kit. He emphasizes the process of developing AI models and deploying them globally. He also showcases the conversion of AI models and the creation of effective edge AI pipelines, with a focus on pre-processing, model conversion, selecting the appropriate inference engine for the target hardware and post-processing.
van Emden shows how Nx can simplify the developer’s life and facilitate a rapid transition from concept to production-ready applications.He provides valuable insights into developing scalable and efficient edge AI solutions, with a strong focus on practical implementation.
Communications Mining Series - Zero to Hero - Session 1DianaGray10
This session provides introduction to UiPath Communication Mining, importance and platform overview. You will acquire a good understand of the phases in Communication Mining as we go over the platform with you. Topics covered:
• Communication Mining Overview
• Why is it important?
• How can it help today’s business and the benefits
• Phases in Communication Mining
• Demo on Platform overview
• Q/A
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
UiPath Test Automation using UiPath Test Suite series, part 6DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 6. In this session, we will cover Test Automation with generative AI and Open AI.
UiPath Test Automation with generative AI and Open AI webinar offers an in-depth exploration of leveraging cutting-edge technologies for test automation within the UiPath platform. Attendees will delve into the integration of generative AI, a test automation solution, with Open AI advanced natural language processing capabilities.
Throughout the session, participants will discover how this synergy empowers testers to automate repetitive tasks, enhance testing accuracy, and expedite the software testing life cycle. Topics covered include the seamless integration process, practical use cases, and the benefits of harnessing AI-driven automation for UiPath testing initiatives. By attending this webinar, testers, and automation professionals can gain valuable insights into harnessing the power of AI to optimize their test automation workflows within the UiPath ecosystem, ultimately driving efficiency and quality in software development processes.
What will you get from this session?
1. Insights into integrating generative AI.
2. Understanding how this integration enhances test automation within the UiPath platform
3. Practical demonstrations
4. Exploration of real-world use cases illustrating the benefits of AI-driven test automation for UiPath
Topics covered:
What is generative AI
Test Automation with generative AI and Open AI.
UiPath integration with generative AI
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slackshyamraj55
Discover the seamless integration of RPA (Robotic Process Automation), COMPOSER, and APM with AWS IDP enhanced with Slack notifications. Explore how these technologies converge to streamline workflows, optimize performance, and ensure secure access, all while leveraging the power of AWS IDP and real-time communication via Slack notifications.
UiPath Test Automation using UiPath Test Suite series, part 5DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 5. In this session, we will cover CI/CD with devops.
Topics covered:
CI/CD with in UiPath
End-to-end overview of CI/CD pipeline with Azure devops
Speaker:
Lyndsey Byblow, Test Suite Sales Engineer @ UiPath, Inc.
How to Get CNIC Information System with Paksim Ga.pptxdanishmna97
Pakdata Cf is a groundbreaking system designed to streamline and facilitate access to CNIC information. This innovative platform leverages advanced technology to provide users with efficient and secure access to their CNIC details.
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc
How does your privacy program stack up against your peers? What challenges are privacy teams tackling and prioritizing in 2024?
In the fifth annual Global Privacy Benchmarks Survey, we asked over 1,800 global privacy professionals and business executives to share their perspectives on the current state of privacy inside and outside of their organizations. This year’s report focused on emerging areas of importance for privacy and compliance professionals, including considerations and implications of Artificial Intelligence (AI) technologies, building brand trust, and different approaches for achieving higher privacy competence scores.
See how organizational priorities and strategic approaches to data security and privacy are evolving around the globe.
This webinar will review:
- The top 10 privacy insights from the fifth annual Global Privacy Benchmarks Survey
- The top challenges for privacy leaders, practitioners, and organizations in 2024
- Key themes to consider in developing and maintaining your privacy program
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
20 Comprehensive Checklist of Designing and Developing a WebsitePixlogix Infotech
Dive into the world of Website Designing and Developing with Pixlogix! Looking to create a stunning online presence? Look no further! Our comprehensive checklist covers everything you need to know to craft a website that stands out. From user-friendly design to seamless functionality, we've got you covered. Don't miss out on this invaluable resource! Check out our checklist now at Pixlogix and start your journey towards a captivating online presence today.
Introducing Milvus Lite: Easy-to-Install, Easy-to-Use vector database for you...Zilliz
Join us to introduce Milvus Lite, a vector database that can run on notebooks and laptops, share the same API with Milvus, and integrate with every popular GenAI framework. This webinar is perfect for developers seeking easy-to-use, well-integrated vector databases for their GenAI apps.
6. Intro to CloudFormation
● Infrastructure as code
● Glue your AWS services together
● Constantly expanding
● Well documented
● Lots of samples available
● JSON syntax
7. An example...
ELB
Health-check
HTTP port
HTTPS port
Cross-Zone
Tags!!
8. CloudFormation at Domain
We took a pre-existing template and extended it
Then we cut it back down significantly
Multi-AZ Load-balanced, AutoScaling & Self-healing
Each platoon has its own template and config
Customised LaunchConfiguration
Uses standard AWS AMIs
Versioned using Git
9. The LaunchConfiguration
● The pivotal step
● Lots of PowerShell
● Pulls pre-built scripts from S3
based on cluster name
10. The Powershell
● Rename server
● Set TZ
● Pull files
● cluster-config.json
● Army config (DSC)
● Platoon config (DSC)
● Find latest app code
● Notify
11. Octopus Deploy
The very last PowerShell Script drives OctopusDeploy
.NET-based, API-first deployment engine
Pulls code from Bamboo/Nuget
Pushes it out to servers
Executes supporting powershell code
12. Gotchas
- CFN-Init does not load windows profiles
- So it can’t use DPAPI or user environment variables
- More services means more stacks
- We have to parameterise, or maintain lots of templates
- Multi-tenancy is tricky
- Luckily, we want the resiliency of separate clusters
- OctopusDSC
- We needed to customise OctopusDSC for AWS
- Octopus Reg/Dereg
- Custom script to clean-up scaled-down instances
13. In Conclusion
You can drive the entire provisioning chain
- (yes, including app deployment)
New cluster ~10-15 min
Auto-scale: <10 min
Cloudformation is epic
Domain’s new Microservices deploy on-commit, with the
capacity to deploy hundreds of times per day to AWS.
14. Want to learn more?
Follow Domain’s tech blog
http://tech.domain.com.au/
And we are hiring!
Big thanks to all the Invader Zim fan sites which contributed images for this presentation!
Editor's Notes
We are Domain.com.au
One of Australia’s leading property classified websites
About four million Australians check us out every month
We’ve been around since ‘99
We’re historically a windows shop with a big, monolithic .NET application, but we’re currently in the middle of a huge project to overhaul almost everything we do, tech-wise
And who am I?
I’m Jason Brown, the DevOps lead at Domain. I’m the field marshall heading up the Robot Army. I came on board about six months ago, and it’s my job to drive the ‘ops’ side of DevOps, and to essentially automate myself of one job and into another, every few weeks. I was recently joined by a couple more skilled engineers, who make things significantly smoother, which means I can come out and talk to groups like this
We’ll cover the CF and Powershell needed to drive the Robot Army
Supporting Code: We’re still mainly a Windows shop, so parts of this will be windows-centric, but the core concepts apply to any platform
We’ll mention the CI and CD pipelines, but not in depth
do jump in if you have questions, it’ll slow me down and stop me skimming over important points
Point one: we’re completely rearchitecting our software on a Microservice architecture, as well as migrating a number of other apps into the Cloud. This means our capacity planning is somewhat at the ‘guesswork’ end of educated guesswork
We could respond to this in a couple of ways
We could just guess
We could over-engineer so that we never hit our limits
We could leverage autoscaling capacities in AWS to make sure we’re at optimum scale all the time - which could also deliver us significant cost savings
We may also incur a significant management overhead, with many different services spread across many different servers, so we need to automate as much as we can.
Point two: We’re doing this with a continuous delivery model backing it up
So we can’t predict what code will be on a given node on a given day. Which means our provisioning pipeline MUST integrate tightly with our delivery pipeline
Point Three: we’re super-agile, and we run light. we need to do this stuff quickly. If we need a new environment, we don’t want to wait a week, but our ops team is very lean
Point Four: We’re on Windows. There aren’t that many shops doing microservices in the cloud on Windows Platforms, and those that do aren’t leveraging every feature they possibly can
There are lots of companies out there doing cloud. There are lots who use autoscaling. There are lots who do automation. And there are lots trying to implement CD. We don’t know many that are doing them all.
The point is though, it’s combining continuous delivery with microservices that is the tricky part. We could be throwing code out literally hundreds of times per week, so making sure that our auto-scaling clusters are always on prime code is a difficult process, and we’ll touch on that towards the end because we think we nailed it
The Robot Army refers to our entire AWS server fleet
it’s made up of smaller Robot Platoons - essentially microclusters consisting of an auto-scaling group, an ELB and two or more EC2 servers, spanning two availability zones.
This is a fairly standard configuration - it’s what you see in, for example, Elastic Beanstalk containers.
But we’ve added a whole lot of smarts under the covers, driven from the CloudFormation template
Worth mentioning on the constantly expanding bit that ELB tagging appeared YESTERDAY (30th Sept) along with the ability to update SNS topics - both of which have been “want” items for a while now.
Will drop this slide if the projector is terrible. CLoudformation is very texty so it’s not great for a slide presentation, but let’s see if it works
Now this will be terribly hard to read but these slides will be available later, I just wanted to show a basic example of a CloudFormation resource
With cross-zone load balancing, your load balancer nodes route traffic to the back-end instances across all Availability Zones. By default the CrossZone property is false.
https://s3-us-west-2.amazonaws.com/cloudformation-templates-us-west-2/AutoScalingMultiAZWithNotifications.template
Why does every cluster have its own template and config script?
Well, the template is actually pretty-much generic. But when we started this project, we weren’t sure how much variance would be needed between individual platoons. As it turns out, the robots are far more generic than we expected them to be
Still, by maintaining separate templates for now, we also maintain a separate revision history for them in git, and we can see where we’ve made changes
The intention is, eventually, to drive ALL our stacks - of which there will be many - from ONE CloudFormation template
This is a vital link in the chain because it’s where CloudFormation meets the Operating System. IN essence what we do here is write a lot of powershell commands and supporting files onto the disk from S3, then execute them
Three key sections within AWS::CloudFormation::Init
“files: : {} writes a few files to the disk, a couple of which came with the template
"commands" : { } contains five powershell commands in total
“services”: {} contains the standard cfn-hup
It’s a bit tricky to go into depth on this LaunchConfig, but at some point in the future there’ll be a detailed blog post on this and we may even end up open-sourcing it, though we’ll go into the powershell on the next slide
cluster-config.json persists some of the parameters we’ve passed to the CloudFormation template onto disk, for later use - it’s written from the CF launchconfiguration directly, using parameters pushed into the CF template
the common config script installs New Relic, The Sysinternals Suite, some DSC and Powershell modules we want common to every node in the entire army
The cluster config DSC script is currently unique per-cluster, and allows us to make significant variations between individual setups. So if one cluster out of ten requires, say, MSMQ, or CGI support, or we can add it here with DSC.
It allows the army as a whole to have minimal configurations across the entire army, with some platoons having extras here and there as needed. We could make quite radical changes here if we wanted (but we don’t)
All of this is itself driven by a powershell script back at home base, making it an easy one-liner to stand up one of these clusters
There’s then a final script, generic across the entire army, called cluster-deploy.ps1 - this picks up cluster-config.json and uses that to find the latest app code from…. <next slide>
This is the final piece in the puzzle. It’s no good having all these beautiful auto-scaling doohickeys if you end up deploying stale code to them - and “stale code” for us can mean literally minutes out of date
This seems to be what makes continuous delivery and Auto Scaling daunting for some organisations - how do you get your latest code onto the servers as they scale (or indeed self-heal)? In linuxworld there’s things like Puppet, chef, mcollective etc.
You could roll your own solution, perhaps using powershell and S3 (as we occasionally do to bring a platoon online for the first time)
You could bake AMIs somewhere in your Continuous Integration pipeline, so your app code is already on-board. You could use EBS snapshots to attach an entire drive of app code.
We chose to be more lean and flexible, so we went with OctopusDeploy
To be honest, if it weren’t for the continuous delivery aspect, if we were still deploying only twice a week, then AMI baking or pulling zipfiles from S3 would be fine for us. But we want the ability to deploy many times a day, either automated or manually, and we want pretty much anyone in the organisation, from a first-day developer to the Technical Director, to be able to deploy code.
Not that we want that happening a lot, but we want it to be possible. So Octopus it is. (and it’s good). We found we can drive many of the variances between platoons from Octopus, and therefore simplify the build pipeline
CFN-Init not loading profiles doesn’t appear to be a problem at first, until you need to use, say, the Data Protection API in windows.
Octopus needs this to register its server agents (called tentacles). We eventually got round it using a DSC module for Octopus which was not even two weeks old when we found it. DSC always load a user profile. Good tip to know.
There’s a balance to be struck between either having lots of parameters in your master template, or having lots of templates, and there are risks on either side. Finding the sweet spot is tricky. We’re starting with one and moving towards the other
Note: multi-tenancy is tricky, but we’re also deliberately isolating our services - other organisations may want to multi-tenant, in which case they’d need to build some extra smarts.
OctopusDSC, while it saved our bacon quite well, had to be modified. By default, it uses Amazon public IP addresses. Because we’ve designed the robot army to be highly secure, none of the individual soldiers has a public IP. So we changed it to use private IPs and sent the code to Octopus for integration into the product - yes, windows shops do open source too.
And finally, autoscaling means a box could be terminated at any time, but Octopus does not automatically de-register dead servers. So we had to roll our own code to handle the cleanup. Luckily: API-first, so easy to do with powershell.
You can have your cake and eat it, CD-wise
Without Cloudformation, this would be significantly harder
We will be open-sourcing components of this pipeline, if not the whole thing
Follow domain’s tech blog, including career opportunities.