This document summarizes a presentation on microservices architectures. It begins with an introduction to microservices and some of the benefits they provide like scalability and faster development. It then discusses challenges like configuration, tooling, discovery, routing, and observability. Examples are provided of microservices architectures at companies like Netflix, Twitter, and Gilt. Key aspects covered include programming languages, containerization, orchestration, and data stores.
It's clear that Docker speeds up development and makes testing and deployment more efficient. As Docker moves into production new use cases and patterns are emerging that address availability and security concerns. With microservices, safety is part of the architecture that developers need to understand and build for. It's no longer good enough to wrap a firewall around an entire app when it goes to production, and have a cold standby in case it breaks.
Opening talk at Monitorama, talks about the problems of monitoring, challenges of creating monitoring tools and why monitoring vendors keep getting disrupted. Ended with a discussion of simulation testing and serverless architectures - Monitorless.
Businesses are speeding up development and automating operations to remain competitive and to get large organizations to scale. Project based monolithic application updates are replaced by product teams owning containerized microservices. This puts developers on call, responsible for pushing code to production, fixing it when it breaks, and managing the cost and security aspects of running their microservices. In this world operations skill-sets are either embedded in the microservices development teams, or building and operating API driven platforms. The platform automates stress testing, canary based deployment, penetration testing and enforces availability and security requirements. There are no meetings or tickets to file in the delivery process for updating a containerized microservice, which can happen many times a day, and takes seconds to complete. The role of site reliability engineering moves from firefighting and fixing outages to buiding tools for finding problems and routing those problems to the right developers. SREs manage the incident lifecycle for customer visible problems, and measure and publish availability metrics. This may sound futuristic but Werner Vogels described this as “You build it, you run it” in 2006.
A rough and researchy presentation where I tried out some new material in front of a local audience. Skipped the usual introduction and talked about some of the problems people run into when they do microservices and miss a few things. More refined version of this talk to be shown at O'Reilly Software Architecture Conference in New York in April.
It's clear that Docker speeds up development and makes testing and deployment more efficient. As Docker moves into production new use cases and patterns are emerging that address availability and security concerns. With microservices, safety is part of the architecture that developers need to understand and build for. It's no longer good enough to wrap a firewall around an entire app when it goes to production, and have a cold standby in case it breaks.
Opening talk at Monitorama, talks about the problems of monitoring, challenges of creating monitoring tools and why monitoring vendors keep getting disrupted. Ended with a discussion of simulation testing and serverless architectures - Monitorless.
Businesses are speeding up development and automating operations to remain competitive and to get large organizations to scale. Project based monolithic application updates are replaced by product teams owning containerized microservices. This puts developers on call, responsible for pushing code to production, fixing it when it breaks, and managing the cost and security aspects of running their microservices. In this world operations skill-sets are either embedded in the microservices development teams, or building and operating API driven platforms. The platform automates stress testing, canary based deployment, penetration testing and enforces availability and security requirements. There are no meetings or tickets to file in the delivery process for updating a containerized microservice, which can happen many times a day, and takes seconds to complete. The role of site reliability engineering moves from firefighting and fixing outages to buiding tools for finding problems and routing those problems to the right developers. SREs manage the incident lifecycle for customer visible problems, and measure and publish availability metrics. This may sound futuristic but Werner Vogels described this as “You build it, you run it” in 2006.
A rough and researchy presentation where I tried out some new material in front of a local audience. Skipped the usual introduction and talked about some of the problems people run into when they do microservices and miss a few things. More refined version of this talk to be shown at O'Reilly Software Architecture Conference in New York in April.
Organizations have accepted that "cloud" is the de-facto platform of the future, and the benefits and flexibility it brings have ushered in a renaissance in software architecture. The disposable infrastructure of cloud has enabled the first "cloud native" architecture, microservices. continuous delivery, a technique that is radically changing how tech-based businesses evolve, amplifies the impact of cloud as an architecture. We expect architectural innovation to continue, with trends such as containerization and software-defined networking providing even more technical options and capability.
Summary of fast development and cloud native architecture along with cost optimization techniques. Presented as opening keynote at the Utility and Cloud Computing 2014 as part of the Cloud Control Workshop.
Sildes of an internal talk given at Twitter similar to a previous webinar for Redhat with the same title.
Speeding up development is a key concern, cloud and technology improvements like Docker speed up key steps that make continuous delivery possible. Breaking up the work into many separate microservices and datastores with stable APIs allows teams to make progress independently so that the organization scales. Monolithic apps are preferred for small projects, built by small teams and when very low latency and high efficiency is the primary requirement. Monitoring microservices is currently a challenge with solutions starting to emerge.
Software application development and delivery often involves multiple development, infrastructure and operations teams, each with their own preferred “tools of the trade” for building, testing and deploying code changes
For years, virtualization and cloud technologies have provided agile, on-demand infrastructure. The advent of Microservices promises even more agility– but what is required to take advantage of Microservices?
Join Electric Cloud CTO Anders Wallgren and Trace3 Principal Consultant - DevOps Marc Hornbeek as they discuss what is required to:
- Overcome culture and architecture challenges created when decomposing monolithic applications into Microservices-based applications.
- Coordinate integration, testing, monitoring, packaging, release approval and deployment of Microservices-based applications over elastic infrastructures
- Create a controlled and auditable delivery pipeline to support
Microservices-based application.
- Prepare for “future” applications, pipelines and patterns.
Lattice: A Cloud-Native Platform for Your Spring ApplicationsMatt Stine
As presented at SpringOne2GX 2015 in Washington, DC.
Lattice is a cloud-native application platform that enables you to run your applications in containers like Docker, on your local machine via Vagrant. Lattice includes features like:
Cluster scheduling
HTTP load balancing
Log aggregation
Health management
Lattice does this by packaging a subset of the components found in the Cloud Foundry elastic runtime. The result is an open, single-tenant environment suitable for rapid application development, similar to Kubernetes and Mesos Applications developed using Lattice should migrate unchanged to full Cloud Foundry deployments.
Lattice can be used by Spring developers to spin up powerful micro-cloud environments on their desktops, and can be useful for developing and testing cloud-native application architectures. Lattice already has deep integration with Spring Cloud and Spring XD, and you’ll have the opportunity to see deep dives into both at this year’s SpringOne 2GX. This session will introduce the basics:
Installing Lattice
Lattice’s Architecture
How Lattice Differs from Cloud Foundry
How to Package and Run Your Spring Apps on Lattice
'The History of Metrics According to me' by Stephen DayDocker, Inc.
Metrics and monitoring are a time honored tradition for any engineering discipline. It is how we ensure the systems we use are working the way we expect. If this is a time honored tradition, why is it not a built into every piece of software we create, from the ground up? With software engineering, usually the trick to solving anything is to make it easier. By solving the hard parts of application metrics in Docker, we should make it more likely that metrics are a part of your services from the start.
DevOps Will Save The World! : Public Safety, Public Policy, and DevOps In Context
Joshua Corman, CTO, Sonatype
Link to video: https://www.youtube.com/watch?v=K-hskShNyoo
How to Effect Change in the Epistemological Wasteland of Application SecurityJames Wickett
From GOTO London 2015
Over the years, application security (appsec) has made progress, but it has also made some considerable mis-steps. Appsec focuses almost solely on developer awareness and secure development training as remediation. This isn't sustainable and arguably does little good. There is a better way, but we have to separate ourselves from the core assumptions we have made that got us here. Lets journey together to find old truths and better approaches.
We will explore ways to make a change for the better across all levels of the development lifecycle, but we will focus on security testing early on in the development process. From this session, you will learn pragmatic approaches and tooling that will affect your development processes and delivery pipelines. You will walk away with code examples and tools that you can put into practice right away for security and rugged testing.
Application Security Epistemology in a Continuous Delivery WorldJames Wickett
CD Summit - Austin, from DevOps Connect
Desc:
Over the years, application security (appsec) has made progress, but it has also made some considerable mis-steps. Appsec focuses almost solely on developer awareness and secure development training as remediation. This isn’t sustainable and arguably does little good. There is a better way, but we have to separate ourselves from the core assumptions we have made that got us here.
http://www.devopsconnect.com/events/cd-summit-austin/
From Zero to DevSecOps in 60 Minutes - DevTalks Romania - Cluj-Napocajerryhargrove
Whether you’re building an application in a DevOps + Security culture, or have already bridged the gap with DevSecOps, the task remains the same: How do you ensure that security best practices are understood, architected for and integrated into your application from day 1 AND remain relevant year 1. During this talk I’ll focus on how to achieve these goals amidst the ever changing landscape of people, process, and technology in the cloud, in the context of various compute environments like instances, containers and serverless functions. and how to do so using off-the-shelf AWS services and features. I’ll complete the story by accompanying this discussion with a reference application architecture and examples. Attendees of this talk will receive actionable best practices and guidance, with specific implementation details for AWS
The Emergent Cloud Security Toolchain for CI/CDJames Wickett
The Emergent Cloud Security Toolchain for CI/CD given at RSA Conference 2018 in San Francisco.
All organizations want to go faster and decrease friction in their cloud software delivery pipeline. Infosec has an opportunity to change their classic approach from blocker to enabler. This talk will discuss hallmarks of CI/CD and some practical examples for adding security testing across different organizations. The talk will cover emergent patterns, practices and toolchains that bring security to the table.
Learning Objectives:
1: Learn the emerging patterns for security in CI/CD pipelines.
2: Receive a pragmatic security toolchain for CI/CD to use in your organization.
3: Understand the real meaning of DevSecOps is without all the hype.
Devops: Who Does What? - Devops Enterprise Summit 2016cornelia davis
Within the IT organizational structures that have dominated the last several decades roles and responsibilities are fairly standardized. But with the dramatic changes that DevOps practices and supporting toolsets bring, many are left feeling a bit off balance - it’s no longer clear who is responsible for even things as “straight-forward” as development or operations.
In this talk I will take traditional roles that are distributed across fairly standard IT structures and sort them into a new organizational context. What is the role of the Enterprise Architect? Who does capacity planning and how? How can change management step out of the way all while still satisfying the requirements of safe deployments? How do agile teams interface with personnel responsible for maintaining legacy systems? I’ll leave the audience with a blueprint for a new organizational structure.
Cloud Native: Designing Change-tolerant Softwarecornelia davis
To see this presentation given live, go to http://bit.ly/DesignPatternsReplay
There is a special (discount) offer in there! :-)
Cloud-native applications are characterized by highly distributed topologies consisting of many relatively small components (yup, usually called microservices). But the thing that sets them apart even more from the previous generation of apps is that they are expected to function flawlessly even while the environment they are running in is constantly changing, or even failing.
All of this requires applying a new set of design patterns and practices and this webinar will introduce the most important ones. The Twelve Factor App (12factor.net) is a high-level articulation of some of these techniques that you may well have heard of, but its descriptions are relatively dense and the industry knowledge has evolved a fair bit since its publication.
Cornelia Davis, Senior Director of Technology at Pivotal, will share best practices for cloud-native applications and clear some of the mystery that shrouds 12-factor today. At the conclusion, attendees will understand what is needed for cloud-native applications, as well as why and how to deliver on those requirements.
Just about all of my current technical content in one 364 slide mega-deck. Source files at https://github.com/adrianco/slides
Sections on:
Scene Setting
State of the Cloud
What Changes?
Product Processes
Microservices
State of the Art
Segmentation
What’s Missing?
Monitoring
Challenges
Migration
Response Times
Serverless
Lock-In
Teraservices
Wrap-Up
Microxchg Analyzing Response Time Distributions for MicroservicesAdrian Cockcroft
Research oriented presentation @Microxchg Berlin Feb 5th 2016. New code to collect histograms of response time and export them to monte-carlo simulation spreadsheet via getguesstimate.com
Organizations have accepted that "cloud" is the de-facto platform of the future, and the benefits and flexibility it brings have ushered in a renaissance in software architecture. The disposable infrastructure of cloud has enabled the first "cloud native" architecture, microservices. continuous delivery, a technique that is radically changing how tech-based businesses evolve, amplifies the impact of cloud as an architecture. We expect architectural innovation to continue, with trends such as containerization and software-defined networking providing even more technical options and capability.
Summary of fast development and cloud native architecture along with cost optimization techniques. Presented as opening keynote at the Utility and Cloud Computing 2014 as part of the Cloud Control Workshop.
Sildes of an internal talk given at Twitter similar to a previous webinar for Redhat with the same title.
Speeding up development is a key concern, cloud and technology improvements like Docker speed up key steps that make continuous delivery possible. Breaking up the work into many separate microservices and datastores with stable APIs allows teams to make progress independently so that the organization scales. Monolithic apps are preferred for small projects, built by small teams and when very low latency and high efficiency is the primary requirement. Monitoring microservices is currently a challenge with solutions starting to emerge.
Software application development and delivery often involves multiple development, infrastructure and operations teams, each with their own preferred “tools of the trade” for building, testing and deploying code changes
For years, virtualization and cloud technologies have provided agile, on-demand infrastructure. The advent of Microservices promises even more agility– but what is required to take advantage of Microservices?
Join Electric Cloud CTO Anders Wallgren and Trace3 Principal Consultant - DevOps Marc Hornbeek as they discuss what is required to:
- Overcome culture and architecture challenges created when decomposing monolithic applications into Microservices-based applications.
- Coordinate integration, testing, monitoring, packaging, release approval and deployment of Microservices-based applications over elastic infrastructures
- Create a controlled and auditable delivery pipeline to support
Microservices-based application.
- Prepare for “future” applications, pipelines and patterns.
Lattice: A Cloud-Native Platform for Your Spring ApplicationsMatt Stine
As presented at SpringOne2GX 2015 in Washington, DC.
Lattice is a cloud-native application platform that enables you to run your applications in containers like Docker, on your local machine via Vagrant. Lattice includes features like:
Cluster scheduling
HTTP load balancing
Log aggregation
Health management
Lattice does this by packaging a subset of the components found in the Cloud Foundry elastic runtime. The result is an open, single-tenant environment suitable for rapid application development, similar to Kubernetes and Mesos Applications developed using Lattice should migrate unchanged to full Cloud Foundry deployments.
Lattice can be used by Spring developers to spin up powerful micro-cloud environments on their desktops, and can be useful for developing and testing cloud-native application architectures. Lattice already has deep integration with Spring Cloud and Spring XD, and you’ll have the opportunity to see deep dives into both at this year’s SpringOne 2GX. This session will introduce the basics:
Installing Lattice
Lattice’s Architecture
How Lattice Differs from Cloud Foundry
How to Package and Run Your Spring Apps on Lattice
'The History of Metrics According to me' by Stephen DayDocker, Inc.
Metrics and monitoring are a time honored tradition for any engineering discipline. It is how we ensure the systems we use are working the way we expect. If this is a time honored tradition, why is it not a built into every piece of software we create, from the ground up? With software engineering, usually the trick to solving anything is to make it easier. By solving the hard parts of application metrics in Docker, we should make it more likely that metrics are a part of your services from the start.
DevOps Will Save The World! : Public Safety, Public Policy, and DevOps In Context
Joshua Corman, CTO, Sonatype
Link to video: https://www.youtube.com/watch?v=K-hskShNyoo
How to Effect Change in the Epistemological Wasteland of Application SecurityJames Wickett
From GOTO London 2015
Over the years, application security (appsec) has made progress, but it has also made some considerable mis-steps. Appsec focuses almost solely on developer awareness and secure development training as remediation. This isn't sustainable and arguably does little good. There is a better way, but we have to separate ourselves from the core assumptions we have made that got us here. Lets journey together to find old truths and better approaches.
We will explore ways to make a change for the better across all levels of the development lifecycle, but we will focus on security testing early on in the development process. From this session, you will learn pragmatic approaches and tooling that will affect your development processes and delivery pipelines. You will walk away with code examples and tools that you can put into practice right away for security and rugged testing.
Application Security Epistemology in a Continuous Delivery WorldJames Wickett
CD Summit - Austin, from DevOps Connect
Desc:
Over the years, application security (appsec) has made progress, but it has also made some considerable mis-steps. Appsec focuses almost solely on developer awareness and secure development training as remediation. This isn’t sustainable and arguably does little good. There is a better way, but we have to separate ourselves from the core assumptions we have made that got us here.
http://www.devopsconnect.com/events/cd-summit-austin/
From Zero to DevSecOps in 60 Minutes - DevTalks Romania - Cluj-Napocajerryhargrove
Whether you’re building an application in a DevOps + Security culture, or have already bridged the gap with DevSecOps, the task remains the same: How do you ensure that security best practices are understood, architected for and integrated into your application from day 1 AND remain relevant year 1. During this talk I’ll focus on how to achieve these goals amidst the ever changing landscape of people, process, and technology in the cloud, in the context of various compute environments like instances, containers and serverless functions. and how to do so using off-the-shelf AWS services and features. I’ll complete the story by accompanying this discussion with a reference application architecture and examples. Attendees of this talk will receive actionable best practices and guidance, with specific implementation details for AWS
The Emergent Cloud Security Toolchain for CI/CDJames Wickett
The Emergent Cloud Security Toolchain for CI/CD given at RSA Conference 2018 in San Francisco.
All organizations want to go faster and decrease friction in their cloud software delivery pipeline. Infosec has an opportunity to change their classic approach from blocker to enabler. This talk will discuss hallmarks of CI/CD and some practical examples for adding security testing across different organizations. The talk will cover emergent patterns, practices and toolchains that bring security to the table.
Learning Objectives:
1: Learn the emerging patterns for security in CI/CD pipelines.
2: Receive a pragmatic security toolchain for CI/CD to use in your organization.
3: Understand the real meaning of DevSecOps is without all the hype.
Devops: Who Does What? - Devops Enterprise Summit 2016cornelia davis
Within the IT organizational structures that have dominated the last several decades roles and responsibilities are fairly standardized. But with the dramatic changes that DevOps practices and supporting toolsets bring, many are left feeling a bit off balance - it’s no longer clear who is responsible for even things as “straight-forward” as development or operations.
In this talk I will take traditional roles that are distributed across fairly standard IT structures and sort them into a new organizational context. What is the role of the Enterprise Architect? Who does capacity planning and how? How can change management step out of the way all while still satisfying the requirements of safe deployments? How do agile teams interface with personnel responsible for maintaining legacy systems? I’ll leave the audience with a blueprint for a new organizational structure.
Cloud Native: Designing Change-tolerant Softwarecornelia davis
To see this presentation given live, go to http://bit.ly/DesignPatternsReplay
There is a special (discount) offer in there! :-)
Cloud-native applications are characterized by highly distributed topologies consisting of many relatively small components (yup, usually called microservices). But the thing that sets them apart even more from the previous generation of apps is that they are expected to function flawlessly even while the environment they are running in is constantly changing, or even failing.
All of this requires applying a new set of design patterns and practices and this webinar will introduce the most important ones. The Twelve Factor App (12factor.net) is a high-level articulation of some of these techniques that you may well have heard of, but its descriptions are relatively dense and the industry knowledge has evolved a fair bit since its publication.
Cornelia Davis, Senior Director of Technology at Pivotal, will share best practices for cloud-native applications and clear some of the mystery that shrouds 12-factor today. At the conclusion, attendees will understand what is needed for cloud-native applications, as well as why and how to deliver on those requirements.
Just about all of my current technical content in one 364 slide mega-deck. Source files at https://github.com/adrianco/slides
Sections on:
Scene Setting
State of the Cloud
What Changes?
Product Processes
Microservices
State of the Art
Segmentation
What’s Missing?
Monitoring
Challenges
Migration
Response Times
Serverless
Lock-In
Teraservices
Wrap-Up
Microxchg Analyzing Response Time Distributions for MicroservicesAdrian Cockcroft
Research oriented presentation @Microxchg Berlin Feb 5th 2016. New code to collect histograms of response time and export them to monte-carlo simulation spreadsheet via getguesstimate.com
Updated slides for 2016 presentation on innovation in large organizations, why microservices and Docker can be useful, thoughts on monitoring for large complex architectures, some discussion of new topics - serverless architectures AWS Lambda and teraservices.
Microservices: What's Missing - O'Reilly Software Architecture New YorkAdrian Cockcroft
Assuming you have already figured out microservices, what else do you need to figure out to get them to work properly. This talk skips my usual introduction to why and what, and goes deeper on how.
There are many ways to manage whether a service can talk to another service. It can be tempting to over-use one segmentation mechanism to implement policy when the real problem is how to coordinate and manage many mechanisms in the physical, cloud and container spaces. This talk summarizes the problem space and opportunities rather than offers solutions.
Presented at the Docker Palo Alto meetup Feb 16th 2016 http://www.meetup.com/Docker-Palo-Alto/events/228277181/
Discussion of how microservices are being applied across both web scale and enterprise/government use cases to help speed up development.
Video available at http://www.ustream.tv/recorded/86151804
Full slide deck for day long discussion of microservices topics. Why use microservices, what options exist and how to migrate to them and address common problems.
Historical view of process and channel oriented programming idioms: CSP 1978, Occam 1983, Pi-Calculus 1993 etc. How they map to Go and some examples of dynamic channel routing using Go to simulate peer-to-peer networks and microservices networks.
Speakers:
Abner Germanow, Sr. Director Strategic Marketing, New Relic
Michael Michael, Sr. Director of Product Management, Apprenda
Hybrid clouds are helping the enterprise achieve greater agility and scalability. But challenges remain.
Application performance issues and friction between ops teams and developers continue, while businesses struggle to enhance their customer experience, reduce time to market, and improve IT manageability.
In this webinar, see how Apprenda’s integration with New Relic can help you:
Get your apps up and running in minutes
Gain valuable insights into performance and availability
Have complete visibility and control over your apps, while adhering to standards set by IT
Deliver a great end-user experience
Make smarter business decisions based on real data
About Apprenda
Apprenda is the leading enterprise Platform as a Service (PaaS) powering the next generation of enterprise software development in public, private and hybrid clouds. With Apprenda, enterprises can securely deliver an entire ecosystem of data, services, applications and APIs to both internal and external customers across any infrastructure.
Containers are lightweight, fast, agile, and solve all dependency issues related to the applications. With such benefits, why aren’t we using containers for stateful apps like databases? Unfortunately, most container platforms treat stateful apps just like stateless, often focusing just on initial provisioning, scale, or failover. To use the cattle vs pets analogy, stateful apps like databases are pets and not cattle. This means losing an instance of a database can lead to data loss, scaling them is not simple, and restoring does not imply simply bringing up another instance on a different host.
In this session, we will review the current state of container support for databases and discuss key aspects of database management like consolidation, data management, node failures, scaling out/back and scaling up/down, etc that container platforms need to address.
1. Reactive:
This kind of monitoring can be achived by the orchestration engine updates the monitoring system.
Example:
Puppet: if any changes to configuration happens it revert back to the actual configuration which management config needs
2. Proactive:
This kind of monitoring can be achived by adding precautionary measures for the known issues, where, if the issue occurs it immidiately starts the precaution to eradicate the fault.
3. Adaptive:
This is better suited for monitoring a frequently changing system like docker containers, as it can adapt itself to the micro services that get intorduced into the containers. Now the question is “Is the adaptive montoring a full solution to the abovementioned challenges?"
- Answer is “NO”
- We need solutions at different levels
Going Cloud Native - It Takes a PlatformChip Childers
There is a lot of buzz about “cloud native.” Becoming cloud native means changing how we think about, develop, and deploy applications. This shift impacts the structure of organizations, as teams align to common business outcomes.
Chip Childers explains why a successful cloud native approach requires a platform: a platform allows you to rethink how IT supports the application development teams. Chip explores what it means to be truly cloud native, what it takes to get there, and how a platform can make it all work.
Keynote at Dockercon Europe Amsterdam Dec 4th, 2014.
Speeding up development with Docker.
Summary of some interesting web scale microservice architectures.
Please send me updates and corrections to the architecture summaries @adrianco
Thanks Adrian
Adrian Cockcroft on his top predictions for the cloud computing industry in 2015 and beyond, as well as how cloud-native applications, continuous-delivery and DevOps techniques, will speed the pace of innovation and disruption.
For more about Adrian be sure to check out his page on Battery Ventures:
https://www.battery.com/our-team/member/adrian-cockcroft/
Follow Adrian on Twitter: @adrianco
Tampere Docker meetup - Happy 5th Birthday DockerSakari Hoisko
Part of official docker meetup events by Docker Inc.
https://events.docker.com/events/docker-bday-5/
Meetup event:
https://www.meetup.com/Docker-Tampere/events/248566945/
Using the SDACK Architecture on Security Event InspectionYu-Lun Chen
The SDACK architecture stands for Spark, Docker, Akka, Cassandra, and Kafka. At TrendMicro, we adopted the SDACK architecture to implement a security event inspection platform for APT attack analysis. In this talk, we will introduce SDACK stack with Spark lambda architecture, Akka and Kafka for streaming data pipeline, Cassandra for time series data, and Docker for microservices. Specifically, we will show you how we Dockerize each SDACK component to facilitate the RD team of algorithms development, help the QA team test the product easily, and use the Docker as a Service strategy to ship our products to customers. Next, we will show you how we monitor each Docker container and adjust the resource usage based on monitoring metrics. And then, we will share our Docker security policy which ensures our products are safety before shipping to customers. After that, we'll show you how we develop an all-in-one Docker based data product and scale it out to multi-host Docker cluster to solve the big data problem. Finally, we will share some challenges we faced during the product development and some lesson learned.
Webinar by ZNetLive & Plesk- Winning the Game for WebOps and DevOps ZNetLive
This webinar presentation illustrates everything that the experts of Plesk & ZNetLive discussed about the opportunities in WebOps and DevOps market along with new features of ZNetLive's Managed WordPress hosting and Cloud VPS, both powered by Plesk Onyx.
To know more, visit- https://www.znetlive.com/
Vasiliy Fomichev - Harness the Power of Containers - SUGCONSUGCON
Get All Your Mongo and Solr Environments under Control: Sitecore xDB has introduced another layer of maintenance complexity – Mongo database. Many .NET developers have only heard about it, let alone, never set it up. Add a Solr Cloud configuration on top of that, and we have a recipe for a huge headache.
The new complexity introduces a learning curve, and a possibility for a wider range of issues:
1. The need for xDB and Solr configurations locally increases the time and complexity of local development environments
2. The higher number of user systems and technologies increases the chances of unforeseen errors due to systems getting out of sync, resulting in issues that are hard to troubleshoot
3. The need for reproducing the same initial setup in many different environments (development, QA, UAT, Staging, Production…etc.) makes that much more room for human error, introducing configuration discrepancies
Bahrain ch9 introduction to docker 5th birthday Walid Shaari
A hands-on workshop will go over the foundations of the containers platform, including an overview of the platform system components: images, containers, repositories, clustering, and orchestration. The strategy is to demonstrate through "live demo, and hands-on exercises." The reuse case of containers in building a portable distributed application cluster running a variety of workloads including HPC workload.
Session 3962: Docking DevOps was originally presented at IBM InterConnect 2015 Feb. 22 - 26, 2016.
The presentation explores the values of Docker and containers and provides insight into areas that IBM has embraced the use of Docker within it's cloud strategy.
The pillars of DevOps are Culture, Automation, Measurement and Sharing. Docker is a rare tool at enables DevOps through all 4 pillars. These slides take a look at how Docker can affect each pillar in your organization through a Lean lens.
Docker Bday #5, SF Edition: Introduction to DockerDocker, Inc.
In celebration of Docker's 5th birthday in March, user groups all around the world hosted birthday events with an introduction to Docker presentation and hands-on-labs. We invited Docker users to recognize where they were on their Docker journey and the goal was to help them take the next step of their journey with the help of mentors. This presentation was done at the beginning of the events (this one is from the San Francisco event in HQ) and gives a run down of the birthday event series, Docker's momentum, a basic explanation of containers, the benefits of using the Docker platform, Docker + Kubernetes and more.
DevOps on AWS: Deep Dive on Continuous Delivery and the AWS Developer ToolsAmazon Web Services
Today’s cutting-edge companies have software release cycles measured in days instead of months. This agility is enabled by the DevOps practice of continuous delivery, which automates building, testing, and deploying all code changes. This automation helps you catch bugs sooner and accelerates developer productivity. In this session, we’ll share the processes that Amazon’s engineers use to practice DevOps and discuss how you can bring these processes to your company by using a new set of AWS tools (AWS CodeCommit, AWS CodePipeline, and AWS CodeDeploy). These services were inspired by Amazon's own internal developer tools and DevOps culture.
DCEU 18: How To Build Your Containerization StrategyDocker, Inc.
Lee Namba - EMEA Professional Services Manager, Docker
The Docker Enterprise container platform helps organizations deploy and manage applications faster and it secures the application pipeline at a lower cost than traditional application delivery models. But it takes more than just great technology to achieve the desired results. The organization and culture of your enterprise directly impacts what you transform, how it’s done, and who does it. Success requires a strategy for how you will govern the container platform environment, how to assess your application estate, what your delivery pipeline will look like, and how to ensure developers, operators, security teams and others play nicely together. In this talk I will cover topics such as different types of workloads (legacy, microservices, FaaS, big data and more), how your org chart can influence whether you deploy CaaS (Containers as a Service) vs CLaaS (Clusters as a Service), how "shifting left" can determine if you can outsource, centralized vs distributed CI/CD and how containers play a role, transforming your pets into cattle, how giant whale balloons are used for onboarding, and a prescriptive and comprehensive methodology for successfully deploying containers into your enterprise.
The Docker Way: modernize traditional applications without action (wu-wei) and create new cloud native microservices applications with naturalness (ziran).
This talk also provides a summary of all the DockerCon EU 2017 announcements: Kubernetes now supported in Docker, MTA, IBM partnership.
Application Centric Microservices from Redhat Summit 2015Ken Owens
When Cisco started envisioning the future of its application development platforms, the ability to create applications that are cloud-native with elastic services, network-aware application policies, and micro-services was strategic to the company. When the decision to build and operate a Cisco cloud service delivery platform for collaboration, video, and Internet of Things (IoT) application development was made, OpenStack and micro-services became central to our application architectures and strategic to our vision as a company. This presentation will look at the journey Cisco developers took to transform to an application-centric OpenStack platform for application development in a secure, network-centric, and completely open source manner. The importance of the platform being Red Hat Enterprise Linux OpenStack Platform and using OpenShift by Red Hat and the contribution to the community will be described. The micro-services architecture and service-oriented DevOps lessons learned for enabling massive scalable and continuous delivery of software will be presented and demoed.
Accelerate Your Automation Testing Effort using TestProject & Docker | Docker...Ajeet Singh Raina
Accelerate Your Test Automation using TestProject & Docker
A recording of a live webinar hosted on May 17th, 2020 - Learn from Docker Captain Ajeet Singh Raina how you can leverage TestProject Agents running in Docker containers, easily setup virtual labs & run tests in parallel.
- Introduction to TestProject
- Why TestProject for Automation?
- A Brief about TestProject Agents
- Why Docker?
- Why run TestProject Agents inside Docker containers?
- Live Demo
- Game with Prizes!
Monitorama - Please, no more Minutes, Milliseconds, Monoliths or Monitoring T...Adrian Cockcroft
Monitorama opening keynote talk on the challenges of Monitoring in a world where we need to deal with continuous delivery, cloud, and automated control feedback loops.
Hack Kid Con - Learn to be a Data Scientist for $1Adrian Cockcroft
Attempt to inspire some kids to pay attention in Math and Science classes so they can get a good job and help fill the skills gap in the years to come.
Slides originally written in April 2013 for a private conference and internal use at Netflix. Publishing now since Heartbleed is another example of an epidemic failure mode.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
3. Why am I here?
%*&!”
By Simon Wardley http://enterpriseitadoption.com/
4. Why am I here?
%*&!”
By Simon Wardley http://enterpriseitadoption.com/
2009
5. Why am I here?
%*&!”
By Simon Wardley http://enterpriseitadoption.com/
2009
6. Why am I here?
@adrianco’s job at the
intersection of cloud
and Enterprise IT,
looking for disruption
and opportunities.
%*&!”
By Simon Wardley http://enterpriseitadoption.com/
2014
2009
7. Why am I here?
@adrianco’s job at the
intersection of cloud
and Enterprise IT,
looking for disruption
and opportunities.
%*&!”
By Simon Wardley http://enterpriseitadoption.com/
2014
2009
Example: Docker
wasn’t on anyone’s
roadmap for 2014. It’s
on everyone’s roadmap
for 2015.
8. 2014 was the year that Enterprises
finally embraced cloud and DevOps.
9. 2014 was the year that Enterprises
finally embraced cloud and DevOps.
10. 2014 was the year that Enterprises
finally embraced cloud and DevOps.
11. 2014 was the year that Enterprises
finally embraced cloud and DevOps.
12. What does @adrianco do?
@adrianco
Technology Due
Diligence on Deals
Presentations at
Conferences
Presentations at
Companies
Technical Advice
for Portfolio
Companies
Program
Committee for
Conferences
Networking with
Interesting PeopleTinkering with
Technologies
Maintain
Relationship with
Cloud Vendors
28. Breaking Down the SILOs
QA DBA
Sys
Adm
Net
Adm
SAN
Adm
DevUX
Prod
Mgr
29. Breaking Down the SILOs
QA DBA
Sys
Adm
Net
Adm
SAN
Adm
DevUX
Prod
Mgr
Product Team Using Monolithic Delivery
Product Team Using Monolithic Delivery
30. Breaking Down the SILOs
QA DBA
Sys
Adm
Net
Adm
SAN
Adm
DevUX
Prod
Mgr
Product Team Using Microservices
Product Team Using Monolithic Delivery
Product Team Using Microservices
Product Team Using Microservices
Product Team Using Monolithic Delivery
31. Breaking Down the SILOs
QA DBA
Sys
Adm
Net
Adm
SAN
Adm
DevUX
Prod
Mgr
Product Team Using Microservices
Product Team Using Monolithic Delivery
Platform TeamProduct Team Using Microservices
Product Team Using Microservices
Product Team Using Monolithic Delivery
32. Breaking Down the SILOs
QA DBA
Sys
Adm
Net
Adm
SAN
Adm
DevUX
Prod
Mgr
Product Team Using Microservices
Product Team Using Monolithic Delivery
Platform Team
A
P
I
Product Team Using Microservices
Product Team Using Microservices
Product Team Using Monolithic Delivery
33. Breaking Down the SILOs
QA DBA
Sys
Adm
Net
Adm
SAN
Adm
DevUX
Prod
Mgr
Product Team Using Microservices
Product Team Using Monolithic Delivery
Platform Team
DevOps is a Re-Org!
A
P
I
Product Team Using Microservices
Product Team Using Microservices
Product Team Using Monolithic Delivery
38. Developer
Developer
Developer
Developer
Developer
Old Release Still
Running
Release Plan
Release Plan
Release Plan
Release Plan
Deploy
Feature to
Production
Deploy
Feature to
Production
Deploy
Feature to
Production
Deploy
Feature to
Production
Immutable microservice deployment
scales, is faster with large teams and
diverse platform components
39. Developer
Developer
Developer
Developer
Developer
Old Release Still
Running
Release Plan
Release Plan
Release Plan
Release Plan
Deploy
Feature to
Production
Deploy
Feature to
Production
Deploy
Feature to
Production
Deploy
Feature to
Production
Bugs
Immutable microservice deployment
scales, is faster with large teams and
diverse platform components
40. Developer
Developer
Developer
Developer
Developer
Old Release Still
Running
Release Plan
Release Plan
Release Plan
Release Plan
Deploy
Feature to
Production
Deploy
Feature to
Production
Deploy
Feature to
Production
Deploy
Feature to
Production
Bugs
Deploy
Feature to
Production
Immutable microservice deployment
scales, is faster with large teams and
diverse platform components
44. Developer Developer
Run What You Wrote
Micro
service
Micro
service
Micro
service
Micro
service
Micro
service
Micro
service
Micro
service
Developer Developer
45. Developer Developer
Run What You Wrote
Micro
service
Micro
service
Micro
service
Micro
service
Micro
service
Micro
service
Micro
service
Developer Developer
Monitoring
Tools
46. DeveloperDeveloper Developer
Run What You Wrote
Micro
service
Micro
service
Micro
service
Micro
service
Micro
service
Micro
service
Micro
service
Developer Developer
Monitoring
Tools
47. DeveloperDeveloper Developer
Run What You Wrote
Micro
service
Micro
service
Micro
service
Micro
service
Micro
service
Micro
service
Micro
service
Developer Developer
Site
Reliability
Monitoring
Tools
Availability
Metrics
99.95% customer
success rate
48. DeveloperDeveloper Developer
Run What You Wrote
Micro
service
Micro
service
Micro
service
Micro
service
Micro
service
Micro
service
Micro
service
Developer Developer
Manager Manager
Site
Reliability
Monitoring
Tools
Availability
Metrics
99.95% customer
success rate
49. DeveloperDeveloper Developer
Run What You Wrote
Micro
service
Micro
service
Micro
service
Micro
service
Micro
service
Micro
service
Micro
service
Developer Developer
Manager Manager
VP
Engineering
Site
Reliability
Monitoring
Tools
Availability
Metrics
99.95% customer
success rate
51. Developing at the Speed of Docker
Developers
• Compile/Build
• Seconds
Extend container
• Package dependencies
• Seconds
PaaS deploy Container
• Docker startup
• Seconds
52. Developing at the Speed of Docker
Speed is addictive, hard to go back to taking much longer to get things done
Developers
• Compile/Build
• Seconds
Extend container
• Package dependencies
• Seconds
PaaS deploy Container
• Docker startup
• Seconds
57. A Microservice Definition
!
Loosely coupled service oriented
architecture with bounded contexts
If every service has to be
updated at the same time
it’s not loosely coupled
58. A Microservice Definition
!
Loosely coupled service oriented
architecture with bounded contexts
If every service has to be
updated at the same time
it’s not loosely coupled
If you have to know too much about surrounding
services you don’t have a bounded context. See the
Domain Driven Design book by Eric Evans.
60. Non-Destructive Production Updates
● “Immutable Code” Service Pattern
● Existing services are unchanged, old code remains in service
● New code deploys as a new service group
● No impact to production until traffic routing changes
● A|B Tests, Feature Flags and Version Routing control traffic
● First users in the test cell are the developer and test engineers
● A cohort of users is added looking for measurable improvement
● Finally make default for everyone, keeping old code for a while
61. Speeding Up The Platform
Datacenter Snowflakes
• Deploy in months
• Live for years
62. Speeding Up The Platform
Datacenter Snowflakes
• Deploy in months
• Live for years
Virtualized and Cloud
• Deploy in minutes
• Live for weeks
63. Speeding Up The Platform
Datacenter Snowflakes
• Deploy in months
• Live for years
Virtualized and Cloud
• Deploy in minutes
• Live for weeks
Docker Containers
• Deploy in seconds
• Live for minutes/hours
64. Speeding Up The Platform
Datacenter Snowflakes
• Deploy in months
• Live for years
Virtualized and Cloud
• Deploy in minutes
• Live for weeks
Docker Containers
• Deploy in seconds
• Live for minutes/hours
AWS Lambda
• Deploy in milliseconds
• Live for seconds
65. Speeding Up The Platform
Speed enables and encourages new microservice architectures
Datacenter Snowflakes
• Deploy in months
• Live for years
Virtualized and Cloud
• Deploy in minutes
• Live for weeks
Docker Containers
• Deploy in seconds
• Live for minutes/hours
AWS Lambda
• Deploy in milliseconds
• Live for seconds
66. With AWS Lambda
compute resources are charged
by the 100ms, not the hour
First 1M node.js executions/month are free
84. Node.js Microservices
Several different approaches
!
Mostly small simple microservices
!
Focus on easy interface with
presentation code in javascript
!
AWS Lambda - preview only
http://senecajs.org/
http://aws.amazon.com/lambda/
85. Adrian’s Tinkering Projects
Model and visualize microservices
Simulate interesting architectures
Generate large scale configurations
Eventually stress test real tools
!
See github.com/adrianco/spigo
Simulate Protocol Interactions in Go
Visualize with D3
ELB Load Balancer
Zuul API Proxy
Karyon
Business
Logic
Staash
Data
Access
Layer
Priam Cassandra
Datastore
Three
Availability
Zones
86. What is Spigo?
Creates and animates microservices
Single Go program on this laptop
Generates 100,000+ instances
About 250,000 messages/sec
Uses Go channels rather than http
Supports social network architecture
Supports NetflixOSS architecture
Simple code patterns to extend
87. Why Build Spigo?
Generate test microservice configurations at scale
Stress monitoring tools display capabilities
!
Eventually (i.e. not implemented yet)
Dynamically vary configuration: autoscale, code push
Simulate microservice, zone, region failures
D3 websocket dynamic browser interface
Timescale: Monitorama Conference June 2015
88. Spigo Nanoservice Structure
func Start(listener chan gotocol.Message) {
...
for {
select {
case msg := <-listener:
switch msg.Imposition {
case gotocol.Hello:
...
case gotocol.NameDrop:
...
case gotocol.Chat:
...
case gotocol.GetResponse:
...
case gotocol.Goodbye:
gotocol.Message{gotocol.Goodbye, nil, time.Now(), name}.GoSend(netflixoss)
return
}
case <-chatTicker.C:
...
}
}
}
90. Web Scale Characteristics
●Brand new Microservices are deployed infrequently
●New versions deployed automatically/frequently
●No real need for general purpose orchestration
●Architectures use hundreds of microservices
●Each deployment is heavily customized
91. Orchestration for Applications
●Standard portable microservice based applications
●New versions deployed automatically/frequently
●Orchestration automated and standardized
●Architectures likely based on tens of microservices
●Opportunity: Docker Hub as the enterprise app store
98. Any Questions?
Disclosure: some of the companies mentioned may be Battery Ventures Portfolio Companies
See www.battery.com for a list of portfolio investments
● Battery Ventures http://www.battery.com
● Adrian’s Tweets @adrianco and Blog http://perfcap.blogspot.com
● Slideshare http://slideshare.com/adriancockcroft
!
● Monitorama Opening Keynote Portland OR - May 7
th
, 2014
● GOTO Chicago Opening Keynote May 20
th
, 2014
● Qcon New York – Speed and Scale - June 11
th
, 2014
● Structure - Cloud Trends - San Francisco - June 19th, 2014
● GOTO Copenhagen/Aarhus – Fast Delivery - Denmark – Sept 25
th
, 2014
● DevOps Enterprise Summit - San Francisco - Oct 21-23rd, 2014 #DOES14
● GOTO Berlin - Migrating to Microservices - Germany - Nov 6th, 2014
● AWS Re:Invent - Cloud Native Cost Optimization - Las Vegas - November 14th, 2014
● O’Reilly Software Architecture Conference - Fast Delivery - Boston March 16th 2015