Creating Packages that Run Anywhere with Chef Habitat
•Download as PPTX, PDF•
2 likes•697 views
See an intro to the Habitat supervisor and packaging system, with a particular focus on what you can do with Habitat TODAY. See a demo of building a Linux package on a Windows workstation, building a Windows package on a Windows workstation, and creating and running a Docker container image from a Habitat package. You can see the demos at these YouTube links Running a Linux Package with Habitat Supervisor: https://www.youtube.com/watch?v=uw-mWBklmFg Creating a Linux Package on a Windows Workstation: https://www.youtube.com/watch?v=uaIe2RuIC4Y Creating a Windows Package on a Windows Workstation: https://www.youtube.com/watch?v=01s3t-S_ae4 Creating and running a Docker Image from a Habitat Package: https://www.youtube.com/watch?v=tILvSfGEO0I
Recommended
More Related Content
What's hot
What's hot (7)
Viewers also liked
Viewers also liked (20)
Similar to Creating Packages that Run Anywhere with Chef Habitat
Similar to Creating Packages that Run Anywhere with Chef Habitat (20)
More from Nell Shamrell-Harrington
More from Nell Shamrell-Harrington (20)
Recently uploaded
Recently uploaded (20)
Creating Packages that Run Anywhere with Chef Habitat
- 1. OPEN335 Nell Shamrell-Harrington Platform Agnosticism with Habitat
- 3. What if you could build software once…
- 4. What if you could build software once… …
- 5. What if you could build software once… …
- 6. What if you could build software once… …
- 7. What if you could build software once… …
- 8. What if you could move a legacy application into the cloud…
- 9. What if you could move a legacy application into the cloud…
- 10. What if you could empower your applications to recover from failure…
- 11. What if you could empower your applications to recover from failure…
- 12. You can!
- 13. Habitat is Application Automation
- 14. Who am I?
- 15. Why am I here?
- 16. Microsoft <3’s Open Source!
- 17. Applications Management App Frameworks Databases & Middleware Infrastructure Linux
- 18. Our Approach to Open Source in the Cloud Integrate Embrace leading Open Source ecosystems and integrate Microsoft products with agility and consistency Release Release key Microsoft technologies into the Open Source domain to build a strong ecosystem Participate Microsoft engineers to participate in communities and contribute to key Open Source projects Enable Enable Linux and Open Source technology to be first class citizens on Microsoft Platforms Open Source Partners & Ecosystem R Server .NET Core Roslyn TypeScript F# PowerShell autorest PowerBI Visuals Office UI Fabric Tools plugins
- 21. Building and running software is painful!
- 22. Building and running software is painful
- 23. Building and running software is painful
- 24. Building and running software is painful
- 25. Habitat builds and runs modern applications
- 31. Building a modern application
- 32. Building a modern application Source Code Repo
- 33. Building a modern application Artifact Source Code Repo
- 34. Building a modern application Artifact Artifact Repo Source Code Repo
- 35. Building a modern application Artifact Bare Metal Container Instance VM Artifact Repo Source Code Repo
- 37. Habitat is a new technology to build, deploy, and manage applications…
- 38. …in any environment from traditional datacenters to containerized microservices
- 39. In Habitat, the application is the unit of automation
- 40. How does it work?
- 41. Packaging an Application with Habitat
- 42. USER Packaging an Application with Habitat
- 43. USER PLAN Packaging an Application with Habitat
- 44. USER ARTIFACTPLAN Packaging an Application with Habitat
- 45. Packaging an Application with Habitat USER ARTIFACTPLAN DEPOT
- 47. Running an Application with Habitat
- 48. Running an Application with Habitat DEPOT
- 49. Running an Application with Habitat ARTIFACTDEPOT
- 50. Running an Application with Habitat ARTIFACTDEPOT SERVICE SUPERVISOR BARE METAL SERVICE SUPERVISOR CONTAINERS SERVICE SUPERVISOR AMI SERVICE SUPERVISOR VM
- 51. Running an Application with Habitat SERVICE SUPERVISOR VM API
- 52. Running an Application with Habitat SERVICE SUPERVISOR VM API Load Balancer
- 53. Running an Application with Habitat SERVICE SUPERVISOR VM SERVICE SUPERVISOR VM SERVICE SUPERVISOR VM SERVICE SUPERVISOR VM
- 54. Running an Application with Habitat SERVICE SUPERVISOR VM SERVICE SUPERVISOR VM SERVICE SUPERVISOR VM SERVICE SUPERVISOR VM
- 55. Habitat has many parts – many of which are still in development
- 56. I’m here to talk about what you can do with Habitat today
- 57. And together we can shape where it goes in the future
- 58. What we will discuss today
- 59. What we will discuss today
- 60. What we will discuss today
- 76. SERVICE SUPERVISOR VM Habitat supervisor Also running MySQL SERVICE SUPERVISOR VM SERVICE SUPERVISOR VM
- 78. SERVICE SUPERVISOR VM Habitat supervisor SERVICE SUPERVISOR VM SERVICE SUPERVISOR VM Let’s elect a leader!
- 79. SERVICE SUPERVISOR VM Habitat supervisor SERVICE SUPERVISOR VM SERVICE SUPERVISOR VM Leader – receives write requests
- 80. SERVICE SUPERVISOR VM Habitat supervisor SERVICE SUPERVISOR VM SERVICE SUPERVISOR VM Leader – receives write requests Followers – receive read requests
- 82. SERVICE SUPERVISOR VM Habitat supervisor SERVICE SUPERVISOR VM SERVICE SUPERVISOR VM Cannot connect
- 84. Habitat supervisor SERVICE SUPERVISOR VM SERVICE SUPERVISOR VM Let’s elect a leader!
- 85. Habitat supervisor SERVICE SUPERVISOR VM SERVICE SUPERVISOR VM Leader – receives write requests
- 86. Habitat supervisor SERVICE SUPERVISOR VM SERVICE SUPERVISOR VM Follower – receives read requests
- 87. Habitat assumes that failures will happen and are normal
- 88. Healthy components will self-organize and re-converge
- 96. How does the Supervisor get something to supervise?
- 98. Habitat packages
- 99. Habitat packages
- 100. Habitat packages
- 102. Habitat plans use Bash for Packages that will run on Linux
- 103. Habitat can also create plans for Windows using Powershell (Preliminary)
- 105. Along with running hart packages with Habitat…
- 106. You can also export them to other formats
- 107. Including container images, by far the most popular is Docker
- 109. Getting artifacts to run anywhere is very difficult
- 110. Containers were supposed to solve this problem, but…
- 114. Current Container Pain Libraries Operating System Application Application Libraries
- 115. Containers with Habitat Libraries Operating System Application Application & Libraries OS
- 119. When you create a container image with Habitat…
- 120. You know exactly what went into the container and what is configurable about the container
- 122. Deploying Containers with Habitat
- 123. Deploying Containers with Habitat
- 124. Deploying Containers with Habitat
- 128. Look for a blog post on using Habitat with Azure Container Services coming soon!
- 130. You can build software once and run it (almost) anywhere
- 131. You can move a legacy application into the cloud (without rewriting it)
- 132. You can empower your applications to recover from failure on their own…
- 133. You can do all these things with
- 134. Habitat is Application Automation
- 135. Why am I here?
- 136. How do I get involved?
- 137. Who am I?
- 138. Who am I? Thank You!
- 139. Who am I? Any Questions?
Editor's Notes
- I’d like to start with a question…
- I have news for you today
- I have news for you today
- Without the need for a central command and control structure? I have news for you today
- With Chef Habitat – a new Open source project from Chef.
- Habitat goes beyond infrastructure automation, beyond cloud automation, and into the world of application automation. We’ll go deeper into what the means in just a moment, but first…
- Let’s cover who I am
- Reason I was asked to speak here today, in the Open Source track…
- I’d like to invite Ken Thompson of Microsoft to say a few words about Microsoft’s commitment to Open Source. Thanks Ken! Now you know that Habitat is an Open Source project, and that Microsoft is committed to Open Source in the industry. The next thing we need to cover what exactly Habitat is.
- Notes coming soon – please check back on http://aka.ms/opensource for latest version
- Thanks Ken! Let’s bring it back to the Habitat open source project in particular and let’s start discussing what, exactly, it is.
- But to do that – we first need to understand the “why” of Habitat.
- The reason Habitat was created is simple – building and running software is very painful
- For example, you need one runtime for a Ruby application and a different runtime for a Java application
- You need different packages for Debian Linux vs. Redhat Linux and I don’t even want to go into the difference in packages between Linux and Windows.
- And you need different types of packages for bare metal VMs and container images. Habitat seeks to alleviate this pain…
- By transforming applications – including legacy applications – into modern applications.
- What I mean by a modern application is that it….
- Reduces complexity in automating the application, rather than making it more complex
- In today’s world, having an application running on one server in your closet is not enough, it needs to be able to scale rapidly.
- How you build a modern application starts with…
- Storing all code – including the application code and all automation code – in the same source code repository.
- We then use that store code to build an artifact – or a software package – that package is then sent to
- And artifact repository. This is a place where you can store software artifacts, find other artifacts, and download those artifacts.
- The artifacts in that repo can then be deployed to a bare metal server, a container, a cloud instance, or a VM, without needing to change the artifact. That artifact can run anywhere. Habitat was designed with this workflow at the forefront. If you are familiar with the 12 factor app, that is exactly the workflow we are embracing. This is the “why” of Habitat…now that we know the WHY
- Now, let’s cover WHAT Habitat is.
- Briefly, Habitat is a new technology to build, deploy, and manage applications
- That run in ANY environment – from traditional datacenters to containerized microservice – from the legacy infrastructure of 20 years ago to the most bleeding edge infrastructure innovations of today.
- The reason for this is that in Habitat the application itself is the unit of automation. That application package we create contains everything needed to deploy, run, and maintain the application.
- And that leads us to this question – how exactly does this work? The clearest way to explain this is to look at the workflows for packaging and running an application with Habitat.
- Packaging an application with Habitat starts with…
- The user – this is you at your workstation. It doesn’t matter if you are using Windows, Mac, or Linux, Habitat works on all of them.
- And on your workstation you are going to create a plan. This contains instructions on how the application should be set up wherever it is deployed. When you are creating a package for a Linux system, you would write this plan in bash. If it’s for a Windows system – and we will see a demo of this in just a bit – you would write that in Powershell. The powershell functionality is not quite production ready yet, but it will be soon.
- Then you use that plan and the compiled application code to create an artifact that contains everything in one place. That artifact is cryptographically signed with a key so you can verify that the artifact came from the place you expected it to come from. We also use this key when we run the application, and I’ll show you that in just a moment.
- And then you can optionally upload that artifact to the public Habitat depot – where you can find Habitat packages by developers all over the world.
- And this is what that public depot looks like –you can access it at app.habitat.sh. In this screencap I was searching for a node Habitat package and there are several available, we’ll use the core/node one down there at the bottom in a demo later in this talk. So that is how you package an application with Habitat…
- Now let’s go through how you would run that application with Habitat. This is where the magic happens.
- If you have your application artifact on the depot, you can find it on that depot.
- And pull that artifact from that depot…
- Onto wherever you want to run it – whether bare metal, containers, a machine image, or a VM. If you are not using the depot, you can also upload it to wherever you want to run it through scp or whatever method you prefer. You then run that package as a service within the supervisor – we’ll go into what I mean by a service and how it interacts with the supervisor later.
- Once you have that application running somewhere – in this example it’s a VM – with the supervisor, you can still get information in and out of it of the application through a restful API.
- And this is really useful if you have something like a load balancer that needs to send traffic to that application, run health checks, and more. No matter where that application is running, you can still get information into and out of it.
- The real magic of the supervisor comes when you have more than one instance of an application running – let’s say we have four VMS all running the same package. The supervisors on each of these VMs form a ring. They will use that key we used to sign the package to decide whether to allow another VM into the ring. They all have to be signed with the same key in order to communicate with each other
- They do this communication over an encrypted GOSSIP protocol which they can then use to self organize into different topologies – we’ll cover those topologies a little later.
- When I last spoke to Jamie Winsor – one of the creators of Habitat – he described it as an umbrella over many components, all designed to allow you to build software once and run it anywhere. Many of these components are still in development. Even though they are very promising…
- I’m here to talk about what you can do with Habitat today – how you can create and run application packages that can run (nearly) anywhere, move existing applications into the cloud, and give your applications the intelligence to recover from failure on their own, without the need for central supervisor or controller.
- I want us to understand what we can do with Habitat today and then together – since this is open source and this project will largely be driven by our community and contributors - we can shape where it goes in the future.
- Today, we will first discuss the Habitat Supervisor – which is what you use to run your application packages. I think this is where the true genius of Habitat shines.
- Then the Habitat packaging format – this is what you use on your workstation to make that software artifact that can run nearly anywhere. This is what you will run with the Supervisor.
- Finally – although you can use Habitat without containers, and we will see that in action, Habitat truly shines when it comes to working with containers. Habitat complements things like Docker and Mesosphere and makes them work even better.
- So let’s start with the Habitat supervisor.
- Briefly – the Habitat Supervisor is an intelligent runtime with deployment coordination and service discovery – we’ll go deeper into what that means in just a moment.
- The supervisor is what allows you to run the application within your hart package natively on HW…
- In a VM or Cloud Instance
- In a container with or without a Container Management Service like Mesosphere.
- One of the main things the supervisor does is act as a process manager…
- And what this means is whenever you pull that HART package onto whatever infrastructure you want to run it on, Habitat will start up and monitor that package.
- Additionally, it will also receive and implement configuration changes. So if you upload a new version of that package to the public depot – say for a security patch – the supervisor will be monitoring that depot and become aware of it, then pull in that new package, install it, and make whatever configuration changes needed for it.
- And it also runs services. Let’s take a closer look by what I mean by services.
- A service is one habitat package running under a supervisor…
- And the simplest example is 1 supervisor running on service on one piece of infrastructure – whether that’s a server, VM or container.
- So one service running would look like this. In this example we have a VM running one service under one supervisor. Let’s take a look of a demo of this simple example.
- The examples we just saw where one supervisor running on one service. But one service is pretty limiting, when we go beyond one service when we scale out, we need a supervisor ring.
- And let’s look at an example of this. Let’s say we start off with one supervisor on a VM running MySQL.
- And we decide we want a MySQL cluster, so we spin up two more VMs and install the MySQL hart package on them. Since that MySQL package on each of them is signed with the same key, they will be allowed to form a supervisor ring.
- What the ring allows these VMs to do is to communicate over each other over a GOSSIP protocol – remember, that communication is all encrypted.
- With a MySql cluster like this, it’s common to use a leader/follower topology. What this means is once we have those three VMS running that MySQL service, they need to elect a leader. Habitat has a built in algorithm for electing a leader in a cluster such as this. And it’s going to run that election algorithm...
- And let’s say this one on the top is elected the leader, that means it will receive the write requests which come to this MySQL cluster. That means…
- The other two VMS are designated as followers, and they receive the read requests that come into that cluster.
- Now let’s say something goes wrong. Something bad happens and the leader goes offline.
- The two other Supervisors will notice this when they cannot connect with the leader over that GOSSIP protocol.
- And they will take it out of that supervisor ring. So now we are down to two VMs, and at the moment both are still followers.
- They will realize that they don’t have a leader, they are not implementing the topology that they have promised to implement. So they will hold another election with that built in election algorithm.
- Let’s say this supervisor wins the election and becomes the leader. It will automatically start receiving write requests.
- And that means this supervisor would be the follower and receive read requests.
- This illustrates that Habitat assumes that failures will happen and that they are normal. We don’t try to anticipate every edge case in the beginning because, frankly, we can’t. There will always be something unforeseen that happens somewhere in an application’s lifecycle.
- The remaining healthy components will self-organize and re-converge on their own. There’s no central coordinate that re-organizes and re-converges them, they have the intelligence to do this on their own.
- The Habitat Supervisor supports two different topologies at this time. We just saw this first one – Leader/Follower.
- And the other topology is the standalone topology – which assumes that every member of the supervisor ring is working as an individual that is in communication with all the others. And these are based on existing standards for IT infrastructure.
- The supervisor is also in charge of handling updates to the package or configuration changes and rolling them out to the rest of the ring.
- It’s going to detect when a new release becomes available on the depot…
- Then deploy it based on a defined update strategy .
- Some of the update strategies that we support or will support soon include all at once – that’s when – whenever a new package becomes available – all the supervisors update to that new package immediately.
- Some strategies coming soon include rollout strategy – that means if we have a supervisor ring with, say, four supervisors – one will update, then the next one will update, then the next one. Only one supervisor is updating at any given time.
- That’s the Supervisor, which is used to supervise and managed applications packaged with Habitat. Let’s now step back a bit and talk about HOW we make that package for the Supervisor to supervise.
- Habitat packages are created through the Habitat Packaging format– this is what you use to create your artifact on your workstation.
- Habitat packages are in a format called the HART format – because we heart you. That stands for Habitat Artifact.
- And these HART packages contain the source code for the application itself – if for example you had a Ruby on Rails application you were automating, you would have all your Ruby on Rails code within this package…
- …along with the application code, these packages also include everything needed to deploy and run the application. This is all kept in one place. Now, enough with the talking about it, let’s see this HART package format in action.
- As we just saw, Habitat plans use Bash for packages that will run on Linux…
- But Habitat can also create plans in Powershell for installing on Windows infrastructure. This is still in the development stages, but I can give you a bit of a preview of building a package for Windows.
- Along with running hart packages with Habitat…
- Habitat also allows you to export your hart packages into other formats.
- And by far the most popular format people export to is a Docker container image.
- Which brings me to Habitat and Containers. Habitat DOES work very well without containers, as we’ve seen, but I think it really shines when we use it WITH containers.
- Getting a software package to run anywhere is very difficult. That’s not a new problem, we all know that.
- And containers were supposed to solve this problem. But…
- There’s still a lot of pain with containers in their current state. Part of this pain is that there is a major learning cliff in between using a container in a development environment and using it in a production environment. Who here uses containers in dev? What about in prod? How long did it take you to get to using them in prod?
- The other major issue is that it is easy for containers to become black boxes – where we deploy them to different environments without fully understanding everything that is inside of them.
- And, among other things, this can cause serious security issues if a container has something in it with a security flaw, but we don’t know or don’t remember to update it.
- Part of the reason for this is traditional containers start by building the operating system, then add in the libraries, then the application libraries, and then finally the application itself at the very end. This adds bloat and complexity to containers.
- Habitat, by contrast, turns the traditional container workflow on it’s head. You start with the application. Once you have the application, you add in the libraries to run that application, and then only at the end do you add in a bare minimum operating system that is just enough to run your application and nothing more.
- Again, Habitat starts with the application, and then the bare minimum operating system comes in later.
- When an app has dependencies, the app itself declares those dependencies and resolves them. We don’t add in those dependencies pre-emptively – the app will pull what it needs and only what it needs.
- And, even when that package is within a container, it still has that exposed API we talked about earlier. Other outside services – such as a load balancer – can still interact with the app easily, even though it is in a container.
- In summary, when you create a container image with Habitat…
- You know exactly what went into the container and exactly what is configurable about the container, it’s not a black box. Now, let’s look at this in action with another demo.
- Once you have that container image, you can run it locally, but you’re going to want to deploy it somewhere, it doesn’t do much good just sitting on your workstation.
- You can deploy it using a container scheduling service such as Kubernetes and Mesosphere…
- Or a Cloud based container service, like AWS ECS or Azure Container Services. We’re going to look at a demo of deploying our container image to Azure Container Service, but before we do I’d like to bring Ken Thompson back to say a few words about Azure Container Service.
- ADD TRANSITION
- Now let’s see this in action, let’s deploy our image to Azure container services
- As we come to the end of this talk, there’s a few things I would like you to take with you…
- You CAN build software ONCE and run it almost ANYWHERE
- You CAN move a legacy application into the cloud without rewriting it
- You can EMPOWER you applications to recover from failure on their own
- With Chef Habitat – a new Open source product from Chef.
- The key to understanding what Habitat is is to realize that it is not Infrastructure automation, it’s not Contain Automation, it’s Application Automation. The application itself is what we are automating.
- And Habitat is 100% Open Source. Where it goes from here is going to be driven largely by our community of users and contributors.
- If you’d like to get involved – and I hope you do! – check out habitat.sh/community and stop by the Chef booth in the exhibit hall! And with that…
- Again, I’m Nell Shamrell-Harrington, a Sr. Engineer at Chef Software. That’s my contact info…
- Are there any questions?
- Are there any questions?