Hands on Habitat workshop for Velocity London October 18 2017

2. Mandi Walls
Technical Community Manager for EMEA
@lnxchk
mandi@chef.io
https://www.habitat.sh/
http://slack.habitat.sh/
Ian Henry @Eeyun___ Habitat Community lead

3. EVERY business is a software business
We’re going to be a software
company with airplanes.
– CIO, Alaska Airlines

5. We Run Infrastructure
To Run Applications
That the Business Requires

6. How Do We Run Applications?
• On a computer
• With an OS
• And some libraries
• And some configuration
• And some way to start it and stop it

7. That isn’t managing
complexity.
It is Adding to it.
Or at least just moving it
around.

8. So. Habitat.
• Reduce snowflakeness
• Support microservices
• Manage container creep
https://www.bonanza.com/listings/Premier-Food-Storage-Containers-20-Piece-Set-Grey/443972348

9. Modern Applications Are Trending Toward
• Immutability
• Platform agnosticism
• Complexity reduction
• Scalability
https://amazingmusthaves.com/products/steel-insulated-food-containers/

10. Ways of Working with Habitat
• Artifacts run themselves via the supervisor
• Export to Docker
• Link your github to habitat builder and let us do the
work
• Build plans from scratch or leverage scaffolding

11. Habitat Goals
• Defer some decisions to runtime
• Do clean room builds
• Repeatable builds
• Distro agnostic packaging system
• Service runtime and discovery
• Configuration exposed via API
• Packages are signed by the system

12. Single Artifact Demo - Habitatize
• Ruby app with HTTP interface, ImageMagick
• Built via scaffolding
• Run locally
• Login to your machine
User: velocity
Pass: London2017

13. Run the Package
• sudo hab start learn-chef/habitatize -
-channel unstable
• Open http://{your machine IP}:8000 in a web
browser

15. Upload a Pic!

16. What’s in This Thing?
• https://github.com/burtlo/habitatize
pkg_name=habitatize
pkg_origin=franklinwebber
pkg_scaffolding="core/scaffolding-ruby"
pkg_version="1.0.0"
pkg_deps=(franklinwebber/imagemagick)

17. Basic Parts of a Simple Habitat App
• Your application code
• A plan file – can be as short as a few lines
Include dependencies or use a scaffolding package
• Habitat installation for running it
(and build, we’ll do that later)
• That’s it!
• Stop with
sudo hab stop learn-chef/habitatize

18. Dependencies?

19. Community
• Teams and individuals contribute packages to
builder
• The “core” packages are maintained by the Habitat
team
• You can use any public package on builder for your
dependencies
• Packages are libraries, runtimes, applications

20. Habitat and Builder
Application Automation Build Machine

21. Lab: Habitat and Builder
• The fastest way to get packages out of the Habitat
workflow is with Builder
• SaaS for automated build and publish
• Code -> Github -> Builder -> Docker Hub
• You’ll need Github and Docker Hub accounts
No worries, you can get them as we go along

22. Our Sample Application
• https://github.com/habitat-sh/sample-node-app
• Fork this to your github account

23. Sign in to Builder
• https://bldr.habitat.sh/#/sign-in

25. Origins
• Are like organizations for grouping habitat
applications
• You can publish to multiple origins as long as you
have the keys – we’ll talk about that later
• For now, create a new unique origin to use
“username-velocitylondon”
• Origins can be public or private
For this lab it doesn’t matter

27. Make the App Our Own
• In github, edit inline in the browser
• https://github.com/YOU/sample-node-
app/blob/master/habitat/plan.sh
• Change
pkg_origin
To your new origin
• Commit changes, add a comment if you want

28. Add Habitat Builder
• As an app in github
• https://github.com/apps/habitat-builder
• Install
• Choose repositories
All, or limit to just the sample-node-app repo

29. Add Docker Hub Integration
• We’ll link Docker Hub to our origin on habitat builder

30. Docker Hub
• https://hub.docker.com/
• Sign up for an account if you don’t have one
• Each time you create a new origin, you’ll need to
hook it to Docker Hub

33. Automatic Builds of Our Sample App
• Habitat Builder will watch our repo for changes
• New builds of our app will then initialize
• We connect Builder to github from “Packages” on
the Builder page

36. Turn on “Publish to Docker Hub”

37. Build It!

40. Our App Shows Up In Docker Hub
• Builder retrieved our code from github
• Built the latest version
• Exported to a Docker container
• Published to Docker Hub

41. Run Our App with Docker
sudo docker pull your-docker-org/your-
docker-repo
sudo run -it -p 8000:8000 your-docker-
org/your-docker-repo
Open {your IP}:8000/ in a browser
(Make sure you stopped habitatize)
sudo hab stop learn-chef/habitatize

43. Make a New Version
• On github, update your plan.sh file to show a new
version of your app – 1.0.2
• Commit your change
• Builder will pick up your change and start a new
build

44. Habitat Studio
Build Locally

45. Application Life
Source
Code
Repo
Artifact
Bare Metal
Container
Cloud Instance
VM
Artifact
Repo

46. Habitat Studio
• Provides a busy box clean
room for your app
• Plus a set of tools for
manipulating and running
harts

47. Why the Studio?
• Declare explicit dependencies
• Ship exactly what you need
• Sign your packages and store artifacts

48. Habitat Plans
• Plan files are where you put together your builds
• They are bash
• Plans are versioned and can be shared on the
habitat depot
• Plans have multiple sections for your needs

49. What’s In A Plan?
pkg_origin=firstnamelastname
pkg_name=national-parks
pkg_description="A sample JavaEE Web app deployed in Tomcat8"
pkg_version=0.1.3
pkg_maintainer="First Last <you@example.com>"
pkg_license=('Apache-2.0')
pkg_source=https://github.com/billmeyer/national-parks/archive/v${pkg_version}.tar.gz
pkg_shasum=56a70d7fc432ad275ba256150bd010ce2581346c848e40ddc0fd0f62c5b17a54
pkg_upstream_url=https://github.com/billmeyer/national-parks
pkg_deps=(core/tomcat8 billmeyer/mongodb)
pkg_build_deps=(core/git core/maven)
pkg_expose=(8080)
pkg_svc_user="root"
pkg_svc_group="root"

50. Plans Have Callbacks
Shell functions executed when building
artifacts:
• do_begin()
• do_download()
• do_verify()
• do_check()
• do_clean()
• do_unpack()
• do_prepare()
• do_build()
• do_install()
• do_strip()
• do_end()

51. Variables
• HAB_CACHE_SRC_PATH
The default path where source archives are downloaded, extracted, &
compiled.
• ${pkg_dirname}
Set to ${pkg_name}-${pkg_version} by default. If a .tar file extracts to a
directory that's different from the filename, then you would need to
override this value to match the directory name created during
extraction.
• ${pkg_filename}
Set manually in the plan.sh
• ${pkg_version}
Set manually in the plan.sh

52. Habitat Tools for Dependencies
• hab pkg path team/app
• You can have build deps and runtime deps
• Build deps get into your hab studio when you are
building
• Runtime deps come along into your hart
Keep artifacts small and under
control!

53. Scaffolding, like for habitatize
• Default core-built dependencies for common
runtimes
• Ruby and Node so far
Go, Python, Java on the way!
pkg_name=MY_APP
pkg_origin=MY_ORIGIN
pkg_version=MY_VERSION
pkg_scaffolding=core/scaffolding-ruby

54. What Gets Built?
• Everything. Sort of.
• Build your own apps from source
• Decide if you want upstream binaries or source for
things like runtime
You don’t have to build Tomcat, but you can
• For COTS, use the binaries and skip steps

55. Habitat Hooks
• Travels with the app
• Tell the app how to start, and if there is something
that needs to be done before start
• Let’s look at a run hook file

56. Run Hook for Tomcat
#!/bin/bash
exec 2>&1
echo "Starting Apache Tomcat"
export JAVA_HOME=$(hab pkg path core/jdk8)
export TOMCAT_HOME="$(hab pkg path core/tomcat8)/tc"
source {{pkg.svc_config_path}}/catalina-opts.conf
echo "$CATALINA_OPTS=$CATALINA_OPTS"
cp "$(hab pkg path {{pkg.origin}}/{{pkg.name}})/{{pkg.name}}.war" "$TOMCAT_HOME/webapps/"
exec ${TOMCAT_HOME}/bin/catalina.sh run

57. Configuration
• Can be manipulated at runtime
• Also travels with the app
• Provides variable substitution and templating

58. A catalina-opts File
{{~#if bind.database}}
{{~#each bind.database.members}}
export CATALINA_OPTS="-
DMONGODB_SERVICE_HOST={{address}} -
DMONGODB_SERVICE_PORT={{port}}"
{{~/each}}
{{~/if}}

59. Builder
• You can share plans with builder, and other hab
users share theirs
• Has team namespacing – our origin
• The core plans are those built by the Habitat team
• https://app.habitat.sh/

60. Caveat - Internet
• You can build your own stuff inside your own
network, sort of, when it’s all on one machine
• This isn’t the best way to use Habitat
• You’ll want internet connectivity on your build box
and on the hosts running habitat artifacts
• Builder provides private origins, linked to github
users

61. Build Output
• By default, it’s a hart – a compressed tarball with
some metadata and a signature
• The hart itself it runnable
• You can export to other formats, like Docker
containers like we saw eariler

62. Runtime
• The hab runtime includes management, service
discovery, other features
• The habs in your application create a mesh so they
can talk to each other
• You can even update your application via the mesh
without restarting every application manually

63. Running a Hart
sudo hab start learn-chef/habitatize --
channel unstable
sudo hab start nathenharvey/national-parks --
listen-gossip 0.0.0.0:9639 --listen-http
0.0.0.0:9632 --peer 127.0.0.1 –bind
database:mongodb.default

64. Setup
• On your remote host
• hab setup
• Set up a default origin? Yes
Enter what you used in builder – username-velocity
• Create a key – Yes
• Github access token!

65. Github Access Token
• Go to your account on github
• Select “settings” from your account on the top right
• Click “Developer Settings” on the left bar
• Click “Personal access tokens” on the left bar
• “Generate new token” button
• Description “habitat” or whatever
Select read:org and user:email
• Generate Token

66. Hab Setup continued
• Copy the github access token into the hab setup

67. HAB_AUTH_TOKEN
• Your github auth token
• Export as an environment variable
• Can be added to ~/.bashrc
• export HAB_AUTH_TOKEN=3db7f……yourtoken

68. Add Keys
• We generated new keys
• We need to add them to Builder
• From your host, run
hab origin key upload –s {YOUR ORIGIN}

69. A place to work
git clone
https://github.com/YOU/sample-node-
app.git
cd sample-node-app
• Your apps have a habitat/ subdirectory for hab-
related files

70. Hab Studio
• hab studio enter
• build
• hab svc start ORIGIN/sample-node-app

71. Habitat Supervisor
• Keeps an eye on your applications
• Also provides and API for viewing what’s going on
with the apps running in the supervisor
• http://{Your IP}:9631/services
• It’s running now for you as part of the studio, but
doesn’t need the studio to run
When you exit the studio, it will shut down
• Also allows for applications to connect to each other

72. Two-Tier Application: National Parks
• Java on Tomcat with MongoDB
• https://blog.chef.io/2016/09/07/deploy-java-web-
app-on-tomcat-8-with-habitat/
• You’ll want a couple terminals open to your host

73. App

74. Decomposing into Habitat
• Microservice architectures lend themselves well to
Habitat
Every service becomes a hab package
• For generic services like databases, the initial
package should be very basic
• Your application layer or a separate package
initializes the data

75. MongoDB
• Already built – we’re just going to use it
sudo hab start nathenharvey/mongodb --
channel unstable
https://bldr.habitat.sh/#/pkgs/nathenha
rvey/mongodb/latest

76. Java App
• https://bldr.habitat.sh/#/pkgs/nathenharvey/national-
parks/latest
• Let’s look at the plan.sh a bit

77. Where Is the Code Coming From
• This app isn’t hooked into the Builder pipeline
• The source code is downloaded from github into the
studio when the package is built
• A checksum can be included in the plan.sh

78. pkg_deps and pkg_build_deps
• Build deps are present in the studio or build
environment but don’t make it into the package
• In this case, we’re not taking Maven with us!
• Also notice the dependency on mongodb!

79. Finding Stuff that Lives in Other Packages
• Check out the setting for JAVA_HOME
• export JAVA_HOME=$(hab pkg path
core/jdk8)
• Java will be included in our package via the
dependencies
• It arrives via another habitat package
• The filesystem locations are different, so we have to
query for where java will live

80. Let’s Start the App
• sudo hab start nathenharvey/national-
parks --channel unstable
• The log output in this case will go to the MongoDB
screen (STDOUT for the supervisor)
hab-sup(MR): Unable to start nathenharvey/national-
parks, hab-
sup(SS)[src/manager/service/spec.rs:208:23]: Missing
required bind(s), database

81. Supervisor Can Help Us
Service
Discovery
Service
Groups

82. Service Groups
• Applications running in the supervisor belong to
service groups
mongodb.default(SR): Configuration recompiled
mongodb.default(SR): Initializing
mongodb.default(SV): Starting service as user=hab,
group=hab
• When connecting services, use the service group
• Groups can be 1 or N instances of an application

83. Uh Oh
national-parks.default(SR): Initializing
national-parks.default hook[init]:(HK): Seeding Mongo Collection
national-parks.default hook[init]:(HK): $MONGOIMPORT_OPTS=--host= --
port=27017
national-parks.default hook[init]:(HK): 2017-10-17T17:52:35.106+0000
connected to: localhost:27017
national-parks.default hook[init]:(HK): 2017-10-17T17:52:35.106+0000
dropping: demo.nationalparks
national-parks.default hook[init]:(HK): 2017-10-17T17:52:35.109+0000 Failed:
not authorized on demo to execute command { drop: "nationalparks" }
national-parks.default hook[init]:(HK): 2017-10-17T17:52:35.109+0000
imported 0 documents
national-parks.default(HK): Initialization failed! 'init' exited with status code 1

84. What is This?
• Our app brings along its seed data
• It needs permissions to load that data into
MongoDB
• The app isn’t authorized to do that!

85. YOLO!
• We’re going to open up mongo to the wild. It’s fine.
• Let’s see how MongoDB is currently configured
hab sup config nathenharvey/mongodb
• In the middle are some security settings
[mongod.security]
cluster_auth_mode = "keyFile"
javascript_enabled = true
key_file = ""

86. Tunables
• The supervisor allows us to update application
config at runtime
• Those mongod.security settings can be changed on
the fly

87. Make a new tunable toml
hab sup config nathenharvey/mongodb >
user.toml

88. Edit user.toml
• Around line 46 are the mongod.security settings
• Update cluster_auth_mode to “”
46 [mongod.security]
47 authorization = "disabled”
48 cluster_auth_mode = "”
49 javascript_enabled = true
50 key_file = ""

89. hab config apply
• Load this new config into the running mongodb:
sudo hab config apply mongodb.default 2 user.toml
• The serial number just has to increase; it’s not
otherwise meaningful
• Must be valid configuration
• The app will reload

90. Now try the Java again
sudo hab start nathenharvey/national-
parks --peer 127.0.0.1 --bind
database:mongodb.default

91. Check the Output
national-parks.default(O): 17-Oct-2017 18:07:58.322 INFO
[main] org.apache.catalina.startup.Catalina.load
Initialization processed in 525 ms
national-parks.default(O): 17-Oct-2017 18:07:58.348 INFO
[main] org.apache.catalina.core.StandardService.startInternal
Starting service Catalina
national-parks.default(O): 17-Oct-2017 18:07:58.348 INFO
[main] org.apache.catalina.core.StandardEngine.startInternal
Starting Servlet Engine: Apache Tomcat/8.5.9
national-parks.default(O): 17-Oct-2017 18:07:58.367 INFO
[localhost-startStop-1]
org.apache.catalina.startup.HostConfig.deployWAR Deploying
web application archive
/hab/pkgs/core/tomcat8/8.5.9/20170215223900/tc/webapps/nation

92. Check the Browser!
• http://{Your IP}:8080/national-parks/

93. Habitat Info
• http://{Your IP}:9631/services

94. Other Places to Find Hints
• https://github.com/habitat-sh/core-plans/
• The plan files for the core packages
• Good for “how do I do x?”

95. Join Us!
• On Slack!
http://slack.habitat.sh
• Online! With Tutorials!
https://www.habitat.sh/
• On Github!
https://github.com/habitat-sh
• This talk
Try it out!
Share your story!
Get some swag!!
goo.gl/WrHQTU

96. Other References
• Summary on The New Stack
https://thenewstack.io/chef-habitat-addresses-issues-
moving-containers-production
• Our YouTube Channel
https://www.youtube.com/user/getchef