Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Techniques and lessons for improvement of
deployment processes
Sander van der Burg
Delft University of Technology, EEMCS,
...
NixOS
A GNU/Linux distribution using the Nix package manager
Sander van der Burg Techniques and lessons for improvement of...
Nix store
Main idea: store all packages
in isolation from each other:
/nix/store/rpdqxnilb0cg...
-firefox-3.5.4
Paths cont...
Nix expressions
openssh.nix
{ stdenv, fetchurl, openssl, zlib }:
stdenv.mkDerivation {
name = "openssh-4.6p1";
src = fetch...
Nix expressions
all-packages.nix
openssh = import ../tools/networking/openssh {
inherit fetchurl stdenv openssl zlib;
};
o...
Disnix
$ disnix-env -s services.nix -i infrastructure.nix -d distribution.nix
Sander van der Burg Techniques and lessons f...
Hydra
Continuous build & integration server
Built upon the Nix package manager
Used at TU Delft for building, integration ...
Nix applications’ advantages
Reliability. Dependencies are always complete, present and
correct. No files overwritten, modi...
Application
How to apply these tools on large codebases?
Sander van der Burg Techniques and lessons for improvement of dep...
Software components
We must write Nix expressions for all components:
“A software component is a unit of composition with
...
Challenges
Contractually specified interfaces. Important for static
verification of compositions.
Explicit context dependenc...
Recommendations
Deal with the deployment complexity from the beginning in
your development process
Make your deployment pr...
Recommendations
Decompose your system and their build processes:
Build times can reduced
Components can be replaced
Common...
Recommendations
Make build-time and run-time properties of components
configurable:
No implicit assumptions, e.g. /usr/bin/...
Recommendations
Isolate your build artifacts:
Store components in separate directories
Not in shared directories, e.g. /us...
Recommendations
Think about a component naming convention:
name-version
name-version-architecture
hash-name-version, e.g.
...
Recommendations
Think about component composition:
Environment variables
Symlink trees
Single folder
Sander van der Burg T...
Recommendations
Granularity of dependencies:
How big should components be?
Big components may trigger unnecessary rebuilds...
Conclusion
With these recommendations, deployment processes can be made
more reliable, reproducible and efficient.
Sander va...
Upcoming SlideShare
Loading in …5
×

Techniques and lessons for improvement of deployment processes

394 views

Published on

Presentation given about deployment techniques at Philips Healthcare.

Published in: Science
  • Be the first to comment

  • Be the first to like this

Techniques and lessons for improvement of deployment processes

  1. 1. Techniques and lessons for improvement of deployment processes Sander van der Burg Delft University of Technology, EEMCS, Department of Software Technology Philips Healthcare, Philips Informatics Infrastructure (PII), Best August 27, 2012 Sander van der Burg Techniques and lessons for improvement of deployment processes
  2. 2. NixOS A GNU/Linux distribution using the Nix package manager Sander van der Burg Techniques and lessons for improvement of deployment processes
  3. 3. Nix store Main idea: store all packages in isolation from each other: /nix/store/rpdqxnilb0cg... -firefox-3.5.4 Paths contain a 160-bit cryptographic hash of all inputs used to build the package: Sources Libraries Compilers Build scripts . . . /nix/store l9w6773m1msy...-openssh-4.6p1 bin ssh sbin sshd smkabrbibqv7...-openssl-0.9.8e lib libssl.so.0.9.8 c6jbqm2mc0a7...-zlib-1.2.3 lib libz.so.1.2.3 im276akmsrhv...-glibc-2.5 lib libc.so.6 Sander van der Burg Techniques and lessons for improvement of deployment processes
  4. 4. Nix expressions openssh.nix { stdenv, fetchurl, openssl, zlib }: stdenv.mkDerivation { name = "openssh-4.6p1"; src = fetchurl { url = http://.../openssh-4.6p1.tar.gz; sha256 = "0fpjlr3bfind0y94bk442x2p..."; }; buildCommand = ’’ tar xjf $src ./configure --prefix=$out --with-openssl=${openssl} make; make install ’’; } Sander van der Burg Techniques and lessons for improvement of deployment processes
  5. 5. Nix expressions all-packages.nix openssh = import ../tools/networking/openssh { inherit fetchurl stdenv openssl zlib; }; openssl = import ../development/libraries/openssl { inherit fetchurl stdenv perl; }; stdenv = ...; openssl = ...; zlib = ...; perl = ...; nix-env -f all-packages.nix -iA openssh Produces a /nix/store/l9w6773m1msy...-openssh-4.6p1 package in the Nix store. Sander van der Burg Techniques and lessons for improvement of deployment processes
  6. 6. Disnix $ disnix-env -s services.nix -i infrastructure.nix -d distribution.nix Sander van der Burg Techniques and lessons for improvement of deployment processes
  7. 7. Hydra Continuous build & integration server Built upon the Nix package manager Used at TU Delft for building, integration and testing of several projects on multiple platforms Sander van der Burg Techniques and lessons for improvement of deployment processes
  8. 8. Nix applications’ advantages Reliability. Dependencies are always complete, present and correct. No files overwritten, modified or removed. Reproducibility. Because side-effects are removed, build functions always yield same result, regardless on what machine they are executed. Generic. We can invoke arbitrary build processes in Nix functions. Nix is supported on many operating systems: Linux, FreeBSD, Mac OS X, Windows (Cygwin) Efficient. Only components that must be built are build. Nix store serves as a cache. Sander van der Burg Techniques and lessons for improvement of deployment processes
  9. 9. Application How to apply these tools on large codebases? Sander van der Burg Techniques and lessons for improvement of deployment processes
  10. 10. Software components We must write Nix expressions for all components: “A software component is a unit of composition with contractually specified interfaces and explicit context dependencies only. A software component can be deployed independently and is subject to composition by third parties.” Sander van der Burg Techniques and lessons for improvement of deployment processes
  11. 11. Challenges Contractually specified interfaces. Important for static verification of compositions. Explicit context dependencies. These are often not always known and may be implicit. Deployed independently. Without explicit dependencies it is hard to guarantee that this is possible. Subject to composition by third parties. Third parties must be able to provide or replace dependencies. Sander van der Burg Techniques and lessons for improvement of deployment processes
  12. 12. Recommendations Deal with the deployment complexity from the beginning in your development process Make your deployment process scriptable Sander van der Burg Techniques and lessons for improvement of deployment processes
  13. 13. Recommendations Decompose your system and their build processes: Build times can reduced Components can be replaced Common components can be shared Sander van der Burg Techniques and lessons for improvement of deployment processes
  14. 14. Recommendations Make build-time and run-time properties of components configurable: No implicit assumptions, e.g. /usr/bin/echo Use environment variables: PATH, CLASSPATH etc. Use configuration files Sander van der Burg Techniques and lessons for improvement of deployment processes
  15. 15. Recommendations Isolate your build artifacts: Store components in separate directories Not in shared directories, e.g. /usr/lib or C:WindowsSystem32 etc. Sander van der Burg Techniques and lessons for improvement of deployment processes
  16. 16. Recommendations Think about a component naming convention: name-version name-version-architecture hash-name-version, e.g. /nix/store/l9w6773m1msy...-openssh-4.6p1 Sander van der Burg Techniques and lessons for improvement of deployment processes
  17. 17. Recommendations Think about component composition: Environment variables Symlink trees Single folder Sander van der Burg Techniques and lessons for improvement of deployment processes
  18. 18. Recommendations Granularity of dependencies: How big should components be? Big components may trigger unnecessary rebuilds Sander van der Burg Techniques and lessons for improvement of deployment processes
  19. 19. Conclusion With these recommendations, deployment processes can be made more reliable, reproducible and efficient. Sander van der Burg Techniques and lessons for improvement of deployment processes

×