Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Deploying .NET services with Disnix
Sander van der Burg
Delft University of Technology, EEMCS,
Department of Software Tech...
Service Oriented Computing
The Service Oriented Computing (SOC) paradigm is very popular
to build distributed applications...
Service Oriented Computing
Systems are composed of distributable components (or services),
working together to achieve a c...
Deployment
Deployment of Service Oriented Systems is complex:
Many types of components. Various operating systems. Multipl...
Deployment
Deployment of Service Oriented Systems is labourious:
Sander van der Burg Deploying .NET services with Disnix
Deployment
Deployment of Service Oriented systems is error-prone
Upgrading is even harder
Sander van der Burg Deploying .N...
Deployment
Deployment of Service Oriented systems is error-prone
Upgrading is even harder
Sander van der Burg Deploying .N...
Deployment
Deployment of Service Oriented systems is error-prone
Upgrading is even harder
Sander van der Burg Deploying .N...
Example case: StaffTracker
Sander van der Burg Deploying .NET services with Disnix
Disnix
A toolset for distributed deployment of service-oriented systems:
Model-driven
Component-agnostic
Supported on mult...
Disnix
Disnix is built on top of the Nix package manager:
Nix expression language to declaratively specify build actions
A...
Disnix: Models
$ disnix-env -s services.nix -i infrastructure.nix -d distribution.nix
Sander van der Burg Deploying .NET s...
Disnix expressions
{ stdenv, fetchurl, unzip, apacheAnt }:
{ staff }:
stdenv.mkDerivation {
name = "StaffService";
src = f...
Disnix expressions
{ stdenv, fetchurl, unzip, apacheAnt }:
{ staff }:
stdenv.mkDerivation {
name = "StaffService";
src = f...
Intra-dependency composition
{pkgs, system}:
rec {
apacheAnt = ...
stdenv = ...
staff = import ../pkgs/staff {
inherit std...
Service model
{distribution, pkgs, system}:
rec {
staff = {
name = "staff";
pkg = pkgs.staff;
type = "mysql-database";
};
...
Service model
{distribution, pkgs, system}:
rec {
staff = {
name = "staff";
pkg = pkgs.staff;
type = "mysql-database";
};
...
Infrastructure model
{
test1 = {
hostname = "test1.net";
tomcatPort = 8080;
mysqlUser = "user";
mysqlPassword = "secret";
...
Distribution model
{infrastructure}:
{
staff = [ infrastructure.test1 ];
StaffService = [ infrastructure.test2 ];
StaffTra...
Disnix: Deployment
Complete deployment process is derived from the models:
Building services from source code
Optionally, ...
Building services
Main idea: store all packages
in isolation from each other:
/nix/store/rpdqxnilb0cg...
-firefox-3.5.4
Pa...
Runtime dependencies
Nix detects runtime dependencies of a build
Nix scans all files of a component for hash-codes of a
dep...
Runtime dependencies
/nix/store
nqapqr5cyk...-glibc-2.9
lib
libc.so.6
ccayqylscm...-jre-1.6.0 16
bin
java
n7s283c5yg...-SD...
Transferring closures
Copy services and intra-dependencies to target machines in
the network
Only components missing on ta...
Transition phase
Inter-dependency graph is derived from specifications
Deactivate obsolete services (none, if activating ne...
Service activation
Every service has a type: mysql-database,
tomcat-webapplication, process, ...
Types are attached to ext...
Atomic upgrading
Two-phase commit protocol mapped onto Nix primitives
Commit-request-phase (distribution phase):
Build sou...
Atomic upgrading
In case of failure during commit-request: system is not
affected
No files overwritten
Will be removed by ga...
Disnix: Examples
Example cases:
Staff Tracker. Toy SOA system which comes in two variants
(PHP/MySQL and Webservices/Java)....
Deploying .NET services
So far, Disnix case studies are deployed on UNIX-like systems,
using portable component technologi...
Requirements
Buildtime support:
Compile Visual Studio solutions and produce assemblies
Invoke MSBuild.exe with right param...
Disnix extensions
How to extend Disnix to support .NET service deployment?
Sander van der Burg Deploying .NET services wit...
Windows support
Nix is supported on Windows through Cygwin:
Offers UNIX implementation on top of Windows
Allows combining U...
Windows support
Several Cygwin utilities map Windows concepts to UNIX concepts
and vica versa:
Can be used to glue UNIX to...
Implementing build support
A .NET Nix build function dotnetenv.buildSolution is
implemented:
Takes source code of a Visual...
Build function implementation
{stdenv, dotnetfx}:
{ name, src, slnFile, targets ? "ReBuild"
, options ? "/p:Configuration=...
Disnix expression
{dotnetenv}:
{zipcodes}:
dotnetenv.buildSolution {
name = "ZipcodeService";
src = ../../../../services/w...
Runtime dependencies
The .NET runtime has complicated means to locate runtime
dependencies.
Sander van der Burg Deploying ...
Resolving runtime dependencies
.NET runtime locaties assemblies:
Determines the correct version of the assembly (strong-na...
Assembly probing
A <codebase> element in the application .config file can specify
its own assembly references:
Private asse...
Resolving runtime dependencies
You can only automatically resolve runtime dependencies in
the basedir as an executable.
.N...
Wrapper class
namespace HelloWorldWrapper {
class HelloWorldWrapper {
private String[] AssemblySearchPaths = {
@"C:cygwinn...
Wrapper class
private Assembly MyResolveEventHandler(object sender, ResolveEventArgs args) {
Assembly MyAssembly;
String a...
Wrapper function
{dotnetenv, MyAssembly1, MyAssembly2}:
dotnetenv.buildWrapper {
name = "My.Test.Assembly";
src = /path/to...
Activation scripts
Activation scripts for mssql-database and iis-webapplication
must be provided
mssql-database invokes OS...
Example case: StaffTracker.NET
Ported from Java / Apache Axis2 to .NET / C# / WCF
Sander van der Burg Deploying .NET servic...
Demo
Sander van der Burg Deploying .NET services with Disnix
Conclusion
With a number of Disnix extensions it’s possible to automatically
deploy a service-oriented system using .NET t...
Limitations
Disnix only manages service components, not infrastructure
Microsoft Windows, SQL server and IIS must be insta...
References
Nix, Nixpkgs, Hydra, NixOS, Disnix, http://nixos.org.
Cygwin, http://www.cygwin.com
StaffTracker.NET example,
ht...
Questions
Sander van der Burg Deploying .NET services with Disnix
Upcoming SlideShare
Loading in …5
×

Deploying .NET services with Disnix

446 views

Published on

Presentation given at Philips Healthcare about deploying services implemented in .NET with Disnix

Published in: Science
  • Be the first to comment

  • Be the first to like this

Deploying .NET services with Disnix

  1. 1. Deploying .NET services with Disnix Sander van der Burg Delft University of Technology, EEMCS, Department of Software Technology Philips Healthcare, Philips Informatics Infrastructure (PII), Best February 2, 2012 Sander van der Burg Deploying .NET services with Disnix
  2. 2. Service Oriented Computing The Service Oriented Computing (SOC) paradigm is very popular to build distributed applications. Sander van der Burg Deploying .NET services with Disnix
  3. 3. Service Oriented Computing Systems are composed of distributable components (or services), working together to achieve a common goal. Sander van der Burg Deploying .NET services with Disnix
  4. 4. Deployment Deployment of Service Oriented Systems is complex: Many types of components. Various operating systems. Multiple variants. Sander van der Burg Deploying .NET services with Disnix
  5. 5. Deployment Deployment of Service Oriented Systems is labourious: Sander van der Burg Deploying .NET services with Disnix
  6. 6. Deployment Deployment of Service Oriented systems is error-prone Upgrading is even harder Sander van der Burg Deploying .NET services with Disnix
  7. 7. Deployment Deployment of Service Oriented systems is error-prone Upgrading is even harder Sander van der Burg Deploying .NET services with Disnix
  8. 8. Deployment Deployment of Service Oriented systems is error-prone Upgrading is even harder Sander van der Burg Deploying .NET services with Disnix Upgrading How to make upgrades safe and reliable? What to do with incoming connections? Rollbacks?
  9. 9. Example case: StaffTracker Sander van der Burg Deploying .NET services with Disnix
  10. 10. Disnix A toolset for distributed deployment of service-oriented systems: Model-driven Component-agnostic Supported on multiple platforms: Linux, FreeBSD, Mac OS X, Windows (through Cygwin) Sander van der Burg Deploying .NET services with Disnix
  11. 11. Disnix Disnix is built on top of the Nix package manager: Nix expression language to declaratively specify build actions Ability to store multiple versions and variants simultaneously Complete dependencies Reliable and atomic upgrades and rollbacks Garbage collector to safely remove components no longer in use Sander van der Burg Deploying .NET services with Disnix
  12. 12. Disnix: Models $ disnix-env -s services.nix -i infrastructure.nix -d distribution.nix Sander van der Burg Deploying .NET services with Disnix
  13. 13. Disnix expressions { stdenv, fetchurl, unzip, apacheAnt }: { staff }: stdenv.mkDerivation { name = "StaffService"; src = fetchurl { url = http://.../StaffService-0.1.zip; sha256 = "0fpjlr3bfind0y94bk442x2p..."; }; buildInputs = [ unzip apacheAnt ]; buildCommand = ’’ unzip $src ant webapp mkdir -p $out/webapps cp *.war $out/webapps cat > $out/conf/Catalina/StaffService.xml <<EOF <Context> <Resource name="jdbc/staff" auth="Container" type="javax.sql.DataSource" url="jdbc:mysql://${staff.target.hostname}/${staff.name}" ... /> </Context> EOF ’’; } Sander van der Burg Deploying .NET services with Disnix
  14. 14. Disnix expressions { stdenv, fetchurl, unzip, apacheAnt }: { staff }: stdenv.mkDerivation { name = "StaffService"; src = fetchurl { url = http://.../StaffService-0.1.zip; sha256 = "0fpjlr3bfind0y94bk442x2p..."; }; buildInputs = [ unzip apacheAnt ]; buildCommand = ’’ unzip $src ant webapp mkdir -p $out/webapps cp *.war $out/webapps cat > $out/conf/Catalina/StaffService.xml <<EOF <Context> <Resource name="jdbc/staff" auth="Container" type="javax.sql.DataSource" url="jdbc:mysql://${staff.target.hostname}/${staff.name}" ... /> </Context> EOF ’’; } Sander van der Burg Deploying .NET services with Disnix Disnix expression Builds and configures a component from its intra-dependencies and inter-dependencies Any build tool can be invoked as long as it is pure For example: Apache Ant, Perl, Python projects are also supported You can also deploy binary software, by patching it Output is produced in a unique directory in the Nix store
  15. 15. Intra-dependency composition {pkgs, system}: rec { apacheAnt = ... stdenv = ... staff = import ../pkgs/staff { inherit stdenv; }; StaffService = import ../pkgs/StaffService { inherit stdenv fetchurl unzip apacheAnt; }; StaffTracker = import ../pkgs/StaffTracker { inherit stdenv fetchurl unzip apacheAnt; }; ... } Composes components locally, by calling functions with intra-dependency arguments. Sander van der Burg Deploying .NET services with Disnix
  16. 16. Service model {distribution, pkgs, system}: rec { staff = { name = "staff"; pkg = pkgs.staff; type = "mysql-database"; }; StaffService = { name = "StaffService"; pkg = pkgs.StaffService; dependsOn = { inherit staff; }; type = "tomcat-webapplication"; }; StaffTracker = { name = "StaffTracker"; pkg = pkgs.StaffTracker; dependsOn = { inherit StaffService ...; }; type = "tomcat-webapplication"; }; ... } Sander van der Burg Deploying .NET services with Disnix
  17. 17. Service model {distribution, pkgs, system}: rec { staff = { name = "staff"; pkg = pkgs.staff; type = "mysql-database"; }; StaffService = { name = "StaffService"; pkg = pkgs.StaffService; dependsOn = { inherit staff; }; type = "tomcat-webapplication"; }; StaffTracker = { name = "StaffTracker"; pkg = pkgs.StaffTracker; dependsOn = { inherit StaffService ...; }; type = "tomcat-webapplication"; }; ... } Sander van der Burg Deploying .NET services with Disnix Service model Defines the distributable components (or services) Refers to the intra-dependency closure of each service Composes components from its inter-dependencies Specifies component types for activation/deactivation
  18. 18. Infrastructure model { test1 = { hostname = "test1.net"; tomcatPort = 8080; mysqlUser = "user"; mysqlPassword = "secret"; mysqlPort = 3306; targetEPR = http://test1.net/.../DisnixService; system = "i686-linux"; }; test2 = { hostname = "test2.net"; tomcatPort = 8080; ... targetEPR = http://test2.net/.../DisnixService; system = "x86_64-linux"; }; } Captures machines in the network and their relevant properties and capabilities. Sander van der Burg Deploying .NET services with Disnix
  19. 19. Distribution model {infrastructure}: { staff = [ infrastructure.test1 ]; StaffService = [ infrastructure.test2 ]; StaffTracker = [ infrastructure.test1 infrastructure.test2 ]; ... } Maps services to machines Sander van der Burg Deploying .NET services with Disnix
  20. 20. Disnix: Deployment Complete deployment process is derived from the models: Building services from source code Optionally, build can be delegated to a capable machine Transferring services (and intra-dependencies) to target machines Deactivation of obsolete services and activation of new services in the right order Optionally, incoming connections can be blocked/queued Sander van der Burg Deploying .NET services with Disnix
  21. 21. Building services Main idea: store all packages in isolation from each other: /nix/store/rpdqxnilb0cg... -firefox-3.5.4 Paths contain a 160-bit cryptographic hash of all inputs used to build the package: Sources Libraries Compilers Build scripts . . . /nix/store l9w6773m1msy...-openssh-4.6p1 bin ssh sbin sshd smkabrbibqv7...-openssl-0.9.8e lib libssl.so.0.9.8 c6jbqm2mc0a7...-zlib-1.2.3 lib libz.so.1.2.3 im276akmsrhv...-glibc-2.5 lib libc.so.6 Sander van der Burg Deploying .NET services with Disnix
  22. 22. Runtime dependencies Nix detects runtime dependencies of a build Nix scans all files of a component for hash-codes of a dependency, e.g. RPATH defined in a ELF header Uses a notion of closures to guarantee completeness: If a specific component needs to be deployed, all its runtime dependencies are deployed first Sander van der Burg Deploying .NET services with Disnix
  23. 23. Runtime dependencies /nix/store nqapqr5cyk...-glibc-2.9 lib libc.so.6 ccayqylscm...-jre-1.6.0 16 bin java n7s283c5yg...-SDS2Util share java SDS2Util.jar y2ssvzcd86...-SDS2EventGenerator bin SDS2EventGenerator share java SDS2EventGenerator.jar Sander van der Burg Deploying .NET services with Disnix
  24. 24. Transferring closures Copy services and intra-dependencies to target machines in the network Only components missing on target machines are transferred Always safe. Files are never overwritten/removed Sander van der Burg Deploying .NET services with Disnix
  25. 25. Transition phase Inter-dependency graph is derived from specifications Deactivate obsolete services (none, if activating new configuration) Activate new services Derive order and dependencies from dependency-graph Sander van der Burg Deploying .NET services with Disnix
  26. 26. Service activation Every service has a type: mysql-database, tomcat-webapplication, process, ... Types are attached to external processes, which take 2 arguments activate or deactivate Nix store component Infrastructure model properties are passed as environment variables Sander van der Burg Deploying .NET services with Disnix
  27. 27. Atomic upgrading Two-phase commit protocol mapped onto Nix primitives Commit-request-phase (distribution phase): Build sources Transfer service closures Commit-phase (transition phase): Deactivating/activating services Optionally block/queue connections from end-users Sander van der Burg Deploying .NET services with Disnix
  28. 28. Atomic upgrading In case of failure during commit-request: system is not affected No files overwritten Will be removed by garbage collector In case of failure during commit: Rollback (transition to previous configuration) Sander van der Burg Deploying .NET services with Disnix
  29. 29. Disnix: Examples Example cases: Staff Tracker. Toy SOA system which comes in two variants (PHP/MySQL and Webservices/Java). ViewVC. Open-source web based Subversion repository viewer (http://viewvc.tigris.org). SDS2. Industrial SOA case study from Philips Research. Java EE based. Sander van der Burg Deploying .NET services with Disnix
  30. 30. Deploying .NET services So far, Disnix case studies are deployed on UNIX-like systems, using portable component technologies Deployment of .NET services on a Windows infrastructure is challenging because of the underlying purely functional deployment model of Nix and its UNIX heritage. Sander van der Burg Deploying .NET services with Disnix
  31. 31. Requirements Buildtime support: Compile Visual Studio solutions and produce assemblies Invoke MSBuild.exe with right parameters The build process must find its buildtime dependencies Runtime support: Assemblies must find its runtime dependencies Integrate .NET CLR dependency resolver with Nix Activation support: Activate web application components in IIS Activate databases in Microsoft SQL server Sander van der Burg Deploying .NET services with Disnix
  32. 32. Disnix extensions How to extend Disnix to support .NET service deployment? Sander van der Burg Deploying .NET services with Disnix
  33. 33. Windows support Nix is supported on Windows through Cygwin: Offers UNIX implementation on top of Windows Allows combining UNIX and Windows utilities Sander van der Burg Deploying .NET services with Disnix
  34. 34. Windows support Several Cygwin utilities map Windows concepts to UNIX concepts and vica versa: Can be used to glue UNIX tooling to Windows processes cygpath, UNIX path names <=> Windows path names Sander van der Burg Deploying .NET services with Disnix
  35. 35. Implementing build support A .NET Nix build function dotnetenv.buildSolution is implemented: Takes source code of a Visual Studio C# solution Produces output of the solution into a unique store path, based on the input arguments of this function Sander van der Burg Deploying .NET services with Disnix
  36. 36. Build function implementation {stdenv, dotnetfx}: { name, src, slnFile, targets ? "ReBuild" , options ? "/p:Configuration=Debug;Platform=Win32" , assemblyInputs ? [] }: stdenv.mkDerivation { inherit name src; buildInputs = [ dotnetfx ]; installPhase = ’’ for i in ${toString assemblyInputs}; do windowsPath=$(cygpath --windows $i) AssemblySearchPaths="$AssemblySearchPaths;$windowsPath" done export AssemblySearchPaths ensureDir $out outPath=$(cygpath --windows $out) MSBuild.exe ${slnFile} /nologo /t:${targets} /p:OutputPath=$outPath ${options} ... ’’; } Sander van der Burg Deploying .NET services with Disnix
  37. 37. Disnix expression {dotnetenv}: {zipcodes}: dotnetenv.buildSolution { name = "ZipcodeService"; src = ../../../../services/webservices/ZipcodeService; baseDir = "ZipcodeService"; slnFile = "ZipcodeService.csproj"; targets = "Package"; preBuild = ’’ sed -e ’s|.SQLEXPRESS|${zipcodes.target.hostname}SQLEXPRESS|’ -e ’s|Initial Catalog=zipcodes|Initial catalog=${zipcodes.name}|’ -e ’s|User ID=sa|User ID=${zipcodes.target.msSqlUsername}|’ -e ’s|Password=admin123$|Password=${zipcodes.target.msSqlPassword}|’ Web.config ’’; } Builds and configures a Visual Studio ASP.NET project Sander van der Burg Deploying .NET services with Disnix
  38. 38. Runtime dependencies The .NET runtime has complicated means to locate runtime dependencies. Sander van der Burg Deploying .NET services with Disnix
  39. 39. Resolving runtime dependencies .NET runtime locaties assemblies: Determines the correct version of the assembly (strong-named assemblies only) Checks whether the assembly has been bound before (strong-named assemblies only) If so, it uses this version Shares same assemblies in memory between applications Checks the Global Assembly Cache (GAC) (strong-named assemblies only) Probes the assembly Checking the <codebase> element in the application .config Probing heuristics Sander van der Burg Deploying .NET services with Disnix
  40. 40. Assembly probing A <codebase> element in the application .config file can specify its own assembly references: Private assemblies can only reside in the directory or a subdirectory of the executable Strong-named assemblies can be invoked from an arbitrary location, including remote locations Referenced assembly is cached, therefore a strong-name is required Sander van der Burg Deploying .NET services with Disnix
  41. 41. Resolving runtime dependencies You can only automatically resolve runtime dependencies in the basedir as an executable. .NET reflection API can load private assemblies from any location .NET CLR fires an AssemblyResolve event if a dependency cannot be found Can be used to load a private runtime assembly from arbitrary locations, e.g. the Nix store Sander van der Burg Deploying .NET services with Disnix
  42. 42. Wrapper class namespace HelloWorldWrapper { class HelloWorldWrapper { private String[] AssemblySearchPaths = { @"C:cygwinnixstore23ga...-ALibrary", @"C:cygwinnixstore833p...-BLibrary" }; private String ExePath = @"C:cygwinnixstore27f2...-ExecutableExecutable.exe"; private String MainClassName = "SomeExecutable.Executable"; public HelloWorldWrapper(string[] args) { AppDomain currentDomain = AppDomain.CurrentDomain; currentDomain.AssemblyResolve += new ResolveEventHandler(MyResolveEventHandler); Assembly exeAssembly = Assembly.LoadFrom(ExePath); Type mainClass = exeAssembly.GetType(MainClassName); MethodInfo mainMethod = mainClass.GetMethod("Main"); mainMethod.Invoke(this, new Object[] {args}); } static void Main(string[] args) { new HelloWorldWrapper(args); } Sander van der Burg Deploying .NET services with Disnix
  43. 43. Wrapper class private Assembly MyResolveEventHandler(object sender, ResolveEventArgs args) { Assembly MyAssembly; String assemblyPath = ""; String requestedAssemblyName = args.Name.Substring(0, args.Name.IndexOf(",")); foreach (String curAssemblyPath in AssemblySearchPaths) { assemblyPath = curAssemblyPath + "/" + requestedAssemblyName + ".dll"; if (File.Exists(assemblyPath)) break; } MyAssembly = Assembly.LoadFrom(assemblyPath); return MyAssembly; } Sander van der Burg Deploying .NET services with Disnix
  44. 44. Wrapper function {dotnetenv, MyAssembly1, MyAssembly2}: dotnetenv.buildWrapper { name = "My.Test.Assembly"; src = /path/to/source/code; slnFile = "Assembly.sln"; assemblyInputs = [ dotnetenv.assembly20Path MyAssembly1 MyAssembly2 ]; namespace = "TestNamespace"; mainClassName = "MyTestApplication"; mainClassFile = "MyTestApplication.cs"; } Builds a Visual Studio project and creates a wrapper around the executable Sander van der Burg Deploying .NET services with Disnix
  45. 45. Activation scripts Activation scripts for mssql-database and iis-webapplication must be provided mssql-database invokes OSQL.EXE to automatically create a database and to import table definitions iis-webapplication invokes MSDeploy.exe to deploy web applications on a IIS server Sander van der Burg Deploying .NET services with Disnix
  46. 46. Example case: StaffTracker.NET Ported from Java / Apache Axis2 to .NET / C# / WCF Sander van der Burg Deploying .NET services with Disnix
  47. 47. Demo Sander van der Burg Deploying .NET services with Disnix
  48. 48. Conclusion With a number of Disnix extensions it’s possible to automatically deploy a service-oriented system using .NET technology Sander van der Burg Deploying .NET services with Disnix
  49. 49. Limitations Disnix only manages service components, not infrastructure Microsoft Windows, SQL server and IIS must be installed by other means .NET framework is not deployed by Nix/Disnix .NET deployment features are relatively new and untested Sander van der Burg Deploying .NET services with Disnix
  50. 50. References Nix, Nixpkgs, Hydra, NixOS, Disnix, http://nixos.org. Cygwin, http://www.cygwin.com StaffTracker.NET example, http://nixos.org/disnix/examples.html. Sander van der Burg Deploying .NET services with Disnix
  51. 51. Questions Sander van der Burg Deploying .NET services with Disnix

×