• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Paremus service fabric
 

Paremus service fabric

on

  • 1,087 views

 

Statistics

Views

Total Views
1,087
Views on SlideShare
1,087
Embed Views
0

Actions

Likes
0
Downloads
10
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Name, Company, Book, Modularising Scala - Neil Modular Private Cloud POC for Investment Bank
  • Certainly this doesn’t factor density. Dynamic languages will produce less code. But Cobol is about the same as C. http://www.embedded.com/design/210602856 Larger systems are more difficult to understand and maintain. Writing new software impedes time-to-market. Wouldn’t it be nice to just assemble software? Can dynamic languages help? Possibly. But they’ll grow in size too.
  • Name, Company, Book, Modularising Scala - Neil Modular Private Cloud POC for Investment Bank
  • Name, Company, Book, Modularising Scala - Neil Modular Private Cloud POC for Investment Bank
  • What does a modular, recovery oriented. Cloud runtime look like
  • Name, Company, Book, Modularising Scala - Neil Modular Private Cloud POC for Investment Bank
  • Name, Company, Book, Modularising Scala - Neil Modular Private Cloud POC for Investment Bank
  • Name, Company, Book, Modularising Scala - Neil Modular Private Cloud POC for Investment Bank

Paremus service fabric Paremus service fabric Presentation Transcript

  • Introduction to The Paremus Services Fabric 19 January 2012 OSGi™ Users’ Forum DC Metro … a distributed application server for running OSGi technology-based applications across anything from a few computing resources to a large-scale Cloud. The Paremus Service Fabric extends OSGi technology’s dynamic module capabilities across multiple network-connected Java Virtual Machines (JVMs) to create an incredibly robust and efficient distributed platform for enterprise applications.
  • But First!
    • … a little context about OSGi technology…
  • The Problem
    • Software Complexity is Increasing at an Alarming Rate
    • Shortened Product Cycles
    • Requirements for Drastically Increased Functionality
    • Increasing Number of Variations of the Same Product
    • Software Development [Today] Largely Consists of Adapting Existing Functionality to Perform in a New Environment.
    • Integrating Many Different Libraries Can be Daunting
    • Monolithic Software Products Undergo Heavy Testing Cycle
    • Unsynchronized Evolution of Different Libraries
    • Key Issue: Today's Software Environments Focus On Writing New Software, Instead of Integrating Existing Software Into New Systems
    • Need Tools That Standardize Integration Aspects of Software so that Reusing Existing Components Becomes Reliable, Robust and Cheap
    From OSGi™ Alliance
  • The Solution – OSGi™
    • The Dynamic Module System for Java™
    • Provides Standardized Primitives That Allow Applications to be Constructed from Small, Reusable and Collaborative Components
    • Components can be Composed Into an Application and Deployed
    • Provides Functions to Change Composition Dynamically on Device of a Variety of Networks, Without Requiring Restarts
    • To Minimize & Manage Coupling, Provides a Service-oriented Architecture that Enables Components to Dynamically Discover Each Other
    • Many Standard Component Interfaces Developed (e.g. HTTP Servers, Configuration, Logging, Security, User Administration, and XML)
    • Provides for Integration of Pre-built, Pre-tested, [Pre-Approved] Component Subsystems
    • Reduces Maintenance Costs and Enables Unique New Aftermarket Opportunities Because Components can be Dynamically Delivered to Devices in the Field
    From OSGi™ Alliance
  • Fine! But What about Cloud [Computing]?
    • I’m glad you asked!
    Let’s Talk about Cloud and OSGi technology.
  • [email_address] The Dawn of Composite Clouds Richard Nicholson: Paremus CEO President of the OSGi Alliance
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011
    • Background & Drivers
    • Atlas Resource Management
    • The Service Fabric
    • Additional Slides
  • Background & Drivers
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Breakdown of OPEX costs Anne Thomas Manes (Gartner) – SOA Symposium: Berlin, October 2010 Environmental Complexity! ‘ Cloud Computing’, ‘Grid’, ‘Virtualisation’ addresses this. ?
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Hardware Cost? Resource Current Image Cloud Image Cloud Target Resource Utilisation 5% 30% 50% Network Loading / Upgrade Cost ? minimal Storage Consumption ? minimal
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Development Breakdown? Activities Saving Code Re-Use % of code re-use Agile Development (modules) smaller parallel teams Increased Test Efficiency % increase in test / release efficiency Effective Outsource of Modules % of modules outsourced / market Increased Project Success failed projects / overruns
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Maintenance Breakdown? Activities % of Maintenance OPEX Initial Install / Deploy _? small Library / Code upgrades - IT Debt / avoiding code rot _? large (Gartner) Ongoing Configure / Re-Configure effort required / cost Failure / Fix / Recover lost business due to service outage Functional Updates effort required / cost
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Observations
    • 1. The Unit of Deployment is...
      • The Unit of Fix / Maintenance
      • The Potential Unit of Reuse
    • 2. Time to Deploy - Directly Influences...
      • Time to Recover a Failed Service / Environment
      • Time to Scale an Environment in Response to Business Load
    Must Optimize Detect Decision Response
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Modules vs. Images  D eployment Artifact Artifact Module Centric Image Centric Machine Image ~ 1 to 2 Gbytes per node ~ 1 to 2 Gbytes per node Application ~ 1 TO 100 Mbytes ~ 1 to 2 Gbytes per node Application Patch ~ 100’s Kbytes ~ 1 to 2 Gbytes per node Property / Config Changes ~ 1 Kbyte ~ 1 to 2 Gbytes per node
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 What does this mean?
    • The Following Assumes:
      • 1 Gbit Network
      • 1000 Physical Nodes
    Action Module Centric Image Centric Deploying an Application ~20 minutes ~3.3 hours Patching / Roll Back ~ 4 minute ~3.3 hours Updating a Configuration / Property < 1 minute ~3.3 hours Time to Scale Resource Horizontally by 10% ~2 minutes 33 minutes Impact on Network Utilization Low High Impact on Business Service Fast Scale & Recovery Slow Scale & Recovery Impact on Operations Low High Detect Decision Response Detect Decision Response ~4 mins. 3.3 hours!
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 What does this mean?
    • The following Assumes:
      • 1,500 Applications
      • 3 Versions of Application
      • 6 Different Configurations of Each Application
        • (Development, UAT, Production, Contingency, etc.)
    Module Centric Image Centric Size of Repository ~150 Gigabyte ~40 Terabyte
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011
      • 120 Billion Lines of Code Written in 1990
      • 250 Billion Lines of Code Written in 2000
      • Lines of Code Double Every 7 Years
      • 50% of Development Time Spent Understanding Code
      • 90% of Software Cost is Maintenance & Evolution
    Source: http://users.jyu.fi/~koskinen/smcosts.htm The Complexity Crisis Source: Burton Group Analyst Kirk Koernschild - http://techdistrict.kirkk.com / Consider this… 2010 2003 1996 1989
  • Why OSGi? Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011
    • Modularization Forces ‘Accidental Complexity’ out of the Development Environment
    • Boundaries Enforce a Clean Interface / Contract between Participants in the Environment: at Each Layer of Structural Hierarchy!
    • Enforces Low Coupling & High Cohesion
    • Unit of Deployment is Unit of Re-use
    • Unit of Deployment is Unit of Maintenance
    Patterns of Modular Architecture Kirk Knoernschild - http://www.kirkk.com/modularity/chapters/ Think about that for a second!
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Design Principles
    • ➟ The Smaller the Deployment Artifacts - the Better!
          • Deploy Bundles
    • ➟ But We Must Manage the ‘Aggregate’ to Avoid Operational Complexity -
          • Manage the Composite Systems
    Must Optimize Detect Decision Response
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Modular Systems are Maintainable Systems Anne Thomas Manes (Gartner) – SOA Symposium: Berlin, October 2010 Paremus Service Fabric Addresses Maintenance Costs, Development Costs, as well as Resource Optimization . OPEX Applicability of VM-based (non-Modular) Private Cloud Solutions.
  • Atlas - Resource Management
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Managing characteristics of population rather than each individual node has a dramatic effect on decreasing runtime complexity - http://adaptevolve.blogspot.com/2008/01/complexity-part-ii-it-all-depends-on.html
      • Manage Populations NOT Instances
      • Atlas, Resource Target State & Dynamic Runtime Assembly
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Off Fabric node Bootstrap Atlas Manager Source Repository for Examples POSH CLI Application UI Service Fabric Nodes Service Fabric Infrastructure Cached Examples Deployed Systems Demo Environment A ‘blue’ Atlas Agent
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Resource Management & Bootstrap
  • [email_address] The Service Fabric Richard Nicholson: Paremus CEO President of the OSGi Alliance
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Introducing the Paremus Service Fabric (1..m) ‘Composite Applications’ (a.k.a. Systems) may be D ynamically Assembled and Run Upon a single Service Fabric (1..n) Compute Resources may Contribute to a Service Fabric An OSGi™ Technology-based Cloud Runtime for the Enterprise
  • Paremus Service Fabric Components Distributed Services Fabric Service Instance Dynamic component wire-up (local) Service Groups Service aggregation Dynamic service wire-up (network) Composite Application (fine-grained) Composite Application (coarse-grained) Web 3.0 / WS wire-up (network)
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Industry Standards
    • OSGi framework - Equinox, Knopflerfish or Felix
    • Paremus OSGi command shell (Posh): OSGi RFC147
    • Paremus Nimble Resolver: OSGi OBR resolver (RFC112)
    • Paremus implementation of OSGi Remote Services Administration
    • OSGi ConfigAdmin
    • Paremus implementation of OSGi Web Archive Bundle (RFC66)
    • RTI implementation of OMG DDS: Used within Service Fabric as a low latency control P2P ‘backplane’ - see http://www.rti.com/company/customers.html
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Self-Describing System Description Running System A Model Driven Runtime
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Target State Runtime State Deploy Starting a System
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Planned Deltas e.g. Configuration changes Unplanned Deltas e.g. Resource failures Target State Runtime State The Service Fabric Responds by Creating the Corresponding System – Deploying all Required Components and ‘Wiring’ these Together. Starting a System
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Runtime State Target State To Change a Runtime System, Change its Model in the Required Manner Re-Configure Updating a System
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Planned Deltas e.g. Configuration changes Unplanned Deltas e.g. Resource failures Target State Runtime State The Service Fabric Responds by Modifying the Running System – Unloading Old Versions and Deploying New Versions of Affected Components Updating a System
  • ❶ Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Each Service Fabric Node Dynamically Deploys and Configures Local Infrastructure Services in Response to the Requirements of the Installed Business Component A self-configuring PaaS Automatically addressing modular & runtime dependencies WAB WAR triggers policy based runtime installation of GlassFish triggers policy based runtime installation of Jetty ❷ ❶ ❷ JEE EAR
  • Systems and Architecture Templates Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 System = + An Architectural template Business Component(s) In-house Architects Select / Define Appropriate Patterns Business Unit Developer A System May Be Optionally Split Into an Architecture Pattern (Template) + Business Components
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 System ( Versioned ) Parts ( Each Versioned ) Scaling Behavior ( How Many Part Instances are Required? Scaling behavior May be Operator-driven or Dynamic in Response to Environmental Conditions. ) + Resource Contract The System Model
    • On-fabric ‘Part’
      • runs outside the fibre’s JVM
      • does not use ConfigAdmin or RSA
      • POJOSR,
      • Java & non-Java Processes
      • even Virtual Machine images
    service wires (remote) + ( Defines Resource Characteristics Required by the Part ) =
    • In-fabric ‘Part’
      • runs in the fibre’s JVM
      • a well-behaved OSGi assembly
      • uses ConfigAdmin & RSA
      • JVM targeted language - Java / Scala
      • Blueprint, DS
      • also WAR & EE
    OSGi - Remote Service / Remote Service Admin
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 In-, On- & Off- Fabric
    • A Java / Scala / OSGi Assembly Running in the ‘fibres’ own OSGi framework.
    • Typically Blueprint or Declarative Services.
    • Configured via ConfigAdmin (ideally).
    • Remote Services via RSA (ideally).
    • Probably not OSGi or Java.
    • Runs in a Different Process Space to ‘fibres’ OSGi framework
    • Configuration via Properties File or Configuration Scripts / API’s.
    • Remote Services Embedded in Artifact.
    • Installation and Lifecycle Management still Performed by Fabric.
    A ‘Part’ within a System may be ‘In-Fabric’, ‘On-Fabric’ or a Reference / Proxy to an Off-Fabric Resource. An In-Fabric ‘Part’ An On-Fabric ‘Part’
  • On-Fabric Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 A System Model
    • Defines:
    • System Name and Version,
    • Whether System is Distributed,
    • Repository Indexes to be used when Assembling Parts, and
    • Runtime Security Group.
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Repositories, Resolvers & REPOPATHs
    • Each Service Fabric Fibre has its own Local OBR Resolver.
    • Upon Agreeing to Host a Part - The ‘Fibre’ Must
      • Resolve Bundle Dependencies
      • Download Required Artifacts
      • Assemble the Part .
    • Resolver Must Resolve Required Part in a Manner that Meets Stated Requirements .
    • Resolver Must Resolve Required Part W.R.T Specified Environment .
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Repositories, Resolvers & REPOPATHs
    • In a Service Fabric…
    • A Resolver Environment is not Defined per OSGi Framework, is not set for the Whole Fabric - BUT Defined for Each System - i.e. a List of Repository Indexes Known as the REPOPATH .
    • Each Fibre’s Resolver Traverses the REPOPATH of the Deployed PART Left to Right through the List.
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Repositories, Resolvers & REPOPATHs A fibre Hosting PART A from System I: its Resolver will use Bundle B-1.6.0 from the Equities Repository. A fibre Hosting PART A from System II: its Resolver will use Bundle B-1.6.1 from the System II Repository.
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 <system name=&quot;gateway.system&quot; version=&quot;1.0“ boundary=&quot;fabric&quot; repopath=&quot;gateway,nimble-rs,aries,aries-ext,fabric,fabric-ext,nimble-cmpn“ xmlns=&quot; http://schema.paremus.com/sf/1 &quot;> <description>Deploys a pricer and gateway component wired together using slp and essencermi</description> <!-- set SystemGroup for security control --> <nature group=&quot;demo&quot; /> <system name=&quot;pricer&quot; boundary=&quot;fibre&quot;> <system.part category=&quot;msf&quot; name=&quot;com.example.pricer&quot;> <property name=&quot;type&quot; value=&quot;firm&quot; /> </system.part> <system.part category=&quot;osgi.active.bundle&quot; name=&quot;com.paremus.dosgi.dsw.essencermi&quot; /> <system.part category=&quot;osgi.active.bundle&quot; name=&quot;com.paremus.dosgi.discovery.slp&quot; /> <system.part category=&quot;osgi.active.bundle&quot; name=&quot;com.paremus.dosgi.topologymanager&quot; /> <replication.handler type=&quot;scalable&quot;> <property name=&quot;scaleFactor&quot; value=&quot;1&quot; type=&quot;float&quot; /> <property name=&quot;fixedDelta&quot; value=&quot;-2&quot; type=&quot;integer&quot; /> <property name=&quot;minimum&quot; value=&quot;1&quot; type=&quot;integer&quot; /> </replication.handler> </system> <system name=&quot;gateway&quot; boundary=&quot;fibre&quot;> <system.part category=&quot;msf&quot; name=&quot;com.example.gateway&quot;> <property name=&quot;id&quot; value=&quot;foo&quot; /> </system.part> <system.part category=&quot;osgi.active.bundle&quot; name=&quot;com.paremus.dosgi.dsw.essencermi&quot; /> <system.part category=&quot;osgi.active.bundle&quot; name=&quot;com.paremus.dosgi.discovery.slp&quot; /> <system.part category=&quot;osgi.active.bundle&quot; name=&quot;com.paremus.dosgi.topologymanager&quot; /> </system> </system> A System Model The Pricer Part. The constituents of each Pricer instance are all deployed/collocated to the same fibre (JVM instance) - hence ‘boundary=fibre’
  • <system name=&quot;gateway.system&quot; version=&quot;1.0“ boundary=&quot;fabric&quot; repopath=&quot;gateway,nimble-rs,aries,aries-ext,fabric,fabric-ext,nimble-cmpn“ xmlns=&quot; http://schema.paremus.com/sf/1 &quot;> <description>Deploys a pricer and gateway component wired together using slp and essencermi</description> <!-- set SystemGroup for security control --> <nature group=&quot;demo&quot; /> <system name=&quot;pricer&quot; boundary=&quot;fibre&quot;> <system.part category=&quot;msf&quot; name=&quot;com.example.pricer&quot;> <property name=&quot;type&quot; value=&quot;firm&quot; /> </system.part> <system.part category=&quot;osgi.active.bundle&quot; name=&quot;com.paremus.dosgi.dsw.essencermi&quot; /> <system.part category=&quot;osgi.active.bundle&quot; name=&quot;com.paremus.dosgi.discovery.slp&quot; /> <system.part category=&quot;osgi.active.bundle&quot; name=&quot;com.paremus.dosgi.topologymanager&quot; /> <replication.handler type=&quot;scalable&quot;> <property name=&quot;scaleFactor&quot; value=&quot;1&quot; type=&quot;float&quot; /> <property name=&quot;fixedDelta&quot; value=&quot;-2&quot; type=&quot;integer&quot; /> <property name=&quot;minimum&quot; value=&quot;1&quot; type=&quot;integer&quot; /> </replication.handler> </system> <system name=&quot;gateway&quot; boundary=&quot;fibre&quot;> <system.part category=&quot;msf&quot; name=&quot;com.example.gateway&quot;> <property name=&quot;id&quot; value=&quot;foo&quot; /> </system.part> <system.part category=&quot;osgi.active.bundle&quot; name=&quot;com.paremus.dosgi.dsw.essencermi&quot; /> <system.part category=&quot;osgi.active.bundle&quot; name=&quot;com.paremus.dosgi.discovery.slp&quot; /> <system.part category=&quot;osgi.active.bundle&quot; name=&quot;com.paremus.dosgi.topologymanager&quot; /> </system> </system> Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 A System Model The Pricer Part. A managed service fabric based built from com.example.pricer. An instance of which is created with attribute ‘firm’
  • <system name=&quot;gateway.system&quot; version=&quot;1.0“ boundary=&quot;fabric&quot; repopath=&quot;gateway,nimble-rs,aries,aries-ext,fabric,fabric-ext,nimble-cmpn“ xmlns=&quot; http://schema.paremus.com/sf/1 &quot;> <description>Deploys a pricer and gateway component wired together using slp and essencermi</description> <!-- set SystemGroup for security control --> <nature group=&quot;demo&quot; /> <system name=&quot;pricer&quot; boundary=&quot;fibre&quot;> <system.part category=&quot;msf&quot; name=&quot;com.example.pricer&quot;> <property name=&quot;type&quot; value=&quot;firm&quot; /> </system.part> <system.part category=&quot;osgi.active.bundle&quot; name=&quot;com.paremus.dosgi.dsw.essencermi&quot; /> <system.part category=&quot;osgi.active.bundle&quot; name=&quot;com.paremus.dosgi.discovery.slp&quot; /> <system.part category=&quot;osgi.active.bundle&quot; name=&quot;com.paremus.dosgi.topologymanager&quot; /> <replication.handler type=&quot;scalable&quot;> <property name=&quot;scaleFactor&quot; value=&quot;1&quot; type=&quot;float&quot; /> <property name=&quot;fixedDelta&quot; value=&quot;-2&quot; type=&quot;integer&quot; /> <property name=&quot;minimum&quot; value=&quot;1&quot; type=&quot;integer&quot; /> </replication.handler> </system> <system name=&quot;gateway&quot; boundary=&quot;fibre&quot;> <system.part category=&quot;msf&quot; name=&quot;com.example.gateway&quot;> <property name=&quot;id&quot; value=&quot;foo&quot; /> </system.part> <system.part category=&quot;osgi.active.bundle&quot; name=&quot;com.paremus.dosgi.dsw.essencermi&quot; /> <system.part category=&quot;osgi.active.bundle&quot; name=&quot;com.paremus.dosgi.discovery.slp&quot; /> <system.part category=&quot;osgi.active.bundle&quot; name=&quot;com.paremus.dosgi.topologymanager&quot; /> </system> </system> Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 A System Model Remote Services using the Paremus implementation of the OSGI Alliance RSA specification. In this example Essence RMI is the protocol - and SLP is used for discover. Normally we recommend DDS - which is the default.
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011
    • Exporting a Service
    • Service E has a service property service.exported.interfaces set.
    • Local Topology Manager is informed of Service E.
    • Based on policy, the Topology Manager informs registered Remote Service Admin services.
    • If a Remote Service Admin can support Service E an appropriate Service Endpoint is created.
    • The framework wires the Service Endpoint to its service; in this case Service E.
    • The Remote Service Admin also creates an Endpoint Description and passes this to registered Discovery Providers.
    • Each Discovery Provider encapsulates the Endpoint Description in the appropriate manner and issues an implementation specific advertisement. NOTE: multiple concurrent discovery providers as supported.
    OSGi Remote Services
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011
    • Importing a Service
    • A Discovery Provider receives a remote-advertisement.
    • The Endpoint Description is unpacked and sent to the local Topology Manager.
    • The Topology Manager is aware that Service B has a reference to Service E, hence the Topology Manager sends the Endpoint Description to the appropriate Remote Service Admin.
    • The Remote Service Admin creates a local proxy for Service E.
    • The framework wiring this to the Service B reference.
    OSGi Remote References
  • <system name=&quot;gateway.system&quot; version=&quot;1.0“ boundary=&quot;fabric&quot; repopath=&quot;gateway,nimble-rs,aries,aries-ext,fabric,fabric-ext,nimble-cmpn“ xmlns=&quot; http://schema.paremus.com/sf/1 &quot;> <description>Deploys a pricer and gateway component wired together using slp and essencermi</description> <!-- set SystemGroup for security control --> <nature group=&quot;demo&quot; /> <system name=&quot;pricer&quot; boundary=&quot;fibre&quot;> <system.part category=&quot;msf&quot; name=&quot;com.example.pricer&quot;> <property name=&quot;type&quot; value=&quot;firm&quot; /> </system.part> <system.part category=&quot;osgi.active.bundle&quot; name=&quot;com.paremus.dosgi.dsw.essencermi&quot; /> <system.part category=&quot;osgi.active.bundle&quot; name=&quot;com.paremus.dosgi.discovery.slp&quot; /> <system.part category=&quot;osgi.active.bundle&quot; name=&quot;com.paremus.dosgi.topologymanager&quot; /> <replication.handler type=&quot;scalable&quot;> <property name=&quot;scaleFactor&quot; value=&quot;1&quot; type=&quot;float&quot; /> <property name=&quot;fixedDelta&quot; value=&quot;-2&quot; type=&quot;integer&quot; /> <property name=&quot;minimum&quot; value=&quot;1&quot; type=&quot;integer&quot; /> </replication.handler> </system> <system name=&quot;gateway&quot; boundary=&quot;fibre&quot;> <system.part category=&quot;msf&quot; name=&quot;com.example.gateway&quot;> <property name=&quot;id&quot; value=&quot;foo&quot; /> </system.part> <system.part category=&quot;osgi.active.bundle&quot; name=&quot;com.paremus.dosgi.dsw.essencermi&quot; /> <system.part category=&quot;osgi.active.bundle&quot; name=&quot;com.paremus.dosgi.discovery.slp&quot; /> <system.part category=&quot;osgi.active.bundle&quot; name=&quot;com.paremus.dosgi.topologymanager&quot; /> </system> </system> Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 A System Model Replication Handlers control the number of Part instances required in the runtime.
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 System Provisioning
    • Push-based Deployment of Artifacts ✘
    • To Scale
      • Need to Avoid Bottlenecks / Choke-points:
      • Don’t Push Artifacts through the Provisioner!
    • Avoid Synchronous Behaviors
    • Also Provisioner Cannot Assume:
      • Resource Population is Static: Compute Resources may come and go.
      • Deployment Artifacts are Static: Feature Upgrades / Patches and rollbacks Occur all the Time.
    • ⟶ Pull-based - Asynchronous ‘Target State’ Provisioning ✔
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Target State Provisioning
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Target State Provisioning
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Demo: Application Agility Screencast of demo is available from YouTube at: http://bit.ly/app-agility_screencast
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011
    • DDS pub/sub Topic-based Messaging is Used for:
      • Service Discovery
      • Dynamic Configuration of Dynamically Deployed Components
      • Optional Low Latency (> 100 μ s) pub/sub Messaging for Hosted Applications
    Dynamic Configuration
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011
    • DDS pub/sub Topic-based Messaging is Used for:
      • Service Discovery
      • Dynamic Configuration of Dynamically Deployed Components
      • Optional Low Latency (> 100μs) pub/sub Messaging for Hosted Applications
    Dynamic Configuration
    • A ‘Robust’ platform is designed to survive cascading failure
    • A ‘Robust’ platform is adaptive & self-repairing: continually attempting to ‘settle’ into the desired state
    • No central or static points of command & control
    Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Adaptive & Recovery Oriented What do we mean by Robust?
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Robustness demonstration Screencast of demo is available from YouTube at: http://bit.ly/robustness_screencast
  • Atlas - Resource Management
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Managing characteristics of population rather than each individual node has a dramatic effect on decreasing runtime complexity - http://adaptevolve.blogspot.com/2008/01/complexity-part-ii-it-all-depends-on.html
      • Manage Populations NOT Instances
      • Atlas, Resource Target State & Dynamic Runtime Assembly
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Off Fabric node Bootstrap Atlas Manager Source Repository for Examples POSH CLI Application UI Service Fabric Nodes Service Fabric Infrastructure Cached Examples Deployed Systems Demo Environment A ‘blue’ Atlas Agent
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Resource Management & Bootstrap
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011
    • On Roadmap (2012):
    • Cobra - (Caching OBR for Availability)
    • Then
      • Federate Service Fabrics across data-centers
      • Enable public Cloud ‘burst’ behaviors for ‘stateless’, ‘latency insensitive’ business components into third party environments - e.g. EC2, Azure etc.
    2012 - A Federated Cloud Runtime...
  • 2012 - The Pervasive Cloud Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Service Fabric enabled Cloud Core (Ambient Assisted Living Mobile / Automotive Smart Grid )
  • 2012 - The Pervasive Cloud Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Why Paremus?
    • Industry leading OSGi platform
    • Cloud Core: The Service Fabric
    • OSGi edge: Nimble
    • Not just OSGi (WAR, EJB, binary)
    • High quality OEM support - ensuring project success
    • Aggressive Pricing
    • ESCROW option available
    • Training: High Quality OSGi Training for internal teams.
    • Tooling: OSS BNDTools project
    • Leading OSGi consulting Services:
    • Dev, Build, Test, Release, Run & Update processes
    • Ongoing guidance to ensure successful project delivery
  • Third Party Comments Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Application servers will look a lot more like elastic application platforms than like today’s products...... the Paremus Service Fabric is a good example The rise of the stack-less stack “ ...Paremus are doing really interesting leading edge work, leading edge in the sense that they are getting to and solving problems before most of us realise they are problems...” Zoe Slattery - IBM Technology Evangelist
  • Additional Slides
  • Getting there... Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011
    • Usually starts with a focused Proof of Concept
    • A single Java application
    • Develop / build / unit test / integration / release chain
    • Initial integration with production
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011
    • Creation of an in-house ‘Heroku type’ PaaS
    • Full Service Fabric Integration with Production Environment
    • Simple WAR and / or Java-based Applications Migrated to Instances of Service Fabric (i.e. a Web ‘farm’)
    • Implement Production Repositories (in-house bundles) and Third Party Repositories (OSS bundles)
    • Implement Pattern Library - Deciding which Middleware and NoSQL Solutions can be Used
    • Steppingstone to Next Phase.
    Getting there...
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Migration - Service Level Integration Integrate with Legacy Services via Usual SOA Mechanisms
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011
    • Transition of Enterprise
    • Start Migration away from ‘silos’ of Batch Grid & Middleware Stovepipes
    • Migration away from Application Servers & Other High ‘Ticket’ Middleware
    • Asynchronous / Parallel Compensational, Transactional – for High ‘Flow’ Systems
    • Enable Federated Data Center and Public Cloud Capabilities
    Getting there...
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Lab49 is a strategy, design and technology consulting firm. Lab49 deliver strategic consulting and build advanced solutions for the world’s leading investment banks, hedge funds and exchanges. “ Lab49 deliver world-class solutions to some of the most sophisticated firms in financial services today. Our innovative approach stems from adopting user-centric practices enabling clients to respond quickly to today’s challenges, as well as realize new business opportunities in the modern world of trading.
    • SDP Foreign Exchange (FX) functionality;
        • Request for Quote (RFQ)
        • Request for Stream (RFS)
        • Execution Blotter
        • Research and News
    • Lab49 SDP Leverages;
        • Paremus OSGi Service Fabric
        • Akka Open Source Agent-Actor Framework
        • Lab49 Design and Engineering
        • HTML5/JavaScript
  • Demo: Single Dealer Platform Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011
    • No Silver Bullets!
    • Consistency, Availability, Partition-ability (CAP) Trade-offs Still Apply. Use most Appropriate Data Service for each Business Application .
    • Unstructured data processing - Hadoop
    • Key / Value - Voldemort
    • Column - Cassandra
    • Graph Database - Neo
    • Relational - Derby, MySQL
    • Distribution to Nodes - BitTorrent
    What about Application Data? http://blog.nahurst.com/visual-guide-to-nosql-systems
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 http://dspace.mit.edu/bitstream/handle/1721.1/60085/BBFRFC66.pdf?sequence=1 Modularity is a Must Have not a Maybe. Patterns of Modular Architecture (Kirk Knoernschild) - http://www.kirkk.com/modularity/chapters/ Diversity and Complexity - Scott E. Page
  • Traditional VM approach Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Hypervisors Image Server sets of static VM images Deployment Service fixed address of Hypervisors
  • Traditional VM approach Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Hypervisors Image Server sets of static VM images Deployment Service fixed address of Hypervisors Static VM images built and deployed to image server
  • Traditional VM approach Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Hypervisors Image Server sets of static VM images Deployment Service fixed address of Hypervisors
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Problems with Approach
    • Resource is Static - Requires Manual Configuration
    • SPoF - Image and Deployment Services
    • If a Hypervisor Fails Hosting a Set of Images - What Does this Mean?
    • Each Change Requires Generation of a New Software Image!
    • Image Explosion
    • Network and Storage Costs
    • Versioning Issues Relating to Images
    • No Runtime Management or Re-configuration of Deployed Services
  • Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Industry Responses - mostly Proprietary
    • Naming Conventions and Versioning for Images?
    • Some Type of Runtime Description?
    • Collection of Image Types Required?
    • High Availability Critical ‘Infrastructure’ Servers
    • BETTER YET
    • Deploy Packages Directly from Software Repository to Blank VMs on HyperV’s
      • Removes Need for VM Image Assembly Tooling Step. (Puppet, CodeChef)
    • BUT
    • Still No Runtime Management or Reconfiguration of Deployed Services
  • BoAML Service Fabric Usage Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011 Hypervisors Image Server Sets of static VM images Service Fabric Deployment Service Dynamic discovery of Hypervisors Service Fabric Runtime Management
  • Service Fabric Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011
    • Service Fabric Selected by BoA to Provide:
      • Runtime Management of Deployed Services on Pre-canned Images
      • An Operationally Simpler and Self-maintaining Environment - So No Additional HA Expense
      • Well Defined Runtime Model - Which Drives Deployment and Recovery
    • But Within BoAML Design
      • Deployment Artifact – ‘Pre-canned’ VM Images??
      • If so, VM Images and Software Artifacts those Images Use Must be Versioned
      • Which Requires SVM Image Assembly Tooling
      • And, Potentially Requires Management of Two Models: VM Deployment Model and Service Fabric System Model
  • Service Fabric Copyright © 2011 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Service Fabric Overview November 2011
    • An Alternative Approach :
      • Service Fabric Deploys Vanilla Virtual Machines into Environment as Required
      • Dynamically Deploys Software Artifacts as Required
      • As Done at Present, Manages Those Software Artifacts
    • Advantages
      • One Runtime Model
      • Only Required Artifacts are Deployed – No VM Image Explosion
      • Lighter on Network and Storage
      • Can Still Ensure ‘Bootstrap’ Appliance Behavior for Sites
      • But also - Via Cobra - Low Maintenance Upgrade Behaviors
  • Thank You