2011 Q4 Technical Update
     and Roadmap
      Paul Fremantle, Co-Founder and CTO
                  @pzfreo #wso2
    http://pzf.fremantle.org http://wso2.com
Agenda
•   Significant updates during 2011
•   Recent releases
•   Focus areas for 2012
•   Roadmap highlights
http://www.flickr.com/photos/andrewfarquhar/
Carbon and Stratos Release History




              © WSO2 2011
MAJOR UPDATES FOR 2011
Carbon Core
• JTA and distributed 2-phase transaction support
• Deployment Synchronizer (including SVN
  support)
• Embedded LDAP directory server
• Kerberos Support
• SAML 2.0 sign-on for all admin consoles

• Message Broker Product
  – Plus embeddable message broker component
• Complex Event Processing Server
Deployment Synchronizer
                              Options:
                              • Use a master to populate the store or do
                                it as part of the development/deployment
                                process
            Dev Environment   • Use WSO2 Registry or SVN as a store
                              • Poll the store for updates or notification
                                (push) model



 SVN-based store                        Registry store




                              Local checkout
Master




                              Server Cluster
                                 © WSO2 2011
Enterprise Service Bus
• Relay / Passthrough Transport
   – Significant performance improvement for load-balancing,
     routing and security gatewaying
• Message Stores and Processors, Embedded Message
  Broker
• Templates
• New mediators
   – Conditional Router, PayloadFactory, URL Rewrite Mediator
• New HL7 support including MLLP
• Enhanced support for service chaining, many improved
  mediators, plus usual bug fixes
Governance Registry
• SCXML based lifecycle support
• New Registry MetaModel Extension format
• UDDI v3 support
• Caching support – significant performance
  update
• Improved Impact Analysis
• Clustering Support
• Plus lots lots more!
Governance
Lifecycles
Meta Model Extensions
<artifactType type="application/vnd.wso2-project+xml" shortName="projects"
singularLabel="Project" pluralLabel="Projects" hasNamespace="false" iconSet="4">
    <storagePath>/projects/@{name}</storagePath>
    <ui>
        <list>
            <column name="Name">
                <data type="path" value="overview_name" href="@{storagePath}"/>
            </column>
        </list>
    </ui>
    <relationships>
        <association type="isPartOf" source="@{assets_entry:value}"/>
    </relationships>
    <content>
        <table name="Overview">
            <field type="text" required="true">
                <name>Name</name>
            </field>
            <field type="text" required="true">
                <name>Project Manager</name>
            </field>
            <field type="text-area">
                <name>Description</name>
            </field>
        </table>
Other major enhancements
• WSO2 Data Services Server
   – Column Storage Services / Cassandra Support
   – Full 2PC distributed transactions
   – Boxcarring
• WSO2 Identity Server
   – XACML Editor
• WSO2 Business Process Server
   – Clustering Support
   – Long running instance management views
• WSO2 Business Activity Monitor
   – Customizable reports, significant updates around
     performance and stability
Carbon Studio
• 12 releases in 2011!
• Full editor for building Carbon and Stratos applications
   –   Webapps
   –   Services
   –   ESB flows
   –   BPEL flows
   –   Registry configurations
• Significant continuous improvements throughout the
  year
   – WSO2 project teams completed full engagements 100%
     using Studio
• More to come in 2012 
Stratos / StratosLive
• Major release update 1.5.0 in June plus 1.5.2 this
  quarter
   –   Cassandra and MySQL multi-tenant support
   –   Billing and Metering
   –   Google Apps Integration
   –   Deployment Synchronizer including SVN based
   –   Elastic Load Balancer
   –   1.5.2 is significantly includes many updates based on
       our experiences running in 24x7 production as
       StratosLive
What is Cassandra?
• Apache Cassandra http://cassandra.apache.org/
• NoSQL column family implementation
• Highly scalable, available and no single point of failure.
• Very high write throughput and good read throughput.
• SQL like query language (from 0.8) and support search
  through secondary indexes
• Tunable consistency and support replication
• Loose/Flexible Schema
Cassandra Approach
Table          Data Items

Books          “Foundation” -> (author=asimov, rank=9, price=14, tag1=“sci-fi” tag2=“future”)
               “I Robot” -> (author=asimov, rank=7, price=14, tag1=“sci-fi” tag2=“robots”)
Tags2Books     “sci-fi” -> 1311031405918=“Foundation”, 1311031405919=“I Robot”
               “future” -> …
Tags2Authors   “sci-fi” -> 1311031405920=“Asimov”
               “future” -> …
Comments       “Foundation” -> (1311031405922=“best book-sanjiva”, 1311031405923=“well I
               disagree-srinath”)
               “I Robot” -> (1311031405924=“Asimov’s best-srinath”, 1311031405928=“I like
               foundation better-sanjiva”)


Ranks          Rank -> (9=“Foundation”, 7=“I Robot”)
Carbon/Stratos
Implementation
http://www.flickr.com/photos/rbs/
Focus Areas for 2012
•   Tooling, Development and Presentation
•   Integration, API Management and Governance
•   Analytics, Data and Monitoring
•   Cloud, Cloud development, Services Store
•   Carbon and Stratos Foundations
Tooling, Development, Presentation
• Carbon Studio 2.0
   – Flow based ESB tooling
   – Better integration into Stratos for deploying cApps
• Jaggery
   – An application development environment in JavaScript
   – Including Data, Presentation and Business Logic
• Gadgets as part of webapps
   – Completely embeddable as part of a JSP, JSSP, or Servlet
• Significant performance improvements for JAX-WS
• Better support for hosting webapps
   – Virtual hosts, etc
ESB Flow Based Tooling
Integration
• ESB API management and REST improvements
    – Already in the trunk
    – Further performance improvements
    – Synapse 2.1 support
• Governance Registry
    – Out of the box metamodel improvements
    – Notification bar
    – S-RAMP support
• Identity Server
    – XACML 2.0, Simple Cloud Identity Management (SCIM)
    – Many further improvements including better embedability and REST
• Data Services
    – Cassandra-based Data Services
• Message Broker
    – Looking at creating a much more scalable and clusterable offering
API Management Solution
API Management Solution
• Available today as a combination of product
  plus services
  – WSO2 ESB, G-Registry, IS, BAM
• OAuth keys, binary relay routing, analytics
  from BAM
• All managed based on policies
• Can include XACML based entitlement
Analytics and Data
• Significant overhaul of BAM
  – New agent model
     • Non-blocking asynchronous binary data publisher
  – Cassandra support for higher scalability
  – New analyzer framework
  – Improved Gadget creation tooling (Gadget IDE)
• Beyond 2.0
  – Include Rules and CEP support to enhance real-
    time analytics
BAM2 Architecture
Data Publisher API (draft)
Event event = new Event();

event.setCorrelation(createCorrelationMap());
event.setEvent(createEventDataMap());
event.setMeta(createMetaDataMap());

EventReceiver eventReceiver = new EventReceiver();
eventReceiver.setSocketTransportEnabled(true);
eventReceiver.setUrl("https://localhost:9443/");
eventReceiver.setUserName("admin");
eventReceiver.setPassword("admin");
eventReceiver.setPort(7611);

ArrayList<Event> events = new ArrayList<Event>();
events.add(event);

BasicConfigurator.configure();
setTrustStoreParams();

AgentConfiguration configuration = new AgentConfiguration();
Agent agent = new Agent(configuration);
agent.publish(events, eventReceiver);
agent.shutdown();



                                 © WSO2 2011
Analyzer configuration screen




            © WSO2 2011
Gadget IDE




   © WSO2 2011
Gadget IDE




   © WSO2 2011
More Data
• Cloud File System
  – HDFS support in Stratos
• Map Reduce Service
  – Hadoop-as-a-Service
Core Carbon and Stratos
• Updates to Equinox level
    – Support for dOSGi
•   Better support for OSGi and Tomcat
•   Improvements to cApp and cApp deployment
•   Multi-language support for Stratos
•   Stratos Integrated development-time support
    – SVN projects, Maven build
• Improved Stratos homepage
• ….
Vertical PaaS
Users
Partners




                  Tenant          Tenant             Tenant    Tenant
                   Apps            Apps               Apps      Apps
Ecosystem Owner




                                     PaaS / Stratos

                                  API Management / API Store


                           Existing business systems as APIs
Automated Cloud Development
• Integrating Redmine, JIRA, SVN, Governance Registry,
  Maven, Stratos and IS together with BPS+ESB
   – Creating a project in Redmine creates SVN, Stratos Project,
     and populates users
   – Checking in a project into SVN with “Deploy” tags initiates
     a review process in BPEL
      • Automatically creates JIRA entries for code review, security review
      • When these are closed, the process progresses through staging
      • Eventually the project is automatically deployed into production
        cloud
• Planned for inclusion in StratosLive in 2012
   – Can be customized for your environment and projects
Shared Service Solution
Key resources
• Me:
  – paul@wso2.com
  – #pzfreo

• OxygenTank
  – http://wso2.org
  – WSO2’s forge and developer portal
  – Documentation, articles, blogs and more

• http://wso2.com

                        © WSO2 2011

WSO2 Quarterly Technical Update

  • 1.
    2011 Q4 TechnicalUpdate and Roadmap Paul Fremantle, Co-Founder and CTO @pzfreo #wso2 http://pzf.fremantle.org http://wso2.com
  • 2.
    Agenda • Significant updates during 2011 • Recent releases • Focus areas for 2012 • Roadmap highlights
  • 3.
  • 4.
    Carbon and StratosRelease History © WSO2 2011
  • 5.
  • 6.
    Carbon Core • JTAand distributed 2-phase transaction support • Deployment Synchronizer (including SVN support) • Embedded LDAP directory server • Kerberos Support • SAML 2.0 sign-on for all admin consoles • Message Broker Product – Plus embeddable message broker component • Complex Event Processing Server
  • 7.
    Deployment Synchronizer Options: • Use a master to populate the store or do it as part of the development/deployment process Dev Environment • Use WSO2 Registry or SVN as a store • Poll the store for updates or notification (push) model SVN-based store Registry store Local checkout Master Server Cluster © WSO2 2011
  • 8.
    Enterprise Service Bus •Relay / Passthrough Transport – Significant performance improvement for load-balancing, routing and security gatewaying • Message Stores and Processors, Embedded Message Broker • Templates • New mediators – Conditional Router, PayloadFactory, URL Rewrite Mediator • New HL7 support including MLLP • Enhanced support for service chaining, many improved mediators, plus usual bug fixes
  • 9.
    Governance Registry • SCXMLbased lifecycle support • New Registry MetaModel Extension format • UDDI v3 support • Caching support – significant performance update • Improved Impact Analysis • Clustering Support • Plus lots lots more!
  • 10.
  • 11.
    Meta Model Extensions <artifactTypetype="application/vnd.wso2-project+xml" shortName="projects" singularLabel="Project" pluralLabel="Projects" hasNamespace="false" iconSet="4"> <storagePath>/projects/@{name}</storagePath> <ui> <list> <column name="Name"> <data type="path" value="overview_name" href="@{storagePath}"/> </column> </list> </ui> <relationships> <association type="isPartOf" source="@{assets_entry:value}"/> </relationships> <content> <table name="Overview"> <field type="text" required="true"> <name>Name</name> </field> <field type="text" required="true"> <name>Project Manager</name> </field> <field type="text-area"> <name>Description</name> </field> </table>
  • 12.
    Other major enhancements •WSO2 Data Services Server – Column Storage Services / Cassandra Support – Full 2PC distributed transactions – Boxcarring • WSO2 Identity Server – XACML Editor • WSO2 Business Process Server – Clustering Support – Long running instance management views • WSO2 Business Activity Monitor – Customizable reports, significant updates around performance and stability
  • 13.
    Carbon Studio • 12releases in 2011! • Full editor for building Carbon and Stratos applications – Webapps – Services – ESB flows – BPEL flows – Registry configurations • Significant continuous improvements throughout the year – WSO2 project teams completed full engagements 100% using Studio • More to come in 2012 
  • 14.
    Stratos / StratosLive •Major release update 1.5.0 in June plus 1.5.2 this quarter – Cassandra and MySQL multi-tenant support – Billing and Metering – Google Apps Integration – Deployment Synchronizer including SVN based – Elastic Load Balancer – 1.5.2 is significantly includes many updates based on our experiences running in 24x7 production as StratosLive
  • 15.
    What is Cassandra? •Apache Cassandra http://cassandra.apache.org/ • NoSQL column family implementation • Highly scalable, available and no single point of failure. • Very high write throughput and good read throughput. • SQL like query language (from 0.8) and support search through secondary indexes • Tunable consistency and support replication • Loose/Flexible Schema
  • 16.
    Cassandra Approach Table Data Items Books “Foundation” -> (author=asimov, rank=9, price=14, tag1=“sci-fi” tag2=“future”) “I Robot” -> (author=asimov, rank=7, price=14, tag1=“sci-fi” tag2=“robots”) Tags2Books “sci-fi” -> 1311031405918=“Foundation”, 1311031405919=“I Robot” “future” -> … Tags2Authors “sci-fi” -> 1311031405920=“Asimov” “future” -> … Comments “Foundation” -> (1311031405922=“best book-sanjiva”, 1311031405923=“well I disagree-srinath”) “I Robot” -> (1311031405924=“Asimov’s best-srinath”, 1311031405928=“I like foundation better-sanjiva”) Ranks Rank -> (9=“Foundation”, 7=“I Robot”)
  • 17.
  • 18.
  • 19.
    Focus Areas for2012 • Tooling, Development and Presentation • Integration, API Management and Governance • Analytics, Data and Monitoring • Cloud, Cloud development, Services Store • Carbon and Stratos Foundations
  • 20.
    Tooling, Development, Presentation •Carbon Studio 2.0 – Flow based ESB tooling – Better integration into Stratos for deploying cApps • Jaggery – An application development environment in JavaScript – Including Data, Presentation and Business Logic • Gadgets as part of webapps – Completely embeddable as part of a JSP, JSSP, or Servlet • Significant performance improvements for JAX-WS • Better support for hosting webapps – Virtual hosts, etc
  • 21.
  • 22.
    Integration • ESB APImanagement and REST improvements – Already in the trunk – Further performance improvements – Synapse 2.1 support • Governance Registry – Out of the box metamodel improvements – Notification bar – S-RAMP support • Identity Server – XACML 2.0, Simple Cloud Identity Management (SCIM) – Many further improvements including better embedability and REST • Data Services – Cassandra-based Data Services • Message Broker – Looking at creating a much more scalable and clusterable offering
  • 23.
  • 24.
    API Management Solution •Available today as a combination of product plus services – WSO2 ESB, G-Registry, IS, BAM • OAuth keys, binary relay routing, analytics from BAM • All managed based on policies • Can include XACML based entitlement
  • 25.
    Analytics and Data •Significant overhaul of BAM – New agent model • Non-blocking asynchronous binary data publisher – Cassandra support for higher scalability – New analyzer framework – Improved Gadget creation tooling (Gadget IDE) • Beyond 2.0 – Include Rules and CEP support to enhance real- time analytics
  • 26.
  • 27.
    Data Publisher API(draft) Event event = new Event(); event.setCorrelation(createCorrelationMap()); event.setEvent(createEventDataMap()); event.setMeta(createMetaDataMap()); EventReceiver eventReceiver = new EventReceiver(); eventReceiver.setSocketTransportEnabled(true); eventReceiver.setUrl("https://localhost:9443/"); eventReceiver.setUserName("admin"); eventReceiver.setPassword("admin"); eventReceiver.setPort(7611); ArrayList<Event> events = new ArrayList<Event>(); events.add(event); BasicConfigurator.configure(); setTrustStoreParams(); AgentConfiguration configuration = new AgentConfiguration(); Agent agent = new Agent(configuration); agent.publish(events, eventReceiver); agent.shutdown(); © WSO2 2011
  • 28.
  • 29.
    Gadget IDE © WSO2 2011
  • 30.
    Gadget IDE © WSO2 2011
  • 31.
    More Data • CloudFile System – HDFS support in Stratos • Map Reduce Service – Hadoop-as-a-Service
  • 32.
    Core Carbon andStratos • Updates to Equinox level – Support for dOSGi • Better support for OSGi and Tomcat • Improvements to cApp and cApp deployment • Multi-language support for Stratos • Stratos Integrated development-time support – SVN projects, Maven build • Improved Stratos homepage • ….
  • 33.
    Vertical PaaS Users Partners Tenant Tenant Tenant Tenant Apps Apps Apps Apps Ecosystem Owner PaaS / Stratos API Management / API Store Existing business systems as APIs
  • 34.
    Automated Cloud Development •Integrating Redmine, JIRA, SVN, Governance Registry, Maven, Stratos and IS together with BPS+ESB – Creating a project in Redmine creates SVN, Stratos Project, and populates users – Checking in a project into SVN with “Deploy” tags initiates a review process in BPEL • Automatically creates JIRA entries for code review, security review • When these are closed, the process progresses through staging • Eventually the project is automatically deployed into production cloud • Planned for inclusion in StratosLive in 2012 – Can be customized for your environment and projects
  • 35.
  • 36.
    Key resources • Me: – paul@wso2.com – #pzfreo • OxygenTank – http://wso2.org – WSO2’s forge and developer portal – Documentation, articles, blogs and more • http://wso2.com © WSO2 2011