SOA Knowledge Kit, Developer Productivity and Performance Comparison Analysis


Published on

Frank Cohen's presentation at the Gartner Application Architecture Development and Integration (AADI) conference 2011. Covers results of an open source comparison of SOA stacks from Oracle, IBM, and TIBCO. Compares developer productivity and performance of the same use case implemented on each stack.

Published in: Technology
  • Feel free to follow up via Twitter or my blog - I do respond in both spaces.
    Are you sure you want to  Yes  No
    Your message goes here
  • slide 17 WID and WPS are not different tools - they work together. WebSphere Integration Developer is the IDE, and WebSphere Process Server is the runtime (which also includes WAS, the app server - so again, your statements about RAD and WAS are redundant, because if you are developing BPEL, you'd only choose WID+WPS.
    Are you sure you want to  Yes  No
    Your message goes here
  • slide 16 I suspect you chose to download RAD with absolutely every optional package for the download to weigh-in at the size you quoted.
    Are you sure you want to  Yes  No
    Your message goes here
  • I don't understand why you chose to use *both* RAD+WAS, and WID+WPS - since everything could be achieved using WPS (a superset of WAS) with WID as the toolkit.
    Are you sure you want to  Yes  No
    Your message goes here
  • Re: slide 15. To my knowledge, RAD 8.0.3 does not require WAS 8.0.3 but can work with several versions of the App Server.
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

SOA Knowledge Kit, Developer Productivity and Performance Comparison Analysis

  1. 1. SOA Knowledge Kit 2011How To Evaluate Highly Scalable SOA Component ArchitectureNovember 29, 2011The Results Are In: Developer Productivity and Performance Comparison AnalysisPushToTest implemented a multi-step business workflow on Oracle, IBM, and TIBCOplatforms to surface developer productivity and application performance differences. Theresults are ideal lessons and patterns for enterprise architects, development managers,and business managers.In this Workshop, PushToTest will present a developers journal of lessons learned andmetrics to compare costs, as well as load and performance results. They will also explorewhy platform choices count – covering the planning, expertise, and knowledge requiredto quickly and flexibly build and deploy highly scalable applications.Not only will they cover the results of the study and the open-sourced code and materialsto run the tests, they will also share the methodology used so you can apply thisknowledge and evaluate the results in your own environment.Come to this Workshop to learn the results. Take the SOA Knowledge Kit back to yourorganization to make the best decisions with the latest information.
  2. 2. SOA Knowledge Kit The Results Are In: Developer Productivity and Performance Comparison AnalysisFrank Cohen, +01 (408) 871-0122, November 29, 2011
  3. 3. http://soakit.pushtotest.comAbout PushToTest‣Our Mission: Test, To Make A More Reliable World‣Make It Easier To Move From Manual To Automated Testing‣Support Advanced Testing, including Agile, Load, Integration‣Open Source Testing (OST) For Everyone3
  4. 4. http://soakit.pushtotest.comResults• ActiveMatrix + BusinessWorks proven to require less time and costs in side-by-side implementation • 29% less time/costs vs. Oracle • 22% less time/costs vs IBM• Greatest productivity gains: $8,000 $7,074 Service Construction $6,469 $6,000 Service Orchestration $5,026 Policy Management $4,000 Security Configuration Change Management $2,000 $0 Oracle IBM4 TIBCO
  5. 5. http://soakit.pushtotest.comWhat We Need‣Models versus Code/Annotations‣Agile, Rapid, Performance‣Significant Maintenance Costs Over Long Term5
  6. 6. http://soakit.pushtotest.comWhat We Get‣Oracle ‣(especially of theout-of-the-box, required community support Nothing worked BEA components) ‣Tools focused on developers, not architects and business analysts ‣Fusion strategy gives more concise installations, still version problems‣TIBCO ‣SOA Oriented Tools and Deployment ‣Tools are more robust, but still have some conflicts ‣Knowledge (Guides, Tutorials) need improvement‣IBM ‣Several Big Platforms Integrated Together ‣Parts Don’t Play Well Together6 ‣Problems With Standards Adherence
  7. 7. http://soakit.pushtotest.comMethodology‣Define A Manufacturing Use Case‣Implement The Use Case‣Make Changes To The Implementation ‣Add HTTPS/SSL Security ‣Change Message Schema ‣Change To Asynchronous Message Delivery‣Run a Functional and Performance Test‣Total Cost of Ownership (TCO) Model Implemented using 2 engineering teams. Team A - TIBCO and Oracle7 Team B - IBM
  8. 8. http://soakit.pushtotest.comDownload Now‣ ‣Compares Developer Productivity between TIBCO, Oracle, IBM ‣Open Source Code Implementations ‣Developer Journals ‣Executive Slide Deck ‣Invitation to SOA Knowledge Kit Workshop - Free Webinar ‣Total Cost of Ownership Model ‣PushToTest TestMaker Platform‣Free Open Source License8
  9. 9. Next let’s talk about Oracle ‣(especially of theout-of-the-box, required community support Nothing worked BEA components) ‣analysts on developers, not architects and business Tools focused9
  10. 10. http://soakit.pushtotest.comOracle Summary‣Oracle 11g has problems not work out of the box ‣Delivered solutions may ‣external services and configurations required knowledge from Special blogs and forums ‣Most improved came from workarounds found by third parties solutions‣Oracle concise, still has versioningtools the packaging of ‣Moresolution creation for engineersproblems‣Quick dive is expected ‣Code adjustments to descriptors, classpaths and general ‣configuration Manual ‣analysts tools are for developers - not for architects and business Oracles ‣Based on the non-standard JDeveloper IDE10
  11. 11. http://soakit.pushtotest.comOracle Getting Started‣From Idea Tothe tools for the task using only the web site was easy Code ‣Identifying Guide straight forward and useful ‣Quickdays for installation Start‣Plan 3 size: 14 Gbyte, No download manager (register each time) ‣Download has problems installing different versions of the ‣application in the environment, not properly described in the Oracle Suite installation guide‣ Tutorials Surface Problems ‣ Web Service creation requires Java coding experience: annotations ‣ Proprietary annotations like oracle.webservices.annotations.PortableWebService not found/included by the tool, workarounds needed ‣ The generated web services failed to generate the correct WSDL11 • Another manual intervention to quick fix the issue was needed
  12. 12. http://soakit.pushtotest.comOracle Learning‣For each of the services we found -severalfor simple solutions - that tutorials ‣didn’tneeded towork workarounds even We actually find as they should have‣chose tutorials on Web Service creation on the Oracle site. The one we Many failed: ‣ws.html#t5 ‣SOAPUI web service mocks can not be called from Oracle12
  13. 13. http://soakit.pushtotest.comOracle Learning‣Found 4 tutorials on building Asynchronous Servers ‣asynch.htm#CBHECBFG‣We used annotations that are not available on the app server ‣@AsyncWebService and @PortableWebService tags ‣import oracle.webservices.annotations.PortableWebService ‣import oracle.webservices.annotations.async.AsyncWebService13
  14. 14. Next let’s talk about IBM ‣Several Big Platforms Integrated Together ‣Parts Don’t Play Well Together ‣Problems With Standards Adherence14
  15. 15. http://soakit.pushtotest.comIBM Summary‣Several Big Platforms Integrated Together Integration Developer, ‣Rational Application Developer, WebSphere Process Server WebSphere Application Server, WebSphere ‣Each7.0 requires WASothers above 1Gbyte, RAD was 8 Gbytes tool 600Mbyte+, ‣WID a day at Disneyland toand RADall the options WAS 8.0.3 7.0 8.0.3 requires ‣LikeDon’t Play Well Togetherexplore‣Parts 7 Refactoring Problems (namespace, name, schema) ‣WID and process development happen in different tools (WID ‣and WPS) Integration ‣RAD cannot develop business process, can’t be deployed to WAS‣Problems With Standards Adherence ‣WID generated the corrupt WSDLs service ‣Could not create the Client for WPS15
  16. 16. http://soakit.pushtotest.comIBM Installation‣Each tool 600Mbyte+, others above 1Gbyte, RAD was 8 Gbytes‣WID 7.0 requires WAS 7.0 and RAD 8.0.3 requires WAS 8.0.3‣Rational Application Developer ‣1.5 hours to install ‣Automatically starts the installation for Websphere Application Server ‣with Googlegosearch.failedthe “reconcile” property asFound and start In the first it got Set and gave a plug-in error. “true” solution the installation manager through command prompt. That again took 1.5 hrs.‣Started the RAD, Encountered another error ‣JVM could not be started ‣SDP) “-Xmx1024m”, I changed it to “-Xmx896m” VM parameters required: Eclipse config file (C:Program FilesIBM16
  17. 17. http://soakit.pushtotest.comIBM Construction‣Can’t Get There from Here ‣WID generated the corrupt WSDLs ‣Could not create the Client for WPS service ‣RAD doesn’t have capability to develop business processes ‣Business process cannot be deployed in WAS ‣WID and WPS process development will happen in different tools Integration and‣WID V7.0 ‣not updated interface, target Namespace, name and schema location Refactoring17
  18. 18. Next let’s talk about TIBCO ‣SOA Oriented Tools and Deployment ‣Tools are more robust, but still have some conflicts ‣Knowledge (Guides, Tutorials) need improvement18
  19. 19. http://soakit.pushtotest.comTIBCO‣SOA Oriented Tools and Deployment ‣abstractingare model oriented, keeping a betterThere is of code and The tools away the underlying technologies. control no need to look at code, just models with SOA Concepts. ‣ Code oriented developers non-SOA has a steep learning curve ‣ The documentation does not help as it is more oriented to describing the functionality rather than showing how to do things‣ Tools are more robust, but still have some conflicts ‣ The modeling language is a proprietary one, reducing intuitiveness. ‣ Eclipse Based IDE that is familiar to many developers ‣ Missed Admin configuration for BW SE and deployed to Tomcat ‣ Engineers used to Message Driven Beans (MDB) approach should be using JMS for Asynchronous Services ‣19 BW WSDL modeling using a single field with complex data type failed
  20. 20. http://soakit.pushtotest.comTIBCO Installation‣Lack of Trial Software ‣installation instructionswas anot available there are no trials and the Getting the tool suite are problem as in the site. ‣order isisveryquick start guide to follow and it seems the installation There no important.‣After registration, the tool suite download was quick ‣Even when the tools are split in 6 different modules.‣Installation using default settings caused some problems ‣problemcoordination between toolscharacter (underscore)Although the So the was related to an invalid is not totally correct. that is widely used.20
  21. 21. http://soakit.pushtotest.comTIBCO Installation‣2.5 hours to learn where to begin ‣architects Much commercial info at the site but no real guide for developers or‣Knowledge Is There, Challenge Is To Find It ‣Documentation abundant and quite good, finding right guide is hard ‣The Eclipse base IDE is a plus in productivity for Eclipse users ‣Failed BW SE to create or select a TIBCO server for deployment ‣Asynchronous use case, references for using JMS, but no how ‣Mediation flow was required to implement asynchronous services ‣business_studio_process_developer_guide.pdf • Helped us realized thatBusiness Studio IDE. Java web services using TIBCO developers must create21
  22. 22. http://soakit.pushtotest.comTCO Model TCO Summary TIBCO Oracle IBM System Setup $609 $844 $750 Design/Review $1,125 $1,125 $0 Learning $1,148 $1,688 $750 Support $0 $0 $375 Service Construction - Normal Serv $375 $234 $750 Service Construction - Async Serv $784 $933 $750 Service Orchestration $281 $891 $938 Deployment $188 $609 $375 Change - Add Security $94 $422 $609 Change - Transport to JMS $188 $188 $516 Change - Complex XSD $234 $141 $281 $5,026 $7,074 $6,469 TIBCO vs Oracle 29% TIBCO vs IBM 22%22
  23. 23. http://soakit.pushtotest.comWhere To Go From Here‣Download the SOA Knowledge Kit ‣‣Attend The Open Source Test Workshop for CIOs, CTOs ‣‣Watch A Screencast and Tutorial ‣ ‣ActiveMatrix Service Grid, ActiveSpaces screencast •‣PushToTest Services Available ‣+01 408 871 0122 (USA), sales@pushtotest.com23