• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
OW2 Clif Use Case OW2con11, Nov 24-25, Paris
 

OW2 Clif Use Case OW2con11, Nov 24-25, Paris

on

  • 600 views

 

Statistics

Views

Total Views
600
Views on SlideShare
600
Embed Views
0

Actions

Likes
0
Downloads
1
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    OW2 Clif Use Case OW2con11, Nov 24-25, Paris OW2 Clif Use Case OW2con11, Nov 24-25, Paris Presentation Transcript

    • Wide-spreading performance testingat Orange with OW2 CLIF:an SOA use caseBruno Dillenseger, Orange LabsAmong the tens of real CLIF use cases at Orange, WSOIis the web-service oriented infrastructure that runs over500 web services consumers and providers, and handlesabout half a billion calls per month for the Orange Group. OW2Con 2011, November 23-24, Orange Labs, Paris. www.ow2.org. 1
    • Testing a service performance andresilience to high traffic Probes measure usage of load injector 1 arbitrary computing resources load injector 2 resource probes resource probes System Under Testtest supervisionExecution, load injector ncontrol andmonitoring of Load injectors :load injectors ● send requests, wait for replies, measure response timesand probes. ● according to a given scenario defining the workload ● for example, emulating the load of a number of real users → virtual users OW2Con 2011, November 23-24, Orange Labs, Paris. www.ow2.org. 2
    • CLIF, an outstanding load testingframeworkAdvanced features ● dynamically adjustable number of virtual users ● support for tests of any scale – from one to millions of virtual users – from one to more than 1000 load injectors ● integrated resources monitoring ● embedded reporting toolVersatility ● OS-independent (Java 1.5+) ● integration to Eclipse ● continuous integration (Hudson/Jenkins) ● command line (through ant and maven) ● custom probes and load injectors OW2Con 2011, November 23-24, Orange Labs, Paris. www.ow2.org. 3
    • Oranges motivation for performancetesting is greatAs an integrated telecommunicationoperator, Orange manages a hugevariety of technologies ● networks, protocols ● equipments ● service platforms...More than 221 millions clients in 42 countries! ● quality of service, user experience and user confidence are key priorities for Orange ● performance issues are critical (testing, sizing, capacity planning) OW2Con 2011, November 23-24, Orange Labs, Paris. www.ow2.org. 4
    • CLIF, an OW2 project lead byOrangeThe CLIF open source project was jointly launched in2003 by INRIA and Orange in ObjectWeb/OW2Maintaining CLIF is strategic for Orange ● versatility/adaptability to almost all technologies – HTTP, SOAP, REST, FTP, DHCP, LDAP, DNS, Diameter, Radius, EAP, GBA, GTPP, TR69, SIP, RTP, proprietary protocols... ● much cheaper than specific commercial tools ● growing confidence (feedback from the community) ● community contributions ● research transfer applied to performance testing – software components, autonomic computing, cloud computing... OW2Con 2011, November 23-24, Orange Labs, Paris. www.ow2.org. 5
    • The WSOI use case: Oranges web-service oriented infrastructure Enablers & Services Enablers & Services Enablers & Services SMS/MMS, Enablers & Services identity, log, Infrastructure address book, accounting, billing... Infrastructure Infrastructure storage... Consumers Consumers Enablers & Services Consumers Enablers & Services Enablers & Services Infrastructure Enablers & Services Infrastructure Infrastructure Infrastructure end-user services: Infrastructure Consumers personal storage, Consumers Consumers Consumers Consumers TV program, musicbefore WSOI Proliferation of specific infrastructureswith WSOI Unified and shared infrastructure Enablers & Services Benefits: ● cost reduction (common security, access control, scaling, WSOI supervision, maintenance) ● ready-to-use and no-cost infrastructure for new services Challenge: Consumers & Partners ● Performance and availability! OW2Con 2011, November 23-24, Orange Labs, Paris. www.ow2.org. 6
    • WSOI performance challenge● Over 500 web services consumers and providers● Half a billion calls per month WSOI - Calls by month 500 000 000 450 000 000 400 000 000 350 000 000 300 000 000 Calls (Million) 250 000 000 200 000 000 150 000 000 100 000 000 50 000 000 0 m 10 j u 10 j u 10 m 11 j u 11 j u 11 ao -09 no -09 ao 10 no -10 11 se t-09 dé -09 ja .-09 fé -10 m .-10 se t-10 oc 1 0 dé -10 ja .-10 fé -11 m .-11 oc 09 av 10 av 11 r.- - - il.- r.- - - il.- .- .- s- s- il . t. t. v. . v. . in in ai ai vr pt vr pt û c û c nv nv ar ar ju IAS Requests SI Requests Month B2B Requests Total Requests OW2Con 2011, November 23-24, Orange Labs, Paris. www.ow2.org. 7
    • Inside WSOI XML appliances Enablers & Services Load balancer WSOI Consumers & Partners Main features: ● XML validation ● routing ● access control ● service level management ● security Configuration server LDAP directory ● encryption, compression ● scalable Mostly supported by hardware (XML appliance) OW2Con 2011, November 23-24, Orange Labs, Paris. www.ow2.org. 8
    • WSOI testbed with CLIF Performance qualification of WSOI: ● request throughput and response times ● WSOI load: – CPU and memory usage on configuration server and LDAP server – XML appliance load (via SNMP monitoring) ● according to a variety of requests and responses sizes SOAP requestsHTTP injectors WSOI Fake services and enablers system probesSNMP probe (cpu, memory) OW2Con 2011, November 23-24, Orange Labs, Paris. www.ow2.org. 9
    • Create a CLIF test project The WSOI "project" will contain all test definitions, input data and results: ● test plans files (.ctp) – definition of injectors and probes to deploy ● scenarios files (.xis) – virtual users behaviors – load profile (number of active virtual users) ● input data – a SOAP request to replay (captured with the XML appliance) ● raw measures (report) ● moving statistics on measures gathered during test executions (stats) OW2Con 2011, November 23-24, Orange Labs, Paris. www.ow2.org. 10
    • Scenario: import necessary plug-ins The FileReader plug-in will load the captured SOAP request and let it available to the HttpInjector. OW2Con 2011, November 23-24, Orange Labs, Paris. www.ow2.org. 11
    • Scenario: define virtual usersbehaviors OW2Con 2011, November 23-24, Orange Labs, Paris. www.ow2.org. 12
    • Scenario: define load profilesEvolution of the number of active virtual users with behavior B0 occording totime (in seconds).Note: the number of active virtual users may be set and changed manuallyalso at test execution time. OW2Con 2011, November 23-24, Orange Labs, Paris. www.ow2.org. 13
    • Define your load injectors and probes OW2Con 2011, November 23-24, Orange Labs, Paris. www.ow2.org. 14
    • Run a testControl of probesand load injectorsMonitoring of probesand load injectors OW2Con 2011, November 23-24, Orange Labs, Paris. www.ow2.org. 15
    • Browse the measuresraw measures fromone test execution CSV-formatted text file with raw CPU usage test plan back-upmoving statisticsfrom monitoring CSV-formatted text file with moving statistics on requests throughput, response times and errors OW2Con 2011, November 23-24, Orange Labs, Paris. www.ow2.org. 16
    • Performance analysis and reporting OW2Con 2011, November 23-24, Orange Labs, Paris. www.ow2.org. 17
    • Advanced usage: self-regulatedload injection on WSOI What is the maximum number of virtual users such that the XML appliance load keeps less than 80%? load injection invocation system control (feedback) system under test probes probes CLIF is enhanced with a load controller alarms, response times, errors alarms, resource consumption, profilingcomponent monitoring the SNMP probe and controlling the load injectors controller saturation criteria load injection policy OW2Con 2011, November 23-24, Orange Labs, Paris. www.ow2.org. 18
    • ConclusionCLIF is not just "yet another load injection" software ● high power and scalable ● versatility – user interfaces, supported protocols, monitored resources ● more advanced features to comeThe WSOI use case for Orange ● qualification of an XML appliance-based SOA infrastructure ● captured SOAP requests replayed with a plain HTTP injector ● full system load monitoring, including the XML appliance3561 CLIF downloads in November 2011 for new 2.0.7production release ● go to clif.ow2.org OW2Con 2011, November 23-24, Orange Labs, Paris. www.ow2.org. 19
    • Questions time clif@ow2.org OW2Con 2011, November 23-24, Orange Labs, Paris. www.ow2.org. 20