Your SlideShare is downloading. ×
IBM Rational Asset Manager Version 7.0.0.2
           Capacity and scalability benchmarks
                                ...
IBM Rational Asset Manager capacity and scalability benchmark, Page 2 of 33



Contents
Abstract ............................
IBM Rational Asset Manager capacity and scalability benchmark, Page 3 of 33




Abstract
The aim of this article is to pro...
IBM Rational Asset Manager capacity and scalability benchmark, Page 4 of 33



Test configuration goals
We intended to exp...
IBM Rational Asset Manager capacity and scalability benchmark, Page 5 of 33


Figure 1. Logical Rational Asset Manager ser...
IBM Rational Asset Manager capacity and scalability benchmark, Page 6 of 33



Test environment
Hardware topology
Figure 2...
IBM Rational Asset Manager capacity and scalability benchmark, Page 7 of 33




The wait times between pages were coded as...
IBM Rational Asset Manager capacity and scalability benchmark, Page 8 of 33


Figure 3. Peak capacity and duration runs


...
IBM Rational Asset Manager capacity and scalability benchmark, Page 9 of 33


   3. Drawing a string from another Rational...
IBM Rational Asset Manager capacity and scalability benchmark, Page 10 of 33




Download Artifacts scenario: In the Downl...
IBM Rational Asset Manager capacity and scalability benchmark, Page 11 of 33


Figure 5. Download artifacts




Keyword Se...
IBM Rational Asset Manager capacity and scalability benchmark, Page 12 of 33




The test structure is pictured in Figure ...
IBM Rational Asset Manager capacity and scalability benchmark, Page 13 of 33


    4. Sign out after you complete the loop...
IBM Rational Asset Manager capacity and scalability benchmark, Page 14 of 33


    3. Using the Basic interface for asset ...
IBM Rational Asset Manager capacity and scalability benchmark, Page 15 of 33



Benchmark results
Figure 9 reflects the ov...
IBM Rational Asset Manager capacity and scalability benchmark, Page 16 of 33




As the results show, vertical scaling is ...
IBM Rational Asset Manager capacity and scalability benchmark, Page 17 of 33


Figure 11. Asset scalability




User scala...
IBM Rational Asset Manager capacity and scalability benchmark, Page 18 of 33


Figure 12. User scalability




Servers res...
IBM Rational Asset Manager capacity and scalability benchmark, Page 19 of 33


Figure 13. CPU use




Figure 14. Memory us...
IBM Rational Asset Manager capacity and scalability benchmark, Page 20 of 33


Figure 15. Disk use




As mentioned previo...
IBM Rational Asset Manager capacity and scalability benchmark, Page 21 of 33


Figure 16. Asset scaling




Although perfo...
IBM Rational Asset Manager capacity and scalability benchmark, Page 22 of 33


Figure 17. Vertical cluster scaling




Fig...
IBM Rational Asset Manager capacity and scalability benchmark, Page 23 of 33


Figure 18. Database scaling




Finally, st...
IBM Rational Asset Manager capacity and scalability benchmark, Page 24 of 33


Although some large assets were included, t...
IBM Rational Asset Manager capacity and scalability benchmark, Page 25 of 33



Appendix: Test results
The test results de...
IBM Rational Asset Manager capacity and scalability benchmark, Page 26 of 33


Assets       Users         Nodes     Instan...
IBM Rational Asset Manager capacity and scalability benchmark, Page 27 of 33


Figure 20. User scalability with 100,000 as...
IBM Rational Asset Manager capacity and scalability benchmark, Page 28 of 33


Figure 22. User scalability with 1,000,000 ...
IBM Rational Asset Manager capacity and scalability benchmark, Page 29 of 33



Throughput scalability

Because the usage ...
IBM Rational Asset Manager capacity and scalability benchmark, Page 30 of 33


Figure 25. Page throughput with 300,000 ass...
IBM Rational Asset Manager capacity and scalability benchmark, Page 31 of 33


Figure 27. Page throughput with 2,000,000 a...
IBM Rational Asset Manager capacity and scalability benchmark, Page 32 of 33



Trademarks
IBM, the IBM logo, and Rational...
IBM Rational Asset Manager capacity and scalability benchmark, Page 33 of 33



About the authors
Dr. Mendel is the techni...
Upcoming SlideShare
Loading in...5
×

IBM Rational Asset Manager Version 7.0.0.2

925

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
925
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
11
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "IBM Rational Asset Manager Version 7.0.0.2"

  1. 1. IBM Rational Asset Manager Version 7.0.0.2 Capacity and scalability benchmarks IBM Corporation Development Gili Mendel, Senior Technical Staff Member Sheehan Anderson, Software Developer Rich Kulp Performance test John Reinstrom, Chief Performance Test Architect Amy Pitts, Performance Tester Skye Bischoff, Software Test Specialist Jennifer Chang, Staff Software Engineer IBM Innovation Center Joanne Bick Lee Bliss Marlon Machado Scott Martin Cindy O’Brien Alan White IBM Software Services for Rational Bryan Miller, Service Offering Development Lead Level: Intermediate March 2008
  2. 2. IBM Rational Asset Manager capacity and scalability benchmark, Page 2 of 33 Contents Abstract ................................................................................................................. 3 Purpose and content................................................................................................. 3 Benchmark test goals ............................................................................................... 3 Test configuration goals ............................................................................................ 4 Benchmark environment ........................................................................................... 4 Rational Asset Manager repository .......................................................................... 4 Test environment ................................................................................................. 6 Benchmark results ................................................................................................. 15 Asset scalability.................................................................................................. 16 User scalability ................................................................................................... 17 Servers resource utilization levels ......................................................................... 18 Appendix: Test results ............................................................................................ 25 Test matrix ........................................................................................................ 25 Trademarks........................................................................................................... 32 Resources ............................................................................................................. 32 About the authors .................................................................................................. 33 Copyright © 2008, IBM® Corporation. All rights reserved. Published by IBM® developerWorks®.
  3. 3. IBM Rational Asset Manager capacity and scalability benchmark, Page 3 of 33 Abstract The aim of this article is to provide guidance in specifying appropriate deployment architecture for IBM® Rational® Asset Manager. The authors provide specific results and analysis of a performance benchmark test that was conducted for Rational Asset Manager Version 7.0.0.2. This report also helps you interpret the results of this benchmark test and explains how to use these results to extrapolate specific hardware requirements for an environment that was not specifically tested. Purpose and content The aim of this article is to guide readers in specifying appropriate deployment architecture for IBM Rational Asset Manager. The authors give specific results and an analysis of a performance benchmark test that was conducted for Rational Asset Manager Version 7.0.0.2. In addition, they provide information to help you interpret the results of this benchmark, as well as to be able to use these results to extrapolate specific hardware requirements for an environment that was not specifically tested. This article does not provide an overview of Java™ 2 Platform, Enterprise Edition (J2EE) scaling, nor does it cover tuning parameters to be used in a large-scale deployment. You can find an overview of Rational Asset Manager as a J2EE application in the Rational Asset Manager 7.0 Capacity Planning Guide 1. For a comprehensive overview of how to tune a large IBM® WebSphere® Application Server cluster for a Rational Asset Manager deployment, consult the Rational Asset Manager Tuning Guide 2. (See the Resources section at the end for links to both.) Benchmark test goals These were the goals of this benchmark test: Determine the level of user loads that could be driven across small, medium, and large cluster configurations. These ranged from a single server node running a single WebSphere container instance up to a 6-server node configuration with 4 WebSphere Application Server instances per node (for a total of 24 instances). Collect comparative data on the impact of vertical versus horizontal scaling on the capacity of Rational Asset Manager. (Horizontal scaling refers to a configuration that expands the cluster by adding server nodes, where as vertical scaling refers to a configuration that increases the number of WebSphere Application Server instances running on a given server node.) Verify that Rational Asset Manager can continue to scale both horizontally and vertically with a large amount of assets. Collect performance data metrics for a mixture of configuration permutations to allow extrapolation or modeling for generic capacity level needs. Copyright © 2008, IBM® Corporation. All rights reserved. Published by IBM® developerWorks®.
  4. 4. IBM Rational Asset Manager capacity and scalability benchmark, Page 4 of 33 Test configuration goals We intended to expose the Rational Asset Manager repository to varying levels of assets by using 9 different configurations that represented small, medium, and large deployments. Our objective was to maintain an average response time of less than one second during our test duration. We also limited the configuration variation to only horizontal and vertical scaling of the WebSphere Application Server instances. The hardware used for the application servers, database, Web server, caching proxy, and load balancer remained constant. Table 1 represents the test matrix that we used to collect data for a particular asset level uploaded to Rational Asset Manager. Table 1. Test matrix used for collecting data for particular asset levels uploaded to Rational Asset Manager Application server configuration 1 node 3 nodes 6 nodes 1 instance Configuration 1 Configuration 4 Configuration 7 2 instances Configuration 2 Configuration 5 Configuration 8 4 instances Configuration 3 Configuration 6 Configuration 9 For each configuration, we exposed the cluster to the maximum number of users that could be sustained for the duration of our tests. Note that the server configuration that we used could support, at most, 4 WebSphere Application Server instances on a single server before performance degraded (see the Rational Asset Manager repository section, which follows, for specific server configurations). Benchmark environment Rational Asset Manager repository Hardware topology The basis for our configuration was 6 IBM AIX application servers, each running multiple instance of WebSphere Application Server. Figure 1 depicts the logical server deployment architecture. In reality, we used logical partitions (LPARs) on an IBM System 5 Enterprise Server, as detailed in Table 2. The communication network was a dedicated 1 Gbps Ethernet network. The storage area network used for the external disks was based on an IBM DS48005 Storage Area Network (SAN). Each server had the OS and paging space on separate internal drives. Copyright © 2008, IBM® Corporation. All rights reserved. Published by IBM® developerWorks®.
  5. 5. IBM Rational Asset Manager capacity and scalability benchmark, Page 5 of 33 Figure 1. Logical Rational Asset Manager server topology Table 2. Hardware components Server System/CPU Memory External disk (DS 4800 SAN) Application server: WebSphere 6 p595 LPARs 15GB Lucene index 24+P Application Server 6.1 Fix Pack 11 4x1.9GHz RAID5 64K segment 2GB Java™ Virtual Machine (JVM) Power5 heap per instance Rational Asset Manager 7.0.0.2 Web Server IHS 6.1 2 p595 LPARs 8GB Fix Pack 11 2x1.9GHz Power5 Database server IBM® DB2® 9.1 1 p570 LPAR 32GB Data 8+P RAID5 log Fix Pack 3a 8x2.2GHz 4+P RAID5 64K Power5+ segment Asset (file) server 1 p570 LPAR 16GB Assets’ .ras files 4x2.2GHz 2x6+P RAID5 64K Power5+ segment Cache proxy edge server 6.1 2 p570 LPARs 8GB 2x2.2GHz Power5+ Load balancer edge server 6.1 1 p550Q LPAR 12GB 3x1.5GHz Power5Q Network deployment WebSphere 1 p520Q LPAR 5GB Application Server 6.1, Fix Pack 11 2x1.5GHz Power5Q Copyright © 2008, IBM® Corporation. All rights reserved. Published by IBM® developerWorks®.
  6. 6. IBM Rational Asset Manager capacity and scalability benchmark, Page 6 of 33 Test environment Hardware topology Figure 2. Logical test server topology Table 3. Hardware components Server System/CPU Memory Workbench Rational 2x3.6GHz Pentium 8GB Performance Tester 7.0.1 Load Driver Rational Agent 2x3.6GHz Pentium 4GB Controller (RAC) Rational Performance Tester 7.0.1 License Server 2x3.6GHz Pentium 8GB User simulation The use cases were designed to represent simple, realistic Rational Asset Manager tasks (see the Users' use case simulation makeup subsection that follows). Each user simulated through IBM Rational Performance Tester was assigned a use case scenario based on the following realistic workload: 50% keyword search 29% faceted search 15% download assets content 5% download assets 1% submit assets Each Rational Performance Tester virtual user (VU) was assigned a non-administrator user ID and password pair that was read from a Rational Performance Tester data pool. In this way, a given simulated user was not logged in multiple times. Copyright © 2008, IBM® Corporation. All rights reserved. Published by IBM® developerWorks®.
  7. 7. IBM Rational Asset Manager capacity and scalability benchmark, Page 7 of 33 The wait times between pages were coded as think time in Rational Performance Tester. The think times were set to 2 minutes between pages in order to produce a page throughput rate of approximately 30 pages per hour per user. The Home, Sign In, and Sign Out pages were, by contrast, given a think time of 1 second to more quickly have the VU signed back in to Rational Asset Manager and performing the use case. In each test scenario, the VU signs in to Rational Asset Manager by using a set of credentials from a Rational Performance Tester data pool of user ID and password pairs, and then enters a 3-iteration loop where they perform the work of their assigned use cases. After the 3 iterations of this loop, the VU signs out of Rational Asset Manager. Because each test is contained inside of a loop in the Rational Performance Tester schedule, the VU immediately re-runs the same test, signing back in to Rational Asset Manager with the same credentials. Performance guidelines Each performance test consisted of both a ramp-up period, during which new users were added in a staggered fashion while existing users executed their test cases. This was immediately followed by a full-load period, during which the full user load executed their use cases. During the ramp-up period, users were added at the rate of 1 per 0.5 second; thus, the full ramp-up period was equal to the number of users divided by 2 seconds. The full-load period always lasted 10 minutes, regardless of the number of users. The goal in choosing this approach was to discover the maximum loads for a given configuration over a short time. It should be noted that runs for a shorter period of time may be able to support higher numbers of users than reported in this benchmark; whereas, runs over a longer period may be able to support fewer users. Figure 3 helps to explain this concept as observed in our test environment. Copyright © 2008, IBM® Corporation. All rights reserved. Published by IBM® developerWorks®.
  8. 8. IBM Rational Asset Manager capacity and scalability benchmark, Page 8 of 33 Figure 3. Peak capacity and duration runs If a test run reported under our performance guidelines had an average response time of 250ms, it was seen that a full load test run could last more than 30 minutes, and the average response time still did not exceed 2 seconds. However, if a response time was reported to be 450ms, a full load run would only last 15 minutes before response times began to exceed 2 seconds. During the benchmark, all runs had an average response time of 325ms ± 125ms. When reading the benchmark results (see Table 5 in the Test matrix section in the Appendix), which displays response times for all runs, keep Figure 3 in mind. If the peak expected capacity is 3,000 users over a 10-minute period, it may be acceptable to choose a configuration that shows a 450ms response time for this level of users. However, if you expect a continuous load of 3,000 users over hours or days, it would be best to choose a configuration that shows a lower response time or a configuration that supports a higher peak user capacity. Users' use case simulation makeup The following subsections describe specific simulated user workload scenarios used in the benchmark test runs. Transitions are marked with a think time. Download Assets scenario: In the Download Assets scenario, the VU executes these actions in a 3-iteration loop after signing into Rational Asset Manager: 1. Navigate to the Search page. 2. Drawing a string from an Rational Performance Tester data pool of search strings, enter it as keyword to search for an asset containing a small file. Copyright © 2008, IBM® Corporation. All rights reserved. Published by IBM® developerWorks®.
  9. 9. IBM Rational Asset Manager capacity and scalability benchmark, Page 9 of 33 3. Drawing a string from another Rational Performance Tester data pool of the Global Unique Identifiers (GUIDs), use it as a GUID to go to the Content tab of the General Details page of an asset containing one small file (4 to 25 bytes). 4. Download the file. 5. Sign out after you complete the loop. Figure 4 shows the test structure. Figure 4. Download assets Copyright © 2008, IBM® Corporation. All rights reserved. Published by IBM® developerWorks®.
  10. 10. IBM Rational Asset Manager capacity and scalability benchmark, Page 10 of 33 Download Artifacts scenario: In the Download Artifacts scenario, after signing in to Rational Asset Manager, the VU executes these actions in a 3-iteration loop: 1. Navigate to the Search page. 2. Drawing a string from the Rational Performance Tester data pool of search strings, enter a keyword for an asset containing a small file. 3. Drawing a string from the Rational Performance Tester data pool of GUIDs, use a GUID to go to the General Details page of an asset containing one small file (4 to 25 bytes). 4. Download the artifact by clicking on the artifact’s link on the asset’s Content page. 5. Sign out after you complete the loop. The test structure is pictured in Figure 5 on the next page. Copyright © 2008, IBM® Corporation. All rights reserved. Published by IBM® developerWorks®.
  11. 11. IBM Rational Asset Manager capacity and scalability benchmark, Page 11 of 33 Figure 5. Download artifacts Keyword Search scenario: In the Keyword Search scenario, after signing in to Rational Asset Manager, the VU executes these actions in a 3-iteration loop: 1. Navigate to the Search page. 2. Drawing a string from the Rational Performance Tester data pool of search strings, enter a keyword for an asset containing a small file. 3. Drawing a string from the Rational Performance Tester data pool of GUIDs, use a GUID to go to the General Details page of an asset. 4. Sign out after you complete the loop. Copyright © 2008, IBM® Corporation. All rights reserved. Published by IBM® developerWorks®.
  12. 12. IBM Rational Asset Manager capacity and scalability benchmark, Page 12 of 33 The test structure is pictured in Figure 6. Figure 6. Keyword search Faceted Search scenario: In the Faceted Search scenario, after signing in to Rational Asset Manager, the VU executes these actions in a 3-iteration loop: 1. Navigate to the Search page. 2. Search for assets of a given Rational Asset Manager Type. In our configuration, there are 15 types defined to Rational Asset Manager, and assets were uploaded equally distributed among the 15 types. In this way, the search will return a results set with a size that is 1/15 the total number of assets. 3. Drawing a string from the Rational Performance Tester data pool of GUIDs, use a GUID to go to the General Details page of an asset. Copyright © 2008, IBM® Corporation. All rights reserved. Published by IBM® developerWorks®.
  13. 13. IBM Rational Asset Manager capacity and scalability benchmark, Page 13 of 33 4. Sign out after you complete the loop. The test structure is pictured in Figure 7. Figure 7. Faceted search Submit Assets scenario: In the Submit Assets scenario, after signing in to Rational Asset Manager, the VU executes these actions in a 3-iteration loop: 1. Navigate to the Submit an Asset page. 2. Define a new asset by entering a Name and Short Description, and then choosing a Type and Community for the asset. These are the minimal fields that must be completed to upload an asset to Rational Asset Manager. Copyright © 2008, IBM® Corporation. All rights reserved. Published by IBM® developerWorks®.
  14. 14. IBM Rational Asset Manager capacity and scalability benchmark, Page 14 of 33 3. Using the Basic interface for asset upload, navigate to the location on the Rational Performance Tester machines that contain a small file to upload (382 bytes), and attach the file. 4. From the Confirm asset page, submit the asset by clicking the Submit button. 5. From the Success page, navigate to the General Details page of the newly submitted asset. 6. Sign out after you complete the loop. The test structure is pictured in Figure 8. Figure 8. Submit assets Copyright © 2008, IBM® Corporation. All rights reserved. Published by IBM® developerWorks®.
  15. 15. IBM Rational Asset Manager capacity and scalability benchmark, Page 15 of 33 Benchmark results Figure 9 reflects the overall scaling characteristics of Rational Asset Manager. The graph was created by using test runs on a repository with 2 million assets. User capacity is the user level (up to 5,000 users in the 2 million-asset tests) normalized by the average response time. As the results show, Rational Asset Manager is able to scale as the number of both nodes and instances is increased. These tests maintained a sub second average response time. These results satisfy our goal of proving the scalability of Rational Asset Manager across various configurations and asset levels. (We have verified similar results by using a configuration with 3 million assets.) You can also see from Figure 9 that a cluster running multiple instances scales more rapidly as we increase the number of nodes. This is evident from the surface graph’s steeper diagonal slope as we increase nodes with multiple WebSphere Application Server instances. This is an important planning consideration. Multiple instances will typically comprise a smaller server cluster for a given capacity. Fewer servers mean less complexity and maintenance costs. Figure 9. Overall Rational Asset Manager scalability (Z axis represents capacity for users) Figure 10 better displays the difference between horizontal and vertical scaling. The X axis for the vertical scaling line (squares) represents instances per a single node; whereas, the X axis for the horizontal scaling line (diamonds) represents nodes each with a single instance. The square line in Figure 10 is comparable to the slope of the left-edge bars in Figure 9. The diamond line in Figure 10 is comparable to the slope of the front bars of Figure 9. Copyright © 2008, IBM® Corporation. All rights reserved. Published by IBM® developerWorks®.
  16. 16. IBM Rational Asset Manager capacity and scalability benchmark, Page 16 of 33 As the results show, vertical scaling is initially logarithmic and could degrade performance with too many instances. Diminishing vertical scaling is predominantly limited by the amount of memory available. With 15GB of memory and a 2GB heap size per instance, we observed that going beyond 4 instances diminished overall performance. Horizontal scaling, on the other hand, continued to provide improved performance. We did not attempt to scale beyond 6 application servers in this benchmark, because the capacity that these servers provided was sufficient to verify our initial goals. This linear horizontal clustering validates that the architecture of Rational Asset Manager scales. This is mainly because each server provides index services only to the WebSphere Application Server instances for that node. Figure 10. Comparison of horizontal and vertical scaling Asset scalability The number of assets stored in a repository can affect performance, although not significantly. The reason for the performance degradation in our tests was that some test scenarios returned (processed) a larger amount of search results as the number of assets grew. Figure 11 depicts the increase in the average response time as the number of assets in the repository was increased. Copyright © 2008, IBM® Corporation. All rights reserved. Published by IBM® developerWorks®.
  17. 17. IBM Rational Asset Manager capacity and scalability benchmark, Page 17 of 33 Figure 11. Asset scalability User scalability It is also important to consider user scalability, while holding the number of assets and configuration constant. Bottlenecks begin to occur on the application server primarily due to excessive garbage collection. This phenomenon likely causes response times to grow exponentially as users are added. Because of the nature of Java™ garbage collection and the long pauses associated with this process, remember that the capacity of a particular configuration over a short duration may be significantly greater than the capacity of the same configuration over a very long duration. Therefore, it is important to consider both of these factors when deciding the level of hardware needed to support particular user loads. (See Figure 12 on the next page.) Copyright © 2008, IBM® Corporation. All rights reserved. Published by IBM® developerWorks®.
  18. 18. IBM Rational Asset Manager capacity and scalability benchmark, Page 18 of 33 Figure 12. User scalability Servers resource utilization levels Disk, memory, and CPU use were also monitored on servers during the test runs. Figures 13, 14, and 15, which follow, show this data for some of the more demanding runs. As the results show, the Web servers, caching proxies, and load balancer were able to handle these large loads without heavy resource usage. Therefore, this article focuses on the resource use of the application servers and the database server. Copyright © 2008, IBM® Corporation. All rights reserved. Published by IBM® developerWorks®.
  19. 19. IBM Rational Asset Manager capacity and scalability benchmark, Page 19 of 33 Figure 13. CPU use Figure 14. Memory use Copyright © 2008, IBM® Corporation. All rights reserved. Published by IBM® developerWorks®.
  20. 20. IBM Rational Asset Manager capacity and scalability benchmark, Page 20 of 33 Figure 15. Disk use As mentioned previously, the number of assets in a repository can have an effect on performance if it is assumed that users’ searches will return a greater number of results. You can see that the application server will use a higher percentage of its CPU capacity; whereas, the database server is not as heavily affected. This is because the indexes against which the searches are performed reside on each application server, not the database server. This helps to avoid the problem of the database becoming a bottleneck in horizontal scaling configurations. (See Figure 16 on the next page.) Copyright © 2008, IBM® Corporation. All rights reserved. Published by IBM® developerWorks®.
  21. 21. IBM Rational Asset Manager capacity and scalability benchmark, Page 21 of 33 Figure 16. Asset scaling Although performance can be improved through vertical clustering, it is important to remember that each additional instance will require more memory. Figure 17 shows that memory use will grow in a linear fashion as instances are added, and this will be the primary bottleneck. Therefore, when adding vertical instances, it is important to observe the total system memory use to ensure that no paging will occur. Additionally, adding more memory to an application server can allow utilization of untapped CPU capacity. Copyright © 2008, IBM® Corporation. All rights reserved. Published by IBM® developerWorks®.
  22. 22. IBM Rational Asset Manager capacity and scalability benchmark, Page 22 of 33 Figure 17. Vertical cluster scaling Figure 18 shows how the single database server scales as the number of instances in a cluster is increased. Bear in mind that, in this benchmark, the database server was very powerful, comprising 8 processors, 32GB of memory, and an 8-disk RAID 5 array on which the data resided. You can see that the scaling with 2,000,000 assets is fairly linear as the number of instances is increased, and that under 5% of the resources were utilized, even with 24 instances. Because there is only a single database server, it is important to use a powerful server if a cluster with many instances is planned. It is also important to have a large number of disks and to configure the data and logs to reside on separate disks. Copyright © 2008, IBM® Corporation. All rights reserved. Published by IBM® developerWorks®.
  23. 23. IBM Rational Asset Manager capacity and scalability benchmark, Page 23 of 33 Figure 18. Database scaling Finally, storage requirements are an important factor to consider when deciding on needed hardware. The three major storage points required for Rational Asset Manager are asset files, the index, and the database. In a clustered environment, there will be only one asset storage location and one database storage location. A single index storage location will exist on each application server. The size of the index is dependent on both the size of the assets and whether the content of the assets is mostly text that can be indexed or binary content that cannot be indexed. This is described by Table 4. The index size will generally grow in a logarithmic fashion. Table 4. Index storage requirements Small asset files Large asset files Small amounts of text in Small Medium asset files Large amounts of text in Medium Large asset files The size of the database is dependent on the size of asset descriptions and other repository activity, such as forums, tagging, and registered users. Metrics are also recorded in the database for many user activities, such as searching and downloading. Therefore, even if no new assets are being added to the repository, the database will grow over time as user activity metrics are recorded. The size of the database will generally grow in a linear fashion. although it is dependent on the repository use. Copyright © 2008, IBM® Corporation. All rights reserved. Published by IBM® developerWorks®.
  24. 24. IBM Rational Asset Manager capacity and scalability benchmark, Page 24 of 33 Although some large assets were included, the average size of an asset in the test environment repository was less than 1KB. If you plan to have larger assets, this should be accounted for when considering the storage requirements for the Persist and Index folders. The database size will not be significantly affected by the size of the assets. Figure 19 shows the storage requirements from the benchmark. Figure 19. Storage requirements Copyright © 2008, IBM® Corporation. All rights reserved. Published by IBM® developerWorks®.
  25. 25. IBM Rational Asset Manager capacity and scalability benchmark, Page 25 of 33 Appendix: Test results The test results described in this section are not a replacement for testing Rational Asset Manager performance in your own environment. Performance and scalability depend on a large number of factors, including but not limited to: hardware setup and tuning, operating system tuning, software tuning, network traffic and latency, end user use models, and the number of assets stored in Rational Asset Manager. It is impossible to cover every single factor affecting performance that exists in the test environment. Therefore, these results are not guaranteed, and it is expected that you will achieve different results in your own environment. This data is intended simply to provide a point of reference from which to begin your own environment design. All tests follow the format described in the User simulation section of this document. Ramp- up time for each test is number of users / 2 seconds. Each test was then driven for an additional 10 minutes with the full user load. Thus these results should not be considered maximum or sustained values. Instead, they represent user loads that can be driven over a short period of time. The average response time for all tests was kept below 450 ms. See the Performance guidelines section for information concerning the impact on the average response time with runs of longer durations. Test matrix Table 5. Benchmark results Assets Users Nodes Instances Response Page hits Page/sec. Total as defined per node time average total maximum run in the User in milliseconds observed minutes Simulation section 100,000 400 1 1 244 3,000 5 14 100,000 700 1 2 250 5,500 7 16 100,000 1,200 1 4 415 11,100 15 20 100,000 1,200 3 1 279 11,200 15 20 100,000 2,100 3 2 340 26,000 22 28 100,000 3,300 3 4 319 51,500 40 38 100,000 2,300 6 1 431 30,000 23 30 100,000 3,500 6 2 218 57,600 40 40 100,000 6,000 6 4 250 136,000 70 60 300,000 400 1 1 258 3,000 5 14 300,000 700 1 2 311 5,600 7 16 300,000 1,100 1 4 352 10,500 13 20 300,000 1,200 3 1 331 11,200 15 20 300,000 2,000 3 2 274 24,100 22 27 300,000 3,300 3 4 345 51,500 35 38 300,000 2,100 6 1 245 26,100 23 28 Copyright © 2008, IBM® Corporation. All rights reserved. Published by IBM® developerWorks®.
  26. 26. IBM Rational Asset Manager capacity and scalability benchmark, Page 26 of 33 Assets Users Nodes Instances Response Page hits Page/sec. Total as defined per node time average total maximum run in the User in milliseconds observed minutes Simulation section 300,000 3,500 6 2 257 57,500 40 40 300,000 6,000 6 4 283 136,300 70 60 1,000,000 400 1 1 358 3,000 5 14 1,000,000 700 1 2 418 5,600 7.5 16 1,000,000 950 1 4 399 8,100 10 18 1,000,000 950 3 1 284 8,000 10 18 1,000,000 1,900 3 2 342 22,100 22 26 1,000,000 3,000 3 4 435 43,300 32 35 1,000,000 2,000 6 1 270 24,100 23 27 1,000,000 3,300 6 2 276 51,500 35 38 1,000,000 5,500 6 4 368 114,600 60 56 2,000,000 400 1 1 407 3,000 5 14 2,000,000 700 1 2 400 5,600 8 16 2,000,000 800 1 4 381 6,600 9 17 2,000,000 800 3 1 309 6,600 9 17 2,000,000 1,700 3 2 386 19,600 21 25 2,000,000 2,700 3 4 437 37,600 31 33 2,000,000 1,900 6 1 346 22,200 22 26 2,000,000 3,100 6 2 357 45,800 35 36 2,000,000 5,000 6 4 400 100,500 59 52 User scalability The number of users that can be driven in a configuration is affected by the number of application server nodes and instances, as well as the number of assets within the repository. The number of assets has an impact in the tests because it is assumed that a repository with a large number of assets will result in users performing search queries that return a larger number of results. In this benchmark, 15% of the users performed searches that returned (number of assets in the repository / 15) results. Figure 20, Figure 21, Figure 22, and Figure 23 show the maximum number of users driven in each configuration with the different asset levels that were tested. Results are fairly comparable between 100,000 and 300,000 assets, with a slight drop off between 300,000 and 1,000,000 assets and with another between 1,000,000 assets and 2,000,000 assets. Copyright © 2008, IBM® Corporation. All rights reserved. Published by IBM® developerWorks®.
  27. 27. IBM Rational Asset Manager capacity and scalability benchmark, Page 27 of 33 Figure 20. User scalability with 100,000 assets Figure 21. User scalability with 300,000 assets Copyright © 2008, IBM® Corporation. All rights reserved. Published by IBM® developerWorks®.
  28. 28. IBM Rational Asset Manager capacity and scalability benchmark, Page 28 of 33 Figure 22. User scalability with 1,000,000 assets Figure 23. User scalability with 2,000,000 assets Copyright © 2008, IBM® Corporation. All rights reserved. Published by IBM® developerWorks®.
  29. 29. IBM Rational Asset Manager capacity and scalability benchmark, Page 29 of 33 Throughput scalability Because the usage model for a “user” can vary widely, it may be preferable to use another measure as a definition of load capacity. The maximum observed page hits per second in each of the 9 configurations and 4 different asset levels can be used to derive and apply a different model for a user to these benchmark results. Figures 24, 25, 26, and 27 show that throughput is similar to user capacity for each configuration and repositories with increasing numbers of assets. Figure 24. Page throughput with 100,000 assets Copyright © 2008, IBM® Corporation. All rights reserved. Published by IBM® developerWorks®.
  30. 30. IBM Rational Asset Manager capacity and scalability benchmark, Page 30 of 33 Figure 25. Page throughput with 300,000 assets Figure 26. Page throughput with 1,000,000 assets Copyright © 2008, IBM® Corporation. All rights reserved. Published by IBM® developerWorks®.
  31. 31. IBM Rational Asset Manager capacity and scalability benchmark, Page 31 of 33 Figure 27. Page throughput with 2,000,000 assets Copyright © 2008, IBM® Corporation. All rights reserved. Published by IBM® developerWorks®.
  32. 32. IBM Rational Asset Manager capacity and scalability benchmark, Page 32 of 33 Trademarks IBM, the IBM logo, and Rational are trademarks of IBM Corporation in the United States, other countries, or both. Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others. Share this article Resources Digg this story Learn Post to del.icio.us • Publications and products mentioned in this article: o Rational Asset Manager 7.0 Capacity Planning Guide1 Slashdot it! o Rational Asset Manager Tuning Guide o IBM System Storage DS4800 o IBM System P Enterprise Servers • Check out the following Rational Asset Manager resources: o IBM Rational Asset Manager V7.0.0: Capacity planning and configuration guide o Rational Asset Manager demos o Maximize reuse of your software assets with Rational Asset Manager, Part 1 o Maximizing reuse of your software assets with Rational Asset Manager, Part 2 o Federated metadata management with IBM Rational and WebSphere software • In the Architecture area on developerWorks, get the resources you need to advance your skills in the architecture arena. • Visit the Rational software area on developerWorks for technical resources and best practices for Rational Software Delivery Platform products. • Subscribe to the developerWorks Rational zone newsletter. Every other week, you'll receive updates on the latest technical resources and best practices for the Rational Software Delivery Platform. • Subscribe to the Rational Edge newsletter for articles on the concepts behind effective software development. • Subscribe to the IBM developerWorks newsletter, a weekly update on the best developerWorks tutorials, articles, downloads, community activities, webcasts and events. • Browse the technology bookstore for books on these and other technical topics. Get products and technologies • Download trial versions of IBM Rational software. • Download IBM product evaluation versions and get your hands on application development tools and middleware products from DB2®, Lotus®, Rational®, Tivoli®, and WebSphere®. Discuss • Rational Asset Manager forum: Ask questions about Rational Asset Manager. • Check out developerWorks blogs and get involved in the developerWorks community. Copyright © 2008, IBM® Corporation. All rights reserved. Published by IBM® developerWorks®.
  33. 33. IBM Rational Asset Manager capacity and scalability benchmark, Page 33 of 33 About the authors Dr. Mendel is the technical lead and manager for IBM Rational Asset Manager software. Sheehan Anderson is a Rational Asset Manager software developer. John Reinstrom is the Chief Performance Test Architect for Rational system and performance tests. Amy Pitts is a software engineer for Rational system and performance tests. Skye Bischoff is a Software Test Specialist on the IBM Rational Performance Tester SVT team, focused on improving customer usability for Rational Performance Tester, Rational SOA Tester, and other Rational products. He has performance-tested IBM Rational Asset Manager in various clustered environments to ensure that it performs quickly, scales, and is resilient. Before working at IBM, Skye worked as a Flash designer and developer at T8Design. He received his B.S. in Computer Science from the University of Northern Iowa. Jennifer Chang is a Software Performance Test Engineer on the IBM Rational Performance Tester SVT team, working on performance testing IBM Rational Asset Manager in clustered and standalone environments. She received both her M.S. in Computer Science and M.A. in Economics from Tufts University, in Medford, Massachusetts. Bryan Miller is a Service Offering Development Lead for the Rational Brand Services team. Copyright © 2008, IBM® Corporation. All rights reserved. Published by IBM® developerWorks®.

×