Paper presented at Malaysian Software Engineering Conference 2013 (MySEC2013) at Malacca

1. PERFORMANCE TESTING STRATEGY
FOR
CLOUD-BASED SYSTEM USING
OPEN SOURCE TESTING TOOL
Assoc. Prof. Dr. Suhaimi Ibrahim
Advanced Inforamtics School
Universiti Teknologi Malaysia

2. Outline
•
•
•
•
•
•
Introduction & Related Works
Overview of System Under Test
Performance Testing Strategy
Performance Test Results
Findings & Discussion
Conclusion

3. Introduction & Related Works
Performance requirements that should be
specified are throughput level, latency, capacity
and ability to operate in lower mode level
(Hussain and Abdulsalam, 2012].
This study illustrates the
strategy employed for
conducting performance
testing for a cloud-based
software by using open
source testing tool via
distributed performance
testing approach
Important tasks during and after the performance
test execution of the cloud should involve
environment deployment, monitoring of
environment status, controller-agent model
implementation, logs analysis, analysis of slow
queries, errors visualization as well as statistics
comparison (Lynch, Cerqueus and Thorpe, 2012)
A framework for testing scalability of web
application on the cloud should include software
that serves as HTTP server, load balancer,
database, cache and benchmark tool (Vasar,
Srirama and Dumas, 2012)
Testing cloud application shall also deal with
quality of workload generation, complexity in data
analysis and service level agreement (Zhou, Zhang
and Ye, 2013)

4. Overview of System Under Test

5. Performance Testing Strategy (1)
Sub-System
Scenario
Number of Virtual Users
(VU)
Client System
Response time to load services
home page
1
Server System
(Intelligent Services
Portal + Content
Aggregator +
Location Aware)
Response time to load services
home page (via client test page)
1, 100, 200, 300, 400, 500,
1000, 1500, 3000

6. Performance Testing Strategy (2)

7. Performance Test Results (1)

8. Performance Test Results (2)
Client Internet
Connection
Server Load
Response Time (seconds)
LAN speed
High
7.01
LAN speed
Low
4.81
Wireless LAN
Low
5.33
1 Mbps wired broadband
High
8.57
1 Mbps wired broadband
Low
5.33
0.44 Mbps wireless
broadband
Low
16.52 – 24.88

9. Findings & Discussion
•
•
•
•
•
•
•
•
The online services system hosted in the cloud platform could response to specified
performance requirement of five seconds from 1 user to 1500 concurrent users without any
break.
At 3000 concurrent users, the system still can response to huge requests without fail but with
delay in response time.
When the server is under heavy load or many concurrent users are accessing the online services
system, a particular end-user may experience slower response time accessing the online
services via the client system.
End-user with fast Internet connection will access the online services faster than those with
slow speed in any server load condition.
Cloud platform allows the virtual web servers to be scaled up in order to meet the performance
requirements for faster response time
Performance testing activities should be planned earlier in the life cycle and not during
execution or after execution completed
Performance test cases should mimic the real-world scenarios to ensure the reliability of test
results
Understanding of the performance requirements will help in selection of suitable performance
test tools to accommodate different usage profile, loads, strategy and environment setup

10. Conclusion
• The study has demonstrated the success of the test strategy in measuring the
performance of cloud-base system with the support open source testing tool
• it is important to have proper planning and come out with realistic scenarios for
performance testing
• Performance testing should not be treated as “when required” or “when
requested” basis. When performance test is done using the right strategy, right
scenarios and appropriate testing tool, any system would be able to deal with
increasing loads
• Future works?
 Host the performance testing tool in the cloud too
 Expanded the performance analysis further to not just on response time,
but also other aspects such as CPU utilization, memory utilization, disk I/O
and network latency
 Explore other possible real-world scenarios for the system usage to ensure
better performance testing coverage