CLOUD TESTING MODEL – BENEFITS, LIMITATIONS AND CHALLENGES

1. ISSN: 2312-7694
Mukesh et al, / International Journal of Computer and Communication System Engineering (IJCCSE), Vol. 2 (5), 2015, 636-640
636 | P a g e
© IJCCSE All Rights Reserved Vol. 02 No.05 Oct 2015 www.ijccse.com
CLOUD TESTING MODEL –
BENEFITS, LIMITATIONS AND CHALLENGES
Dr. Mukesh Chandra Negi
Project Manager, TechMahindra Ltd.
PhD, MCA, M.Sc., APGDCA, PGDCA, CIC
Mukesh_negi77@yahoo.com, Mukesh.negi0910@outlook.com
Abstract— Software testing has always become a
challenging part in the field of IT, especially when related with
enterprise applications in comparison with standalone
applications, where lots of integrations and complexities are
involved including other related challenges, like security, costs,
infrastructure, scalability and growth in future. The role of
software testing is very significant in the area of enterprise
applications world. Criticality of testing is less but the role is
very high. In a typical On-Premise IT environment, you have an
internal infrastructure for testing in almost every organization,
which includes hardware’s, software’s, networking, security and
all related infrastructure and facility managements like all other
development, QA, prod etc. environments. Most of time this
testing environment sits idle and required only when you are
working on scalability of your applications or doing some
functional or nonfunctional testing’s. With time and for
scalability, it’s required to increase or upgrade the testing
infrastructure with time to time which incurs extra cost. Cloud
testing is an era in the field of testing where you can save your
cost as well perform different testing’s more efficiently and
dynamically according to your current requirements and in
future as well. Despite the benefits, there are certain limitations
and challenges also associated. My article will focus on different
limitations and challenges associated with the cloud testing
model.
I. INTRODUCTION
Traditionally, testing environment always seen as an evil
since most of the time it sits idle and in conjunction it
required lots of infrastructure and resources. You need
everything, a space, non IT infra setup, electricity, backup,
cooling, servers, networking, security, IT manpower etc. for
your testing environment. Your business also grows with
time and in parallel you have to upgrade as well increase IT
resources of you testing environments which incur extra cost
time to time. As your requirements start growing with time,
you have to spend more on testing’s and it’s infra to make it
sustain as per the current updates and requirements. Cloud
testing is a model where your complete testing environment
is outsourced to a third party service provider as On-Demand
model, and it eliminate your upfront capital expenditures on
testing infra setup. It gives you great benefits like lower your
costs, on demand flexibility, scalability, availability,
maintainability, freedom from infra management, pay-as-use
model etc. but in parallel, Cloud testing has limitations as
well, and it introduces lots of challenges as well like lack of
standards, data integrity and security, layer testing challenges
etc.
If you see one of the reports published by Fujitsu in 2010,
almost more than 55% of applications and infrastructure over
public cloud was occupied by the Testing and Development
environments.
Source: Fujitsu 2010
II. TaaS (Testing as a Service)
TaaS concept in testing is similar like other cloud models
SaaS; PaaS & IaaS where you can completely outsources you
testing IT infrastructure and resources and just use and pay
On-Demand basis whenever required. It’s a completely
outsourcing model where you get the Testing services from
some third party Testing Service Provider. You just have to
give your exact infrastructure, applications and resources
requirements and time for how long you want the same, and

2. ISSN: 2312-7694
Mukesh et al, / International Journal of Computer and Communication System Engineering (IJCCSE), Vol. 2 (5), 2015, 636-640
637 | P a g e
© IJCCSE All Rights Reserved Vol. 02 No.05 Oct 2015 www.ijccse.com
based on that you have to pay only for required resources and
timing only. It’s a great benefit in terms of cost and
scalability where you don’t need to setup and internal on
premises testing environment and not need to worry about
management and scalability of same in future.
Below is the graphical representation of same. There is a
TaaS service provider at the end, which runs and provides
services to you 24*7*365 and you as an end user or client can
access it from the cloud web interface anytime from
anywhere.
III. BENEFIT OF CLOUD TESTING
Below are certain benefits of using cloud testing model
A. Scalability (Future Growth)
Like cloud other delivery models, one of the great benefit
of cloud based testing is scalability. You can easily up and
down your requirements anytime as per your requirements.
Especially useful where business demands keep altered
frequently.
B. Less Cost
Cost is another major benefit of cloud based testing. You
don’t need to invest heavy on any on premises based
infrastructure and applications licensing. Just take the
services as per your requirements whenever you want and
pay for that only.
C. Live replica & Availability
This is another major benefit of cloud testing. In a generic on
premises testing environments, you don’t have an exact
replica of your production environments, which creates lots
of issues during test environment testing’s and live rollout
due to this simulation problems. With cloud testing. you have
flexibility to choose and perform testing on exact same
environment and resources as you have on your live
production environment, which help in great way to catch
and fix lots of issues, which you can’t able to handle in on
premises less resources testing scenarios.
D. Easy Customization
This is another great advantage. Testing involves lots of
permutation and combinations scenarios where you need to
test your applications on different software’s, infrastructures,
browsers etc. which is very tough in traditional testing
environments due to cost and lots of different factors, but
with cloud testing option you can easily achieve this.
E. Better Time Management
You time management will be better as you don’t need to
worry and waste your time on and testing related
procurement, management, resourcing, scalability,
availability etc. and you can contribute that time and efforts
to your actual business.
F. Agile Development & Testing
During the agile development, you have different teams from
different locations are involved, and during testing,
irrespective of the team, project & location you need the
contribution of all. With cloud testing, you all can work
together from anywhere at same time.
G. Assured Performance and Peak Load Testing
Cloud testing can give you an actual picture of
performance resources your application actually needed at
peak time, which is not possible in your traditional on
premises IT infrastructure testing environment due to
limitations of resources. On cloud testing model, you have
option to identify peak load of your application by taking
resources from the big pool of resources.
IV.DIFFERENT FORMS OF CLOUD
TESTINGS

3. ISSN: 2312-7694
Mukesh et al, / International Journal of Computer and Communication System Engineering (IJCCSE), Vol. 2 (5), 2015, 636-640
638 | P a g e
© IJCCSE All Rights Reserved Vol. 02 No.05 Oct 2015 www.ijccse.com
Generally a cloud testing can be categorized in two
different categories as below –
 Functional Testing
 Non Functional Testing
Functional testing
Functional testing is a verification of the business
requirement functionalities which you have developed as per
the client requirements, if it’s functioning as per the
expectations and giving you the desired results.
Broadly, you can further categorized it into three sub
categories as below –
System Verification Testing – During this phase, you
have to make sure each and every module you have
developed are working and giving you the expected output.
System Integration or Interoperability Testing –
During this testing you have to check integration testing’s of
modules you have verified during system verification testing.
It’s actually a integration testing where complete end to end
flow tested.
User Acceptance Testing – During this phase you have
to carry out different functioning cases testing with the actual
end users to make sure all are working fine, and in case of
any gap identified you have to fix, deploy and test the again.
Non Functionality Testing
It’s all apart from functional testing’s. Here you don’t need to
check any user or client desired functionalities. You have to
concentrate on all rest parts to make sure your designed
system and solution is able to sustain the client requirements
as per the user expectations, governance, risk and
compliance. Broadly, you can divide it into below sub
testing’s –
Availability Testing – Here cloud vendor has to make
sure services are available all the time 24*7.
Scalability Testing – Here you have to make sure the
solution you have designed is capable to scale up and down
the system as per your need at any time in near future.
Disaster Recovery Testing – Here you have to test
failover environment of your live environment to make sure
your business with continue in case of any disaster at main
live location.
Security Testing – Here you have to make sure there are
no security gaps in your system and solution, and it’s
designed to work as proper governance, risks and
compliance.
Performance, Load and Stress Testing – Here you
have to test if you designed testing solution is actually able to
sustain the load and demand of client especially at peak load
time.
Multi Tenancy Testing – Since you cloud applications if
accessed and used by different users from different locations,
departments and organizations, so you have to make sure
sufficient security, access, authorization and authentications
are in place.
Overall structure you can represent as below –
V. CLOUD TESTING LIMITATIONS
AND CHALLENGES
As I mentioned earlier, Despite lots of benefits, Cloud testing
has some challenges and limitations as well which need to be
studied further to make it an integrated solution which any
organization can adopt without any fear of risks & threats.
Below are few existing challenges and limitations as of now.
A. Lack of Standards

4. ISSN: 2312-7694
Mukesh et al, / International Journal of Computer and Communication System Engineering (IJCCSE), Vol. 2 (5), 2015, 636-640
639 | P a g e
© IJCCSE All Rights Reserved Vol. 02 No.05 Oct 2015 www.ijccse.com
Lack of standards is still a big issue in the field of cloud
computing and testing since there is no universal standards
exist as of now. All public cloud providers have their own
process and mechanism in terms of infrastructure,
architecture, cost, integrations etc. and there is no universal
standard which governs the integration of your on premises
private cloud with the public cloud.
B. Security
Security is still another big challenge in the field of cloud
computing, especially in public cloud. You don’t have
control over your data as it’s completely resides in the service
provider network and premises. Data integrity, account
hackings, data loss insecurity, shared technology, insecure
API etc. are same common security factors in public cloud.
C. Dependency on internet
This is also one of the major challenge and dependency on
cloud computing and testing where to perform any kind of
work or testing on your environments you need a good speed
internet connection.
D. Understanding of the Cloud
This is another challenge before adoption of cloud testing.
You have to make sure you have enough resources who know
everything about cloud testing and able to perform work on
same or you have to provide good hands on training to your
staff before adoption and implementation of cloud testing
model.
E. Selection of a Service Provider
Selection of a good public cloud service provider is also a big
challenge. There are uncountable no of public cloud service
providers in the market and to identify one out of them
according to your need and requirements is still a big
challenge.
F. Infrastructure Requirements
Some cloud service providers in the market provide limited
capabilities and functionalities in terms of servers,
configurations, applications, storage, networking, bandwidth
etc., which is also a challenge.
G. Testing Strategy & Usage
Preparing your strategy of testing over cloud testing and the
identification of actual demand of your testing requirements
is also a big challenge in cloud testing model.
H. Service Level Agreements with Service
Provider
Due to lack of standards, there still lack of good service level
agreements almost with all public cloud service provides.
You have to discuss with all providers in terms of their
service level agreements with respect to availability,
scalability etc. as per your testing needs to filter most suited
provider.
I. Performance issues in Public Cloud
Since public cloud is a multitenant model where all if the
resources are shared by different cloud service provider
clients, so performance demand in public cloud is also a big
challenge since sometime it happened that due to some glitch
in another client applications it may impact your applications
performance as well due to shared resources model.
J. Layer Testing
Since the complete control of infrastructure is completely
owned and controlled by the service provider, so layer testing
is still a big challenge in public cloud testing as during layer
testing’s, you need the details of different configuration and
parameters which you may further need to tune as per your
requirements related with networking, security, database etc,
but you don’t have any control over that.

5. ISSN: 2312-7694
Mukesh et al, / International Journal of Computer and Communication System Engineering (IJCCSE), Vol. 2 (5), 2015, 636-640
640 | P a g e
© IJCCSE All Rights Reserved Vol. 02 No.05 Oct 2015 www.ijccse.com
VI. CONCLUSION
Cloud testing model emerged as a great innovation in the
field of IT testing, but since the complexities of the demand
and future is changing, there has to be some way to make
cloud testing process more agile and flexible. There may be
some standards are required to govern the cloud testing
model, as well to challenge the current limitations and
challenges. TaaS has to be accepted as one of the cloud
delivery models apart from SaaS, IaaS and PaaS, and all the
similar limitations and challenges has to be addressed and
fixed in parallel with all cloud delivery model.
REFERENCES
[1] A. Vanitha Katherine & Dr.K. Alagarsamy (2012),
“Conventional Software Testing Vs. Cloud Testing”,
International Journal of Scientific & Engineering Research,
Vol. 3, No. 9.
[2] AppLabs,“Testing the Cloud,”white paper,Internet:
www.applabs.com/html/Testing the Cloud 786 .html
[3] “Confidence In Cloud Grows, Paving Way For New
Levels Of Business Efficiency,” Fujitsu, 2010.
[4] Cloud Based Testing: Need of Testing in Cloud Platforms
by Vinit B. Mohata1, Dhananjay M.Dakhane2, Ravindra
L.Pardhi3 : International Journal of Application or Innovation
in Engineering & Management (IJAIEM): Volume 2, Issue 3,
March 2013
[5] Eljona Proko & Ilia Ninka (2012), “Analysis and Strategy
for the Performance Testing in Cloud Computing”, Global
Journal of Computer Science and Technology Cloud &
Distributed, Vol. 12, No. 10, Version 1.0, USA.
[6] Jerry Gao, Xiaoying Bai, and Wei-Tek Tsai, “Cloud
Testing- Issues, Challenges, Needs and Practice” An
International Journal (SEIJ), Vol. 1, No. 1, SEPTEMBER
2011.
[7] M. Oriol and F. Ullah, “YETI on the cloud,” in Third
International Conference on Software Testing, Verification,
and Validation Workshops, 2010, pp. 434–437.
[8[Nancy Gohring, “The Cloud: A Threat to Incumbents,
Opportunity for Startups,” IDG News, Sept. 28, 2010.
[9] Leah Muthoni Riungu, Ossi Taipale, Kari Smolander,
“Software Testing as an Online Service: Observations from
Practice,” In Third International Conference on Software
Testing, Verification, and Validation Workshops (ICSTW),
418-423, 2010.
[10] Paul Ashwood, “Why Your IT Organization Should
Move from Traditional application Testing to Testing-as-a-
Service (TaaS)”, HP Enterprises.
[11] P. Joshi, H. S. Gunawi, and K. Sen, “PreFail: a
Programmable Tool for Multiple-Failure Injection,” in
Proceedings of the 2011 ACM international conference on
Object oriented programming systems languages and
applications, 2011, pp. 171–188.
[12] Software Testing in Cloud Platform: A Survey by
A.Vanitha, Katherine, K. Alagarsamy : International Journal
of Computer Applications (0975 – 8887)Volume 46– No.6,
May 2012
[13] Thomas J. Bittman, “Test and Quality Management: The
First Frontier for Private Cloud,” Gartner, Inc., Sept. 3, 2010.
[14] Xiaoying Bai, Muyang Li, Bin Chen, Wei-Tek Tsai,
Jerry Gao. Cloud Testing Tools. Proceedings of The 6th
IEEE International Symposium on Service Oriented System
Engineering (SOSE 2011).
[15] Abbas, Waseem, Nasim Abbas, and Uzma Majeed.
"PERFORMANCE ENHANCEMENT OF END-TO-END
QUALITY OF SERVICE IN WCDMA WIRELESS
NETWORKS." Science International 26.2 (2014).