Getting Started with Cloud Testing

In this article we are going to see how to perform “cloud testing”. To understand this we need to know the cloud computing concept first. This article explains what is cloud computing, what is cloud testing, and the major challenges with testing in cloud.

Cloud computing introduction

Several years back the industry witnessed a new buzzword and technology called “virtualization”. With the advent of virtualization, the ideology of sharing computing resources across multiple operating systems to reduce capital costs, increase scalability and enable easy administration of the IT infrastructure became the backbone of several enterprises.

In the past few years, IT has witnessed an evolution of virtualization in the form of cloud computing. Any novice to cloud must picture cloud computing as a model that views everything “As a Service”. It is obvious to define cloud computing as a total solution that delivers IT as a Service. Like virtualization, its motto is resource sharing whereby allocation and availability are on demand via Internet.

cloud computing and testing

[image credit kezber.com]

Real Life Example to Explain Cloud Computing:

You can understand the theory of cloud computing taking a simple example of having a meal at a restaurant. As an example consider having a meal at home versus that of having at a restaurant. While the objective is purely the consumption of a meal, the approach or model used in both these cases vary greatly.

At home, you need to make sure you have all the ingredients required for the meal, you need to make sure that the required preparation to cook the particular dish is done. You cook the food and the responsibility of keeping the surroundings tidy post the dish is cooked solely belongs to you.

Now on the other hand, having a meal at a restaurant can be described as catering as a service. As a customer, you only go to the restaurant, order the dish of your choice and pay for the services. The onus of possessing the ingredients, preparation for the dish, cooking the dish, its presentation, serving the dish to you, clearing and cleaning the table once done, packing any excessive leftover food all falls on the restaurant administration.

This example can be viewed as similar to the ideologies and principles of cloud computing – “On demand availability”, “As a service” and “Pay for the usage”.

Cloud delivery models

Cloud testing layers

[image credit hyphenet.com]

The cloud has mainly three types of delivery models or components that provide “as a Service” capabilities:

  • Infrastructure as a Service (IaaS): This is the most fundamental layer which forms the building block of the cloud. It mainly comprises of physical resources such as Storage, Network devices, compute servers, etc. All these computing resources are available on demand, where a user pays as per his usage.
  • Platform as a Service (PaaS): The central layer in the cloud Is the platform. Here there is no control over the underlying infrastructure but you can access the deployed applications. Hence this provides the entire runtime environment on demand which could be a development or test environment. Hence in this model, most commonly you would have a VM that contains the complete environment like OS, required middleware, etc. available when you need it.
  • Software as a Service (SaaS): The topmost layer here is the application layer, which is generally visible to any user. Here, applications / products are available to the user on demand via the internet. Hence instead of having to acquire licenses for a particular user, this proves to be the most cost effective way of making sure that the license is always in use. Examples of this are Gmail, Google docs, Photoshop, etc.

Types of cloud

There are 3 types of cloud formations: public, private and hybrid clouds:

  • Public clouds are where services are available to everyone, where the resources are allocated and provisioned dynamically as per request.
  • Private clouds are generally managed within firewall rules of a particular organization and are available only to the users within the company.
  • Hybrid clouds are a mixture of private and public clouds. Organizations can decide what services they want to expose to everyone and what services they want to expose to only the users within the organization.

Cloud testing- Need for it

Cloud testing refers to testing of resources such as hardware, software, etc. that are available on demand. Even the testing here can be viewed “as a service”. For cloud offerings it’s essential to make sure that the service (product) not only meets its functional requirements but also nonfunctional requirements. With the range of applications in the cloud it is now becoming essential to develop cloud testing strategy.

top applications in the cloud

Benefits of Cloud Testing:

The need for cloud testing is easily visible with the benefits, which are far too many. Below we attempt to discuss the most apparent benefits which explain why cloud is the need of the hour:

  1. Dynamic availability of testing environment: The normal testing approach in any organization is to invest in the hardware/ software infrastructure needed for the testing. Almost most of you will agree that the environment supplied to testing teams very rarely matches a customer environment on account of rapidly changing requirements, as a result of which it is very challenging for companies to keep up. Cloud is the answer to this problem whereby, users can easily replicate a customer environment and find defects early in the cycle.
  1. Low cost: Another angle to the previous point is that when companies invest in the infrastructure, the usual case it many of their servers are not utilized all the time. As a result, they may have to incur additional expenses on license renewal. Transitioning to cloud helps in this scenario as well, as users can commission appliances as and when they want, saving huge costs for an organization.
  1. Easily customizable: With the usage of cloud, it’s a rather easy task for organizations to emulate an end-user centric environment by customizing it as per use saving cost and time. Test teams can easily perform load and performance testing scenarios in various permutations and combinations like – different OSes, browsers, configurations, etc.
  1. Scalability: One of the most attractive features of the cloud whereby the computing resources can be increased or decreased wherever it is necessary. Extremely useful in situations where the business requirements keep altering frequently.

Cloud testing deep dive

Testing with respect to a cloud will employ both traditional and new age methods to perform testing. Testing and cloud are equally large domains in their own right. Let’s learn how to view the two together in the below sections.

------------

1) Cloud testing forms:

Cloud testing can be broadly divided into four different categories based on what they aim to do :

  • Testing of the whole cloud: The cloud is viewed as a whole entity based on its features and testing is carried out based on that.
  • Testing within a cloud : This is the testing carried out inside of the cloud by checking each of its internal features
  • Testing across clouds: Based on specifications, here the testing is carried out on the different types of cloud like public, private and hybrid clouds.
  • SaaS testing in cloud: Functional and nonfunctional testing is performed based on requirements.

2) Cloud testing environments:

There are 3 types of cloud testing environments:

  • Private or public environment where the quality of applications deployed in them need to be validated.
  • A hybrid environment, where the quality of applications deployed in them need to be validated.
  • A test environment which is cloud based, where the quality of applications deployed in the cloud need to be validated.

3) Types of testing performed in cloud:

Testing in a cloud has to not only ensure that the functional requirements are met, but a strong emphasis needs to be laid on nonfunctional testing as well. Let’s take a look at the different kinds of testing that are performed.

Functional testing:

Goes without saying, that functional testing has to be performed to make sure that offering provides the services that the user is paying for. Functional tests ensure that the business requirements are being met. Some of the functional tests are described below:

  • System Verification Testing: This makes sure that the various modules function correctly with one another, thus making sure that their behavior is as expected.
  • Acceptance Testing: Here the cloud based solution is handed over to the users to make sure it meets their expectations.
  • Interoperability Testing: Any application must have the flexibility to work without any issues not only in different platforms, but also must work seamlessly when moving from cloud infrastructure to another.

Non-Functional testing:

Non-functional tests mainly focus on the web application based tests ensuring that they meet the desired requirements. Here are few forms of nonfunctional tests discussed below :

  • Availability Testing: The cloud supervisor/ vendor has to make sure that the cloud is available round the clock. As there could be many mission critical activities going on, the administrator has to make sure that there is no adverse impact to the consumers
  • Multi Tenancy Testing: Here, multiple users use a cloud offering. Testing must be performed to ensure that there is sufficient security and access control of data when multiple users are using a single instance.
  • Performance Testing: Verification of the response time needs to be done to ensure that everything is intact even when there is a large amount of requests to be satisfied. The network latency is also one of the critical factors to evaluate performance. Also workload balancing needs to be done when there is a reduction in load, by decommissioning resources. Thus load and stress testing are done in the cloud offering to make sure applications are performing optimally with increase/decrease in load and stress.
  • Security testing: Since with the cloud everything is available anytime, it’s essential to make sure that all user sensitive information has no unauthorized access and the privacy of users remains intact. When maintaining the applications in cloud, user data integrity must also be verified.
  • Disaster recovery Testing: As already stated in availability testing, the cloud has to be available at all times and if there are any kind of failures like network outages, breakdown due to extreme load, system failures, etc, measure how fast the failure is indicated and any data loss during this period.
  • Scalability Testing: Test to make sure that offering has the capability to provide scale up or scale down facilities as per the need.

Cloud testing tools:

In this section, some of the different tools used in various kinds testing performed in a cloud are mentioned. The details of the tools are out of the scope of this article.

Tools for Performance Testing in the Cloud:

Many of the tools are basically used for performance, load, stress testing. Some of these tools below can also be used for functional tests:

  • SOASTA CloudTest
  • LoadStorm
  • CloudTestGo
  • AppPerfect.
  • Jmeter
  • Cloudslueth
  • CloudTestGo
  • AppPerfect

Cloud Security Testing Tools:

There are also some security related tools which are commonly used:

  • Nessus
  • Wireshark
  • Nmap

You can refer the internet for details on various tools and how they can be used in a cloud infrastructure. One such link is found here.

Cloud Testing Challenges

As exciting as it cloud sounds, all is not hunky-dory here. There are some challenges with relying and using cloud as an infrastructure as well. Let’s take a look at some of the primary concerns using the cloud.

Challenge #1. With everything available on demand to any user, security is a primary issue for the businesses as currently there is still a lot of discussion and research going on in the industry to set up security standards. User privacy protection, security standards on cloud, security of applications running within the cloud, security testing techniques are some of the primary issues that need to be addressed in the cloud infrastructure.

Challenge #2. Another big challenge is the performance of an application in a cloud: specifically in private clouds. It will be shared across many users and hence could lead to delays. Also in case of some maintenance or outage related activities, the bandwidth may seem insufficient.

Challenge #3. Sometimes for testing purposes, we require certain configurations: with respect to servers, storage or networking which may not be supported by the cloud provider. This sometimes makes it difficult to emulate customer environments.

Challenge #4. Another commonly faced challenge is with respect to integration testing whereby the testers test the network, database, servers, etc. In such situations the tester will not have control on the underlying environment. Secondly, the challenge is doubled when there has to be an interaction between these components because the tester will have to anticipate risks like crashes, network breakdown or servers going kaput.

Conclusion

Cloud computing has today become one of those “big bangs” in the industry. Most organizations are now leaning to adopting the cloud because of its flexibility, scalability and reduced costs.

Using the cloud for testing is immensely helping organizations to acquire the required tools, software licenses, infrastructures at a very low cost without having to set it up themselves and then worry about its maximum utilization.

Evidently, cloud is here to stay for a long time and cloud testing has a bright future as well. :)

About the author: This is a guest article by Sneha Nadig. She is working as a Test lead with over 7 years of experience in manual and automation testing projects.

Are you working on cloud testing? Please share your experience. Or got a question? feel free to post it in comments.

Recommended reading

37 comments ↓

#1 smita

very useful. can you also explain how to performance test in the cloud?

#2 Amandeep Singh

Very useful explanation of testing in the cloud…

Cheers!

#3 Krishna

what is difference between testing a cloud and cloud testing?

#4 Swati

Very educative. Thank you Sneha. I thoroughly enjoyed reading it. It was my morning dose of cloud 101.

#5 jim

V.good article, in case of any more mitigation measures to the outlined cloud testing challenges. Please let me know..

#6 Sneha

Thank you all for your valuable time to read the article. Will try to keep improvising and make it more relevant.

#7 Sneha

@Smita : On a very high abstract level, I have touched upon the performance testing done in a cloud. Getting into the details would probably require another article :) You can use that as a baseline and search for your needs specifically. There are several tools available to facilitate this as well.

#8 Sneha

@Krishna : A very thin line really. This particular comparison has leaded to a lot of internal discussions, more often heated in nature. Frankly, I find these jargons tend to confuse a person even more. Even when you search the internet, I have observed that the answer is not very straight-forward. After reading up a lot of different articles and visualizing the overall picture , I am attempting to answer your question with the most satisfactory answer that I could myself agree upon :

When you refer to Testing a cloud, you basically intend to test the infrastructure, components or applications that are residing within the cloud. This testing SHOULD be performed for any application, infrastructure that is hosted the cloud or in the process of being there. This activity has to be carried out at any cost and is almost natural as having a glass of water after a meal.

On the other hand, cloud testing can viewed as “Testing as a Service” model whereby the testing activities to meet the business goals are done by people who don’t belong to the organization. These could be people who provide testing services, but use the cloud infrastructure to simulate the required environment. Off-course we have to take this information with a proverbial grain of salt because some amount of testing will be done in house by the employees.
In some other cases, the employees could also be trained by these testing service providers to do these testing activities.
So the most right way of answering this now would be that, a certain area of testing that probably doesn’t require too much know-how of the internal system works is ported out to these service providers, who carry out the testing activities. Performance testing is a popular example.

I hope this explanation was satisfactory.

#9 Chris

Great post Sneha! One of my friends has just employed an apprentice who’s going to be taking care of their cloud testing, so I’ll make sure I send this over to them.

#10 ganesh

good information on cloud

#11 Gayathri

Got an understanding of what cloud testing is about. The basics at least since I am non-IT side. Many thanks for the information.

#12 Raj

Very Helpful in understanding the concepts!!

Thank you…

#13 Nandan

It is informative. Thanks

#14 Sneha

Many thanks to all of you, who’ve taken time out to read. I’ll try to write more with respect to this.

#15 kishore

Hi,
Is there any course/training available for the same? This is indeed a good post and very useful. How would someone get into this domain?

#16 Sajid

Hi Sneha,
Same question….

Is there any course/training available for the same? This is indeed a good post and very useful. How would someone get into this domain OR get the project based on Cloud in a organization?

Based on your research, can u please elaborate testing in cloud computing as a career Or Testers demand in cloud computing ?

#17 Honey

Hello Sneha,
Thanks for spending your valuable time in knowledge sharing. I did like your article, very basic but really helpful to beginners. Out of your vaste research done around this area if you could share some material (may not be your own) or links would be grand. It really helps everyone who want to understand the banging word these days every where “Cloud cloud cloud”.
Well done! Thanks.

#18 sumanth

This is really interesting article.Is there any good training institute in Bangalore who can teach cloud computing testing.

#19 Forty Niner

Very Good Article. I have some clarifications though.
Scalability testing, Disaster testing, performance testing and all you need to do even with on premise equipment as well so why are we saying it is unique to Cloud testing. It is basically rerunning your same tests on the cloud gear. What am I missing?

I guess I am looking for the additional tests I have to do if the Application Under Test is now moving to the cloud. From what you have outlines above, I think I just need to run the same tests and make sure the numbers are acceptable as they will be different. Please help clarify.

#20 Chandralina

Good one Neha .Thanks.

#21 Phani

Very nice article.. Informative

#22 saurabh

Hi Sneha,

I am looking to schedule a knowledge sharing session, please can you inbox me your contact details.

Thanks
Saurabh Sachdeva

#23 raghav

I want to join cloud testing course.
Please let me know the availability in Bengaluru.

#24 Nandana

Hi Sneha,

The article is very useful. I’m looking for the data about automation testing on Cloud. The information like tools which support the automation testing on cloud computing, automation test approach and automation challenges. Could you please give me some information on the above.

Thanks in advance.

Thanks,
Nandana

#25 Yogita

Hi Sneha,

This article and cloud information is very helpful.All points about cloud is clearly mentioned.I am searching Some information of security testing, in security testing how to cloud working can you share please.

Thanks,
Yogita

#26 Pallavi Majumder

Hi Sneha,

Great post!!
However, can you please explain why Unit Testing is not one of the options . Unit tests do have a lot of advantages then why are they not used as often as System/Integration tests in cloud applications

#27 Ravi K

Describe “Cloud Testing Challenges” very well.

Nice article

#28 Tarun Aarya

An informative article! I like the way you explained and target all the basics things… also further I would like to ask is there any another article do you have guys based on software testing service like mobile testing, cloud testing whatever I found one “Cloud Testing Services?—?Key Informational Guide” at medium.

#29 sharmi

Hi Guys,

I’m very new to this field, so far i understand what is cloud testing, i have take over a task to do a web, android and ios cloud testing. Is there anyone can guide me what type of requirement we need to find a good software for this

email:sharmilahdevi290895@gmail.com

#30 James McAvoy

Love the way you explained the points of cloud testing services. Thanks!

#31 Jim Fernandes

Hi Sneha, loved the article. Was wondering why you havent mentioned Oracle’s TaaS (Testing as a Service) app. Would love to hear your thoughts on this particular tool.

#32 Lokesh Gunti

Hi all,

Can anyone say Difference between Testing on the cloud, Testing with in the cloud, Testing of the cloud

#33 Richa

Very informative article …I want to join the course

#34 Arun Kumar

Hi Sneha,
Great insight.
As You know most of the organizations are going for SAAS now a days, one challenge for them is how much and what all testing needs to be done on the SAAS solution to ensure is works as per their needs. For ex performance testing is must as there are many infrastructure parameter which could have impacted the performance testing of an application for the user/who is buying SAAS solution.

#35 pramod

very useful and good learning

#36 Dev

Live the Could Life!!

#37 Dev

Cloud Life!!

Leave a Comment