Getting Started with Cloud Testing

In this article, we are going to learn how to perform “cloud testing”. In order to understand this, we need to know the cloud computing concept first. This article explains about cloud computing, cloud testing, and the major challenges with testing in the cloud.

Cloud Computing Introduction

Several years ago, the industry witnessed a new buzzword and technology called “virtualization”. With the advent of virtualization, the ideology of sharing computing resources across multiple operating systems in order to increase scalability, reduce capital costs and enable easy administration of the IT infrastructure, it became the backbone of several enterprises.

In the past few years, IT has witnessed an evolution of virtualization in the form of cloud computing. Any novice to the cloud must picture cloud computing as a model that views everything “As a Service”. It is obvious to define cloud computing as a total solution that delivers IT as a Service. Like virtualization, its motto is resource sharing whereby allocation and availability are on demand via the Internet.

cloud computing and testing

[image credit kezber.com]

Real Life Example to Explain Cloud Computing:

We can easily understand the theory of cloud computing by taking a simple example of having a meal at a restaurant. For example, consider having a meal at home versus that of having at a restaurant. While the objective is purely the consumption of a meal, the approach or model used in both these cases vary to a great extent.

At home, you need to make sure whether you have all the ingredients required for the meal, you need to make sure that the required preparation to cook the particular dish is done. You cook the food and the responsibility of keeping the surroundings tidy post the dish is cooked solely belongs to you.

Now, on the other hand, having a meal at a restaurant can be described as catering which is a service. As a customer, you only go to the restaurant, order the dish of your choice and pay for the services. The onus of possessing the ingredients, preparation for the dish, cooking the dish, its presentation, serving the dish to you, clearing and cleaning the table once done, packing any excessive leftover food all fall on the restaurant’s administration.

This example can be viewed as similar to the ideologies and the principles of cloud computing – “On-demand availability”, “As a service” and “Pay for the usage”.

Cloud Delivery Models

Cloud testing layers

[image credit hyphenet.com]

The cloud mainly has three types of delivery models or components that provide “as a Service” capabilities:

  • Infrastructure as a Service (IaaS): This is the most fundamental layer which forms the building block of the cloud. It mainly comprises of physical resources such as Storage, Network devices, computing servers, etc. All these computing resources are available on demand, where a user pays for it as per his usage.
  • Platform as a Service (PaaS): The central layer in the cloud Is the platform. Here there is no control over the underlying infrastructure, but you can access the deployed applications. Hence this provides the entire run-time environment on demand which could be a development or a test environment. Hence in this model, most commonly you would have a VM that contains the complete environment like OS, required middleware, etc. available when you need it.
  • Software as a Service (SaaS): The topmost layer here is the application layer, which will generally visible to any user. Here, applications/products are available to the users on demand via the internet. Hence instead of having to acquire the license for a particular user, this proves to be the most cost-effective way of making sure that the license is always in use. Examples of this are Gmail, Google Docs, Photoshop, etc.

Types of Cloud

There are 3 types of cloud formations namely Public, Private and Hybrid clouds:

  • Public clouds are the ones, where the services are available to everyone, where the resources are allocated and provisioned dynamically as per the request.
  • Private clouds are generally managed within firewall rules of a particular organization and are available only to the users within the company.
  • Hybrid clouds are a mixture of both private and public clouds. Organizations can decide what services they want to expose to everyone and what services they want to expose only to the users within the organization.

Cloud Testing- Need For It

Cloud testing refers to testing of resources such as hardware, software, etc. that are available on demand. Even the testing here can be viewed “as a service”. For cloud offerings, it’s essential to make sure that the service (product) not only meets its functional requirements but also the non-functional requirements. With a range of applications in the cloud, it is now becoming essential to develop the cloud testing strategy.

top applications in the cloud

Benefits of Cloud Testing

The need for cloud testing is easily visible with the benefits that we derive from it, which are far too many. Below we have attempted to discuss the most apparent benefits which explain why Cloud is the need of the hour:

  1. Dynamic availability of testing environment: The normal testing approach in any organization is to invest in the hardware/ software infrastructure required for testing. Almost, many of you will agree that the environment supplied to the testing teams very rarely match a customer environment on account of rapidly changing requirements, as a result of which it is very challenging for the companies to keep it up. Cloud is the only answer to this problem whereby, users can easily replicate a customer environment and find defects early in the cycle.
  1. Low cost: Another angle to the previous point is that when companies invest in the infrastructure, the usual case of it is many of their servers are not utilized all the time. As a result, they may have to incur additional expenses on license renewal. Transitioning to cloud helps in this scenario as well, as users can commission appliances as and when they want, thereby saving huge costs for an organization.
  1. Easily customizable: With the use of cloud, it’s an easy task for organizations to emulate an end-user centric environment by customizing it as per the usage thereby, saving cost and time. Test teams can easily perform load and performance testing scenarios in various permutations and combinations like – different OSes, browsers, configurations, etc.
  1. Scalability: This is one of the most attractive features of the cloud whereby the computing resources can be increased or decreased wherever it is necessary. This is widely used in the situations where the business requirements keep altering frequently.

Cloud Testing Deep Dive

Testing with respect to a cloud will employ both traditional and new age methods to perform testing. Testing and cloud are equally large domains in their own path. Let’s learn how to view the both together in the below sections.


1) Cloud Testing Forms:

Cloud testing can be broadly divided into four different categories based on what they aim to do :

  • Testing of the whole cloud: The cloud is viewed as a whole entity based on its features and testing is carried out based on that.
  • Testing within a cloud: This is the testing that is carried out inside the cloud by checking each of its internal features
  • Testing across the clouds: Based on the specifications, here the testing is carried out on the different type of clouds-like public, private and hybrid clouds.
  • SaaS testing in cloud: Functional and non-functional testing is performed based on requirements.

2) Cloud Testing Environments:

There are 3 types of cloud testing environments:

  • A private or public environment where the quality of applications deployed in them needs to be validated.
  • A hybrid environment, where the quality of applications deployed in them needs to be validated.
  • A test environment which is cloud-based, where the quality of applications deployed in the cloud needs to be validated.

3) Types of Testing Performed in the Cloud:

Testing in a cloud has to not only ensure that the functional requirements are met, but a strong emphasis needs to be laid on nonfunctional testing as well. Let’s take a look at the different kinds of testing that are performed.

Functional Testing:

Functional testing has to be performed to make sure that offering provides the services that the user is paying for. Functional tests ensure that the business requirements are being met.

Some of the functional tests are described below:

  • System Verification Testing: This ensures whether the various modules function correctly with one another, thus making sure that their behavior is as expected.
  • Acceptance Testing: Here the cloud-based solution is handed over to the users to make sure it meets their expectations.
  • Interoperability Testing: Any application must have the flexibility to work without any issues not only on different platforms but it must also work seamlessly when moving from cloud infrastructure to another.

Non-Functional Testing:

Non-functional tests mainly focus on the web application based tests ensuring that they meet the desired requirements.

Here are few forms of nonfunctional tests discussed below:

#1) Availability Testing: The cloud supervisor/vendor has to make sure that the cloud is available round the clock. As there could be many mission-critical activities going on, the administrator has to make sure that there is no adverse impact to the consumers

#2) Multi-Tenancy Testing: Here, multiple users use a cloud offering. Testing must be performed to ensure that there is sufficient security and access control of the data when multiple users are using a single instance.

#3) Performance Testing: Verification of the response time needs to be done to ensure that everything is intact even when there is a lot of requests to be satisfied. The network latency is also one of the critical factors to evaluate performance. Also, workload balancing needs to be done when there is a reduction in load, by decommissioning resources. Thus load and stress testing are done in the cloud offering to make sure applications are performing optimally with increase/decrease in load and stress.

#4) Security Testing: As everything is available anytime with Cloud, it’s essential to make sure that all user sensitive information has no unauthorized access and the privacy of users remains intact. When maintaining the applications in the cloud, user data integrity must also be verified.

#5) Disaster recovery Testing: As already stated in availability testing, the cloud has to be available at all times and if there are any kind of failures like network outages, breakdown due to extreme load, system failures, etc, measure how fast the failure is indicated and if any data loss occurs during this period.

#6) Scalability Testing: Test to make sure that offering has the capability to provide scale up or scale down facilities as per the need.

Cloud Testing Tools

Some of the tools used in different kinds of testing performed in a cloud environment are mentioned in this section. The details of the tools are out of the scope of this article.

Tools for Performance Testing in the Cloud:

Many of the tools are basically used for performance, load, stress testing. Some of these tools below can also be used for functional tests:

  • SOASTA CloudTest
  • LoadStorm
  • CloudTestGo
  • AppPerfect.
  • Jmeter
  • Cloudslueth
  • CloudTestGo
  • AppPerfect

Cloud Security Testing Tools:

There are also some security related tools which are commonly used:

  • Nessus
  • Wireshark
  • Nmap

You can refer the internet for details on various tools and how they can be used in a cloud infrastructure. One such link is found here.

Cloud Testing Challenges

As exciting as the cloud sounds, all is not hunky-dory here. There are some challenges with relying on and using the cloud as an infrastructure as well. Let’s take a look at some of the primary concerns while using the cloud.

Challenge #1. With everything available on demand to any user, security is a primary issue for the businesses as currently there is still a lot of discussion and research going on in the industry to set up security standards. User privacy protection, security standards on the cloud, security of applications running in the cloud, security testing techniques are some of the primary issues that need to be addressed in the cloud infrastructure.

Challenge #2. Another big challenge is the performance of an application in a cloud, specifically in private clouds. It will be shared across many users and hence could lead to delays as well. Also in case of some maintenance or outage-related activities, the bandwidth may seem to be insufficient.

Challenge #3. Sometimes for testing purposes, we require certain configurations with respect to servers, storage or networking which may not be supported by the cloud provider. This sometimes makes it difficult to emulate customer environments.

Challenge #4. Another commonly faced challenge is with respect to integration testing whereby the testers test the network, database, servers, etc. In such situations, the tester will not have control over the underlying environment. Secondly, the challenge is doubled when there has to be an interaction between these components because the tester will have to anticipate risks like crashes, network breakdown or servers going kaput.

Conclusion

Today, Cloud Computing has become one of those “big bangs” in the industry. Most organizations are now leaning towards adopting the cloud because of its flexibility, scalability and reduced costs.

Using the cloud for testing is immensely helping organizations to acquire the required tools, software licenses, infrastructures at a very low cost without having to set it up themselves and later worry about its maximum utilization.

Evidently, cloud is here to stay for a long time and cloud testing has a bright future as well. :)

About the author: This is a guest article by Sneha Nadig. She is working as a Test lead with over 7 years of experience in manual and automation testing projects.

Are you working on Cloud testing? Please share your experience. Or got a question? feel free to post it in comments.




Recommended reading

39 comments ↓

#1 smita

very useful. can you also explain how to performance test in the cloud?

#2 Amandeep Singh

Very useful explanation of testing in the cloud…

Cheers!

#3 Krishna

what is difference between testing a cloud and cloud testing?

#4 Swati

Very educative. Thank you Sneha. I thoroughly enjoyed reading it. It was my morning dose of cloud 101.

#5 jim

V.good article, in case of any more mitigation measures to the outlined cloud testing challenges. Please let me know..

#6 Sneha

Thank you all for your valuable time to read the article. Will try to keep improvising and make it more relevant.

#7 Sneha

@Smita : On a very high abstract level, I have touched upon the performance testing done in a cloud. Getting into the details would probably require another article :) You can use that as a baseline and search for your needs specifically. There are several tools available to facilitate this as well.

#8 Sneha

@Krishna : A very thin line really. This particular comparison has leaded to a lot of internal discussions, more often heated in nature. Frankly, I find these jargons tend to confuse a person even more. Even when you search the internet, I have observed that the answer is not very straight-forward. After reading up a lot of different articles and visualizing the overall picture , I am attempting to answer your question with the most satisfactory answer that I could myself agree upon :

When you refer to Testing a cloud, you basically intend to test the infrastructure, components or applications that are residing within the cloud. This testing SHOULD be performed for any application, infrastructure that is hosted the cloud or in the process of being there. This activity has to be carried out at any cost and is almost natural as having a glass of water after a meal.

On the other hand, cloud testing can viewed as “Testing as a Service” model whereby the testing activities to meet the business goals are done by people who don’t belong to the organization. These could be people who provide testing services, but use the cloud infrastructure to simulate the required environment. Off-course we have to take this information with a proverbial grain of salt because some amount of testing will be done in house by the employees.
In some other cases, the employees could also be trained by these testing service providers to do these testing activities.
So the most right way of answering this now would be that, a certain area of testing that probably doesn’t require too much know-how of the internal system works is ported out to these service providers, who carry out the testing activities. Performance testing is a popular example.

I hope this explanation was satisfactory.

#9 Chris

Great post Sneha! One of my friends has just employed an apprentice who’s going to be taking care of their cloud testing, so I’ll make sure I send this over to them.

#10 ganesh

good information on cloud

#11 Gayathri

Got an understanding of what cloud testing is about. The basics at least since I am non-IT side. Many thanks for the information.

#12 Raj

Very Helpful in understanding the concepts!!

Thank you…

#13 Nandan

It is informative. Thanks

#14 Sneha

Many thanks to all of you, who’ve taken time out to read. I’ll try to write more with respect to this.

#15 kishore

Hi,
Is there any course/training available for the same? This is indeed a good post and very useful. How would someone get into this domain?

#16 Sajid

Hi Sneha,
Same question….

Is there any course/training available for the same? This is indeed a good post and very useful. How would someone get into this domain OR get the project based on Cloud in a organization?

Based on your research, can u please elaborate testing in cloud computing as a career Or Testers demand in cloud computing ?

#17 Honey

Hello Sneha,
Thanks for spending your valuable time in knowledge sharing. I did like your article, very basic but really helpful to beginners. Out of your vaste research done around this area if you could share some material (may not be your own) or links would be grand. It really helps everyone who want to understand the banging word these days every where “Cloud cloud cloud”.
Well done! Thanks.

#18 sumanth

This is really interesting article.Is there any good training institute in Bangalore who can teach cloud computing testing.

#19 Forty Niner

Very Good Article. I have some clarifications though.
Scalability testing, Disaster testing, performance testing and all you need to do even with on premise equipment as well so why are we saying it is unique to Cloud testing. It is basically rerunning your same tests on the cloud gear. What am I missing?

I guess I am looking for the additional tests I have to do if the Application Under Test is now moving to the cloud. From what you have outlines above, I think I just need to run the same tests and make sure the numbers are acceptable as they will be different. Please help clarify.

#20 Chandralina

Good one Neha .Thanks.

#21 Phani

Very nice article.. Informative

#22 saurabh

Hi Sneha,

I am looking to schedule a knowledge sharing session, please can you inbox me your contact details.

Thanks
Saurabh Sachdeva

#23 raghav

I want to join cloud testing course.
Please let me know the availability in Bengaluru.

#24 Nandana

Hi Sneha,

The article is very useful. I’m looking for the data about automation testing on Cloud. The information like tools which support the automation testing on cloud computing, automation test approach and automation challenges. Could you please give me some information on the above.

Thanks in advance.

Thanks,
Nandana

#25 Yogita

Hi Sneha,

This article and cloud information is very helpful.All points about cloud is clearly mentioned.I am searching Some information of security testing, in security testing how to cloud working can you share please.

Thanks,
Yogita

#26 Pallavi Majumder

Hi Sneha,

Great post!!
However, can you please explain why Unit Testing is not one of the options . Unit tests do have a lot of advantages then why are they not used as often as System/Integration tests in cloud applications

#27 Ravi K

Describe “Cloud Testing Challenges” very well.

Nice article

#28 Tarun Aarya

An informative article! I like the way you explained and target all the basics things… also further I would like to ask is there any another article do you have guys based on software testing service like mobile testing, cloud testing whatever I found one “Cloud Testing Services?—?Key Informational Guide” at medium.

#29 sharmi

Hi Guys,

I’m very new to this field, so far i understand what is cloud testing, i have take over a task to do a web, android and ios cloud testing. Is there anyone can guide me what type of requirement we need to find a good software for this

email:sharmilahdevi290895@gmail.com

#30 James McAvoy

Love the way you explained the points of cloud testing services. Thanks!

#31 Jim Fernandes

Hi Sneha, loved the article. Was wondering why you havent mentioned Oracle’s TaaS (Testing as a Service) app. Would love to hear your thoughts on this particular tool.

#32 Lokesh Gunti

Hi all,

Can anyone say Difference between Testing on the cloud, Testing with in the cloud, Testing of the cloud

#33 Richa

Very informative article …I want to join the course

#34 Arun Kumar

Hi Sneha,
Great insight.
As You know most of the organizations are going for SAAS now a days, one challenge for them is how much and what all testing needs to be done on the SAAS solution to ensure is works as per their needs. For ex performance testing is must as there are many infrastructure parameter which could have impacted the performance testing of an application for the user/who is buying SAAS solution.

#35 pramod

very useful and good learning

#36 Dev

Live the Could Life!!

#37 Dev

Cloud Life!!

#38 surfcloudtech

Performance testing is most important in cloud testing as we have to check the scalability of the system. We can also test the vertical and horizontal scaling in cloud computing.

#39 Abhijeet

To learn Cloud Testing, should I also learn Cloud Computing to get a better idea of cloud testing

Leave a Comment