Getting Started with Cloud Testing

In this article, we are going to learn how to perform “Cloud Testing”. In order to understand this, we need to know the Cloud Computing concept first. This article explains about Cloud Computing, Cloud Testing, and the major challenges with testing in the cloud.

Cloud Computing Introduction

Several years ago, the industry witnessed a new buzzword and technology called “virtualization”. With the advent of virtualization, the ideology of sharing computing resources across multiple operating systems in order to increase scalability, reduce capital costs and enable easy administration of the IT infrastructure, it became the backbone of several enterprises.

In the past few years, IT has witnessed an evolution of virtualization in the form of cloud computing. Any novice to the cloud must picture cloud computing as a model that views everything “As a Service”.

It is obvious to define cloud computing as a total solution that delivers IT as a Service. Like virtualization, its motto is resource sharing whereby allocation and availability are on-demand via the Internet.

[image source]

Real-Life Example to Explain Cloud Computing:

We can easily understand the theory of Cloud Computing by taking a simple example of having a meal at a restaurant. For Example, consider having a meal at home versus that of having at a restaurant. While the objective is purely the consumption of a meal, the approach or model used in both these cases varies to a great extent.

At home, you need to make sure whether you have all the ingredients required for the meal, you need to make sure that the required preparation to cook the particular dish is done. You cook the food and the responsibility of keeping the surroundings tidy post the dish is cooked solely belongs to you.

Now, on the other hand, having a meal at a restaurant can be described as catering which is a service. As a customer, you only go to the restaurant, order the dish of your choice and pay for the services.

The bonus of possessing the ingredients, preparation for the dish, cooking the dish, its presentation, serving the dish to you, clearing and cleaning the table once done, packing any excessive leftover food all fall on the restaurant’s administration.

This example can be viewed as similar to the ideologies and the principles of Cloud Computing – “On-demand availability”, “As a service” and “Pay for the usage”.

Cloud Delivery Models

[image source]

The cloud mainly has three types of Delivery Models or components that provide “as a Service” capabilities:

• Infrastructure as a Service (IaaS): This is the most fundamental layer which forms the building block of the cloud. It mainly comprises of physical resources such as Storage, Network devices, computing servers, etc. All these computing resources are available on-demand, where a user pays for it as per his usage.

• Platform as a Service (PaaS): The central layer in the cloud Is the platform. Here there is no control over the underlying infrastructure, but you can access the deployed applications. Hence this provides the entire run-time environment on demand which could be a development or a test environment. Hence in this model, most commonly you would have a VM that contains a complete environment like OS, required middleware, etc. available when you need it.

• Software as a Service (SaaS): The topmost layer here is the application layer, which will generally visible to any user. Here, applications/products are available to the users on-demand via the internet. Hence instead of having to acquire the license for a particular user, this proves to be the most cost-effective way of making sure that the license is always in use. Examples of this are Gmail, Google Docs, Photoshop, etc.

Types Of Cloud

There are 3 types of cloud formations namely Public, Private and Hybrid clouds:

• Public clouds are the ones, where the services are available to everyone, where the resources are allocated and provisioned dynamically as per the request.
• Private clouds are generally managed within the firewall rules of a particular organization and are available only to the users within the company.
• Hybrid clouds are a mixture of both private and public clouds. Organizations can decide what services they want to expose to everyone and what services they want to expose only to the users within the organization.

Cloud Testing- Need For It

Cloud Testing refers to the testing of resources such as hardware, software, etc. that are available on demand. Even the testing here can be viewed “as a service”. For cloud offerings, it’s essential to make sure that the service (product) not only meets its functional requirements but also the non-functional requirements. With a range of applications in the cloud, it is now becoming essential to develop the Cloud Testing strategy.

Benefits Of Cloud Testing

The need for cloud testing is easily visible with the benefits that we derive from it, which are far too many. Below we have attempted to discuss the most apparent benefits which explain why Cloud is the need of the hour:

1. Dynamic availability of testing environment: The normal testing approach in any organization is to invest in the hardware/software infrastructure required for testing. Almost, many of you will agree that the environment supplied to the testing teams very rarely match a customer environment on account of rapidly changing requirements, as a result of which it is very challenging for the companies to keep it up. Cloud is the only answer to this problem whereby, users can easily replicate a customer environment and find defects early in the cycle.

2. Low cost: Another angle to the previous point is that when companies invest in the infrastructure, the usual cause of it is many of their servers are not utilized all the time. As a result, they may have to incur additional expenses on license renewal. Transitioning to the cloud helps in this scenario as well, as users can commission appliances as and when they want, thereby saving huge costs for an organization.

3. Easily customizable: With the use of cloud, it’s an easy task for organizations to emulate an end-user centric environment by customizing it as per the usage thereby, saving cost and time. Test teams can easily perform load and performance testing scenarios in various permutations and combinations like – different OSes, browsers, configurations, etc.

4. Scalability: This is one of the most attractive features of the cloud whereby the computing resources can be increased or decreased wherever it is necessary. This is widely used in situations where the business requirements keep altering frequently.

Cloud Testing Deep Dive

Testing with respect to a cloud will employ both traditional and new age methods to perform testing. Testing and cloud are equally large domains in their own path. Let’s learn how to view both together in the below sections.

#1) Cloud Testing Forms

Cloud Testing can be broadly divided into four different categories based on what they aim to do :

• Testing of the whole cloud: The cloud is viewed as a whole entity based on its features and testing is carried out based on that.
• Testing within a cloud: This is the testing that is carried out inside the cloud by checking each of its internal features
• Testing across the clouds: Based on the specifications, here the testing is carried out on the different types of clouds-like public, private and hybrid clouds.
• SaaS testing in cloud: Functional and non-functional testing is performed based on requirements.

#2) Cloud Testing Environments

There are 3 types of Cloud Testing Environments:

• A private or public environment where the quality of applications deployed in them needs to be validated.
• A hybrid environment, where the quality of applications deployed in them needs to be validated.
• A test environment that is cloud-based, where the quality of applications deployed in the cloud needs to be validated.

#3) Types Of Testing Performed In The Cloud

Testing in a cloud has to not only ensure that the functional requirements are met, but a strong emphasis needs to be laid on nonfunctional testing as well. Let’s take a look at the different kinds of testing that are performed.

Functional Testing:

Functional Testing has to be performed to make sure that the offering provides the services that the user is paying for. Functional tests ensure that the business requirements are being met.

Some of the functional tests are described below:

• System Verification Testing: This ensures whether the various modules function correctly with one another, thus making sure that their behavior is as expected.
• Acceptance Testing: Here the cloud-based solution is handed over to the users to make sure it meets their expectations.
• Interoperability Testing: Any application must have the flexibility to work without any issues not only on different platforms but it must also work seamlessly when moving from cloud infrastructure to another.

Non-Functional Testing:

Non-functional tests mainly focus on a web application based tests ensuring that they meet the desired requirements.

Here are a few forms of non-functional tests discussed below:

#1) Availability Testing: The cloud supervisor/vendor has to make sure that the cloud is available round the clock. As there could be many mission-critical activities going on, the administrator has to make sure that there is no adverse impact to the consumers

#2) Multi-Tenancy Testing: Here, multiple users use a cloud offering. Testing must be performed to ensure that there is sufficient security and access control of the data when multiple users are using a single instance.

#3) Performance Testing: Verification of the response time needs to be done to ensure that everything is intact even when there is a lot of requests to be satisfied. The network latency is also one of the critical factors to evaluate performance.

Also, workload balancing needs to be done when there is a reduction in load, by decommissioning resources. Thus load and stress testing are done in the cloud offering to make sure applications are performing optimally with an increase/decrease in load and stress.

#4) Security Testing: As everything is available anytime with Cloud, it’s essential to make sure that all user sensitive information has no unauthorized access and the privacy of users remains intact. When maintaining the applications in the cloud, user data integrity must also be verified.

#5) Disaster Recovery Testing: As already stated in availability testing, the cloud has to be available at all times and if there are any kind of failures like network outages, breakdown due to extreme load, system failures, etc, measure how fast the failure is indicated and if any data loss occurs during this period.

#6) Scalability Testing: Test to make sure that offering has the capability to provide scale up or scale down facilities as per the need.

Cloud Testing Tools

Some of the tools used in different kinds of testing performed in a cloud environment are mentioned in this section. The details of the tools are out of the scope of this article.

Tools for Performance Testing in the Cloud:

Many of the tools are basically used for performance, load, stress testing. Some of these tools below can also be used for functional tests:

• SOASTA CloudTest
• LoadStorm
• CloudTestGo
• AppPerfect.
• Jmeter
• Cloudslueth
• CloudTestGo
• AppPerfect

Cloud Security Testing Tools:

There are also some security-related tools which are commonly used:

• Nessus
• Wireshark
• Nmap

=> Click here for a complete list of Cloud-Based Automation Testing Tools

Cloud Testing Challenges

As exciting as the cloud sounds, all is not hunky-dory here. There are some challenges with relying on and using the cloud as infrastructure as well. Let’s take a look at some of the primary concerns while using the cloud.

Challenge #1) With everything available on-demand to any user, security is a primary issue for the businesses as currently there is still a lot of discussion and research going on in the industry to set up security standards. User privacy protection, security standards on the cloud, the security of applications running in the cloud, Security Testing techniques are some of the primary issues that need to be addressed in the cloud infrastructure.

Challenge #2) Another big challenge is the performance of an application in a cloud, specifically in private clouds. It will be shared across many users and hence could lead to delays as well. Also in case of some maintenance or outage-related activities, the bandwidth may seem to be insufficient.

Challenge #3) Sometimes for testing purposes, we require certain configurations with respect to servers, storage or networking which may not be supported by the cloud provider. This sometimes makes it difficult to emulate customer environments.

Challenge #4) Another commonly faced challenge is with respect to integration testing whereby the testers test the network, database, servers, etc. In such situations, the tester will not have control over the underlying environment. Secondly, the challenge is doubled when there has to be an interaction between these components because the tester will have to anticipate risks like crashes, network breakdown or servers going kaput.

Conclusion

Today, Cloud Computing has become one of those “big bangs” in the industry. Most organizations are now leaning towards adopting the cloud because of its flexibility, scalability and reduced costs.

Using the cloud for testing is immensely helping organizations to acquire the required tools, software licenses, infrastructures at a very low cost without having to set it up themselves and later worry about its maximum utilization.

Evidently, a cloud is here to stay for a long time and cloud testing has a bright future as well. :)

About the author: This is a guest article by Sneha Nadig. She is working as a Test lead with over 7 years of experience in manual and automation testing projects.

Are you working on Cloud Testing? Please share your experience. Or got a question? feel free to post it in comments.