In this article we are going to see how to perform “cloud testing”. To understand this we need to know the cloud computing concept first. This article explains what is cloud computing, what is cloud testing, and the major challenges with testing in cloud.
Cloud computing introduction
Several years back the industry witnessed a new buzzword and technology called “virtualization”. With the advent of virtualization, the ideology of sharing computing resources across multiple operating systems to reduce capital costs, increase scalability and enable easy administration of the IT infrastructure became the backbone of several enterprises.
In the past few years, IT has witnessed an evolution of virtualization in the form of cloud computing. Any novice to cloud must picture cloud computing as a model that views everything “As a Service”. It is obvious to define cloud computing as a total solution that delivers IT as a Service. Like virtualization, its motto is resource sharing whereby allocation and availability are on demand via Internet.
[image credit kezber.com]
Real Life Example to Explain Cloud Computing:
You can understand the theory of cloud computing taking a simple example of having a meal at a restaurant. As an example consider having a meal at home versus that of having at a restaurant. While the objective is purely the consumption of a meal, the approach or model used in both these cases vary greatly.
At home, you need to make sure you have all the ingredients required for the meal, you need to make sure that the required preparation to cook the particular dish is done. You cook the food and the responsibility of keeping the surroundings tidy post the dish is cooked solely belongs to you.
Now on the other hand, having a meal at a restaurant can be described as catering as a service. As a customer, you only go to the restaurant, order the dish of your choice and pay for the services. The onus of possessing the ingredients, preparation for the dish, cooking the dish, its presentation, serving the dish to you, clearing and cleaning the table once done, packing any excessive leftover food all falls on the restaurant administration.
This example can be viewed as similar to the ideologies and principles of cloud computing – “On demand availability”, “As a service” and “Pay for the usage”.
What You Will Learn:
Cloud delivery models
[image credit hyphenet.com]
The cloud has mainly three types of delivery models or components that provide “as a Service” capabilities:
- Infrastructure as a Service (IaaS): This is the most fundamental layer which forms the building block of the cloud. It mainly comprises of physical resources such as Storage, Network devices, compute servers, etc. All these computing resources are available on demand, where a user pays as per his usage.
- Platform as a Service (PaaS): The central layer in the cloud Is the platform. Here there is no control over the underlying infrastructure but you can access the deployed applications. Hence this provides the entire runtime environment on demand which could be a development or test environment. Hence in this model, most commonly you would have a VM that contains the complete environment like OS, required middleware, etc. available when you need it.
- Software as a Service (SaaS): The topmost layer here is the application layer, which is generally visible to any user. Here, applications / products are available to the user on demand via the internet. Hence instead of having to acquire licenses for a particular user, this proves to be the most cost effective way of making sure that the license is always in use. Examples of this are Gmail, Google docs, Photoshop, etc.
Types of cloud
There are 3 types of cloud formations: public, private and hybrid clouds:
- Public clouds are where services are available to everyone, where the resources are allocated and provisioned dynamically as per request.
- Private clouds are generally managed within firewall rules of a particular organization and are available only to the users within the company.
- Hybrid clouds are a mixture of private and public clouds. Organizations can decide what services they want to expose to everyone and what services they want to expose to only the users within the organization.
Cloud testing- Need for it
Cloud testing refers to testing of resources such as hardware, software, etc. that are available on demand. Even the testing here can be viewed “as a service”. For cloud offerings it’s essential to make sure that the service (product) not only meets its functional requirements but also nonfunctional requirements. With the range of applications in the cloud it is now becoming essential to develop cloud testing strategy.
Benefits of Cloud Testing:
The need for cloud testing is easily visible with the benefits, which are far too many. Below we attempt to discuss the most apparent benefits which explain why cloud is the need of the hour:
- Dynamic availability of testing environment: The normal testing approach in any organization is to invest in the hardware/ software infrastructure needed for the testing. Almost most of you will agree that the environment supplied to testing teams very rarely matches a customer environment on account of rapidly changing requirements, as a result of which it is very challenging for companies to keep up. Cloud is the answer to this problem whereby, users can easily replicate a customer environment and find defects early in the cycle.
- Low cost: Another angle to the previous point is that when companies invest in the infrastructure, the usual case it many of their servers are not utilized all the time. As a result, they may have to incur additional expenses on license renewal. Transitioning to cloud helps in this scenario as well, as users can commission appliances as and when they want, saving huge costs for an organization.
- Easily customizable: With the usage of cloud, it’s a rather easy task for organizations to emulate an end-user centric environment by customizing it as per use saving cost and time. Test teams can easily perform load and performance testing scenarios in various permutations and combinations like – different OSes, browsers, configurations, etc.
- Scalability: One of the most attractive features of the cloud whereby the computing resources can be increased or decreased wherever it is necessary. Extremely useful in situations where the business requirements keep altering frequently.
Cloud testing deep dive
Testing with respect to a cloud will employ both traditional and new age methods to perform testing. Testing and cloud are equally large domains in their own right. Let’s learn how to view the two together in the below sections.
1) Cloud testing forms:
Cloud testing can be broadly divided into four different categories based on what they aim to do :
- Testing of the whole cloud: The cloud is viewed as a whole entity based on its features and testing is carried out based on that.
- Testing within a cloud : This is the testing carried out inside of the cloud by checking each of its internal features
- Testing across clouds: Based on specifications, here the testing is carried out on the different types of cloud like public, private and hybrid clouds.
- SaaS testing in cloud: Functional and nonfunctional testing is performed based on requirements.
2) Cloud testing environments:
There are 3 types of cloud testing environments:
- Private or public environment where the quality of applications deployed in them need to be validated.
- A hybrid environment, where the quality of applications deployed in them need to be validated.
- A test environment which is cloud based, where the quality of applications deployed in the cloud need to be validated.
3) Types of testing performed in cloud:
Testing in a cloud has to not only ensure that the functional requirements are met, but a strong emphasis needs to be laid on nonfunctional testing as well. Let’s take a look at the different kinds of testing that are performed.
Goes without saying, that functional testing has to be performed to make sure that offering provides the services that the user is paying for. Functional tests ensure that the business requirements are being met. Some of the functional tests are described below:
- System Verification Testing: This makes sure that the various modules function correctly with one another, thus making sure that their behavior is as expected.
- Acceptance Testing: Here the cloud based solution is handed over to the users to make sure it meets their expectations.
- Interoperability Testing: Any application must have the flexibility to work without any issues not only in different platforms, but also must work seamlessly when moving from cloud infrastructure to another.
Non-functional tests mainly focus on the web application based tests ensuring that they meet the desired requirements. Here are few forms of nonfunctional tests discussed below :
- Availability Testing: The cloud supervisor/ vendor has to make sure that the cloud is available round the clock. As there could be many mission critical activities going on, the administrator has to make sure that there is no adverse impact to the consumers
- Multi Tenancy Testing: Here, multiple users use a cloud offering. Testing must be performed to ensure that there is sufficient security and access control of data when multiple users are using a single instance.
- Performance Testing: Verification of the response time needs to be done to ensure that everything is intact even when there is a large amount of requests to be satisfied. The network latency is also one of the critical factors to evaluate performance. Also workload balancing needs to be done when there is a reduction in load, by decommissioning resources. Thus load and stress testing are done in the cloud offering to make sure applications are performing optimally with increase/decrease in load and stress.
- Security testing: Since with the cloud everything is available anytime, it’s essential to make sure that all user sensitive information has no unauthorized access and the privacy of users remains intact. When maintaining the applications in cloud, user data integrity must also be verified.
- Disaster recovery Testing: As already stated in availability testing, the cloud has to be available at all times and if there are any kind of failures like network outages, breakdown due to extreme load, system failures, etc, measure how fast the failure is indicated and any data loss during this period.
- Scalability Testing: Test to make sure that offering has the capability to provide scale up or scale down facilities as per the need.
Cloud testing tools:
In this section, some of the different tools used in various kinds testing performed in a cloud are mentioned. The details of the tools are out of the scope of this article.
Tools for Performance Testing in the Cloud:
Many of the tools are basically used for performance, load, stress testing. Some of these tools below can also be used for functional tests:
- SOASTA CloudTest
Cloud Security Testing Tools:
There are also some security related tools which are commonly used:
You can refer the internet for details on various tools and how they can be used in a cloud infrastructure. One such link is found here.
Cloud Testing Challenges
As exciting as it cloud sounds, all is not hunky-dory here. There are some challenges with relying and using cloud as an infrastructure as well. Let’s take a look at some of the primary concerns using the cloud.
Challenge #1. With everything available on demand to any user, security is a primary issue for the businesses as currently there is still a lot of discussion and research going on in the industry to set up security standards. User privacy protection, security standards on cloud, security of applications running within the cloud, security testing techniques are some of the primary issues that need to be addressed in the cloud infrastructure.
Challenge #2. Another big challenge is the performance of an application in a cloud: specifically in private clouds. It will be shared across many users and hence could lead to delays. Also in case of some maintenance or outage related activities, the bandwidth may seem insufficient.
Challenge #3. Sometimes for testing purposes, we require certain configurations: with respect to servers, storage or networking which may not be supported by the cloud provider. This sometimes makes it difficult to emulate customer environments.
Challenge #4. Another commonly faced challenge is with respect to integration testing whereby the testers test the network, database, servers, etc. In such situations the tester will not have control on the underlying environment. Secondly, the challenge is doubled when there has to be an interaction between these components because the tester will have to anticipate risks like crashes, network breakdown or servers going kaput.
Cloud computing has today become one of those “big bangs” in the industry. Most organizations are now leaning to adopting the cloud because of its flexibility, scalability and reduced costs.
Using the cloud for testing is immensely helping organizations to acquire the required tools, software licenses, infrastructures at a very low cost without having to set it up themselves and then worry about its maximum utilization.
Evidently, cloud is here to stay for a long time and cloud testing has a bright future as well. :)
About the author: This is a guest article by Sneha Nadig. She is working as a Test lead with over 7 years of experience in manual and automation testing projects.
Are you working on cloud testing? Please share your experience. Or got a question? feel free to post it in comments.