System Testing – A Beginner’s Guide

So…do you really think it will take that huge amount of time to test, what you call system testing, even after spending a lot of efforts on integration testing?

The client we recently approached for the project was not convinced about the estimation we provided for each testing effort.

I had to chime in with an example –

Mike, I would like to elaborate our efforts and importance of system testing with an example.

Shoot, he replied. 

A car manufacturer does not produce the car as a whole car. Each component of the car is manufactured separately, like seats, steering, mirror, break, cable, engine, car frame, wheels etc. 

After manufacturing each item, it is tested independently whether it is working the way it is supposed to work and that is called Unit testing.

Now, when each part is assembled with another part, that assembled combination is checked if assembling has not produced any side effect to the functionality of each component and whether both components are working together as expected and that is called integration testing.

Once all the parts are assembled and the car is ready, it is not ready actually.

The whole car needs to be checked for different aspects as per the requirements defined like if car can be driven smoothly, breaks, gears and other functionality working properly, car does not show any sign of tiredness after being driven for 2500 miles continuously, color of car is generally accepted and liked, car can be driven on any kind of roads like smooth and rough, sloppy and straight etc and this whole effort of testing is called System testing and it has nothing to do with integration testing.

The example worked the way it was expected and the client was convinced about efforts required for ST (system testing).

I narrated the example here to encourage the importance of ST.

system test

So, what is System Testing?

Definition: As per IEEE, it is – The process of testing of an integrated hardware and software system to verify that the system meets its specified requirements.

Verification: Confirmation by examination and provisions of objective evidence that specified requirements have been fulfilled.

Approach:

It is performed when integration testing is completed.

System Testing 1

It is mainly a black box type testing. This testing evaluates working of the system from a user point of view, with the help of specification document. It does not require any internal knowledge of system like design or structure of the code.

It contains functional and non-functional areas of application/product.

Focus criteria:

It mainly focuses on following:

  1. External interfaces
  2. Multiprogram and complex functionalities
  3. Security
  4. Recovery
  5. Performance
  6. Operator and user’s smooth interaction with system
  7. Installability
  8. Documentation
  9. Usability
  10. Load / Stress

Why System Testing?

#1. It is very important to complete a full test cycle and ST is the stage where it is done.

#2. ST is performed in an environment which is similar to the production environment and hence stakeholders can get a good idea of the user’s reaction.

#3. It helps to minimize after-deployment troubleshooting and support calls.

#4. In this STLC stage Application Architecture and Business requirements, both are tested.

How to perform System test?

It is basically a part of software testing and test plan should always contain specific space for this testing.

To test the system as a whole, requirements and expectations should be clear and the tester needs to understand the real-time usage of application too.

Also, most used third-party tools, version of OSes, flavours and architecture of OSes can affect system’s functionality, performance, security, recoverability or installability.

Therefore, while testing system a clear picture of how the application is going to be used and what kind of issues it can face in real time can be helpful. In addition to that, a requirements document is as important as understanding the application.

Clear and updated requirements document can save tester from a number of misunderstandings, assumptions and questions.

In short, a pointed and crisp requirement document with latest updates along with an understanding of real-time application usage can make ST more fruitful.

Example Test Scenarios

System testing sample test scenarios for an eCommerce Site:

1. If the site launches properly with all the relevant pages, features and logo

2. If the user can register/login to the site

3. If the user can see products available, can add products to his cart can do payment and can get confirmation via e-mail or SMS or call.

4. If the major functionality like searching, filtering, sorting, adding, changing, wishlist etc work as expected

4. If number of users (defined as in requirement document) can access the site simultaneously

5. If the site launches properly in all major browsers and their latest versions

6. If the transactions are being done on the site via specific user are secure enough

7. If the site launches properly on all the supported platforms like Windows, Linux, Mobile etc.

8. If the user manual/guide return policy, privacy policy and terms of using the site is available as a separate document and useful to any newbie or first time user.

9. If the content of pages is properly aligned, well managed and without spelling mistakes.

10. If session timeout is implemented and working as expected

11. If a user is satisfied after using the site or in other words user does not find it difficult to use the site.


Types of System Test:

ST is called a superset of all types of testing as all the major types of testing are covered in it. Although a focus on types of testing may vary on the basis of product, organization processes, timeline and requirements.

The overall it can be defined as below: (click on image to enlarge)

System Testing 2

Installation Testing: To make sure that product/software can be installed on specific or support defined system, can be configured and can be brought into an operational mode.

Functionality Testing: To make sure that functionality of product is working as per the requirements defined, within the capabilities of the system,
Recoverability Testing: To make sure how well the system recovers from various input errors and other failure situations.

Interoperability Testing: To make sure whether the system can operate well with third-party products or not.

Performance Testing: To make sure system’s performance under the various condition, in terms of performance characteristics.

Scalability Testing: To make sure system’s scaling abilities in various terms like user scaling, geographic scaling and resource scaling.

Reliability Testing: To make sure system can be operated for longer duration without developing failures.

Regression Testing: To make sure system’s stability as it passes through an integration of different subsystems and maintenance tasks.

Documentation Testing: To make sure that system’s user guide and other help topics documents are correct and usable.

Security Testing: To make sure that system does not allow unauthorized access to data and resources.

Usability Testing: To make sure that system is easy to use, learn and operate.

Tips for fruitful System testing:

Tip #1: Replicate real-time scenarios rather than doing ideal testing as the system is going to be used by an end user and not by the trained tester.

Tip #2: Verify system’s response in various terms as the human does not like to wait or to see wrong data.

Tip #3: Install and configure the system as per the documentation because that is what end user is going to do.

Tip #4: Involving people from different areas like business analysts, developers, testers, customers can end in a better system.

Tip #5: Regular testing is the only way to make sure that littlest change in the code to fix the bug has not inserted another critical bug into the system.

Entry Criteria:

  • Software development -> completed
  • All modules are integrated fully and integration is tested
  • Unit testing -> completed
  • Production like test environment is ready

What is System Integration Testing?

It is a type of testing in which system’s ability to maintain data integrity and operation in coordination with other systems in the same environment, is being checked.

Example of System Integration Testing:

Let’s take an example of well-known online ticket booking site – http://irctc.co.in.

This is a ticket booking facility; online shopping facility interacts with PayPal. Overall you can consider it as A*B*C=R.

Now on the system level, online ticket booking facility, online shopping facility and online payment option facility can be system tested independently, followed by check perform Integration tests for each of them. And then the whole system needs to be system tested.

So where does System Integration testing come into the picture?

The web portal http://Irctc.co.in is a combination of systems. You may perform tests at the same level (single system, the system of systems), but at each level, you may want to focus on different risks (integration problems, independent functionality).

  • While testing Online Ticket booking facility, you may verify if you are able to book tickets online. You may also consider integration problems e.g. Ticket booking facility integrates back-end with front-end (UI). For example, how front-end behaves when the database server is slow to respond?
  • Testing of Online ticket booking facility with online shopping facility. You may verify that online shopping facility is available for the users logged into the system to book tickets online. You may also consider verification of integration in online shopping facility. For example, if the user is able to select and buy a product without a hassle.
  • Testing of online ticket booking facility’s integration with PayPal. You may verify whether, after booking tickets, money was transferred from your PayPal account to Online Ticket Booking account. You may also consider verification of integration in PayPal. For example, what if system puts two entries in a database after debiting money for once only?

Difference between System Testing and System Integration Testing:

The main difference is –

  • System Testing looks after single system’s integrity with relevant environment
  • System Integration Testing looks after multiple systems’ integrity with each other, being in the same environment.

Thus, ST is the beginning of real testing where you test a product as a whole and not a module/feature.

With this, we are ending our System Testing session and again ready to discuss with your readers. As usual, please pour in your comments and views on the post and we would like to connect with our readers to make the software testing community stronger.

About the author: This guide is written by STH team member Bhumika Mehta. She is a project lead, carrying more than 7 years of experience in software testing.




Recommended reading

30 comments ↓

#1 Aparna

Useful info..

#2 Ravichandra Kammar

Thanks for Clear Explanation, please share on Performance Testing by Using JMeter.

#3 Supriya

very important and probably the most time consuming part in the STLC?

#4 veda

gud & nice one

#5 Inthiyaz

Detailed and very informative. thanks for sharing!

#6 Rupali

thanks for giving this very useful info

#7 chandrasekar

Hi,

Thanks for explaining about the testing levels with clear examples. This kind of examples save our time to understand the concepts in a minute. Because this kind concepts and all we can’t imagine while learning and studying by self. Really awesome! This is the best way of teaching too. Thanks a lot once again

#8 karthik R

it’s a useful information for beginners like me

#9 Bhavin Shah

Amazing info.. Thanks a ton…!!

#10 Bhumika Mehta

@Bhavin, @Karthik, @Chandrasekar, @Roopali, @Inthiyaz, @Veda, @Supriya, @Aparna

Glad to know that the article was useful and thanks a lot for your continuous readership with us.

#11 shikha

what should be the integration scenario based on above write system testing scenario

#12 Md. Lokman Hekim

Yes, very good article. Waiting more informative article about software testing.

#13 Suresh Uva

Thanks for this Nice article..

#14 kanika

all the articles are to the point yet covers everything thing minutely. Keep it up.

#15 Matt

Thanks a lot for article. It is really helpful for beginning testers!

#16 deepak bhatt

very very useful article. Example provided above fully explains about system testing in detail

#17 Abhishek

thanks mam to share useful information its really very very useful information for me . thanks a lot mam.

#18 Abhishek

mam, now i m persuing mca 5th sem and i want to develop our career in testing field. so please guide me mam what i do?

#19 KK
#20 Anita

Really, its very good explain to explain.Thank u so much

#21 goutami

Thank u so much..

#22 Muntasir Abdullah

Nice description. This article give me clear concepts about system test.
Thank you so much.

#23 Debi Prasad Pal

This gives me a clear idea about sys test overall.

#24 Hussain

Very detailed and informative article.

#25 Faiz

thank you really its very useful information. i like it thank you very much admin..<3

#26 Elias Mujurizi in Uganda

Hi
thanks very much for this simple and clear details on system testing. its a very useful information.

#27 swetha

Very informative. Thankyou for posting.

#28 jaisimha reddy

i am from non-IT sector but i wanna change my carrer into Testing field so it is useful to me please post more info on Testing…Thanks !

#29 monika

Really this blog give me clear information about system testing with good example.

#30 yvette

very detailed information, thanks

Leave a Comment