FitNesse Acceptance Testing Framework Tutorial

In this article, we will know in detail about FitNesse, a collaboration tool for testers and developers. Let’s get started.  

The world is moving towards Agile in the present times. Early and continuous feedback is crucial for any scrum team. Because the world is rapidly changing, the mindset of testers has also changed.

Instead of just “finding bugs, breaking software, measuring requirement”, testers are now thinking of “delivering the quality, right at first time, test without the UI or test even before the UI is available”.

FitNesse – A Collaboration Tool for Testers and Developers

Testers are also now required to respond to changes and hence it is important to come out of the black box testing technique and not wait until the UI is developed; instead, begin testing the intermediate deliverables as well.

Why

“NOW THIS IS AGILE PERSPECTIVE”.

Whenever we build software, the lowest layers of tests are kept at the unit/component level. Unit tests will be carried out by the development team. These unit tests are technology oriented and are mostly written in the same language as the system under which the test has been written.

These unit tests are written with “X unit” test tool. We say in the testing world that if our unit test is rock solid, defects are identified much earlier and the testing above the unit testing layer becomes easy in a stable environment. When we talk in Agile, we say that if a team is mastering the art of TDD (Test Driven Development), the Unit level tests provide the quickest feedback.

The layer above the unit/component layer is the Acceptance test layer which is carried out by the business. These are functional tests that have more coverage than the unit tests and are most often executed by non-developers. These tests test the layer behind the presentation layer or the APIs. These APIs or methods when being tested give feedback and by the time GUI is developed, most of the functionalities are tested.

FitNesse is an example of this Automated Acceptance Test Layer.

What is FitNesse

FitNesse is “A fully integrated standalone wiki and acceptance testing framework”. It is an Open source, wiki web server. Wiki- because it allows you to create your own web pages on which test tables are created. These test tables are nothing but test data.

The intention is to support agile style black box acceptance and regression testing. It’s also a collaboration tool because testers collaborate with developers to prepare the test suite.

Why should I use FitNesse

The Agile test team can use FitNesse to prepare test suits that will test the methods in the code. FitNesse is analogous to Junit in a way that it also tests the methods, but it’s different than Junit because the tests are in the form of simple tables which can be used by both developers and non-developers.

Benefits

• early feedback by executing automated acceptance tests as often as required.
• Test results are deterministic because they are highlighted in either red or green.
• Test data can be designed to suit quality needs.
• The tests are written in simple language and easy to understand as they are written in tabular form.
• These tables are defined in terms of input and expected outputs.
• See all FitNesse features here.

So what all can I create

With FitNesse, you can create Tests and Suites. The terms are very much the same as those used in the testing world. Tests are single script and suit is collection/group of tests. When you create a suit and execute it, the advantage is that all the tests in that suit are executed. So you need a proper plan to arrange your tests in a suit.

Downloading and Configuring FitNesse

=> To download FitNesse, Click Here

(Note: Click on any image for an enlarged view)

Download the latest version of fitnesse-standalone.jar and save it to your local drive.

Open the command prompt and execute the jar file. For ease, I have created a batch file:

Once the jar file is executed, FitNesse will start as shown in the image below: (click on the image for an enlarged view)

To open FitNesse, open your browser and type: http://localhost:<portnumber>. In this case, the port number is 2222.

The page received is shown below: (click on image for enlarged view)

So here, if you can see the Tests dropdown, we can create a “Suite page” as well as a “Test Page”. When you create a suite, all the test scripts within that suite will be executed.

For explanation purposes, I am taking an example of a Test Page.

FitNesse Example 

As of now, we are testing a simple calculator program below.

Here is the code in java, which has 4 methods:

• addition ()
• minus ()
• multiply ()
• divide ()

(Please see that FitNesse works with any language of your choice. For explanation purposes, I have used java)

This code in the FitNesse world is called “Fixture”.

Fixtures are nothing but the sample code – or a link between FitNesse and the application under test. So whenever we want to test a method, we have to write a fixture and this fixture will invoke, and test the method.

So the “Fixture” code for our example is as follows:

publicclass Calculator {
privateint first,second;

publicvoid setFirst(int first)
{
this.first=first;
}

publicvoid setSecond(int second)
{
this.second=second;
}

publicint addition()
{
return (first+second);
}

publicint minus()
{
return (first-second);
}

publicint multiply()
{
return (first*second);
}

publicfloatdivide()
{
return (first/second);
}
} 

The code for the eclipse is shown as: (click on image for an enlarged view)

We will need the class file, so make sure that you compile it.

Write your test in FitNesse

Step #1) Let’s go back to the browser where we have the FitNesse front page.

On the front page, click on “Test Page”, enter the name of the test and click on “Save” button. In our case, it’s “Calculator”

Step #2) In your URL, append the name of your test with a dot “.” Operator.

Like: http://localhost:2222/FrontPage.Calculator

Step #3) Click on the Edit button and enter the lines shown below

Here are the Lines entered

!define TEST_SYSTEM {slim}

!path F:\Eclipse\TestFitness\bin

!|Calculator|

|first|second|addition?|minus?|multiply?|divide?|

|4   |2     |6       |2     |8       |2.0   |

|10   |5     |15       |5     |50       |2.0   |

|10   |10   |20       |0     |100     |1.0   |

Let’s understand the lines one by one.

a) The first line says FitNesse to use the SLIM test system.

(SLIM – Stands for Simple List Invocation Method. As per the SLIM test system, all the table processing is done by FitNesse. SLIM has SLIM Runner and SLIM Executer. SLIM Runner breaks the test pages into simple instructions and these instructions are passed to SLIM Executer which directs the fixture code to call the system under tests)

b) The second line defines the location of the class file. In this case, the java code is compiled and the class file is kept at location “path F:\Eclipse\TestFitness\bin”

c) The third line states the name of the Class. In our case its “Calculator”

d) Now comes the fourth line:

The first two columns |first|second| are the parameters or the inputs to the java method.

The next 4 columns, which are followed by “?” addition?|minus?|multiply?|divide?| are the methods in the java class. These methods will return a value which would be compared against the expected values.

e) The lines:

|4   |2     |6       |2     |8       |2.0   |

|10  |5     |15       |5     |50     |2.0   |

|10   |10   |20       |0     |100     |1.0   |

Are the test cases or should I say Test data for our method?

The first line:

|first|second|addition?|minus?|multiply?|divide?|

|4   |2            |6            |2            |8                |2.0   |

Will take 4 as the first parameter and 2 as the second parameter and will pass these two values in the addition method of the java class. The method will execute and will return value. This returned value will be compared against the expected value written under the “addition?” which is |6             |

In a similar fashion, FitNesse will pass the first 2 parameters in the minus method of the java class and returns a value. This value will be compared against the expected value against |2         |

In the same way, multiply? and divide? will work by taking the values of the first and second parameters and returns value which is compared against |8           |2.0   | respectively

In a similar fashion, the below 2 rows (or I should say the test cases) are executed.

|10   |5     |15       |5     |50       |2.0   |

|10   |10   |20       |0     |100     |1.0   |

Step #4) Once you have edited your tests, click on the save button and your page will look like:

Step #5) To run the tests, click on the Test button and we get the result as follows: (click on the image for an enlarged view)

For the first row (which is our first test case), the green color highlights that the values, returned from the method addition (), minus (), multiply () and divide () are matching with what is expected i.e. 6, 2, 8 and 2.0 respectively. Similarly, for the second row (which is the second test case) all the values returned from the methods are matching.

Step #6) Now to demonstrate, let me change a few of the expected values to some other values (the values are incorrect, but I have done it purposefully for explanation)

As of now, I have:

• Changed the expected value for addition() for the first test case to be 7
• Changed the expected value for minus() for the second test case
• Changed the expected value for divide () for the third test case.

Step #7) Run the test by clicking the “Test” button. The above tests should fail. (click on image for enlarged view)

The red color highlights that these tests failed.

Some insights on Fixture/Table styles

We have seen that in FitNesse the tests are executed by executing rows in a table. Hence to execute different kinds of tests (or I should say to test different kinds of methods), we would need to use different kinds of tables. We use the below Fixtures/table styles most of the times:

• Column Fixture – is most widely used (and is used in the above example). Here the rows of data represent different sets of inputs and their expected output.
• Row Fixtures – It is used for testing queries which returns some set of values.
• Action Fixtures – It is used to run tests for a sequence of events. These events can be like clicking on a button, checking values

Recommendation

I have tried to demonstrate the concepts so that we can start exploring FitNesse. The tester’s mindset also needs to be changed and broadened. We have to stop restricting ourselves to look inside the code. According to me, as we are testing the code, why don’t we try to see the code and test then and there?

Start sharpening your programming skills and emphasis more on building logic and rather learning syntax. Once you are well versed with programming concepts and have practice on implementing them, exploring FitNesse will be easier.

Conclusion

Testing in agile comes in 4 flavors:

1. Automated Unit testing – By using Junit
2. Automated acceptance verification test – By using FitNesse
3. Automated UI / regression tests – by using Selenium or QTP
4. Manual testing

We should try to push the maximum of our testing in the unit and the acceptance layer. Till now we have been trying to keep most of our testing for the UI layer using tools like QTP and Selenium, but the disadvantage here is that these functionalities could not be tested unless the UI is developed. By the time you find a defect, the developers have moved into some other feature development.

On the other hand, if we can test the APIs soon after it has been written, developers can fix it instantly. This would also result in less effort when we test the GUI. Because all the functionalities are tested, testing the GUI becomes easy.

With Agile, the mindset of testers also needs a change and they have to come out of their routine set of testing and now you should look at the code and try to identify defects even if the UI is not available.

About the author: This is a guest article by STH team member Shilpa C. Roy. She has been working in the software testing field for the past 9+ years in domains like Internet advertising, Investment Banking, and Telecom.

Let us know your experience, doubts or queries in the comments section below. We would love to hear from you.