Now the world is moving to Agile. Early and continuous feedback is
imperial for any scrum team. Because the world is changing, the mindset
of testers also needs to be changed.
Instead of “finding bugs, breaking software, measuring requirement”, testers are now thinking of “delivering the quality, right at first time, test without the UI or test even before the UI is available”.
Testers are also now required to respond to change, and hence it is important to come out of black box testing technique and not waiting until the UI is developed; instead, begin testing the intermediately deliverables as well.
“NOW THIS IS VERY AGILE PERSPECTIVE”.
Whenever we build software, the lowest layers of tests are kept at the unit/component level. Unit tests are carried out by the development team. These unit tests are very much technology oriented and mostly written in the same language in with the system under test is written.
These unit tests are written with “X unit” test tool. We say in the testing world that if our unit test is rock solid, defects are identified much earlier and the testing above the unit testing layer becomes easy in a stable environment. And when we talk in Agile, we say that if a team is mastered the art of TDD (Test Driven Development), the Unit level tests provide the quickest feedback.
The layer above the unit/component layer is the Acceptance tests layer which is carried out the business. These are functional tests have more coverage than the unit tests and are most often executed by the non-developers. These tests test the layer behind the presentation layer or the APIs. These APIs or methods when are tested, gives a quick feedback and by the time GUI is developed, most of the functionalities are tested.
FitNesse is an example of this Automated Acceptance Tests Layer.
Its intention is to support agile style of black box acceptance and regression testing. It’s also a collaboration tool because testers collaborate with developers to prepare the test suite.
Benefits:
In FitNesse you can create Tests and Suite. The terms are very much the same as used in the testing world. Tests are single script and suit is collection / group of tests. When you create a suit and execute it, the advantage is that all the tests in that suit are executed. So you need a proper planning to arrange your tests in suit.
Download the latest version of fitnesse-standalone.jar and save it to your local drive.
Once the jar file is executed, FitNesse is started as shown below: (click on image for enlarged view)
To open FitNesse, open your browser and type: http://localhost:<portnumber>. In this case the port number is 2222.
The page received is shown below: (click on image for enlarged view)
So here, if you can see the Tests dropdown, we can create a “Suite page” as well as a “Test Page”. When you create a suite, all the test scripts within that suite will be executed.
For explanation purpose, I am taking an example of Test Page.
Here is the code in java, which has 4 methods:
This code in FitNesse world is called “Fixture”.
Fixtures are nothing but the sample code – or a link between FitNesse and the application under test. So whenever we want to test a method, we have to write a fixture and this fixture will invoke, and test the method.
So the “Fixture” code for our example is as follows:
We would need the class file, so make sure that you compile it.
In the front page, click on “Test Page”, enter the name of the test and click on “Save” button. In our case, it’s “Calculator”
Step #2) In your URL, append the name of your test with a dot “.” Operator.
Like: http://localhost:2222/FrontPage.Calculator
Step #3) Click on the Edit button and enter the lines shown below
Here are the Lines entered:
!define TEST_SYSTEM {slim}
!path D:\Lifefray\FitnesseTests\bin
!| import |
| calc |
| calc.Calculator |
| first | second | addition? | minus? | multiply? | divide? |
| 4 | 2 | 6 | 2 | 8 | 2.0 |
| 10 | 5 | 15 | 5 | 50 | 2.0 |
| 10 | 10 | 20 | 0 | 100 | 1.0 |
Let’s understand the lines one by one.
a) The first line says FitNesse to use SLIM test system.
(SLIM – Stands for Simple List Invocation Method. By saying SLIM test system, all the table processing is done by FitNesse. SLIM has SLIM Runner and SLIM Executer. SLIM Runner breaks the test pages into simple instructions and these instructions are passed to SLIM Executer which directs the fixture code to call the system under tests)
b) The second line defines the location of the class file. In this case, the java code is compiled and the class file is kept at location “path D:\Lifefray\FitnesseTests\bin”
c)The third & fourth lines states importing of package name "calc" . Fifth line is the name of the Class with Package name. In our case class name is “Calculator”
d) Now comes the sixth line:
The first two columns |first|second| are the parameters or the inputs to the java method.
The next 4 columns which are followed by “?” addition?|minus?|multiply?|divide?| are the methods in the java class. These methods will return a value which would be compared against the expected values.
e) The lines:
|4 |2 |6 |2 |8 |2.0 |
|10 |5 |15 |5 |50 |2.0 |
|10 |10 |20 |0 |100 |1.0 |
Are the test cases or I should say Test data for our method.
The first line:
|first|second|addition?|minus?|multiply?|divide?|
|4 |2 |6 |2 |8 |2.0 |
Will take 4 as first parameter and 2 as second parameter and will pass these two values in the addition method of the java class. The method will execute and will return a value. This returned value will be compared against the expected value written under the “addition?” which is |6 |
In the similar fashion, FitNesse will pass the first 2 parameters in the minus? Method of the java class and returns a value. This value will be compared against the expected value against |2 |
In the same way multiply? and divide? will work by taking the values of the first and second parameter and returns value which are compared against |8 |2.0 | respectively
In the similar fashion, the below 2 rows (or I should say, the test cases) are executed.
|10 |5 |15 |5 |50 |2.0 |
|10 |10 |20 |0 |100 |1.0 |
Step #4) Once you have edited your tests, click on the save button and your page will look like:
Step #5) To run the tests, Click on the Test button, and we get the result as follows: (click on image for enlarged view)
For the first row (which is our first test case), the green color highlights that the values, returned from the method addition (), minus (), multiply () and divide () are matching with what is expected i.e. 6, 2, 8 and 2.0 respectively. Similarly for the second row (which is second test case) all the values returned from the methods are matching.
Step #6) Now to demonstrate, let me change few of the expected values to some other values (the values are incorrect, but I have done it purposefully for explanation)
As of now I have:
The red color highlights that these tests failed.
Start sharpening your programming skills and emphasis more on building the logic and rather learning syntax. Once you are well versed with programming concepts and have practice on implementing it, exploring FitNesse will become easier.
We should try to push maximum of our testing in the unit and the acceptance layer. Till now we have been trying to keep most of our testing for the UI layer using tools like QTP and Selenium, but the disadvantage here is that the these functionalities could not be tested unless the UI is developed. By the time you find a defect, the developers have moved into some other feature development.
On the other hand, if we can test the APIs soon after it has been written, developers can fix it instantly. This would also result in less effort when we test the GUI. Because all the functionalities are tested, testing the GUI becomes easy.
With Agile, the mindset of testers also needs a change and they have to come out of their routine set of testing and now you should look inside code and try to identify defects even the UI is not available.
Note - Above article published first appeard in http://www.softwaretestinghelp.com. Some updates done from my side i.e Configurations changes, code changes with new screenshots which will help for new users without any errors or minimum errors.
Instead of “finding bugs, breaking software, measuring requirement”, testers are now thinking of “delivering the quality, right at first time, test without the UI or test even before the UI is available”.
Testers are also now required to respond to change, and hence it is important to come out of black box testing technique and not waiting until the UI is developed; instead, begin testing the intermediately deliverables as well.
But WHY?
“NOW THIS IS VERY AGILE PERSPECTIVE”.
Whenever we build software, the lowest layers of tests are kept at the unit/component level. Unit tests are carried out by the development team. These unit tests are very much technology oriented and mostly written in the same language in with the system under test is written.
These unit tests are written with “X unit” test tool. We say in the testing world that if our unit test is rock solid, defects are identified much earlier and the testing above the unit testing layer becomes easy in a stable environment. And when we talk in Agile, we say that if a team is mastered the art of TDD (Test Driven Development), the Unit level tests provide the quickest feedback.
The layer above the unit/component layer is the Acceptance tests layer which is carried out the business. These are functional tests have more coverage than the unit tests and are most often executed by the non-developers. These tests test the layer behind the presentation layer or the APIs. These APIs or methods when are tested, gives a quick feedback and by the time GUI is developed, most of the functionalities are tested.
FitNesse is an example of this Automated Acceptance Tests Layer.
What is FitNesse?
FitNesse is “A fully integrated standalone wiki, and acceptance testing framework”. It is Open source, wiki web server. Wiki- because it allows creating your own web pages on which test tables are created. These test tables are nothing but the test data.Its intention is to support agile style of black box acceptance and regression testing. It’s also a collaboration tool because testers collaborate with developers to prepare the test suite.
Why should I use FitNesse?
Agile test team can use FitNesse to prepare test suits that will test the methods in the code. FitNesse is analogous to Junit in a way that it also tests the methods, but it’s different than Junit because the tests are in the form of simple tables which can be used by both developers and non-developers.Benefits:
- Early feedback, by executing the automated acceptance tests as often as required.
- Tests results are deterministic because they are highlighted in either Red or Green.
- Test data can be designed to suite the quality needs.
- The tests are written in simple language and easy to understand as they are written in tabular form.
- These tables are defined in terms of input and expected outputs.
- See all FitNesse features here.
So what all can I create?
In FitNesse you can create Tests and Suite. The terms are very much the same as used in the testing world. Tests are single script and suit is collection / group of tests. When you create a suit and execute it, the advantage is that all the tests in that suit are executed. So you need a proper planning to arrange your tests in suit.
Downloading and Configuring FitNesse:
=> To download FitNesse, Click HereDownload the latest version of fitnesse-standalone.jar and save it to your local drive.
Open command prompt and execute the jar file. For ease, I have created a batch file:
Once the jar file is executed, FitNesse is started as shown below: (click on image for enlarged view)
The page received is shown below: (click on image for enlarged view)
So here, if you can see the Tests dropdown, we can create a “Suite page” as well as a “Test Page”. When you create a suite, all the test scripts within that suite will be executed.
For explanation purpose, I am taking an example of Test Page.
FitNesse Example – The stuff to Test:
As of now, we are testing a simple calculator program shown below.Here is the code in java, which has 4 methods:
- addition ()
- minus ()
- multiply ()
- divide ()
This code in FitNesse world is called “Fixture”.
Fixtures are nothing but the sample code – or a link between FitNesse and the application under test. So whenever we want to test a method, we have to write a fixture and this fixture will invoke, and test the method.
So the “Fixture” code for our example is as follows:
package calc;
public class Calculator {
private int
first,second;
public void
setFirst(int first)
{
this.first=first;
}
public void
setSecond(int second)
{
this.second=second;
}
public int
addition()
{
return
(first+second);
}
public int minus()
{
return
(first-second);
}
public int
multiply()
{
return
(first*second);
}
public float
divide()
{
return
(first/second);
}
}
We would need the class file, so make sure that you compile it.
Writing your test in FitNesse:
Step #1) Let’s go back to the browser where we have the FitNesse front page.In the front page, click on “Test Page”, enter the name of the test and click on “Save” button. In our case, it’s “Calculator”
Step #2) In your URL, append the name of your test with a dot “.” Operator.
Like: http://localhost:2222/FrontPage.Calculator
Step #3) Click on the Edit button and enter the lines shown below
Here are the Lines entered:
!define TEST_SYSTEM {slim}
!path D:\Lifefray\FitnesseTests\bin
!| import |
| calc |
| calc.Calculator |
| first | second | addition? | minus? | multiply? | divide? |
| 4 | 2 | 6 | 2 | 8 | 2.0 |
| 10 | 5 | 15 | 5 | 50 | 2.0 |
| 10 | 10 | 20 | 0 | 100 | 1.0 |
Let’s understand the lines one by one.
a) The first line says FitNesse to use SLIM test system.
(SLIM – Stands for Simple List Invocation Method. By saying SLIM test system, all the table processing is done by FitNesse. SLIM has SLIM Runner and SLIM Executer. SLIM Runner breaks the test pages into simple instructions and these instructions are passed to SLIM Executer which directs the fixture code to call the system under tests)
b) The second line defines the location of the class file. In this case, the java code is compiled and the class file is kept at location “path D:\Lifefray\FitnesseTests\bin”
c)The third & fourth lines states importing of package name "calc" . Fifth line is the name of the Class with Package name. In our case class name is “Calculator”
d) Now comes the sixth line:
The first two columns |first|second| are the parameters or the inputs to the java method.
The next 4 columns which are followed by “?” addition?|minus?|multiply?|divide?| are the methods in the java class. These methods will return a value which would be compared against the expected values.
e) The lines:
|4 |2 |6 |2 |8 |2.0 |
|10 |5 |15 |5 |50 |2.0 |
|10 |10 |20 |0 |100 |1.0 |
Are the test cases or I should say Test data for our method.
The first line:
|first|second|addition?|minus?|multiply?|divide?|
|4 |2 |6 |2 |8 |2.0 |
Will take 4 as first parameter and 2 as second parameter and will pass these two values in the addition method of the java class. The method will execute and will return a value. This returned value will be compared against the expected value written under the “addition?” which is |6 |
In the similar fashion, FitNesse will pass the first 2 parameters in the minus? Method of the java class and returns a value. This value will be compared against the expected value against |2 |
In the same way multiply? and divide? will work by taking the values of the first and second parameter and returns value which are compared against |8 |2.0 | respectively
In the similar fashion, the below 2 rows (or I should say, the test cases) are executed.
|10 |5 |15 |5 |50 |2.0 |
|10 |10 |20 |0 |100 |1.0 |
Step #4) Once you have edited your tests, click on the save button and your page will look like:
Step #5) To run the tests, Click on the Test button, and we get the result as follows: (click on image for enlarged view)
For the first row (which is our first test case), the green color highlights that the values, returned from the method addition (), minus (), multiply () and divide () are matching with what is expected i.e. 6, 2, 8 and 2.0 respectively. Similarly for the second row (which is second test case) all the values returned from the methods are matching.
Step #6) Now to demonstrate, let me change few of the expected values to some other values (the values are incorrect, but I have done it purposefully for explanation)
As of now I have:
- Changed the expected values for minus() for the second test case
The red color highlights that these tests failed.
Some insights on Fixture/Table styles:
We have seen that In FitNesse the tests are executed by executing rows in a table. Hence to execute different kinds of tests (or I should say to test different kinds of methods), we would need to use different kinds of tables. We use the below Fixture/table styles most often:- Column Fixture – is most widely used (and is used in the above example). Here the rows of data represent different sets of input and its expected output.
- Row Fixtures – It is used for testing queries which returns some set of values.
- Action Fixtures – It is used to run tests for sequence of events. These events can be like clicking on a button, checking values
Recommendation:
I have tried to demonstrate the concepts, so that we can start exploring more on FitNesse. The tester’s mindset also needs to be changed and have to be broadened. We have to stop restricting ourselves to look inside the code. I feel; ultimately we are testing the code, so why don’t we try to see the code and test then and there?Start sharpening your programming skills and emphasis more on building the logic and rather learning syntax. Once you are well versed with programming concepts and have practice on implementing it, exploring FitNesse will become easier.
Conclusion:
Testing in agile comes in 4 flavors:- Automated Unit testing – By using Junit
- Automated acceptance verification test – By using FitNesse
- Automated UI / regression tests – by using Selenium or QTP
- Manual testing
We should try to push maximum of our testing in the unit and the acceptance layer. Till now we have been trying to keep most of our testing for the UI layer using tools like QTP and Selenium, but the disadvantage here is that the these functionalities could not be tested unless the UI is developed. By the time you find a defect, the developers have moved into some other feature development.
On the other hand, if we can test the APIs soon after it has been written, developers can fix it instantly. This would also result in less effort when we test the GUI. Because all the functionalities are tested, testing the GUI becomes easy.
With Agile, the mindset of testers also needs a change and they have to come out of their routine set of testing and now you should look inside code and try to identify defects even the UI is not available.
Note - Above article published first appeard in http://www.softwaretestinghelp.com. Some updates done from my side i.e Configurations changes, code changes with new screenshots which will help for new users without any errors or minimum errors.
