Working Correctly with Test Data?

Requirements documentation most commonly focuses on functional requirements such as use cases and descriptions of functions. As a tester, you will often lack an analysis of the data the system will handle. In this article you will learn more about how you can work so that the test data is not forgotten during testing.

Imagine that you are going to test a car wash facility. You have clear specifications which define all the possible usage situations and alternative flows to these: How do customers choose the wash program, how the customer selects payment methods, how the customer cancels and starts over, how the system leads the vehicle to position itself in the right place in the wash-bay, etc.

With the above specifications you have an excellent foundation to write a set of test cases. When it’s time to test, you rent a car of a popular model and run through all the possible scenarios. Everything works and you feel satisfied with your testing effort. Time for deployment.

However, you now begin to feel a nagging concern. What if it worked just for this particular car? What happens to a vehicle with a longer wheelbase; is it going to be equally washed everywhere? Will a car with a steeper tailgate get scraped by the nozzles?

So, you realise you have to try more cars, but which ones? You realise that to be absolutely sure that the testing covers all cases, you must map out every vehicle on the roads. You look in the requirements documentation but receive very little guidance. What will you do now?

You might have never tested a car wash, yet on many occasions found yourself in similar situations when testing systems which handled financial instruments, pensions, railroad yards, PCs or anything else under the sun.

What’s happened here is that the requirements work in the project has focused on use cases and functions but then neglected to develop an overview of what data the system will perform. Unfortunately, experience shows that you can expect serious problems in operation when new features bump into existing data which was not expected, or when an existing system is expected to handle new types of data.

Each project is unique but here are some tips in the form of a step-by-step description on how to work to ensure that your testing has the maximum data coverage.

1. Identify the need for test data early

Raise the issue of test data as early as possible, as early as the test planning phase. Be prepared to drive the issue. Find out if the requirements engineers have specified what types of data the system will handle. This is not always the case and you will probably have to deeply analyze the need for test data.

Examine whether there is existing data from the production environment that can be reused directly or used after conversion. Document the results, preferably in the test plan or test strategy.

2. Thorough surveys during test design

Analyzing the potential test data should happen early in the test design phase. Work wide, both theoretically and practically. Look in the documentation available. Data models and architecture descriptions can both be good starting points. Set a meeting with architects and developers, in which you try to determine what kinds of test data is needed, based on the system. Check for any normal combinations of test data that may be present.

It is also wise to look for extreme situations. Try to identify the most complex cases, cases which have “extra everything”, as well as situations with almost no data. For example, if you are testing banking services for individuals, it can be interesting to test with typical bank customers but also using some ‘extreme’ customers who own all types of accounts and who use all kinds of banking services.

Test design techniques such as equivalent partitioning and boundary value analysis are very useful when looking for data cases. Try to figure out which situations are handled identically in the system. Identify test data which is right on the boundaries between these identical classes.

If the calculation of premiums in a system for car insurance were the same regardless of vehicle model, it would sufficient to do all tests on only one vehicle. However, since there are probably different premium conditions for different vehicles, there must be a set of data in the database for each class to be tested.

Only when you get this far in the survey will you know how much work you have ahead of you in order to obtain relevant data.

 

3. Create test data

After you have done your research you will need to create test data and enter it into the system. It is advisable to create test data either during test design or during the test execution phase.

If you are completely certain what data you need, you can do it in one single action during the test design phase. You can also make it an ongoing operation in which the creation of data is done as part of test execution. If you need large amounts of data it can be useful to write (or get a developer to write for you) a script to create the data.

All the interesting test data may already exist in a database that comes from the production environment. In this case, the work will consist of seeking out interesting data rather than creating it. You can either pick random data or do database searches, for example, using SQL.

 

4. Execute tests

Run test cases using all the relevant data you have. During test execution, it may be necessary to run through a number of test cases for each set of test data created.

If, for example, you are testing a pensions system, you can check a client’s pension commitment for all test customers you have in the system; young and old, high income and low income, men and women, overseas workers, government employees, unemployed, sick leave, part-time employees and so on. This can be well represented using a test matrix.

During this part of the work you will almost certainly come up with new cases – add these as well.

 

5. Save data

During the project’s final stage, it is important to remember to save the test data which is used so that it is easily accessible in the future. This will save you time in future. Logical, but often overlooked.

 

6. Conclude with confidence

Let’s go back to the car wash analogy. If you have better test data you will obviously be able to test to much greater effect.

Start with a document study. Look at car manufacturers online and in various car catalogues. Create lists of what you find, then partition vehicles into different classes which are expected to have the same or rather close characteristics in the car wash. You can base your vehicle classes on things such as vehicle type, length, width, weight, shape, number of wheels, etc.

Pick one car in each class to represent the entire class.

Go to a car rental firm with your list of typical models. Arrange a short-term contract – with damage waiver. Only when you have returned the cars (hopefully cleaner than when you hired them) can you be pretty sure that all the ‘normal’ types of cars work.

Invite a few colleagues in the project with knowledge of how the system works to try find different, more or less ‘extreme’ car models, for example a heavier utility van or a beach buggy. Locate an example of the car in the vehicle register and offer the owners a free wash. Once done, you will know that different extreme variations work in your car wash as well.

Now that you’ve tested all the use cases for typical models and different extreme vehicles you can recommend deployment with good confidence.

Sound complicated? Remember it’s not much different when it comes to the world of software and not cars.