Streamline and Supercharge your Exploratory Testing

This is the final article in our little series about Exploratory Testing. If you haven’t done so yet, read the first two pieces, Start exploring Exploratory Testing and How to get started with Exploratory Testing.

Make exploratory testing more efficient with pair testing

In pair testing, two testers sit together and use a single computer to test, analyze and explore the test object. It works just like pair programming, where two developers also share a single computer.

In both cases, working in pairs leads to more and better ideas than if you sit by yourself and work alone. Two people observe any errors they find, which makes it easier to reproduce the defect. Additionally, it facilitates knowledge transfer and cross-pollination of ideas between testers.

Another advantage of pair testing is that the testers have to explain their ideas and thoughts to each other in order to avoid misunderstandings. When you discuss ideas together, you come across new insights that you might not have thought of yourself. Cooperation during testing boosts the testers’ creativity.

We’ve sung pair testing’s praises many times before; you can read all about it here and here.

Plan for success in pair testing

To succeed in pair testing, you need teams that can work together, that understand the technique, and appreciate the value of cooperation. The test manager is responsible for laying the foundation for a well-functioning work environment.

Let one tester sit at the keyboard while the other tester manages notes and retrieves documents as needed. It is important that testers keep some form of testing documentation so they can review what areas they’ve tested and what defects they’ve found. Keep the testers engaged and on their toes by reversing their roles, so they do something they’re not always doing. When they find an error, the tester who’s not at the keyboard can try to reproduce it on another computer, so that testing doesn’t have to stop.

The typical length of a test session is 60 minutes, but this obviously depends on the situation. Remember that it can be difficult to stay fully focused for a long time, but if you can’t avoid lengthy sessions, at least try to take breaks now and then. For the tests to be effective, it’s essential that testers are fully focused on the task throughout the session.

When exploratory testing fits

You can use exploratory testing at any time when you have access to experienced, skilled testers. In some situations, however, it fits exceptionally well.

If the requirements and other documents that form the basis for the tests are incomplete or change frequently, it can be difficult or near to impossible to write traditional test cases. In such situations, an experienced tester who is familiar with the system can still do a good job using exploratory testing principles.

Similarly, if you’re on a very tight schedule, exploratory testing will help you quickly get started with test execution, thus allowing you to provide early bug reports and give feedback to the developers faster.

It is also a fact that systems tend to build up an immunity to test cases. By testing a system with the same set of test cases again and again, you will “wear out” the test cases, and consequently the team will find fewer and fewer errors with them. When this happens, exploratory testing is a good technique to use in order to find new approaches.

Exploratory testing is also suitable for installation tests (tests of a newly-installed system) in order to quickly check if the system is sufficiently stable so you know that it’s worthwhile to continue testing. When you do daily builds and daily tests, you can use exploratory testing before you start testing with test cases.

Benefits of exploratory testing

There are several benefits to exploratory testing: you spend less time on preparation, and get started with test execution more quickly. This means you can report defects back to the development team sooner, and hopefully you’ll find the serious defects faster. Exploratory testing gives you plenty of opportunities to adapt and improve work processes, once you’ve gotten to know the system and its weaknesses. You can then shift your focus to areas where there seem to be more problems, and the testers’ work will be more fun and stimulating. When you repeat exploratory testing, you won’t execute test cases in exactly the same way, thus increasing the chance that you’ll find new errors. This will also prevent the system from growing immune to the tired, old test cases.

Disadvantages of exploratory testing

Of course there are also some drawbacks to the Exploratory Testing approach. It is harder (although certainly not impossible) to keep track of what the team has tested. It’s important to stay well-organized when using exploratory testing, thus making the test manager’s role very important indeed.

Exploratory testing also places greater demands on the tester’s expertise, experience and skills than scripted testing does. The technique demands that the tester understands the system well, which, of course, is also a natural consequence of exploration. An experienced test leader can lead less experienced testers through exploratory testing, but it is often a very tough job. Inexperienced testers often seek much support when they are in doubt.

When you repeat exploratory testing, the test cases won’t be performed in exactly the same way, which can make it more difficult to reproduce the steps that led to an error, and more difficult to verify that the bugs have really been fixed. During regression testing it is usually considered that the test cases should be performed in the same way all the time without alterations. If possible, create automated tests for regression testing and use exploratory testing when it is better to be creative.

Another disadvantage is that you don’t write test cases and review them in advance, so you don’t have the opportunity to spot conflicting requirements during test design, or to prevent faulty solutions from being implemented or to find faulty test cases

And finally, always remember that there’s no sure-fire recipe for success in every organization, so you should analyze and evaluate what works best in your business.