How to Set Goals for Session Based Testing (SBT)

The previous blog post introduced tour testing as a way testers can perform exploratory testing in a structured manner. In this article, I bring you a second approach that has been devised by testers to make exploratory testing easier and more effective.

If you read our last post, you’ll remember how I compared exploratory testing to being a tourist visiting a new city. By getting a guidebook or booking a tour, you’ll get the chance to see all the highlights of new place with less chance of getting lost or wasting time wandering in uninteresting areas. That’s exactly what tour testing is all about.

The new framework I’ll be discussing here is called session based testing.

What is Session Based Testing?

Session based testing (SBT) is an alternative framework developed by James and Jonathan Bach for doing exploratory testing. Like tour testing it brings a structured approach, however it is time-based rather than context-based.

If tour testing is like going on a tour of a new city, session based testing is like freediving. You dive into the water holding your breath and can explore only so much of the seafloor before you need to surface again to take in some air.

The obvious benefit of this self-imposed constraint in SBT is that you force yourself to concentrate on the task at hand and avoid all kinds of waffling in your quest to find booty (or, in our case, software bugs).

Session based testing also makes it possible to produce meaningful metrics, which can be presented to senior managers and help the whole team to address any flaws in the product and the QA process.

Let’s dive deeper into SBT by comparing to exploratory testing in general…

Comparing Session Based Testing to Exploratory Testing

As I mentioned on several occasions in this blog, exploratory testing is a technique for testing software which doesn’t use any scripts and depends entirely on the tester’s intuition as they interact with the product with very little or no preconceived notions about the way it’s supposed to work or behave.

In fact, this temporary suspension of familiarity and detached approach to software is what characterises exploratory testing (and not, as it is mistakenly assumed by many testers, random and ad hoc testing). Besides this particular mindset, testers have to depend on their skill and experience, which in turn informs the quality of their hunches.

As you can tell, these qualities are all intangibles which can only be acquired through hard work and continuous practice. Skilled explorers are hard to come by, which is why frameworks like tour testing and session based testing are so helpful for many testers, even seasoned ones who are hard-pressed for time and resources.

SBT brings structure to the ineffable experience or “pure” exploratory testing. It’s a variation on this basic technique you can use to help you come up with realistic and practical expectations for the work that will be done during exploration and how it will be reported to the rest of the team.

By using sessions as the basic unit of exploratory testing, you basically force yourself to focus for a time-limited period (usually 45 minutes up to a couple of hours) and carry out the exploration according to a charter that has been prepared beforehand. Compare this method with the Pomodoro technique used for increasing productivity; in fact SBT could be seen as a cross between the Pomodoro technique and exploratory testing.

Components of Session Based Testing (SBT)

Session based testing consists of the following five components:

Mission: This can be a simple one-liner with describes the purpose of the session and its duration. It serves to focus the tester’s attention when exploring the system.
Charter: This can be a checklist of the features that will be explored during the session, a list of goals that have to be achieved, or an agenda detailing what will be done during the session. Charters can be based on the requirements, test plan, or the results and observations made during previous sessions.
Session: This is a pre-determined period of uninterrupted testing which can range anywhere from 45 minutes to a couple of hours. Sessions can be recorded by jotting down notes or using screen-capture tools.
Session Report: Contains a record of the test session and usually includes: the charter, the functionality or scenario tested, notes about the testing process, a list of bugs and issues found, and various metrics (see next section).
Debriefing: This is a short meeting held between the tester and the Scrum master (or Product Owner) which summarises the results of the session(s). Jonathan Bach offers a structure for the debriefing meeting based on the acronym PROOF:
- Past – What happened during the session?
- Results – What was achieved during the session? (See the Session Report and metrics section.)
- Obstacles – What got in the way of good testing?
- Outlook – What still needs to be done?
- Feelings – How does the tester feel about all this?

Session Based Testing (SBT) metrics to watch out for

The session report will contain a mix of quantitative and qualitative results which will be discussed between stakeholders during the debriefing held at the end of a session or series of sessions.

Among the session metrics which serve to give a representation of the status of system following SBT are the following:

Number of sessions completed (J. Bach suggests having at least 3 sessions per day)
Number of issues found
Functionality covered
% of session time spent setting up for testing
% of session time spent testing
% of session time spent investigating problems

Summary

Session based testing brings enhanced accountability to exploratory testing, which is especially useful for testers who are still finding their feet with this type of testing. However, even veterans will find that SBT enables them to speed up bug discovery and offers tighter management of the time they spend testing.

SBT also introduces a finer level of control to your testing process, improve productivity, and error detection. These benefits come in handy when working with requirements that change frequently, are incomplete, or even be missing altogether.