Thoughts on Software Test Documentation and IEEE Standard 829

Finding the right balance between documenting too little or too much is maybe the hardest part of a test leader’s work. It’s vital to bear in mind that the most important task is to test and documentation is secondary, so if constrained by time, testing is more important. However, you often find documentation rules that are set by your organization or your customer. Maybe you work in a regulated area and have to comply with standards such as those for the military, medical or banking industries.

If the system under test is to be maintained for ages, there is one need for documentation. If the system on the other hand is a smartphone app that is only used for a specific campaign, there is another, lighter need for documentation.

How much test documentation you actually do need depends on a lot of factors, such as the criticality of the system, defined requirements for maintainability, development method and not least, your own personal preferences.

If you don’t have a test plan template, you will end up inventing the template yourself. This might lead to unnecessary problems due to making mistakes that the template would have solved. For instance, in one project where I worked as a test leader, the quality of the system under test was so poor that we should have returned it immediately to the external supplier without finishing the tests.

However I had missed something; defining criteria for when to stop testing if the testing does not run smoothly. This is something that a good test-plan template can solve. In another project I learned the importance of defining what not to test. If you only define what to test, readers often get the idea that you are going to test other areas as well, and you just did not remember to write these down. A good template can help you with this and much more.

And now to the good news, there is already a standard available that covers the test plan template as well as templates for test cases, bug reports and other frequently used documents within testing. The standard is called IEEE 829, and it’s a standard for software test documentation. It is developed by the organization IEEE which solely develops standards for different areas. There is also a standard called 830 that is aimed at Requirements management.

Why should I use a standard, you may think. Besides from not having to invent the wheel each time, a good template makes it easier to work on different projects within the company. Basically you recognize the headings when you compare documents from two projects. The customer and supplier can share the same template for greater understanding. And best of all, the test plan in IEEE 829 is not rocket science, in fact, if you ever have seen a test plan, you will recognize the headings.

On the other hand, the danger of documentation templates is that they quickly provide an excuse for us to stop thinking creatively and practically, especially in the case of ‘one size fits all’ templates, which as we know, are a misnomer if there ever was one because all projects are different and no one template could ever fit them all.

A practical tip is to not delete headings. Instead write ‘not applicable’ as the only text below the heading. A very common situation is to use an old test plan as template when creating new test plans.. If the author has deleted headings for one project, the new plan may lack information that is important to other projects.

A few tips on the subject of documentation:

• Do not duplicate information! If information is elsewhere, refer to it. Information may be in tools or databases or in other documents..

• Introductory items may be generally defined elsewhere.

• Tailor the outlines from the standard to your actual needs.

• Test documentation should be a tool, not a burden. The most important thing is its usefulness.

• If you have multiple releases per year, you need a lighter approach to documentation. Normally you cannot write 10 pages of test plans for each sprint or iteration if this is done frequently. Usually there are lots of things that do not change from iteration to iteration. Move these parts to a test strategy instead. This helps you keep the test plan short.

• IEEE contains templates for test-cases and bug reports as well. Compare these headings and the structure to the one in your test management tool. A common situation is to use Word or similar for creating the test plan and the test report. All other information can be stored in your test management tool. May I suggest you use ReQtest?