Let’s Repeal Acceptance Tests

This is a guest post from our partner Seavus. You can also find the original article in Swedish on their website. The author, Aida Taye, is an experience consultant within testing. Here, she writes about her experience in the questioning of structured acceptance tests when working agile.

For eight years, I have been working with tests and acceptance testing. Both in organizations which consider themselves having an agile way of working and in more traditional organizations. I have been the test manager for individual, complex systems as well as for End2End projects where several systems in a flow have been replaced at the same time. Although, the main reason why I might see things differently when it comes to acceptance testing, is my experience in working as an acceptance test manager on the receiving end.

Over the years, an increasing amount of companies have embraced an agile or semi agile way of working, which in many cases has improved their testing process. By increasing the understanding of conducting early tests, the acceptance test managers have been included earlier in the process, resulting in minimized risks for the delivery. However, despite the increased understanding for an agile way of working, I continuously receive the same questions, such as:

• Do we need to have a plan prior to the acceptance tests?
• Do we really need an acceptance test period?
• Do the acceptance tests need to be documented?
• In an agile world, do we really need to appoint specific test persons with a joint responsibility for the tests?

These questions are often asked by experienced people working with tests in order to question the need of structured acceptance tests when working agile. It has made me wonder why we have so different views on how acceptance testing needs to be handled.

Clearly, many still equate waterfalls as a structured way of working and mistakes an ad hoc work style as agile. As soon as documentation and structure is discussed, it is addressed as not being agile.

Where in the agile manifesto is this contradictory? If this is claimed, you have missed the whole point. Working with agile testing (as with all agile work) is to identify the need and question why you do the things you do. Are we documenting it just because, or is there a need of documentation? If so, at what level? The same applies to the other questions.

There is a difference in the acceptance tests made in a delivery organization and the acceptance tests made in the receiving organization. They have different purposes and perspectives, and thus the test procedure may differ. The acceptance test manager has a responsibility to consider how the tests should be designed by reviewing the current situation. Has the organization purchased a standard system to be implemented directly as it is, or is there a need to adapt it in order to support a more complex need? Is it a system developed in-house and to what extent can the receiving organization be involved in the process? These are the prerequisites influencing the development of acceptance tests.

In system development, acceptance tests can be used in an early stage and to continuously test individual releases. However, this is not the same as assuring the quality of entire flows or the work process. Some organizations and situations need one or more minor acceptance test periods in addition to ongoing tests of what is being delivered. In a delivery organization, you have an interest of testing the features and flows in the application you are delivering. In a receiving organization, the need may differ. The acceptance tests may include tests of End2End integrations or tests to ensure that the application supports the work processes and special needs that the organization has. In particular, the receiver has a need to ensure that the delivery corresponds to what has been ordered.

The acceptance test manager will also review the need for documentation. The flows and alternate flows can be so complex that there is a need to document and tick off what has been delivered and tested. This does not necessarily mean that all test cases should be documented. Again, it all depends on the need.

A phrase which can be misunderstood is that “testing is everyone’s responsibility”. This can sometimes lead to the opposite, when no one is feeling responsible and takes ownership of the testing. As an acceptance test manager, you need to think about who is testing what and from which perspective. Having several acceptance testers does not automatically mean better quality assurance. Instead, the acceptance test manager need to consider what the purpose of the tests is and what you want to assure. Is it the appearance, the functionality, the flow or the support of the work process you want to focus on? Based on this, it will be easier to know who should be acceptance testers.

Instead of just concluding that planning, structure and documentation is non-agile phenomenon, we need to consider what the needs are and why. Otherwise, we risk falling back into working ad hoc. A structured work process and an agile one is not in contradiction. This applies to acceptance testing too. Instead, the receiver’s acceptance tests should be included in the agile development and, in an early stage, find a way to include the receiver without sacrificing the quality assurance.

Seavus® is a software development and consulting company with a proven track-record in providing successful enterprise-wide business solutions. The company has over 700 IT experts worldwide and offers a variety of products and service options, successfully covering the European and US market from several offices in the world. Their expanding portfolio covers: BSS/OSS, CRM, CEM, Business Intelligence solutions, ALM, embedded programming, business and consumer products, mobile and gaming solutions, managed services, as well as custom development, consultancy and resourcing. Seavus’ portfolio includes over 3.500 customers, among which leading worldwide telecoms and handset manufacturers, organizations from banking and finance industry, consumer electronics, technology, education, government, health, etc. Seavus in Stockholm has about 50 employees and was founded in 2000.