5 Things You Need To Build A Rock-Solid UAT Test Plan

In our series on Testing concepts and fundamentals, this blog post is going to address an important testing practice called User Acceptance Testing and the UAT test plan.

User Acceptance Testing or UAT Testing is an inherent part of all Software Testing – irrespective of methodology. Whether you’re using Agile development methodologies or sticking it out with Waterfall, any software product that you build needs to undergo User Acceptance.

By definition, UAT is the point at which a user accepts that the product meets requirements and design standards. Therefore, UAT is conducted after all development and functional testing (read System Testing) is complete, and just before the product is pushed forward to the release or implementation phase. I’m sure you know all this.

So important is UAT to the success of a product upon release, that it has its own Software Development Lifecycle phase. There is a separate branch of Software Testing that is dedicated to User Acceptance Testing. And there are specific skill sets and career paths dedicated to UAT that allow Testers and Test managers specialise in this discipline.

“It is possible to get UAT right – when you’ve done your utmost to plan ahead and plan well.”

Coming as it does at such a critical juncture of the project delivery cycle, it is imperative that you get all the support you need to plan and execute UAT successfully. And with next to no slippages or issues.

It is possible to get UAT right – when you’ve done your utmost to plan ahead and plan well. A true reflection of your efforts to adequately prepare for the critical UAT phase of your projects is in how well your UAT test plan is received, accepted and approved. The success of the plan is also directly proportionate to the success of your team’s Testing efforts – they’re going to (more or less) execute to the plan after all.

Therefore, in this post, we’ll cover what it takes to build that rock solid UAT test plan.

You’ll get to know all the essentials you need to get your UAT test plan right. We’ll also discuss how you can optimise your efforts when building your UAT test plan.

What is User Acceptance Testing?

To the uninitiated, UAT represents essentially the last frontier for Testing to catch any unseemly bugs prior to a product release to customers. The idea is that you won’t have the chance to rectify any glaring defects with your product beyond this phase in the project.

UAT is also the point at which the ‘user’ accepts the product as meeting all requirements. By user, we mean those that commissioned the product in the first place. These may be people that directly use the product, or who expect their customers to use it.

For example,

• If you’re building a staff facing front end for a core banking system, ‘users’ are staff in the various offices of the bank that will use the staff front end on a daily basis.
• If you’re building a customer facing mobile app for a retail bank, ‘users’ are the business proposition team that have requested the functionality served by the app. They in turn have captured their requirements based on interactions with or feedback from customers or the market.

UAT Test Plan:

Now that we’ve established who the users are, UAT represents the cycle of testing traditionally performed by users themselves to ascertain that the product works as expected, and meets all the requirements that they provided at the beginning of the project.

The actual testing can be conducted completely by users themselves. This would require a certain amount of dedicated resource allocation from the sponsoring user team.

But, this isn’t always possible. Due to any number of reasons, you may not always have adequate user time available to complete a thorough UAT cycle. What is increasingly common, therefore, is that the users appoint a dedicated team of testers that they outsource UAT testing effort to.

Note the important difference here: users these days outsource the execution of UAT – not the decision making that comes with it. They appoint a responsible UAT manager who works with the project team to come up with a UAT test plan, and recruits a team of experienced testers to execute the test cases.

Therefore, it is possible that the users hand over Responsibility for planning and executing UAT to a dedicated testing team. The users are still Accountable for signing off a product that has completed UAT. That is, they need to be able to express utmost faith in the product having met all its intended original requirements before it is greenlighted for release.

Depending on the software development methodology – Waterfall, Agile, or something in between – how UAT is planned and executed may vary. However, the essence of what UAT represents and the goals it aims to achieve remain the same irrespective of how you bake UAT into your delivery.

Let us now look at the key enablers for building a rock solid UAT test plan…

How it works in agile…

Key enablers for building a rock solid UAT test plan:

Signed off (or approved) requirements

Yup – I know the Agile advocates among you are cringing at this headline. Doesn’t signed off represent Waterfall ways of working? Agile represents fluid project scope and requirements, right?

Not so much. Let me explain.

If you are working on Waterfall projects (I don’t recommend Waterfall methodology in today’s ever changing Digital landscape, but to each his own), it is true you will have ‘definite’ (note the quotes) and signed off requirements before even kicking off development. So using these signed off requirements to plan and execute UAT is, therefore, straightforward and logical.

On Agile projects, things can get a bit convoluted.

Does your Agile project have development sprints, but dedicated test cycles to help polish the product before release? With experience, I can tell you 60% to 80% of all Agile projects run some level of dedicated System Testing and User Acceptance Testing cycles to catch the unseemly bugs that need to be caught before deploying the product to customers.

Why is this necessary? Because, when you’re working Sprints, you don’t really have the time to ensure 100% quality in your product. Especially if it is new product development, Sprints primarily focus on building the core product features as quickly as possible to get these right before worrying about release quality.

So there’s usually going to be a fair bit of testing that is left to the pre-release prep so the Scrum team can focus on the 80% (of the famous 80-20 rule) first. And however you clothe it – call it Sprint, call it a dedicated Test Cycle, call it just ‘Testing’ or delineate System Testing from User Acceptance, use actual users or delegate testers – you need UAT at the end of such projects.

“There can be no confusion about ‘what is the requirement?’ when your testers and developers are arguing about the validity of a defect.”

So coming back to the point about Signed off or Approved requirements, even on Agile projects, you can build a repository of signed off requirements. When a user story has been absorbed into a sprint for delivery, essentially that particular story is signed off. The Product Owner has provided their requirements for the Sprint – albeit in the form of a ‘story’ and ‘acceptance criteria’. And these requirements are maintained centrally (insert your favourite Agile Requirements management tool) to track delivery.

These are, then, your signed off and approved requirements.

Why is sign-off or approval important? User Acceptance Testing basically seeks to ensure the product meets the requirements of those who pay for it to be built. When the people that pay specify their requirements clearly and are willing to approve or sign off, you then get a tangible set of base requirements to compare and test the product against.

Having signed off requirements therefore will help you build a plan to meet those exact requirements. There can be no confusion about ‘what is the requirement?’ when your testers and developers are arguing about the validity of a defect.

Test scenarios MUST be reviewed and signed off by the User or Product Owner

This is a follow on to the previous point.

We have established that end users should really be testing the product, and signing off User Acceptance Testing. We have also established that this doesn’t commonly happen – at least, users don’t directly perform UAT testing, and if they do, users don’t necessarily complete all UAT testing. They mostly recruit specialist UAT testers to do their job.

The thing with specialist testers, however, is they don’t always understand everything about the product to be tested. This can be because they are new to the product, line of business or even simply that they don’t have the same view that the business sponsors and corresponding end users have about the business needs the product is expected to fulfil, target audience, design standards and a host of other requirements.

So how do you help a UAT tester be effective? By defining the boundaries and parameters that they are expected to test, of course.

And who better to validate that the testing scope covers all that needs to be covered? To give you the answer, it is the same people that define the requirements in the first place – the same people that will use the product being built.

Get the product owner or your users to review test scenarios in scope for your testing in detail. Provide them a walkthrough if you have to. On Waterfall projects, this is a daunting task – especially if your project is large and complex. On Agile, when you lock in requirements and corresponding test scenarios/cases within each sprint, you make your job that much easier.

On Agile again, when you have a dedicated UAT cycle after the Sprints, it is still a good idea to build your Test scope iteratively through the Sprints. This way, you can be ready when the UAT cycle begins and you won’t have to try and build this humongous test case database all at once.

Tooling

Self-explanatory but let’s dwell on this one for a moment.

A good test case repository will help you save hundreds of hours of redundant effort that could be wasted in writing existing test cases from scratch, preparing test scripts etc. I’ve previously written about improving your Test Case Management practices and this is still relevant.

Tooling plays a key part in making UAT simpler, faster and better. Be it a dedicated Test Case repository, adequate Requirements Traceability or a robust Bug tracking tool – whether these are standalone tools that integrate well or an integrated tool like our own ReQtest, understand your tooling requirements and fulfill them. So UAT test plan tools are very important for making faster and better.

Your team will thank you for making their lives easier and their work more productive.

UAT Test Plan Template

UAT templates

Commitment from your development team to plan and deliver code on time

On Waterfall projects, and on Agile projects with dedicated Testing cycles, this is a real challenge. Despite the expectation that code that passes unit testing and system testing should really be pristine and free from functional defects, you and I both know the reality.

Often, I’ve seen projects continue to deliver code into System testing, and sometimes into UAT. We used to advocate a minimum number of ‘clean weeks’ for testing – i.e., the weeks before completion of the test cycle when no new code will be allowed into the system.

For example, let’s take a medium sized project with about 4 weeks’ clean SIT and 2 weeks’ clean UAT. This means that the last 4 weeks of SIT will not allow any new code. And the last 2 weeks of UAT likewise.

What this does, is it allows the Testing team to work with code that is reasonably stable – and familiar. You do, however, need your development team to work with you on this. In fact, you need wider programme management to accept such practices in the first place for them to be effective in implementation.

Push for a commitment from your dev team to adhere to the clean weeks’ policy for testing. And when it looks like they might slip, raise a risk to the project management. This will help you and your project mitigate any slippages in delivery dates and expose the issues to management so they can be resolved. And save a ton of heartburn later in testing when everyone’s breathing down your neck to finish testing on time.

Entry & Exit Criteria

Again, leads from the previous point.

With so much going on in a project, especially if you are working on a high profile one, it’s often not clear to everyone in your project (not just your testers but everyone) what are the minimum criteria that need to be met before Testing can begin, and before Testing can end.

Why is this important? Entry and Exit criteria for Testing cycles define the absolute minimum that the UAT manager will accept (on behalf of the sponsor) before they can begin testing the product, and before they will recommend sign off of Testing for release.

Entry criteria could be that the requirements are signed off, Test plan is signed off, test scenarios and test cases reviewed approved by key stakeholders etc. Exit criteria to move the product into release could be that 100% of all test cases have been executed, 100% of all critical and high defects have been fixed, 90% of all medium defects are fixed, 95% of all test cases have passed etc.

When you have Entry and Exit Criteria defined on your UAT test plan and agreed by all key stakeholders (including your engineers), you then have a clear path to define progress of Testing when you’re in full swing. And there will be no surprises.

---

Bringing it all together

There may be other equally important enablers out there, and some may be specific to your particular project or organisational situation. The point with this exercise is to give you a minimum set you can begin to work with, and build a successful UAT test plan from there.

So go ahead, go build your rock solid UAT test plan today. And remember, the better you are prepared, the better your chances of succeeding.

Would you like to share other things that will help build a rock solid UAT test plan? Use the comments section below and let’s have a healthy discussion.