Testing

May 2, 2017

Understanding the Difference Between Software Verification and Validation

It’s frustrating to argue with clients and other departments on the difference between software verification and validation. Wouldn’t you agree? Everyone wants functional software at the end of the day. Yet arguments over these terms slow down the whole process.

You can turn the situation around at your next software development meeting. That means more projects shipped out the door and less stress for you at the office. You can also demonstrate – in black and white – that your software is fully operational.

After reading this article, you will walk away with a clear understanding of these terms. Even more important, you will know the common mistakes that hurt verification and validation efforts. Finally, you’ll find steps on help you create a checklist to guide your team with software validation and verification.

Section 1: What Does SW Verification and Validation Mean Anyway?

In software development, defining terms correctly is the difference between code that works and error messages. While people will never be quite that precise, it’s worth clarifying terms up front on a software project.

In my experience, vaguely defined development terms frustrate everyone. The lead developer refers to validation in one way while your project manager has a different approach. Taking five minutes at the start of the project to define your terms will save you hours of effort later. Let’s start with (SW validation) software validation.

1.1 What Is Software Validation?

Simply stated: software validation is comparing what you built compared to the requirements. Imagine you won the lottery and commissioned a construction firm to build a mansion for you. Validation in that case would involve questions like, “The blueprints say there would be five full bathrooms. Did that actually get built into the building?” If you’re spending serious money for a dream home, you will expect a simple and clear answer.

In my experience, the validation process goes well when the requirements are clearly defined up front. If you are asked to build “a web app that accepts payments,” you are going to have a tough time validating the app. In comparison, “a web app that accepts payments with PayPal and Stripe” is much easier to validate.

That’s not all though.

Other organizations define software validation differently by focusing specifically on business requirements. In this case, technical details like performance in a test environment are not the end goal. Instead, you need to show that the app, system or product will perform its intended use when put into production.

Here’s how you can put this definition in practice on your next software project.

Firstly, write a one sentence definition of software validation that reflects your organization’s terminology and focus.

Secondly, write two or three questions you could ask to determine to help someone begin the validation process. (You can also reference the checklist section at the end of this article for additional ideas.)

1.2 What Is Software Verification?

Here’s the deal: software verification is all about the process you use to build your software. For example: your software testing method and your cybersecurity approach would be part of the story.

Here’s the deal: if you and your team can show that verify that you’re using a good process, you are much more likely to produce working software. Sure, it’s possible that you could whip up some code on a weekend that might “do the job.” That approach tends to lead to a superficial solution and cause long term problems.

For example: you build an app with a “just get it done” mindset. You skip most of the best practices for software verification. All those extra steps just slow you down, right? In reality, those processes make sure you cover all your bases and overcome your software blind spots.

You’re probably wondering if there’s an official definition for software verification. There is no final word defining the concept. We can round out the definition of the concept by looking at widely referenced definitions from Wikipedia:

“Software verification asks the question, “Are we building the product, right?”; that is, does the software conform to its specification.”

But wait, there’s more to the idea. Software experts have come up with two broad approaches to verification. As Wikipedia defines it:

Dynamic verification is performed during the execution of software, and dynamically checks its behaviour.

Static verification is the process of checking that software meets requirements by inspecting the code before it runs.

In my opinion, software verification is the mark of the professional developer. For example, a code inspection may reveal that you have failed to create code for error messages or logs. Without those resources, supporting the application in production becomes much more difficult.

How can you actually use this? Use the same approach outlined above for software validation.

Firstly, write a one sentence definition of software verification that reflects your organization’s terminology and focus. (If your company’s developers are relatively junior, you may need to start with a basic definition.)

1.3 What If Your Company Uses Different Definitions?

You’re probably wondering: “Hey, my boss uses these terms differently. I’m so confused.” Relax, we’ve all been there.
It’s true that each company defines software development jargon differently. In my experience, each company (and department!) tend to develop their own acronyms and abbreviations over time. Finding your way through this linguistic maze doesn’t have to be hard. Let’s look at how you can get it done.

Look, you have to directly ask questions and get your company’s definition clear.

The important point is that your approach includes the concepts of software verification and validation, even if you happen to use different labels for those terms.

Not sure how to bring up this question? Concerned that you’re going to look like a newbie for asking questions about terminology. No problem! Here are two settings where you can ask about these terms:

In your next one on one meeting with your boss, come prepared with a copy of the definitions of these terms. Ask: “Do we use these terms or do we have specialized company terms for these ideas?”
At your next project kick off meeting, it’s natural for people to ask many questions as the team comes together. When the project manager asks, “Are there any questions?” – that’s your moment. Ask the question with this approach: “I define software verification and validation in these terms. Does everyone else use that approach as well?

Section 2: Software Validation Mistakes

Now: getting your definitions straight is just the first step in the journey. You also need some practical ways to put these concepts into action on your next project. Our starting point is to look at ways the most common mistakes that hold people from effective software validation.

In my opinion, it is helpful to study mistakes when you’re learning a process. Why? Mistakes are often easier to remember because they tap into our psychology. Nobody wants to “lose face” when it comes to developing software. You might not become the world’s best software validation expert… But you can avoid embarrassing mistakes.

You’re going to learn both about the mistakes you’re making as an individual and that mistakes your team is probably making. If you wrap your mind around both of these, your software validation efforts are practically guaranteed to improve.

2.1 Software Validation Mistakes: Your Blind Spots

We all have blind spots in our work. By experience, training or habit, you’re probably used to paying attention to some problems more than others. When it comes to software validation, here are the some of the blind spots that you may not be aware of.

Mistake 1: The Face Value Fallacy

A customer tells you about one of their requirements… You take a few quick notes and get to work. Sound fair?

In my opinion, that’s not going to work. It’s important to ask a few questions to properly clarify what the customer really means. For example: they say they want “high performance on mobile devices.” But how do you know if your approach will satisfy the client? You can’t! You have to ask questions to get to the bottom of what the client really wants.

Resource: Use the Five Why’s technique to improve your software requirements: To Get To The Root Of A Hard Problem, Just Ask “Why” Five Times.

Mistake 2: The Procrastination Trap

If you overlook software validation until the end of the project, the whole process becomes much more difficult. Here’s the deal with this blind spot – it stems from a misunderstanding of validation.

With this blind spot, you miss the value of software validation because you think it’s an afterthought or overhead. That it’s not “real work.” In my opinion, that’s a mistake because the code may be great but none of that matters if you can’t show how your success meets the customer’s requirements.

Tip: Use this article to fight procrastination – Six Scientifically Supported Ways to Crush Procrastination.

Sadly, individual blind spots are the smallest kind of mistake you face. That’s why we’re going to look at mistakes at the team level next.

2.2 Software Validation Mistakes Your Team Makes

Boom! You just wasted a thirty-minute team meeting because the team did not have a common software validation approach. Here are some of the software validation mistakes that teams make.

Mistake 1. No Central Validation Resource

Trading emails back and forth to validate a piece of software is frustrating. There’s always that nagging fear that somebody will forget something important. The solution is to use a shared resource to organize your team’s validation in one place. A shared Google Doc would be a basic solution. Using ReQtest would be our recommendation.

Mistake 2. No Scale Validation

Ever been to a weekend hackathon? It’s a fun way to throw together an app quickly. It’s the “pure distilled essence” of the popular minimum viable product concept. Here’s the deal: that works if you are a startup or want a rough prototype.

If you’re building software for an insurance company, that approach isn’t enough. Your software validation approach needs to cover the scale question. For example, what happens to the software when it has a large volume of transactions? Is it vulnerable to a denial of service attack?

To solve this problem, you need to adopt a different mindset about your work. Take a step back from the work of grinding out code. Look at your software development team like an executive or consultant.

Is the team working together effectively? Are there good processes to guide the validation process? Those are the kinds of questions you need to ask to take your team’s quality to the next level.

By now, you have the fundamentals of software validation done. You know how to avoid the most common mistakes that hurt validation. Next, let’s turn to software verification mistakes.

Furthermore, the testing team is well prepared, well ahead of time, to run the tests.

Section 3: Software Verification Mistakes

It’s tough to build working, reliable software if your development habits are full of novice mistakes. For example: you read through the first few lines of code, they check out and then you move on. That approach might be enough for the most basic software – like a script you use in Excel. But it’s not going to cut it when it comes to verifying software mistakes.

3.1 Software Verification Mistakes: Code Reviews

In my opinion, code reviews are one of the best ways to achieve software verification. Check out the most common mistakes with this verification approach.

Code Review Mistake 1: No Prioritization

Let’s say you have 4,000 lines of code to review in the next two days. How would you approach that problem? Absent any guidance, each developer will take their own approach. Some might focus on database issues while others are more concerned with security. You can solve this problem by focusing your verification efforts on the top priorities.

Code Review Mistake 2: Filling in The Blanks Mentally

Silently reading through your code should be fine, right? Wrong. This approach to code review is flawed because you will tend to miss critical, easy to miss coding mistakes. For example: a missing closing bracket in a snippet of HTML. It’s much better to review the code out loud, especially for sensitive components.

Resource: Adapt this technique to improve your manual code reviews – The Proofreading Technique That Will Change Your Life.

3.2 Software Verification Mistakes: Ineffective Automation

Software professionals love to automate tasks. In fact, I’ve seen some developers happily spend an afternoon automating a thirty minute, routine task.

Section 4: Building Your Checklist

We’re almost done with this world wind tour of software verification and validation. There’s just one more point to cover – checklists.
As accomplished surgeon Atul Gawande notes in his book, “The Checklist Manifesto,” check lists are one of the best ways to improve performance. He has convinced highly trained health professionals to adopt checklists in order to reduce medical errors.

You know which profession has the best checklists? Airline pilots! Their checklists are an excellent model because they include two important qualities:

They’re sorted by priority: A well-written checklist focuses on the most important points. When your plane is in trouble, you need to focus on the most important causes of failure.
They’re short: A check list with ten points or less is a good rule of thumb. Any longer and you risk the resource being ignored. If your checklist is longer than one page, that’s a problem!

Now, let’s get down to putting together some practical ideas of your checklist.

4.1 Questions to Build Your Software Validation/Verification Checklist

Here’s the deal – book a thirty-minute meeting with your team to build a checklist. The purpose is simple: improve the quality of your work. Come prepared with these questions and whatever else you can think of:

What are common themes that have caused us to miss software delivery deadlines in the past year?
What are the underlying causes of complaints we receive from users?
What are the most important 2 or 3 methods used by the team’s best software tester?
What automated software testing tools do you we have available?
How can we improve the quality and usefulness of our requirements?

Use these resources to continue developing your checklists:

Conclusion

We’ve covered a lot of ground in this discussion. You’ve learned the answers to “what is software verification?” and “what is software validation?” We’ve also looked at typical mistakes and some of the ways you can solve those problems. Finally, we covered how to create checklists to improve the quality of your software. Now you can finally head home early on Friday afternoon!

Share with us one example of a software verification (or validation) mistake you’ve seen. What did you learn from that mistake? Let us know in the comments below.