5 Commonly Made Mistakes When Choosing a Test Management Tool

Test management tools are used for test cases, test plan, test runs and report the status of software testing and quality assurance activities. The software testing market has grown significantly in the past few years and many software testing tools have emerged on the market. These testing tools play a crucial role in the work done by quality assurance teams to streamline and make their testing and requirements processes more efficient.

So how do you deal with the variety of choices you have when searching for the ideal test management tool? To minimise confusion and increase your chances of finding the right tool, I compiled this list of five common mistakes that are committed when selecting a test management tool.

Mistake #1: Not considering the advantages and disadvantages of license models

Generally, software testing teams already know whether they’ll opt for a software as a service (SaaS) solution or to buy/lease an on-premises software package. SaaS solutions — especially in combination with cloud computing — are the most popular choice among companies, however it pays to look into the pros and cons of the two types.

Typically, testing is more intense during some periods of the year. If you have 4 annual releases, usage will go up due to more acceptance testing, involving more resources form the business side. During other times, usage will go down, since the tool is used for refining requirements and preparing for the next test period. SaaS solutions let you impact the size of the invoice by allowing you to scale up or down the number of users as needed. You also do not have to have your own servers and you’ll keep administration to a minimum since the SaaS provider installs all necessary updates.

On the downside there’s an element of risk involved when using cloud services, however this is offset by the ability to add/remove licences as necessary; adjusting for busier or slower periods and minimising costs related to hardware maintenance and software upgrades. Centralized administration is mostly good but it also means a risk for uncontrollable downtime during updates. You cannot judge the performance until you have tried the solution. If the SaaS vendor gets many customers it might affect the performance. Finally the SaaS vendor might decide to end development or change direction, but this applies to on-premise solutions as well.

On-premise solutions aren’t always safer. It is very common to not upgrade the software and operating system to the latest versions, due to lack of resources or competence. Most data security problems arise from within the company rather than from an external attack. On-premise solutions are often charged in bundles of 10, 50 or 100 users. If the need gets smaller, you still have to buy for the larger amount of users.

Open source solutions seldom cover all the needs for QA. There is never such a thing as a free lunch. The cost of customizing, implementing and supporting an open source solution is often considerable but seldom considered. The solution and all supporting software all have to be continuously updated. These kinds of solutions are often created by developers for developers, leaving the business side with unmet, unsatisfied requirements.

For these reasons it is important to consider all the factors involved before choosing one type of testing solution over the other.

Mistake #2: Assuming the test management tool you choose is easy to master

The product with all the bells and whistles may be an attractive choice, but can you handle it? In many cases, when a company picks a test management tool it has to spend a considerable amount of time figuring out the different functions and features the tool provides.

You cannot assume that the steps described above will be completed in a short time, especially if no one on your team has had prior experience with a particular tool.

Training time needs to be factored in as one of the costs of choosing a new test management tool since a prolonged period of adaptation to new utility can cause a major disruption in the timeframes of your projects.

BONUS – The short ramp-up time is one of the reasons why we think ReQtest is a better product than others out there.

Mistake #3: Being seduced by features and neglecting support

As mentioned above, getting used to a new test management tool can take some time and in the process of learning how to use it, it’s easy to find yourself in a position where extra help is needed.

When using a new product on a trial basis, it is important to validate the different levels of support that are provided by the vendor and to get an accurately picture of the reliability and speed of response.

Avoid being distracted by feature-packed solutions that offer no real support in how to deploy them effectively; the level of vendor involvement post-purchase is an important indication of the quality of the software you’re using.

Mistake #4: Basing purchase decision only on retail price

A very common (yet understandable) mistake is to base your decision to buy a particular test management tool solely on its sticker price. Company budgets often dictate the choice of testing packages that is made, however it is important to get a more holistic idea of the cost you will incur for a particular product.

Just like when buying a car; when choosing a test management tool, make sure you factor in the running costs.

Simply adding up licence and support costs doesn’t cut it. You have to obtain full details from the vendor about the costs of data hosting and storage, server backup and maintenance, and future software upgrades. A maintenance fee of 15-18 % of the initial cost is quite normal when it comes to on-premise solutions. Plus extra costs when upgrading to a new major release. It is also very common to have licenses for other products as well, for instance Microsoft Sharepoint, SQL server or other technology needed to use the tool.

Invisible costs like training time and installation should also be considered in order to determine all the factors that impact the final cost of the product you choose.

Mistake #5: Adapting your work strategy to the test case management tool

One of the most serious mistakes that can be committed when choosing a test management tool is to adapt your work processes to the testing tool rather than the other way round.

Whilst this might not be an issue for new companies who are still calibrating to their ideal workflow; established companies need to take into account existing work processes and try not to disrupt them.

Tailoring a test management package to the methods used by testers is the foundation stone of good user/product fit. In this case, a company has to apply to itself the principles it follows when creating offers for its customers; if you are the customer then make sure to find the testing tool that best complements your work strategy.

BONUS – How to use ReQtest in a Agile project

Finding an optimal solution

By steering clear of making the mistakes listed in this article, quality assurance teams can navigate better through the plethora of test management tools available on the market and pinpoint the solution that works best for them.

Test management has a critical role in the overall success of software and website development; therefore investing in an optimal solution is not only important for quality assurance, but it is an investment whose benefits will be reaped by the entire company.

Have you ever witnessed these mistakes in action? What were the consequences? Do you know of teams that ended up lumped with a tool they didn’t like and barely used? Let us know in the comments below!