Why You Need a Good Manual Testing Strategy for Your Startup

Manual Testing receives a lot of flak these days. The reasons are plenty, and most of them irrelevant, but why is this happening? Because ’tis the season for looking cool by insulting manual testing.

• Your project costs are running high? Blame Manual Testing.
• There are more bugs than you can fix? Blame Manual Testing.
• The milk in the office fridge has turned sour? Blame Manual Testing.

Ok, that last one was a bit of an exaggeration, but you see why.

“Of the many fads we have watched come and go in IT, the one about blaming manual testing for any and all of our woes has endured longer than I expected.”

One of the side effects of this, is how the widespread (and totally baseless) dislike for or embarrassment of manual testing has affected tech startups.

In this article, I take a look at manual testing and benefits it brings for startups. We’ll discuss why you need to incorporate manual testing in your IT roadmap if you’re a startup, and how it can help you reach your goals.

Limitations of a startup

Let’s begin by looking at some of the limitations a startup needs to work with.

Why? So you can begin to appreciate the insane boundaries within which startups need to work – at least until they get enough funding to stretch their feet just a little. And appreciating these limitations will help provide you an understanding of what practices work best in a startup. And we’ll look through the typical Resource, Time, Cost and Tooling lenses that any IT project team is judged by, so we can draw parallels easily.

Resources

You can’t hire 20 headcounts dedicated to testing on a given project. It is true that most startups are a small group of people that take on multiple roles, mainly because they can’t afford a small army.

Time

You can’t take 3 months to setup a team. And you can’t take eons to deliver.  You decide today, you start work today, you deliver yesterday – if that is all possible. In startup universe, every minute counts.

Money

Doesn’t flow from a bottomless cup – there’s a finite amount of it, and unless it’s yours (and most often even if it is yours), you are held accountable to every single penny. And no – you don’t get personal catering.

Tooling

You can’t have all the best tools in the world; you simply can’t afford them all. Startups thrive on tools from other startups in general, because these are generally free to use (side effect: they’re also buggy because they’re usually very early alpha versions). Where they absolutely need to, startups go for per user licensing or pay-per-use if that is at all possible.

Automation isn’t, and cannot accomplish, everything (although it is necessary)

Test Automation is quite beneficial to your Testing efforts. Then again, automation is also limited in the benefits it offers.

Remember, test automation is programmed by human beings. So your automation code can only execute test cases you’ve programmed it to run.

If you want to test something ad hoc or a new functionality for which automation code doesn’t exist, you have two choices: write automation code and then run it, or run the test straightaway.

And if you are a startup (even if you aren’t), that choice depends on how much time, money and resource you have, not to mention the tools you have at your disposal.

Manual Testing benefits for a Startup

Good startups do not ignore any traditional IT project task or activity just because it is traditional. Good startups consider what benefits any activity brings them, and if it can’t be replaced by something faster, better, cheaper; or eliminated altogether.

In that vein, let us see why you need to build Manual Testing into your plan if you are a startup, and how manual testing can help:

#1: Manual Testing is essential

Manual Testing is fundamental to an IT project’s success. It’s silly even to consider delivering your project without sufficient manual testing efforts built in. As we’ve seen above, and as you will know from your own experience, the world isn’t yet remotely ready to pass on Manual Testing.

Manual Testing puts code through real-world testing. This is something test automation code cannot by itself do. Automation code usually looks to execute a scenario and looks for a specific outcome. It cannot learn to adapt, change behaviour, expect different outcomes.

For instance, let’s say you built a test script that checks whether you can upload an image onto your profile. A rudimentary script will allow the user to upload an image and select OK.

Now, imagine you’re working on improving this journey. You now need to add additional steps that check the size of the image, run it against a database to compare the image to a government issued id to confirm it’s definitely you in that image.

Suddenly, the journey becomes more complicated. There are inline validations as well as links to external databases. Your automation code now needs tending. And you’ll find that it needs constant tending in real Agile work environments like this. Constant tending equals revisiting the automation code and continually revising it to test for the updated journey.

Now, think about how much effort you need to expend in revising your automation code for a product that has thousands of test cases that will be impacted by functional changes to your product.

Eventually, once you’ve built the journey, Test Automation will help you run regression tests against the finalised journey in a jiffy. Until then, until you have built a journey you consider worthwhile for your MVP, a typical startup wants to just move ahead as quickly as possible to the next iteration. Manual testing is the way to go here.

There is nothing to replace the human brain (just yet), and until the machines take over completely, you need human beings to run intuitive manual tests that help discover bugs hidden deep in your code that won’t be quashed with automation. Because automation code won’t know to look for unscripted anomalies. Machine learning may make this possible in the future, but the stress there is on may.

#2: Manual Testing dominates early stages of Product Development

If you’re a startup, then you’re working hard to build a product iteration good enough:

• to either demo to prospective investors, or
• to launch Alpha, Beta versions to market,

In both cases, speed is of essence. You need to constantly code, test and loop back to code.

Obviously, you need skilled Scrum team members who can work efficiently to build automation code alongside other dev efforts. But you also need your team to have the skills to perform Exploratory, White/Grey/Black Box testing and whatever else is needed to move your project forward at any given point.

Such on the fly testing efforts are integral to successful early stage product development. And when you think Exploratory or any other alternatives, you know you need some manual testing effort.

#3: Automation equals regression – almost always

Automation helps with testing tremendously.

Test automation removes a lot of the redundant testing effort from under the skilled tester’s work pile. This frees up a good tester to focus on the more challenging manual testing efforts, and to plan more test automation.

Using automation code to run repetitive regression tests that help you validate that a change hasn’t introduced any bugs to existing functionality improves your team’s test efficiency by manifold.

Ultimately, however, that is the major utility of automation tests. When your cash and resource strapped and looking to move to a rough but high value MVP in time for a further round of funding, you’re trying to uncover and fix critical bugs that impede progress; not polish the product for a market launch.

In your quest for the MVP, therefore, your best friend is manual testing. You find the critical bug that impedes progress, fix it, move forward. You find the next one, rinse, repeat.

Good Test Automation practices will help you build automation code alongside such MVP-focused activities. But it is important to note that some of the bottleneck bugs you fix will never be reproduced, ever again. So you need to balance your test automation strategy to focus on meaningful automation.

#4: Think Integration testing; think Manual Testing

Test Automation is very helpful in running unit tests – essentially the smallest testable components in an application. When you step outside this unit testing well, however, you’ll notice that the term ‘Testing’ encompasses way more than just unit testing.

Can you think of an app that doesn’t talk to any interfaces outside of its container? If yes, all you need is Test Automation. Even then, there are limitations to how much you can automate.

In a connected world, the reality is everything connects to everything. You can automate what you know – in your case, testing for the system or piece of code that you or your team are working to develop. Beyond this circle of control, your code interacts with code from other systems, third parties, that you have no control over.

It has been proven time and again that Automation is mainly suitable for self-contained unit tests. Beyond this, when you need to perform end-to-end system and integration tests, efforts at test automation don’t deliver incremental results that justify the effort that goes into build the automation code. Manual testing is still the best way to run integration and interface tests. Especially when you’re working with legacy systems.

Bringing it All Together

Everything said, what works will vary based on team, situation and a host of other factors. So there isn’t a one size fits all approach that we can adopt to make Manual Testing work for your Startup.

That isn’t the intention behind this post.

I merely tried to help you see the benefits Manual Testing can bring – even to your Startup, as surprising as it may sound to some of us.

Hopefully, you can go back today and begin to appreciate how manual testing works to help your progress. Or, you can pat yourself on the back for sticking it out with manual testing despite the perceived negativity towards this art.

Whatever your situation, I hope you’re now in a place where you can help others NOT shun manual testing on face value.

What is your Manual Testing strategy for your startup? And what are the challenges you face with building it into your plan? Share your views below so others can benefit.