The Building Blocks of a Test Automation Strategy

Strategy and planning are fundamental building blocks of software development life-cycle; whereas quality is always measured with the grade. Fulfilling business expectation and customer satisfaction with no defects is the main aim of testing. To achieve high-quality software within a short time is one of the objectives of every software company.

In the test planning stage, along with all the other elements of testing, automation testing and test automation strategy are always decision-making topics. Expecting 100% automation from day one is an unrealistic automation test strategy.

On the other hand, lack of envisioning for enhancement is one of the reasons of automation testing failure. To build a cost-effective automation testing strategy with a result-oriented approach is always a key to success in automation testing.

Automation Test Life Cycle:

You can build a good test automation strategy, once you understand automation test life cycle. Automation test life cycle has following phases:

• Automation Feasibility Analysis: In this phase, you check the feasibility of automation. This includes short listing the test cases for automation and selecting the right test tool.

• Test Strategy: In test strategy, you select the test automation framework. In this phase, you create a test plan and test automation suite in your test management tool.

• Environment Set up: In this phase you set up the testing environment and acquire the required hardware and software to execute the automated test cases.

• Test Script Development: In this phase, you start creating the automation test scripts. Make sure that your scripts are reusable, well-structured and well documented.

• Test Case Execution: In this phase, you execute your test scripts.

• Test Result Generation and Analysis: This is the last phase of test automation life cycle. In this phase, you analyse the output of test cases and share the reports with stakeholders.

Why Build Test Automation Strategy?

In this agile age, software project managers and test managers want to deliver a thoroughly tested application more quickly and efficiently.

In their attempt to do more with less, they seek different ways to reduce the testing effort and time – without compromising on quality. This typically leads to consider test automation. Moreover, there are various scenarios where manual execution of test cases is not possible – say, you want to see the behaviour of application when 1,000 users are using it simultaneously.

You might also feel the need to automate your regression test cases; as manual execution of the same test cases again and again might cause the tester to either lose interest or create the tunnel vision.

Having said this, we have understood that in various situations the need of automation testing arises. However, you can not completely switch to automation testing and threw away the manual testing. Rather, you need to come up with an intelligent test automation strategy to fully enjoy the benefits of automation testing and achieve the desired level of success.

Let’s have a look at the building blocks of test automation strategy.

Building Blocks of Testing Automation Strategy:

Following is the list of building blocks of automation test strategy:

1. Selecting the Right Test Automation Tool

Selecting the right tool is the key factor that guarantees the success of your test automation strategy. You need to select one out of thee several test automation tools available in the market. First, identify your testing requirements and shortlist the tools that provide your required features.

In order to select the best fit for your project, you consider other parameters such as licensing cost of tool, maintenance cost, training and support, tools extensibility and tools performance and stability. There are plenty of cost-effective tools available in the market for automating different kinds of application like windows application, web application, website, mobile web applications and native mobile apps.

Following are a few automation tools which serve the purpose of automating any kind of application:

• Selenium
• HP QTP
• Microsoft Coded UI Framework

Days are gone when you have to write scripts for automation of test cases. These tools might scare away your testers. Why not use ‘Capture and Replay’ tools?

Yes, there are many tools available in the market that make your life hassle free by allowing you to record test scenarios, define the expected result and replay the test case when needed. If your testing team is reluctant to do script writing but test automation is your need – especially load testing, you might give the following tools a try:

• Test complete
• WAPT
• Ranorex
• Microsoft Test Manager

2. Shortlisting Test Cases for Automation

A smart test manager understands that all test cases cannot be automated. Instead, he manages the balance between manual and automation testing. For this purpose, test manager needs to thoroughly analyse the test plan and shortlist the test cases which need automation. You can use the following criteria to shortlist the test cases for automation:

• Which scenarios are tedious but can be automated?
• Which regression test cases be automated?
• Can we automate test data entry procedure?
• Which test cases are required to perform on cross platforms? Can we automate those?
• Can we automate all kinds of validations in the application?
• Can we automate all CRUD (create, retrieve, update, delete)  operations in the application?

3. Automation Test Environment

Setting up your test environment is another building block of test automation strategy. Establish your automation test environment by identifying the requirements of test environment and  by acquiring the required tools with their licenses.

The rule of thumb for setting up the environment is as follows – development and test environment should be replica of stage environment and stage environment should be replica of production environment. Here, the replica means same configuration of hardware and software on each server.

4. Reusability of automation scripts

At organisational level, code or script and even the framework should be reusable within cross-functional teams. Your test automation strategy should include the reusability of test scripts, if you are using scripts for test automation. This will help to reduce scripting development efforts.

There are many general functionalities which are common in applications – especially if your company is taking projects related to a single domain. For example, if your company develops web applications, you can create a process to re-use testing scripts for login and validation test cases.

5. Time box conversion of manual test cases

Agility should be adopted with the time box activity. Converting manual test cases into automated test cases also helps to cut down the time for doing BVT or smoke testing. It gives quick confidence to the development team and stakeholder about working shippable product.

6. Execution and Management of Test Cases

Your test automation strategy should also define the process of execution and management of test cases. After executing a test case, assign a status ‘Pass or Fail’ with it. In automated testing, this status is assigned to the test cases automatically as per defined success criteria.

7. Result-oriented approach

Outcome of each automation test run should be result oriented and measurable. This will give confidence about robust automation suite.

Different kinds of KPI and dashboards will help stakeholders to quantify and qualify the software by using test metrics. This will also highlight the quality standards of particular software before it is launched in the market.

8. Lesson learnt and adopts to change

Lesson learnt while developing software should be captured and documented for future reference. Periodic review of test strategy and based on the positive and negative lesson learnt, test strategy documents needs to be updated. Continuously improve your test automation strategy by learning from those lessons.

9. Set Objectives

You need to clearly define some parameters against which you can measure the success of your test automation strategy. You need to be specific while defining the target. For example, you can set a target of 30% faster time to market, 40% reduction in testing cost, 70% faster regression testing.

10. Automation Maturity Model

Automation maturity model can be the key element to revamp automation test strategy. Automation maturity model can be divided into phases like Initial Phase, Managed Phase, Define Phase, Measured Phase and Optimize Phase. In Initial phase, prerequisites of automation should be completed.

• In Managed Phase, automation can be achieved without any central infrastructure. In Define Phase, central automated process across the application life-cycle should be built.
• In Measured Phase, collected and analysed metrics of the automated process should be measured against business goals.
• In Optimize Phase, self-service automation, self-learning using analytics and self-remediation should be addressed in automation maturity model.

By understanding the above automation test strategy elements, you can resolve or avoid below challenges

Deciding automation scope is directly related to the development of an automation suit. Unclear business requirements lead to failure of automation test suit and does not give expected outcome.

A test analyst can scope out automation test cases by discussing queries with SME. Most of the time; automation testing covers 90% of test coverage as there are some limitations due to which full 100% automation is not possible.

Test experts can evaluate the suggested top five automation test tools. According to the nature of an application, he can recommend tools and techniques that can guarantee the best results. Most of the time, automation testing is targeted for regression testing, functional testing, smoke testing, load and performance testing. Choosing proper tool based on the kind of software application is an important part to achieve expected result.

To avoid high maintenance cost and time, one should revisit and update automation test suite based on the latest version of the product over the period of time. This activity is a continuous activity to remove stale test cases and upgrade the automation suit by adding new functionality.

Instead of setting long-term goals, short-term goals should be targeted and those should be revisited periodically, i.e. monthly or quarterly. This will help to build a continuous iterative incremental model for making the necessary changes in automation test strategies.

While implementing the test strategy elements below points can be easily achievable:

Test automation scope can be enterprise oriented, product oriented and project oriented. Enterprise oriented scope covers different product line and projects in organisation. Product oriented scope covers specific product line of an application. And project oriented scope covers specific project and its test process.

Automation test suite can be scalable and covers the maximum number of test cases under automation test suite. Common functionality can be considered while designing automation framework so that automation suite will be active and working across the functional area.

Automation framework consists of keyword-driven framework, data driven framework and hybrid framework. Based on the application or software one can choose the correct framework. Result oriented approach will give quick confidence about shippable software to stakeholders.

Major loss of time and efforts will be reduced due to the iterative and incremental improvement in automation suite. Maintaining agility and building continuous automation test suite is always a challenge to test automation team; due to which they do not understand from where to begin. Automation test team should convert last iterated manual test cases into automated test cases in each sprint.

Robust automation framework works on Windows and Web applications, across platforms and in different functional departments. Periodically maintained and upgraded strategy will keep the main objective of automation test strategy alive. Quick ROI (return on investment) should be possible due to automation test strategy.

Recap

So let’s quickly touch base cost-effective and empowered testing automation strategy with result oriented approach. This can be achieved by identifying functional and non-functional automation test cases, dedicated skilled automation team, Time box activity with optimisation and iterative incremental methodology for shippable product within no time.

The long-term benefits of deciding proper automation test strategy will be scalability, reusability of common framework across the functional area, agility for continuous improvement, shippable product to end customer and thousands of satisfied customers.