Why writing an automation test plan is important?
“A bad plan is better than no plan, and the most important quality of any plan is the flexibility to change.” — Judson L Moore
Surveys suggest that 50% of the testing projects don’t have automation and 80% of these projects don’t have an automation test plan.
Another well-known fact is that the risk of a project failing without a proper plan is far more.
The automation test plan is uncharted territory for many of us (especially if we are new to automation) and we start automation without a proper automation test plan.
Another factor to consider before starting automation is to consider if automation will be profitable for the project/organization or not. Hence creating an automation test plan and estimating the return on investment (ROI) is of prime importance before starting automation. This is where things get tricky and the automation plan needs to be well thought out and well-articulated.
What is an Automation Testing Plan?
An automation test plan is a detailed document that describes the automation test scope, objectives, goals, estimations, and resources required and enlists the responsibilities of the automation testing team members. An automation test plan acts as a blueprint to conduct automation testing in a project. It also contains risks and contingency planning.
Benefits of writing an Automation Testing Plan
It justifies the ROI of implementing automation.
Give stakeholders an articulated automation approach.
It gives direction to automation and ensures automation is measurable and traceable.
Drafting an Automation Test Plan - Step by step guide
Following is a step-by-step guide and a template that we can use in our automation projects. We can arrange our plan in different sections to comprehend the various aspects of automation testing.
Test Automation Planning
1. Objective
This section specifies the objective of the automation test plan document and set stakeholder expectations from this document.
2. Project overview
This section gives a general overview of the project to the stakeholders. It helps the stakeholders to understand the project better and understand the automation plan with the project in the prospective.
3. Assumptions
Specify any assumptions that are there while drafting the automation plan. The assumptions need to be called out to the stakeholders. All assumptions affecting any phase of automation need to be called out here.
4. Return on Investment (ROI)
Not everything needs to or should be automated. Automation can be a costly affair and will give a negative ROI if the project is not a valid case of automation. Also, a project can experience negative ROI if automation is not planned properly.
This section should explain how the automation project will justify the investment verse if the same work is done manually throughout the life cycle of the product.
The formula to calculate ROI is -
ROI = Lifetime cost saved by automation - [Cost of developing automation + Lifetime cost of maintaining automation] + Lifetime cost of manual effort saved by automation
5. Automation Plan
This section defines what to automate and how automation will be planned. This section defines the foundation of automation planning. This section shall be covered the following items -
The type of testing automation will cover — Unit, Integration, End to End, etc.
Automation approach according to the tech stack — UI, API, DB, etc.
Automation Tool selection as per testing layers
6. Automation design
This section provides details about how the automation implementation will be designed. In either case of in-house framework development or commercial tool implementation, the technical design of the automation implementation needs to be thoroughly reviewed so that framework can remain scalable and maintainable for its complete life cycle.
Not having a technical design document will lead to a low-quality framework because lack of -
Unstructured modules/Monolithic application
Improper design patterns
Not following best coding practices
No formal review process
Low code reusability
No modularity at the function level
No separate test phase of automation framework
7. Automation execution
This section defines how automation will be implemented in the projects for automation test case creation and execution. Following considerations need to be called out in this section
How to implement at the project level?
What will be the branching strategy to manage automation code and test cases?
How will the test data be managed?
How will the test environment be managed?
8. Defect Management
The ultimate purpose of automation is to maintain the quality of the product and raise defects when they are introduced in the code. It is essential to articulate how automation defects will be captured and reported in the defect management tool.
Defect Life-cycle
Defect Management life cycle contains the following stages :
Following are the description of the common statuses a defect can remain in its lifecycle.
9. Test Automation Reporting
The automation framework should be designed to produce a report that clearly articulates the automation run. It should specify test cases executed, validations performed, and pass/fail test cases.
A good test report has the following components :
Project Information includes the name and description of the project.
Description of the test objective including the aim and enlisting the types of tests.
Test Summary including the test case execution states.
Defect report with the description, priority, severity, and status of the defect management process.
10. Automation Maintenance
This section defined the maintenance of the automation framework. Automation maintenance activity includes -
Enhancing the expectation handling and error logging mechanism.
Making the framework more robust and scalable.
Adding new functionalities in the framework to handle more use cases and test cases.
Code refactoring.
The above template of the automation test plan can fit most of our automation projects. This gives a comprehensive view of how to approach automation testing.
Read further -