Project teams desire to include automation testing in their project without understanding both the advantages and disadvantages of each type of testing - Automation and Manual Testing. It is not always beneficial to include automation testing with analysing the user case we are trying to solve, skill set in the team, calculating ROI etc. Lets understand some guiding principals when to pick what type of testing and benefits we would draw from our approach.
When manual testing is better than automation testing
Manual testing shall be preferred over automation testing when -
There are a limited number of times a functionality needs to be tested. Low repetition of tests eliminates the need for automation.
The team doesn’t have the required skill set to automate.
Code is unstable and is anticipated to change in the future.
The project plan doesn’t have a time window to wait till the automation framework is created and stabilized.
Advantages of Manual testing over Automation testing
Good manual testing forms the base of stable automation testing.
Automation may not give ROI in every case.
Automation gives returns late in the software lifecycle.
Automation is a difficult skill and getting skilled resources can be a challenge.
The automation script needs regular maintenance.
Automation may not identify issues other than where validations and assertions are not scripted.
Automation may give false alerts.
Exploratory testing is not possible in automation.
Manual testing can start from scratch whereas automation takes a longer time to start.
When automation testing is better than manual testing
When there is a return on investment (ROI) doing automation is more than manual testing. Automation testers/Test Lead/Test Managers need to take conscious calls when choosing automation over manual testing. The decision should be based on the factor called ROI which measures the effort and time in terms of dollar value) spent on automation will save cost if the same testing is done manually throughout the lifecycle of the product.
The project code is stable and anticipating a low number of changes. With every change in code, automation scripts need to be updated accordingly hence affecting ROI.
Desired skill set available in the team.
The team managers should have a long-term vision and strategy for automation testing.
Advantages of Automation testing over Manual testing
Early detection of defects & reduce time to market — a good automation suite can be leveraged and can give quick results compared to manual testing which will take a long time to share results.
Consistent results — Automation will give consistent results and eliminate manual errors.
Saves cost — In general, where there is the repetition of testing, automation will give better results than manual testing.
Reduce manual overhead — Automation can run anytime without any manual dependencies.
Team with advanced skill set — The automation skill set is considered more valuable in the software industry than manual testing.
Teams blindly aspire to start automation testing without analysing which out of two - Manual and Automation testing fit their purpose. Choosing automation testing without analysing is like shooting in the dark. Teams could land-up spending more than they could possible get in return. Implementation of automation is a strategic step that has to be well analysed and estimated. We also have to consider not everything cannot be automated and long term return of automation should be positive. On the contrary automation provide many implicit benefits as well like team aspire to learn automation skills that keep them motivated, it also introduce automation mindset within team and enhance testers' skill in coding. Deciding to choose automation or manual testing should be well thought out considering which method provide better results, return on investment, long term explicit and implicit benefits etc.