The Paradox of Testing
What if we get to know what are the testing areas we can focus on most, to get 100% testing coverage, and with the least effort? Or how can we test our application end-to-end with the least efforts, in the quickest way, and yet ensuring 100% coverage? Is it possible to prepare a test plan to support these factors?
Yes, it is, and it's pretty simple. The answer is the Test pyramid. Below we shall cover a framework to implement the Test Pyramid.
Implementing the test pyramid is the answer to these questions. For that, rather than defining the Test pyramid directly, let us try to understand it by -
Understanding the concept behind Test Pyramid,
Learn how to implement it in manual and automation testing.
Lets first understand the concept behind the Test pyramid, which is the three-tier tech stack of the application's architectural design.
The three-tier tech stack of AUT (application under test)
Application development is designed to be segregated into tiers of the tech stack. This is done to develop the application in an engineered way. It ensures that each tier has a specific function to perform, hence segregation of responsibilities is achieved. This provides multiple benefits like independent development, modularisation, loosely coupled design, and microservice architecture. It helps the development team to make changes easily to the software without worrying about affecting another tech tier.
In general, all applications can be divided into below three-tiered tech stacks.
Back-end tier - This is where data of the application is stored in databases.
Middle tier -This tier performs operations like read, write, delete, modify the database and expose the data to the front-end.
Front-end tier- This tier is where the end-user interacts with the system. User’s operations are performed on the front-end and the middle tier carries these operations to perform them on the back-end tier.
As these tiers have their own set of responsibilities, they use different technologies to accomplish them. For example, the back-end tier will need PL SQL and skills on database-related technologies, the middle tier will need skills in programming languages (like java or python) and APIs, and the front-end tier will need skills in designing, HTML, CSS, etc. Hence the skill set required in development at each tech tier is different and often different sets of team members work at each tier. Developers who are skilled at all three tiers are called full-stack developers.
Testing Layers - How we can use tech tiers to plan manual and automation testing?
We all agree that if we understand the functionality and tech stack of the application under test (AUT) better, the testing produces better results. Now that you have understood how software design and development work by dividing development into three tech tiers, let's use this information to understand how we can use this tech stack to draft a test plan using the test pyramid for manual and automation testing.
Test Pyramid
Test pyramid provides a way to test each of the tech tiers of the application under test. It helps the testing team to draft a plan to perform testing in the most optimal and cost-effective way by saving time and effort and yet increase coverage.
Test Pyramid states that testing can be divided into three different testing layers namely -
Unit testing
Service or Integration testing
UI Testing
The principle behind the test pyramid is that the speed and efficiency at which testing can happen reduces as we move up the pyramid hence we should test more at lower layers to achieve maximum coverage in minimum time, and test only those functionality in higher layers which are not possible to test at lower layers.
This means that the tests on the lower layers are less expensive and faster to run as compared to tests on the higher level. Thus, a test plan should have more unit tests and fewer UI tests.
Testing characteristics of each layer of the Test Pyramid
The Unit test layer - In the unit test, the smallest testable component of a software application is tested and this testing is done by developers. This is the lowest layer of the test pyramid and we can test fastest and hence we should test most here to achieve maximum coverage in minimum time. That is the reason why the width of this layer is depicted widest among others. This testing layer brings maximum efficiency. Though it has a disadvantage that it tests the system in isolation and hence integration testing of back-end with middle tier, and middle-tier with front-end tier cannot be done here. Hence this leaves the scope to test in other layers.
The Service or Integration test layer - This testing involves checking the calls to the databases, web services, etc. to ensure if the data retrieval systems are working properly. It sits in the middle and is faster than the UI layer. Also, it covers the disadvantage of unit tests as it tests the units in combination and back-end integration with the middle tier.
The UI Test layer - This is testing the user journeys on the UI of the application. This is the slowest to test but is most important as users interact with the system here. Hence, this needs to be thoroughly tested. It provides the guarantee that the system is working end-to-end and implicitly tests all three layers of the application - front-end, middle-tier, and back-end.
Using Test Pyramid to implement Automation and Manaual Testing
If you have understood by now, you should be able to relate the AUT tech tier with test pyramid layers. The Test Pyramid is a way to divide testing based on three tech stack tiers (back-end, middle tier, front-end) of the AUT and test the tiers based on the inherent behavior of each tech tier.
To implement the test pyramid, the task for the testers is to step back and analyze the application’s tech tiers at a higher level. Testers can take help from architects and tech lead in the project to understand and draw the high-level application’s tech stack diagram. Most modern applications’ architecture is based on the three-tier tech stack defined in previous sections. Once we are clear with the AUT tech stack, we can plan to implement automation and manual testing based on the application’s tech tiers.
Integrating tech tiers with the test pyramid gives a way to test the tiers of the application which are quicker and easier to test than the tiers which are slower and inefficient to test.
We have to follow the principle of the test pyramid to do unit testing on the lowest tier (back-end) and once testing is completed move to the upper tier (Middle tier) to do Integration Testing, and then to the highest tier (front-end tier) to perform UI testing. By covering testing at lower tiers first, will ensure minimum time spent on the front-end tier for UI testing which is least productive.
Implementing the test pyramid can be done by defining a plan for testing at each tier of the tech stack. For this, we have to relate the tech stack (back-end, middle tier, front-end) of the application with the test pyramid (Unit, integration, and UI testing) as shown in the picture. Then starting from the lowest layer first, i.e. Unit Test layer moving to higher layers of Integration Test and UI Test, decide how much time to spend at each layer to bring maximum coverage in minimum possible time. Although we can see that the UI test layer is where we get most integration and it implicitly tests all three tech stack tiers, the problem is speed and stability. UI test is most time-consuming and is least stable, especially in the case of automation testing. Hence, this is where the test pyramid comes into the picture. Implement the test pyramid to move the testing from UI testing to Integration and Unit testing. As a result, the more testing we do at the lower layers of the test pyramid, the more coverage we can achieve in a quick time.
Other advantages of implementing Test Pyramid
Other than bring speed, quality, and coverage to testing, implementing the test pyramid brings the following benefits to manual and automation testing -
If we test the front-end tier directly by UI testing, it will take the maximum time to test all the scenarios and integration.
The defect fixes will also be expensive as UI testing will start late in the cycle when the front-end tier is developed.
Also, any defect found directly in the front-end tier can force changes to other tiers if we test the font-end directly. On the contrary, if we start with the unit testing to fully test the back-end tier first and then move to the middle tier to fully test the integration layer, this will enable us to reduce the UI testing layer scope to test the front-end tier. This approach will give the most optimum performance in terms of speed and coverage.
The other advantage of implementing the test pyramid is as the end-user UI functionality development happens at the end, we can start testing as soon as the back-end and middle tiers are developed.
It is also important to understand that Test Pyramid brings the same benefits for manual and automation testing and it is not the model only applicable to automation testing.
Select Automation testing tool using the Test Pyramid
It is important to understand the tech stack of your application under test and then the test pyramid is the ultimate guide and bring a top-down approach for automation tool selection. As different tools are designed to work on different tech tiers of the application, the selection of tools should be done after the implementation of the test automation pyramid. This will give a clear picture to the team of what type of tools should be used for testing. Here is the list of some of the tools as per tech stack and layers of the test pyramid.
List of tool for automation for front-end tier to perform UI testing
Selenium
HP UFT
Cucumber
Ranorex
List of tool for automation for middle tier to perform API and integration testing
Rest Assured
Karate
Postman
SoapUI
apigee
List of tool for automation for back-end technologies to perform Unit testing
xUnit (nUnit/sUnit/jUnit/phpUnit)
PIT (for Java)
EMMA