The concept of test automation has recently become very popular in the IT industry. When thinking about developing a website more and more people are aware of the importance of testing. Many of them have heard about its importance but have little idea of what it entails or the actual benefits of using it. It’s actually a very complex process that affects the project on many levels and requires specific approach to fully utilise its potential.
Test automation in Software Development
So what is test automation and how does it actually work? As the name suggests the idea is to automate certain repetitive testing tasks. Let’s look at an example:
Brian is a manual tester – he comes to work everyday and does manual checks on an application in order to check if there are any new issues. The developers work on the application daily and make a lot of changes and add new features so he has to test the login process, the registration process, the purchase process and about a hundred other functionalities. This is an extremely repetitive work so Brian is very bored and sometimes omits some checks when he believes they are not necessary. It actually takes him around 3 hours to check the whole application and then he reports any new issues that he has found so it can be decided later if they are important or if they should be fixed at all.
Let’s see how would his day look like if Brian’s project invested in test automation at the very beginning.
Brian is an automation tester – he comes to work everyday and checks the automated test results in order to prevent any new issues. Instead of doing those checks manually he just presses one button and focuses on the other quality aspects of the project while the machine does the testing for him. He’s very happy because he feels he can expand his knowledge as a specialist instead of doing boring repetitive tasks. His colleagues are also happy because they can see the tests results after every change in the app without asking Brian to check that himself.
Of course this example is quite simple and usually the test automation is not so straight forward but, on the whole it grasps the concept of test automation quite well.
When to automate?
Before starting a test automation process it is crucial to ask yourself whether this is the right project to for it. A perfect target for automation is an application which will be maintained and developed for a long period of time but will not undergo a lot of changes during the process. Both aspects are actually quite vital as writing and applying test automation in the project requires time and effort so if the project is short then the effort may be not worth it.
A very important aspect is also choosing which part of system should be automated. Usually one starts with the elements that finished and will not be changed anytime soon. Frequent changes in the application require additional effort in updating the tests and some of the tests may need to be deleted in order to write new ones.
Levels of automation
There are actually a few levels on which you can apply automation in the project. Let’s focus on the three most basic ones: unit tests, integration/API tests, and end-to-end tests also known as user interface tests.
Unit tests are on the lowest level of the test automation pyramid – they check the code logic directly. Let’s say that a developer created a piece of code that is responsible for calculating a discount for certain products. A unit test would call the code logic itself and verify whether given inputs provide correct outputs. In this case when another developer changes the code and makes an error the unit test would fail. Unit tests are the cornerstone of each successful IT project – they are extremely fast to execute, fully reliable (as code will either pass or fail, there is no room for doubts) and easy to maintain.
Another level of test automation are the integration tests. A web application often consists of many sub-modules that interact with each other via the application programming interface (API) that is basically a set of predefined methods of communication between various components. Some pages also communicate with external components like bank payment methods or social media which also require a very specific request to be sent and expect an equally specific response.
So it is quite easy to imagine a situation when one developer responsible for a certain module changes the communication protocol and this is not reflected in another module. In this case a good test coverage can ensure that such issues are found quickly and fixed before they reach the end user. This level of test automation is usually slower and more difficult to maintain than the unit tests but it is still reasonably fast so a lot of these tests can easily be written and maintained.
Finally the top of the test automation pyramid – user interface tests. Those tests mimic end user behavior. They open up a browser/application and perform actions in a similar fashion as the actual user. These kind of tests are very valuable because they can pick up errors on many levels. It doesn’t matter whether two components communicate badly, whether the code logic is incorrect or whether a button is missing – the result will be still different than the one expected and the test would pick it up.
However, with great power comes great price… as these kinds of tests are the most expensive to create and to maintain as they are prone to multilevel changes. A simple change in design can often require an update in the test. They also execute rather slowly as they have to do everything in the same manner the user would do it – i.e. they have to wait for the page to load and for the data to show up in the browser.
Finally, they can sometimes show incorrect results due to brief network overloads or page scripts not loading correctly. Unless you have a lot of resources or you are extremely focused on finding every single minor issue it is recommended to keep the number of those tests limited as they are slow (often taking a few hours or more if you have hundreds of them) and require more effort to update and check their results.
The risk of test automation
Test automation is a powerful tool however, and requires involvement from the whole team. For each critical functionality the writing of new tests has to be taken into account. Before making a change in the application the team has to assess its impact on current automated tests and plan updating those tests alongside making the actual change.
Basically, setting up test automation takes some time and expert knowledge and it slows down the overall development process. It is quite easy to do it incorrectly as it requires the support of business, development team and some measure of experience. An inexperienced test automation engineer can write brittle tests that take a very long time and do not provide the expected feedback because of random failures which require additional investigation.
The benefits of test automation
There are plenty of benefits of test automation balancing out or even outweighing the risks described above. First of all it enables a fast and reliable tool to verify all covered functionalities. It also lacks the human factor, where a tester can omit a check when tired, or make a mistake – a machine will keep checking every single test case relentlessly. Doing any kind of changes in the application is much easier if you have automated checks that can afterwards inform you whether all the core functionalities still work. It also makes it easier for new team members to join the team – they may not know the application or code but they will still be notified if they break anything significant. It is also impossible to achieve a continuous deployment process without proper test automation.
Delivering the code to the production team continuously requires good test coverage and a solid automation process to ensure that the test quality does not deteriorate and the new code is ready for the end users. Finally the cost of web development actually lowers in time – setting up test automation and maintaining it can be costly but manually repeating the same tests hundreds of times every day will be even more costly in the long run.
