Automated Testing Best Practices

Automating tests has the following benefits:

• Efficiency: the speed with which a machine executes tests is considerably higher than the speed of a human being, which makes the testing process quicker.
• Precision: a machine is less likely to make mistakes than a human being, which increases the effectiveness and reliability of the tests.
• Increased coverage: having scripts that execute tests independently allows for teams to change the focus of manual testing resources to other functionalities in the system.

However, it is a common error to try to focus only on the advantages that come with test automation, and ignore the risks involved in not following an appropriate process when designing and implementing automated tests. Not considering these risks may lead to an automation project not being successful; that is to say, that the objectives that were set at the beginning of the project are not met in due form.

Below, we will discuss 7 automated testing best practices which must be taken into consideration during a test automation process in order to minimize the risks and make the most out of the advantages offered by test automation:

1. 1. Feasibility of the automation project
2. 2. Planning times and defining resources
3. 3. Selecting the appropriate tool
4. 4. More parameterization means more maintainability
5. 5. Best locator practices
6. 6. Running tests over the same dataset
7. 7. Test automation as a complement to manual tests

1. Feasibility of the automation project

One of the worst situations that can arise is that time and resources are dedicated to test automation for a system that is not yet ready to be automated, or for test cases to be automated that add no value to the project.

You should keep in mind that test automation is an investment of time which is carried out to bring efficiency to the test execution process. If the time dedicated to maintaining scripts far exceeds the time it takes for manual tests to be carried out because, for example, the system is not stable, then you are not obtaining any benefits by automating tests.

The same applies when the objectives related to what will be tested when implementing automated tests are not clearly defined. Vague or unclear definitions regarding the scope of the tests can cause for resources to be invested in automating parts of the system which should not have been prioritized.

These types of situations can be easily avoided by conducting an initial feasibility analysis for the project, and by clearly defining the scope and the objectives you want to meet by adopting automated tests.

2. Planning times and defining resources

Having defined the project’s scope and objectives, the next crucial step is to define the resources that will be assigned to the project.

One of the most common causes of a testing project not moving beyond the initial planning stage is not having a clear distribution of the resources assigned to the project.

After all, if who will be participating in the project and how much time they will invest in it is not clear enough, the progress will be slower because the focus will be on other tasks that are not related to automation that do have well-defined, clear priorities.

Another very common error is to only automate tests when there is time to spare. It should be clearly stated from the get-go of a test automation introduction process that the activity is an investment for the future. If no resources are assigned specifically for the project, there will be no tangible results.

3. Selecting the appropriate tool

Selecting the appropriate tool or framework to be used during the test automation process is, without a doubt, one of the automated testing best practices, and one of the most important aspects to consider when defining a test automation plan. This decision can mean that the project is successful according to what was planned and within the agreed deadlines, or it can cause the project to fail.

However, choosing the appropriate tool is not an easy task. Having said that, below you will find the most important aspects that need to be considered when selecting the test automation tool that will be used during the automation process.

The first element to consider is that there is no one tool that is the best option and can be adapted to all contexts. Because of this reason, the elements we mention will be the ones helping you make the right decisions depending on the context involved.

Having said this, the ideal tool or framework is the one that best adapts to both the needs of the projects and the needs of the testing team in charge of carrying out the automation tasks.

Nowadays, there is a wide variety of automation tools. From these options, we will need to choose the ones that best adapt to the project timeline, the scope (required technical challenges) and the desired coverage (number of test cases to be automated, or whether cross-browser testing is necessary or not, for example).

On the other hand, as we mentioned before, it is very clear that the knowledge and technical skills the automation team has are key when selecting the tool to be used. These become even more relevant if the team does not have advanced programming skills. In these cases, it is better to select a “codeless” tool, which will allow for them to create and maintain automation scripts intuitively and without the need to write any code.

It is important to point out that there are also some frameworks which integrate both options, allowing testers to choose whether they want to write code or create the scripts in an assisted and intuitive way. The advantage these tools have is that they allow for you to combine a team with diverse programming skills, achieving very good results in a short space of time.

4. More parameterization means more maintainability

Once the automation scripts have been created, they need to be maintained. Given that systems under test are being constantly developed, it is expected for certain changes to the system to entail updates to the automation scripts.

Updating automation scripts takes a lot of effort, even more so if the system is medium or large-sized.

For this reason, one of the test automation best practices is to correctly parameterize during the development of the automation scripts. More parameterization means less effort when maintaining automation scripts and a higher chance of reusing code.

As mentioned above, as the system grows, the number of test cases to be automated also grows, so parameterization becomes invaluable. In these cases, it is recommended to go on to the next parameterization level and create an automation framework customized for the system. This implies creating high-level functions to handle the system that allow for test cases to be created in a faster, more efficient and standardized way, thus increasing the robustness of the tests and minimizing the maintenance efforts posed by any new changes.

5. Best locator practices

If the tests to be automated are user interface tests, it is very likely that most of the effort will need to be focused on locating elements in the UI. This is the reason why choosing the correct strategy to locate elements is, undoubtedly, one of the most important automation practices to be considered. An incorrect strategy when locating elements can mean increasing the level of effort involved in maintaining scripts, and it might end up being the reason why the objectives defined at the beginning are not met.

Because the system under test is constantly being developed, it is important to give priority to locating the elements that are less likely to be changed; that is to say, the ones that will probably not change in future versions of the system (for example, “ID”, “name” or “class”). However, there are times where there are no unique locators for the elements in the system, which means having to use complex locators such as CSS or XPath.

Once an element has been detected to not have a unique identifier, it is necessary to consider certain aspects when choosing the CSS or XPath locator. Selectors that are too general might result in tests returning false positives (scripts that validate the elements are present when they are really not present in the interface), while using selectors that are too specific can cause false negatives (the smallest changes cause the automated tests to fail because the element cannot be found, even though it is present in the user interface). A successful strategy is somewhere between these two extremes: to generate a selector that is as simple as possible and only applies to the element you are trying to locate.

Considering the time it will take to implement scripts in relation to the time it takes to locate elements in the interface, it is recommended to analyze the system during the planning stages. If the elements in a system do not have unique identifiers, it is necessary to know so before implementing automated tests so that you can more accurately estimate the necessary effort, or even to work together with the development team to evaluate if the locators can be improved.

6. Running tests over the same dataset

Depending on the context and the project you want to introduce automation best practices, it is important to consider certain aspects related to the system data that will be involved in the automated tests.

There are certain contexts in which the initial and the final state of a system after performing a test is the same. However, in some other cases, the final state of the system data is altered after executing tests. Given that an automated script will need to start with the same dataset every time, we need to know that, when facing these kinds of systems, the second execution of the same script will fail because the necessary dataset in the precondition has changed and does not exist anymore.

To deal with this situation, it is crucial to detect which one of these contexts we will be facing. If the system or functionality under test required a specific dataset, it will be useful to consider doing one of the following:

• Undo the changes that were made in the system during an automated script using specific instructions, and integrating these at the end of the script. In other words, including instructions that return the system to its original state at the end of a script.
• Grouping scripts with related testing workflows. For example, group test cases which add and remove a record, thus guaranteeing that both will be executed in this order. This way, the element that was added is deleted in the same flow, thus ensuring that the final state of the system is the same as the initial one was.
• Work together with the development team to automatically reinitialize data, so that this can be integrated into the automated tests. For example, if you are automating user interface test cases, an alternative option would be to have a button somewhere in the testing environment which would allow you to reset the data. If this were the case, it could be achieved just by creating a function which clicks on the button, and then calling that function at the end of each script. This alternative is the most recommendable one out of the three, because it makes the automation process more robust. Having a way to reset the data within a function in a testing project allows for it to be executed after each test case (“@AfterTestCase”), and to return the system to its initial state in case a test cannot be completely executed for any reason.

7. Test automation as a complement to manual tests

As mentioned before, the advantages of automating tests are very clear. However, there still are some ways in which human beings conquer machines, such as our ability to distinguish and reason. Likewise, there are tests in which the cost-benefit ratio when automating is not favorable, and it is necessary to continue to execute the tests manually.

Considering this, automated tests clearly should not replace manual tests; at least, not all of them. Tests that are not repetitive, which involve human intervention, or tests that will take too long to design and/or maintain, will still need to be tested manually.

It is crucial to keep this in mind when planning the automation process. Not doing so can mean wasting hours automating tasks which we should not be automating. Investing part of the testing effort in automating tests that should be executed manually is a very common error, so it is important to consider this right at the start of the automation process.

By taking into account all of the above, and applying the advice we mentioned, you will considerably increase the effectiveness and efficacy of an automation project. However, keep in mind that even though it is important to consider them all throughout the implementation process, the earlier you put them into practice, the better results you will have.