July 1, 2020

Automated Testing can provide a great Return on Investment (ROI). However, not all scripts are suitable for automation. Here are some top tips to help avoid falling into an automated testing mess.


Try to allocate a separate budget to the software automated testing solution. It should be considered its own project.


Don’t group Performance Testing Automation with Regression and Functional Automated Testing solutions. Even though both use automated test tools they require a very different skill set in application and should be budgeted separately. A case in point being test environments - a regression testing environment does not need to be scalable but a performance test environment does.  In some businesses this difference equates to millions of pounds.


Produce a measurable test strategy that clearly measures goals and scope of the automated testing activities.


Ensure that you conduct test tool comparisons to verify that you get the best product to meet your requirements; the cheapest is not always the best value for money, neither is the most expensive necessarily the best option.


Do not assume the test tool demo conducted by a vendor against a system on their laptop will actually work on your system, it may well not.


Check the job market to verify the rates and availability for technical resource that specialise in a given test tool. It is quite often the case that an automated tool that is unknown and cheaper is more expensive due to a lack of qualified and experienced resources on the market.  This can result in a business being tied into a vendor’s expensive services solution.


Don’t take a tool vendor's first price - most of them expect a haggle. If the price isn't negotiable, how about the support services or the licencing period?


If you are using a service provider to assist in automated test tool evaluations ensure that they are independent and can provide you with a neutral perspective during your evaluations.


Don’t start with outsourcing - start by implementing your own in-house solution.  Make it measurable, controllable and prove its ROI.  After you have made it work in-house you can then safely scale the process with outsourcing. Get them to follow your proven process which ensures you are always in control. The last thing you want is to be tied to a company without the ability to move your service requirements to other companies easily and efficiently.


Don’t attempt to automate everything.  Automated testing should only be conducted against a sub set of functionality that will prove a ROI and focuses on real business cases.


Remember every automated test script produced will always require maintenance and re scripting. When a test script is completed it generally stays stable for a short period of time. Typically every test script needs to be verified between code releases. This is one of the biggest hidden costs of automated testing.


When using an automated functional testing tool ensure that your application's objects are mapped (GUIMap or Object Map) this will help immeasurably to reduce recoding requirements and make maintenance easier and cheaper.


Automated testing is not a one-off development activity, it is an on-going process.  You can save time and money with automated testing through faster test cycles, however you will have to typically pay more for resources who specialise in a test tool and you will need them to continue their services in order to keep the automated testing process efficient.


In-house resource can be trained in the use of automated testing tools, however, we would always recommend using experienced people involved in managing the technical process.


Get experts in test automation such as Testing Performance Ltd to help you produce a measurable and effective test strategy and to build your automated testing framework.


Part of the strategy should be to ensure that you are never tied to a service company and can move between companies or in-house testing solutions.

Posted on:

July 1, 2020


Test insight blog


Is there a project You'd like to discuss?

related insights

Artificial Intelligence (AI) and Machine Learning (ML) in Performance Testing

The Differences between Usability and Accessibility Testing

Why Incorporate Non-Functional Testing Early in the Software Development Cycle ?

Benefits / Drawbacks of Performance Testing in Test / Scaled Down Environments

Incorporating Performance Testing within CI/CD Pipelines

How to Obtain Stakeholder Buy-In for Non-Functional Testing

Troubleshooting Performance Issues in Test Environments: A Real-World Scenario

Demystifying Database Tuning - Unraveling Database Performance

‍Functional Test Automation: Why companies often feel let down by the outcome of their investment

The OWASP Top Ten - The Top 10 Web Application Security Risks

Avoiding Artificial Bottlenecks / Performance Issues in Performance Testing

Accessibility Guidelines - Understanding WCAG 2.1, the Upcoming WCAG 2.2 and Future WCAG 3.0 Updates

What is Volumetric Analysis ?

The Performance Testing Cycle Explained

Service Level Agreements vs. Non-Functional Requirements for Performance Testing

Applying Automated Test Solutions

Combining Performance Testing and Chaos Engineering

Non-Functional Testing Strategy for Performance

Explaining Penetration Testing

Explaining Performance Testing

Explaining Accessibility Testing

Silk Central Upgrade - "It's just a simple upgrade...."

Virtual Machine LoadRunner Load Generators on Azure Setup

How Selenium WebDriver can be used for Performance Testing

Performance Testing with SSO, OAuth

What is Automated Software Testing?

Load Testing and Performance Testing Tools

10 Top Tips for Automated Performance Scripts