July 1, 2023

In practice, Functional Test Automation can deliver a number of benefits to an organisation that is prepared to invest. The time to test is decreased, the quality and accuracy of the test execution increases and when the Functional Test Automation is frequently executed, costs can decrease as well compared to manual functional testing.

However, Functional Test Automation can sometimes fail to deliver and can cause frustration amongst project management who have invested in a long-term outcome. Often the underlying problem is the automated functional test analysts spending more time maintaining their test automation scripts (mainly due to application code changes and automated script writers having to keep up with these changes in their script) than it would take to execute manually.

Take a test pack with 100 regressions test cases in it. Manual execution at the rate of 16 per day would take 6 days. When these tests are automated, if 50% of these fail because the application has been updated, then 50 scripts corrected at the rate of 4 per day would take 12.5 days to resolve.

While the problem may appear to reside with the functional automated test engineers who have not created robust test automation, generally the problem lies with management who have failed to invest correctly to achieve the desired outcome.


There can be many reasons to invest in functional test automation, the common ones are:

  • Improve Testing Efficiency.
  • Cost Savings.
  • Improve Quality.
  • Decrease Time to Market.
  • Risk Reduction.
Business or Test Analyst Testing
  • Technical Resource - Largely unskilled in Test Automation.
  • Dedicated Resources - No.
  • Resource Costs - Limited investment in time and resources.
  • Internal / External Resources - Customer self-sufficient with internal resources.
  • Test Automation Tool - Visual scripting.
  • Test Tool Costs - Medium.
  • Outcome - One time use scripts, good for creating test data or one-time tests.
Technical Level Testing
  • Technical Resource - Technical Tester / Automation Engineer.
  • Dedicated Resources - Yes.
  • Resource Costs - Significant investment in time and resources.
  • Internal / External Resources - Blend between investment in a partner and self sufficiency.
  • Test Automation Tool - Uses a technical automation tool.
  • Test Tool Costs - High.
  • Outcome - Regression test up to date with current sprint -3.
Developer Level Testing
  • Technical Resource - Developer Level Technician.
  • Dedicated Resources - Yes.
  • Resource Costs - Very large investment in time and resources.
  • Internal / External Resources - Investment in a partner.
  • Test Automation Tool - Uses technical automation tool integrated with IDE (Visual Studio, Eclipse, IntelliJ, etc).
  • Test Tool Costs - Low.
  • Outcome - Automation Test Pack with many scripts, up-to-date for Current Sprint / Sprint -1.

Functional Test Automation can often be deemed to fail when investment made does not match the outcome desired, for instance, trying to achieve technical testing outcomes with Business or Test Analysts who are more used to manual testing.

Posted on:

July 1, 2023


Automated testing


Is there a project You'd like to discuss?

related insights

Artificial Intelligence (AI) and Machine Learning (ML) in Performance Testing

The Differences between Usability and Accessibility Testing

Why Incorporate Non-Functional Testing Early in the Software Development Cycle ?

Benefits / Drawbacks of Performance Testing in Test / Scaled Down Environments

Incorporating Performance Testing within CI/CD Pipelines

How to Obtain Stakeholder Buy-In for Non-Functional Testing

Troubleshooting Performance Issues in Test Environments: A Real-World Scenario

Demystifying Database Tuning - Unraveling Database Performance

The OWASP Top Ten - The Top 10 Web Application Security Risks

Avoiding Artificial Bottlenecks / Performance Issues in Performance Testing

Accessibility Guidelines - Understanding WCAG 2.1, the Upcoming WCAG 2.2 and Future WCAG 3.0 Updates

What is Volumetric Analysis ?

The Performance Testing Cycle Explained

Service Level Agreements vs. Non-Functional Requirements for Performance Testing

Applying Automated Test Solutions

Combining Performance Testing and Chaos Engineering

Non-Functional Testing Strategy for Performance

Explaining Penetration Testing

Explaining Performance Testing

Explaining Accessibility Testing

Silk Central Upgrade - "It's just a simple upgrade...."

Virtual Machine LoadRunner Load Generators on Azure Setup

How Selenium WebDriver can be used for Performance Testing

Performance Testing with SSO, OAuth

16 Tips Before You Automate

What is Automated Software Testing?

Load Testing and Performance Testing Tools

10 Top Tips for Automated Performance Scripts