WHY COMPANIES OFTEN FEEL LET DOWN BY THE OUTCOME OF THEIR INVESTMENT

In practice, Functional Test Automation can deliver a number of benefits to an organisation that is prepared to invest. The time to test is decreased, the quality and accuracy of the test execution increases and when the Functional Test Automation is frequently executed, costs can decrease as well compared to manual functional testing.

However, Functional Test Automation can sometimes fail to deliver and can cause frustration amongst project management who have invested in a long-term outcome. Often the underlying problem is the automated functional test analysts spending more time maintaining their test automation scripts (mainly due to application code changes and automated script writers having to keep up with these changes in their script) than it would take to execute manually.

Take a test pack with 100 regressions test cases in it. Manual execution at the rate of 16 per day would take 6 days. When these tests are automated, if 50% of these fail because the application has been updated, then 50 scripts corrected at the rate of 4 per day would take 12.5 days to resolve.

While the problem may appear to reside with the functional automated test engineers who have not created robust test automation, generally the problem lies with management who have failed to invest correctly to achieve the desired outcome.

WHAT DOES YOUR ORGANISATION WANT TO ACHEIVE FROM FUNCTIONAL TEST AUTOMATION ?

There can be many reasons to invest in functional test automation, the common ones are:

• Improve Testing Efficiency.
• Cost Savings.
• Improve Quality.
• Decrease Time to Market.
• Risk Reduction.

WHAT AUTOMATION DO YOU GET ?

Business or Test Analyst Testing

• Technical Resource - Largely unskilled in Test Automation.
• Dedicated Resources - No.
• Resource Costs - Limited investment in time and resources.
• Internal / External Resources - Customer self-sufficient with internal resources.
• Test Automation Tool - Visual scripting.
• Test Tool Costs - Medium.
• Outcome - One time use scripts, good for creating test data or one-time tests.

Technical Level Testing

• Technical Resource - Technical Tester / Automation Engineer.
• Dedicated Resources - Yes.
• Resource Costs - Significant investment in time and resources.
• Internal / External Resources - Blend between investment in a partner and self sufficiency.
• Test Automation Tool - Uses a technical automation tool.
• Test Tool Costs - High.
• Outcome - Regression test up to date with current sprint -3.

Developer Level Testing

• Technical Resource - Developer Level Technician.
• Dedicated Resources - Yes.
• Resource Costs - Very large investment in time and resources.
• Internal / External Resources - Investment in a partner.
• Test Automation Tool - Uses technical automation tool integrated with IDE (Visual Studio, Eclipse, IntelliJ, etc).
• Test Tool Costs - Low.
• Outcome - Automation Test Pack with many scripts, up-to-date for Current Sprint / Sprint -1.‍
  

Functional Test Automation can often be deemed to fail when investment made does not match the outcome desired, for instance, trying to achieve technical testing outcomes with Business or Test Analysts who are more used to manual testing.