October 10, 2023
INTRODUCTION

Incorporating non-functional test (or indeed any type of testing) early in the software development lifecycle is a great idea, as the earlier in the development cycle defects and issues are found and remediated, the lower the cost of that remediation is, as issues would not have had less time to bed and become an integral part of the application.

WHAT IS NON-FUNCTIONAL TESTING ?

When we describe non-functional testing, we are referring to such aspects as performance, reliability, scalability, security, rather than functional testing, which verifies that the software functionally does what is expected.

How would we go about incorporating non-functional testing early in the software development cycle ?

DEFINE NON-FUNCTIONAL REQUIREMENTS EARLY

At the start of the project, we should define non-functional requirements as well as functional requirements. In order to be able to test, we are going to need some sort of indication / definition of the sort of system behaviour which would be acceptable in terms of aspects such as response times, memory utilisation, latency, acceptable error rates and so on.

It is however not common that non-functional requirements are initially defined at the start of the project, and such requirements tend to be retrofitted at a later date depending on emerging and changing customer requirements into non-functional aspects.

INVOLVE / EDUCATE STAKEHOLDERS ABOUT NON-FUNCTIONAL TESTING

By initially involving all stakeholders, including performance engineers, security experts and business analysts in the requirement gathering process the requirements are known from the beginning and the system can be designed with clear non-functional objectives which will not need to be reworked at a later date.

From the outset it is important all non-functional requirements are well understood by all stakeholders, especially everyone in the development and testing process, and the impact of failing these requirements is communicated and realised.

Educate wider project staff about the impact of non-functional issues can have overall success of a project.

INCORPORATE AUTOMATED NON-FUNCTIONAL TESTING INTO CI/CD CYCLES

Incorporate non-functional testing tasks straight into the overall development process.

Performance testing is often executed as part of the continuous integration and continuous delivery (CI/CD) pipeline, which can give swift feedback on the performance of any recent code changes. When choosing the performance testing tool, you should check the compatibility of the tool when used with the CI/CD tool you are using (e.g. Jenkins, TeamCity, etc...)

Implement a monitoring solution in the CI/CD pipeline (using tools such as DataDog, New Relic, AppDynamics, etc...) to monitor the performance of the deployed application in real-time and send alerts when issues or performance drops below the required standard. Can your performance testing tool integrate with a monitoring / test reporting tool ?

Security testing tools, such as SAST or Static Application Security Testing tools can find potential security defects and vulnerabilities simply by scanning code without needing to deploy the application. As such, they can also be run as part of CI/CD cycles.

Performance testing results from the CI/CD process must be interpreted and presented in a format that can be understood by all stakeholders in the team.

CONCLUSION

By including non-functional testing early in the software development cycle, teams can accelerate non-functional testing and the production of quality releases and identify and remediate non-functional issues early before they manifest into well integrated and therefore hard to remove issues.

Posted on:

October 10, 2023

in

Performance testing

category.

Is there a project You'd like to discuss?

related insights

HTML, CSS and ARIA in Website Accessibility

Explaining DevSecOps

Artificial Intelligence (AI) and Machine Learning (ML) in Performance Testing

The Differences between Usability and Accessibility Testing

Benefits / Drawbacks of Production Performance Testing in Test / Scaled Down Environments

Incorporating Performance Testing within CI/CD Pipelines

How to Obtain Stakeholder Buy-In for Non-Functional Testing

Troubleshooting Performance Issues in Test Environments: A Real-World Scenario

‍Functional Test Automation: Why companies often feel let down by the outcome of their investment

The OWASP Top Ten - The Top 10 Web Application Security Risks

Avoiding Artificial Bottlenecks / Performance Issues in Performance Testing

Accessibility Guidelines - Testing for Accessibility and Understanding WCAG 2.1, the Upcoming WCAG 2.2 and Future WCAG 3.0 Updates

What is Volumetric Analysis ?

The Performance Testing Cycle / Process Explained

Service Level Agreements vs. Non-Functional Requirements for Performance Testing

Applying Automated Test Solutions

Combining Performance Testing and Chaos Engineering

Non-Functional Testing Strategy for Performance

Explaining Penetration Testing

Explaining Performance Testing

Explaining Accessibility Testing

Silk Central Upgrade - A simple upgrade ?

Virtual Machine LoadRunner Load Generators on Azure Setup

How Selenium WebDriver can be used for Performance Testing and its Pros and Cons

Performance Testing with SSO, OAuth

16 Tips Before You Automate

What is Automated Software Testing?

Load Testing and Performance Testing Tools

10 Top Tips for Creating Automated Performance Test Scripts