July 14, 2023
WHY PERFORMANCE TEST ?

Whilst an application may work functionally, if it cannot handle the required load of system users and their activity without exhibiting poor performance / errors, then it may not be fit for purpose. Often we look to validate the functional requirements of an application without taking into account the importance of fulfilling non-functional requirements

NON-FUNCTIONAL REQUIREMENTS

So how does a non-functional requirement differ from a functional requirement? 

•  FUNCTIONAL REQUIREMENTS – Defined as something an application must allow the user to do. Focused on user requirements – Defined as a functional use case, for example, the system must allow users to reset their password by clicking the Forgot Password link.

•  NON-FUNCTIONAL REQUIREMENTS – Not related to any specific piece of functionality within the application, but rather how the application should behave and perform when used. Focused on user expectations – Defined as a behaviour / quality attribute, such as for example the system must handle 200 concurrent users without performance degradation.

TYPES OF PERFORMANCE TEST

In performance testing we seek to validate the application is able to perform under varying degrees of load without error and within response time guidelines.

The application should be resilient enough to handle not only anticipated system load in terms on concurrent users / user activity, but other types of testing are available to help ensure the performance on the application-under-test.

 LOAD TESTING – Is the application performant under anticipated normal and peak loads ?

 STRESS TESTING – Can the application handle loads above and beyond the anticipated load ? How much can we increase load to the system before encountering application errors and unacceptable performance ?

 SOAK / STABILITY TESTING – Can the application handle loadover a prolonged period of time without encountering memory issues ?

 SPIKE TESTING – How does the application handle sudden surges of activity ?

 SCALABILITY TESTING – Can we scale up with extra system resources to handle extra user demand ?

 AVAILABILITY TESTING – Can the system automatically and seamlessly failover with no loss of performance when we inject failures (through methods such as chaos engineering) ?

THE PERFORMANCE TEST PROCESS

To plan and execute performance tests, we need to follow a number of steps to ensure that we are creating and executing representative performance test scenarios which allow us to test not just the performance, but also the resiliency of the application under test. 

i). TEST ENVIRONMENT – Understand the test environmentand the technologies and protocols you will be working with. Your chosen performancetest tools must be able to work with the protocols you require to generate trafficthrough (e.g. HTTP/HTML, SAP, RTE, IBMMQ, Web Services, etc…). Which tools canbe used to monitor the performance of the application under test ?

ii). DEFINE ACTIVITY / SCENARIOS – Can we identify the anticipated production system load so we can perform a representative load test? What type of testing should we conduct to achieve all of our testing objectives (performance, stress, soak / stability, etc…) ?

iii). PERFORMANCE METRICS – Which performance metrics / key performance indicators are we going to use to evaluate the ultimate failure / success of the test ? Do we already have SLAs or non-functional requirements defined which will allow us to determine test success or failure ?

iv). SCRIPT / EXECUTE TESTS - Script with the testing tool to replicate the scenarios mentioned previously. Perform an initial set of baseline tests, if possible, to give an indication of starting performance, which we can compare subsequent testing runs to. Perform a cycle of testing, fixing / tuning the application and analysing results.

v). REPORTING - The performance tester prepares and presents a summary of the test, with detailed reporting into the test activity, the findings of the test and the conclusions reached on whether the system meets the non-functional requirements and SLAs defined previously.

TOOLING OPTIONS

Depending on the technologies under test and the level of protocol support and testing tool support required required, there are a number of commercial and open-source performance testing tools and performance monitoring solutions which we can use to conduct the performance test exercise. 

•  OPEN SOURCE TOOLS – Apache JMeter, Grafana Labs K6, Gatling open-source.

•  COMMERCIAL TOOLS – OpenText LoadRunner, TricentisNeoLoad. 

HOW CAN FIMATIX HELP ?

Whether you need advice defining the volumetrics for a performance test, non-functional performance requirements to gauge test success or failure with or whether you need expertise to plan, write and execute performance testing to ensure production readiness then FIMATIX can help. 

Posted on:

July 14, 2023

in

Performance testing

category.

Is there a project You'd like to discuss?

related insights

Securing cloud-based applications

Usability and Accessibility Testing - What is the Difference ?

Incorporating Non-Functional Testing Early in the Software Development Cycle

Benefits / Drawbacks of Performance Testing in Test / Scaled Down Environments

Incorporating Performance Testing within CI/CD Pipelines

Obtaining Buy-In for Non-Functional Testing

Troubleshooting Performance Issues in Test Environments: A Real-World Scenario

Demystifying Database Tuning - Unraveling Database Performance

‍Functional Test Automation: Why companies often feel let down by the outcome of their investment

The OWASP Top Ten - The Top 10 Web Application Security Risks

Avoiding Artificial Bottlenecks / Performance Issues in Performance Testing

Accessibility Guidelines - Understanding WCAG 2.1, the Upcoming WCAG 2.2 and Future WCAG 3.0 Updates

What is Volumetric Analysis ?

The Performance Testing Cycle Explained

Service Level Agreements vs. Non-Functional Requirements for Performance Testing

Applying Automated Test Solutions

Combining Performance Testing and Chaos Engineering

Non-Functional Testing Strategy for Performance

Explaining Penetration Testing

Explaining Accessibility Testing

Explaining Penetration Testing

Silk Central Upgrade - "It's just a simple upgrade...."

Virtual Machine LoadRunner Load Generators on Azure Setup

How Selenium WebDriver can be used for Performance Testing

Performance Testing with SSO, OAuth

16 Tips Before You Automate

What is Automated Software Testing?

Load Testing and Performance Testing Tools

10 Top Tips for Automated Performance Scripts