WHY PERFORMANCE TEST ?

Whilst an application may work functionally, if it cannot handle the required load of system users and their activity without exhibiting poor performance / errors, then it may not be fit for purpose. Often we look to validate the functional requirements of an application without taking into account the importance of fulfilling non-functional requirements. 

NON-FUNCTIONAL REQUIREMENTS

So how does a non-functional requirement differ from a functional requirement? 

•  FUNCTIONAL REQUIREMENTS – Defined as something an application must allow the user to do. Focused on user requirements – Defined as a functional use case, for example, the system must allow users to reset their password by clicking the Forgot Password link.

•  NON-FUNCTIONAL REQUIREMENTS – Not related to any specific piece of functionality within the application, but rather how the application should behave and perform when used. Focused on user expectations – Defined as a behaviour / quality attribute, such as for example the system must handle 200 concurrent users without performance degradation.

TYPES OF PERFORMANCE TEST

In performance testing we seek to validate the application is able to perform under varying degrees of load without error and within response time guidelines.

The application should be resilient enough to handle not only anticipated system load in terms on concurrent users / user activity, but other types of testing are available to help ensure the performance on the application-under-test.

•  LOAD TESTING – Is the application performant under anticipated normal and peak loads ?

•  STRESS TESTING – Can the application handle loads above and beyond the anticipated load ? How much can we increase load to the system before encountering application errors and unacceptable performance ?

•  SOAK / STABILITY TESTING – Can the application handle loadover a prolonged period of time without encountering memory issues ?

•  SPIKE TESTING – How does the application handle sudden surges of activity ?

•  SCALABILITY TESTING – Can we scale up with extra system resources to handle extra user demand ?

•  AVAILABILITY TESTING – Can the system automatically and seamlessly failover with no loss of performance when we inject failures (through methods such as chaos engineering) ?

THE PERFORMANCE TEST PROCESS

To plan and execute performance tests, we need to follow a number of steps to ensure that we are creating and executing representative performance test scenarios which allow us to test not just the performance, but also the resiliency of the application under test. 

i). TEST ENVIRONMENT – Understand the test environmentand the technologies and protocols you will be working with. Your chosen performancetest tools must be able to work with the protocols you require to generate trafficthrough (e.g. HTTP/HTML, SAP, RTE, IBMMQ, Web Services, etc…). Which tools canbe used to monitor the performance of the application under test ?

ii). DEFINE ACTIVITY / SCENARIOS – Can we identify the anticipated production system load so we can perform a representative load test? What type of testing should we conduct to achieve all of our testing objectives (performance, stress, soak / stability, etc…) ?

iii). PERFORMANCE METRICS – Which performance metrics / key performance indicators are we going to use to evaluate the ultimate failure / success of the test ? Do we already have SLAs or non-functional requirements defined which will allow us to determine test success or failure ?

iv). SCRIPT / EXECUTE TESTS - Script with the testing tool to replicate the scenarios mentioned previously. Perform an initial set of baseline tests, if possible, to give an indication of starting performance, which we can compare subsequent testing runs to. Perform a cycle of testing, fixing / tuning the application and analysing results.

v). REPORTING - The performance tester prepares and presents a summary of the test, with detailed reporting into the test activity, the findings of the test and the conclusions reached on whether the system meets the non-functional requirements and SLAs defined previously.

TOOLING OPTIONS

Depending on the technologies under test and the level of protocol support and testing tool support required required, there are a number of commercial and open-source performance testing tools and performance monitoring solutions which we can use to conduct the performance test exercise. 

•  OPEN SOURCE TOOLS – Apache JMeter, Grafana Labs K6, Gatling open-source.

•  COMMERCIAL TOOLS – OpenText LoadRunner, TricentisNeoLoad. 

HOW CAN FIMATIX HELP ?

Whether you need advice defining the volumetrics for a performance test, non-functional performance requirements to gauge test success or failure with or whether you need expertise to plan, write and execute performance testing to ensure production readiness then FIMATIX can help.