Continuous integration (CI) is the practice of frequently merging working copies early in the developing process. Among a wide variety of testing disciplines, performance can boost development process as part of CI.

Performance tests consume lots of memory, CPU and the machine resources in general. By itself, performance is expensive. But, you can do much for free. By using JMeter as the main performance tool and continuously integrating everything on Jenkins, great results can be accomplished.

How to set a performance test in Jenkins?

Apart from being the butler from Scooby Doo, Jenkins

• is an open-source automation server
• can be used to automate all sorts of testing-related tasks
• can be installed and run standalone with any machine that has the JRE installed
• supports version control tools (aka Git)
• has a functionality that can be extended with plugins
• builds/tests can be triggered by various means

As performance testers we want:

• our tests to run in the background without the need to keep an eye on them
• not to use our personal machine resources
• to run tests every day and have an automated process that can trigger the run
• presentable test results
• our tests to sync with Git or another version control tool
• to be notified if and when something goes wrong

Knowing what we want and who can help us, it just needs to be done.

Build jobs are the core of Jenkins and the Jenkins Pipeline job is crucial for executing performance tests and creating different kinds of reports.

Why Pipeline?

An important part of performance is how the system behaves with a different load (requests per second or number of concurrent users). Pipeline gives you an opportunity to stage different runs depending on your key performance indicators.

Plugins, you say?

Jenkins extends its functionality to plugins. When it comes to performance, there are two that can give you a great overview and display of test results. They are closely connected but they both add something new to the overall image. One of them is Performance Plugin and another one is HTML report.

With Performance plugin you will:

• get a performance trend for transactions/requests response time in every job
• be able to compere the current and the previous build using the metrics such as: mean, median, average, percentile and/or deviation
• get a display of all failed requests

With HTML report you will:

• Generate graphs that come as a part of JMeter (transactions per second, response time vs. number of threads, errors vs threads etc.)
• be able to see an application index for every request
• get a records of all the errors and an error percentage pie chart for every stage run

With these two combined, you will have enough data to get an overview of your application performance, write an outstanding report and track the progress.

Anything else?

With Jenkins plugins there is always something more:

• You can automatically send an email if there is any threshold indicator that is not satisfied (a large number of errors for example),
• You can integrate test results with test management tool (QTest, Zephyr or similar)

To wrap it all up

You will need some time to integrate JMeter scripts with Jenkins plugins and make them work together. It is not an effortless process but it offers great satisfaction in the end. It makes performance much easier without usage of any additional resources.