A heavy load#
As mentioned in my previous post , I recently had the pleasure of encountering a particular performance issue in a Google App Engine application that prompted us to enhance our testing practices when it came to high traffic scenarios. We have since made the enhancements thanks to all the additional logging we added to the problematic APIs, but in order for us to better gauge the improvements in the changes we made, we needed to be able to easily simulate such scenarios.
So, we decided to add some load testing to our development process. For some additional context, we already have our development environment set up to mimic our production environment as closely as possible, App Engine deployment with autoscaling. All we needed now was to get the actual testing in place. But in order to do that, we first needed to find a tool that would meet our requirements:
- Simulate multiple concurrent users hitting our backend APIs
- Generate reports on the performance of said APIs
- Set thresholds for acceptable performance
- Be easy to set up and use
There are a lot of really popular and well-documented tools that meet these simple requirements, like Locust , however, since our team was a lot more familiar with JavaScript, we decided to go with Grafana K6 instead. And I cannot recommend it enough.
K9 (-3)#
$ k6 run script.js
/\ Grafana /‾‾/
/\ / \ |\ __ / /
/ \/ \ | |/ / / ‾‾\
/ \ | ( | (‾) |
/ __________ \ |_|\_\ \_____/
execution: local
script: http_get.js
output: -
scenarios: (100.00%) 1 scenario, 1 max VUs, 10m30s max duration (incl. graceful stop):
* default: 1 iterations for each of 1 VUs (maxDuration: 10m0s, gracefulStop: 30s)
█ TOTAL RESULTS
HTTP
http_req_duration.......................................................: avg=117.55ms min=117.55ms med=117.55ms max=117.55ms p(90)=117.55ms p(95)=117.55ms
{ expected_response:true }............................................: avg=117.55ms min=117.55ms med=117.55ms max=117.55ms p(90)=117.55ms p(95)=117.55ms
http_req_failed.........................................................: 0.00% 0 out of 1
http_reqs...............................................................: 1 2.768749/s
EXECUTION
iteration_duration......................................................: avg=361.09ms min=361.09ms med=361.09ms max=361.09ms p(90)=361.09ms p(95)=361.09ms
iterations..............................................................: 1 2.768749/s
NETWORK
data_received...........................................................: 6.8 kB 19 kB/s
data_sent...............................................................: 541 B 1.5 kB/s
running (00m03.8s), 0/1 VUs, 1 complete and 0 interrupted iterations
default ✓ [======================================] 1 VUs 00m03.8s/10m0s 1/1 iters, 1 per VU
We will get to the code later. But above is a quick example of what an example output of a K6 test looks like (you are gonna have to scroll to the side a bit to see the numbers).
As you can see, there is a lot of useful information here. You get to see the scenario you set up, along with a bunch of stats about the http requests that were made. For example:
- You can compare the median and average response times with the P90 to see if the majority of users are getting the expected response time.
- You can see the number of requests that failed when your backend is exposed to a certain amount of traffic.
- You can set up thresholds for acceptable performance, and K6 will fail the test if those thresholds are not met.
- The list goes on.
There’s a whole section of their documents dedicated to the Metrics that K6 provides, and even how to make custom metrics in their documentation here , so do check it out if you are interested in that.
If you have never done load testing before, you will be very surprised to see how little traffic it would take to bring your backend down if you are not prepared for it.
Imagine you are writing a new feature to be able to generate an aggregated report based on customer transactions in the backend for an online store. You would probably implement the feature, run it in your localhost, test the API out a few times to make sure input validation works and you get the expected result, and call it a day. But what if someone performs the report generation during peak hours? Black Friday sales? You would be trying to access data from hundreds of records to generate the report, while traffic is at its highest. What took maybe just over a second to generate on your localhost may now take tens of seconds, all while your backend is trying to handle all the checkouts. What if checkouts start failing? And it is at this point in time that you realize that maybe you should re-think your approach to how you want your backend to generate reports.
Making the load#
Now that the introductions are out of the way, let’s look at how K6 works. K6 operates on the concept of Virtual Users (VUs). These VUs are simulated users that will hit your backend APIs concurrently.
All you need to do is define the following:
- Define a setup function (optional) to prepare any data you need for the test.
- Define the configuration options for the test. This is where you set the number of VUs, duration of the test, and any thresholds you want to set.
- Define the main function that will be executed for each VU. This is where you will make the actual HTTP requests to your backend APIs.
Options#
For a better flow, we will start with the configuration first.
const thresholds = {
http_req_duration: ["p(95)<500"], // 95% of requests should be below 500ms
http_req_failed: ["rate<0.01"], // Less than 1% of requests should fail
};
const shortRunOptions = {
stages: [
{ duration: "30s", target: 20 }, // Ramp up to 20 users over 30 seconds
{ duration: "1m", target: 20 }, // Stay at 20 users for 1 minute
{ duration: "30s", target: 0 }, // Ramp down to 0 users over 30 seconds
],
thresholds,
};
Like the comments in the code snippet shows, we essentially configured the test to:
- Ramp up to 20 VUs for the first 30 seconds.
- Then, Stay at the 20 VUs for 1 minute.
- Then, ramp back down to 0 VUs over the next 30 seconds.
- We also set thresholds for the test. We want 95% of the requests to be responded to in less than 500ms, and we want less than 1% of requests to fail.
It’s really intuitive to set up, and you can have multiple scenarios like the one above set up and reuse them wherever you need.
The example above is a very simple configuration. It can get a lot more complicated and there are a lot more options that you can set. More details can be found in their docs .
Setup#
Now, we move onto the setup function.
export function setup() {
const token = getAuthToken(hostname, authUser, authPassword);
return { token };
}
Here, all we are doing is just getting an authentication token that we will be using for all of the VUs. Again, this is optional. You can easily set this up in the default function so that the auth process is repeated for each request. We did it this way because we wanted to really target the performance of a particular API that gets performed multiple times in sequence per session, so we did not want the auth process to be included.
Default function#
This is the final part of our K6 script, the default function.
export default function (data) {
// Pick a random query for each iteration
const query = getRandomFromQueries(someQueries);
const url = `${hostname}/v1/${endpoint}?${query}`;
const params = {
headers: {
Authorization: `Bearer ${data.token}`,
},
};
const res = http.get(url, params);
check(res, { 200: (r) => r.status === 200 });
sleep(1); // Simulate user think time
}
The above is quite literally all it would take to actually call the API. It is no different from any other API call you would make on the frontend. And with just these lines of code, we can easily start our testing.
Now we execute our test with k6 run {filename} and you will see your load tests running in the terminal.
Loading complete#
This is just a bare-bones implementation of the minimum you would need to start load testing your backend with K6. Everything above, coupled with the logs we added was more than enough for us to see the performance improvements we made to our backend APIs. Like all other tools in the software development world, you can get more details regarding running and testing with K6 in their documentation .
One thing I am definitely more interested in looking at is the test lifecycle and all the fun ways I could test my backend applications in the future.
