Contents ▼

Testing with Loadster Engine

If you’re load testing on a private network that’s not reachable from Loadster’s cloud regions, you will need to host your own engines to actually generate the load. Don’t worry, this is pretty easy and it’s included in your plan!

Download Loadster Engine for Windows & Mac →

How It Works

Most of the time, an engine just sits there quietly and waits for instructions to run a test. When you launch a test from your Loadster Dashboard or from Loadster Workbench, the engine comes to life, spinning up virtual users and generating load on your site.

As the test runs, the engine streams back lots of data which Loadster compiles into test results and displays in the live charts on your test dashboard.

Installing Loadster Engine

Loadster Engine is built for Windows, Mac, and Linux/Docker.

Running load tests can hog quite a bit of CPU, so we recommend you install the engine on a machine with at least 2 CPU cores, and preferably more. You should also have at least 2 GB of RAM free. A fast network connection is also essential. Obviously, the more powerful a machine you run the engine on, the more v-users you will be able to reliably simulate with your self-hosted engine.

Windows

Loadster Engine for Windows has an MSI installer. Download and install it like any other Windows application. After installation, simply run the Loadster Engine from the Start menu.

Get Loadster Engine for Windows →

Mac

Loadster Engine for Mac version is a standard DMG disk image. Download and open the disk image. After mounting the image, drag the Loadster Engine application to your Applications folder and double-click it to run the engine.

Get Loadster Engine for Mac →

Linux/Docker

Loadster Engine for Linux is packaged in a Docker image. You can also run this Docker image from Windows or Mac if you have Docker and prefer to run it that way.

$ docker pull quay.io/loadster/loadster-engine
$ docker run -t -p1998:1998 --env ENGINE_KEY=$ENGINE_KEY quay.io/loadster/loadster-engine

You can obtain an $ENGINE_KEY from the Settings page of your dashboard, which is necessary for identifying the engine with Loadster’s cloud services. If you run multiple engines, don’t use the same key for each!

Exposing port 1998 with -p1998:1998 is only necessary if you want to control the engine from Loadster Workbench.

Registering Loadster Engine for Windows & Mac

Before you can run tests with your engine, you’ll have to register it so that Loadster knows about it.

Each self-hosted Loadster Engine requires a unique Engine Key so it can communicate with Loadster’s cloud services. There is no extra cost for creating keys, becauseĀ self-hosted engines are included in your Loadster plan.

First, go to Settings to create your engine keys.

Then, when you run your engine locally, enable Connect to the Loadster API. You’ll be prompted to enter the engine key. Enter it.

Enter a valid key to uniquely identify this engine

After entering a key, your engine will connect to the Loadster API and start waiting for instructions.

Your engine is ready to use!

Be careful not to use the same key on multiple engines or confusion will ensue! If you run multiple self-hosted engines, generate a unique key for each of them.

Running Tests With Loadster Engine

Once you’ve installed an engine, using it is very simple. In your test scenario, just select the engine for each of your virtual user groups you want to run on it.

Select one of your self-hosted engines when running a test

Now, when you start your test, Loadster will send instructions to that engine (via the Engine Key) to generate the load.

Tuning Your Operating System

Load testing is resource intensive, and Loadster Engine is a resource intensive application. It tries hard to max out the available resources, particularly CPU and network sockets. While the default settings for your OS may be fine for most things, and even for small to moderate load tests, there may be a time when you need to push your systems harder than the default settings will allow.

Tuning Windows Systems

The main resource constraint Windows suffers from is a limited range of ephemeral ports.

Loadster Engine uses ephemeral ports temporarily, to bind a socket for an outbound connection to the application or website you are testing. In a load test, it’s not uncommon to go through thousands of ephemeral ports per minute.

By default (as far as we can tell), most Windows systems only allow ports from 1024 to 5000 to be used for this purpose. Furthermore, it can take up to 4 minutes after a socket is closed before the port is recycled. Thus, if you run more than 1000 requests per minute for a sustained period, it is possible to run out of ephemeral ports.

When this happens, Loadster Engine sometimes reports errors like address already in use: connect that indicate it cannot open a port.

Increasing the Ephemeral Port Range

Modern versions of Windows have a Registry key that tells what port range to allow for user processes. By default, the range is quite small. You can add or update a Registry key to modify the range.

Adding a DWORD value to the Windows registry

To update the Registry key:

  • Run regedit.exe, as Administrator.
  • Drill down to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip and right-click Parameters.
  • Add a DWORD (32-bit) value. Enter the name MaxUserPort, with the decimal value of 65534.
  • After adding the new key, restart your system.

When your system starts up again, Windows will have a much wider range of ports to use for load testing.

Changing the TIME_WAIT Window

After a TCP connection is closed, it goes into TIME_WAIT status for a while before the OS is able to reuse the port. When running load tests, it is possible for all available outbound ports to become tied up. To avoid this, you can tell Windows to recycle the connections more aggressively.

  • Run regedit.exe, as Administrator.
  • Drill down to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip and right-click Parameters.
  • Add or update the TcpTimedWaitDelay property and give it a decimal value of 30.
  • Restart your system.

Once your system starts up again, Windows will only leave connections in TIME_WAIT status for 30 seconds before recycling them, instead of 4 minutes.

Tuning Mac OS X Systems

This section is coming soon. If you are having issues running large load tests on Loadster Engine for Mac, please contact us and we will work with you one-on-one.

Tuning Linux Systems

In our experience, most Linux distributions have default network settings that are fairly generous and sometimes not much configuration is needed unless you are really pushing the limits. However, there are a few parameters that sometimes need to be adjusted.

Maximum Open File Descriptors

One of the important settings for Linux systems processing a large amount of throughput is file-max. This is the kernel’s limit of how many file descriptors can be open at one time. File descriptors are used for network sockets as well as local file reading/writing, along with a few other things.

You can determine the current limit on your Linux machine with the following:

# cat /proc/sys/fs/file-max

We suggest at least 100000 as a general guideline for load testing. To set it temporarily, run (as root):

# echo "100000" > /proc/sys/fs/file-max

Setting it permanently depends on your Linux variant, but can usually be done by editing /etc/sysctl.conf.

Ephemeral Port Range

In a load test, the Loadster Engine opens and closes many outbound HTTP connections. Each time it opens a new connection, it uses a different ephemeral port. After the connection is closed, the Linux kernel may take a little while before it can reuse that same port. Most modern Linux variants have a reasonably large range of ports from which to choose from, but if you find yourself getting errors like “address already in use”, you may need to adjust the local port range.

To determine the current local port range, run:

# cat /proc/sys/net/ipv4/ip_local_port_range

Newer Linux systems usually have a default range of 32768 to 61000, meaning that any port number in this range can be used for outbound connections.

In most cases, you can safely change the lower bound to 10000 and the upper bound to 65000, but that really depends on what other services you are running on the machine and what other ports might be in use.

To adjust the local port range, run (as root):

# echo "10000 65000" > /proc/sys/net/ipv4/ip_local_port_range

Setting it permanently depends on your Linux variant, but can usually be done by editing /etc/sysctl.conf.