Contents ▼

Testing with Loadster Workbench

Loadster Workbench is a desktop application for Mac and Windows.

Originally, Loadster Workbench was our main solution, but now most of our customers prefer to do everything from the browser. We still support Loadster Workbench for our customers who enjoy a native desktop experience.

Loadster Workbench is equally suited to cloud testing or on-premises testing. For smaller tests (up to 50 v-users), Loadster Workbench can generate the load itself. For larger tests, it can either control your self-hosted engines, or it can quickly launch cloud engines all over the world.

Getting Started

When you first run Loadster Workbench, you’ll be prompted to sign in or create a Loadster account. This is essential because your Loadster account has your plan and Loadster Fuel.

If you already have a Loadster account, choose “Use Existing Account”. If not, choose “Create Free Account” and create one.

Project Repository

Your Loadster projects are stored in a repository. By default, the repository is hosted on our cloud services, so that every member of your team running Loadster can access the same projects.

Example of the Project Repository

Projects

A Loadster project is simply a container for related scripts, scenarios, and datasets. To create a new project, choose New Project… from the File menu.

Scripts

A Loadster script represents a path of a single user through your site or application. It is a sequence of HTTP steps and wait steps. In Loadster Workbench, you can record scripts or manually create them. You can also play them back with a single user, and edit them until they accurately simulate user behavior. Depending on the site, application, or API you are testing, your script may be very simple or it may be fairly complex.

Scenarios

A Loadster scenario is a recipe for a load test. It specifies how many v-users should be running each of your scripts, for how long, and how the load should ramp up and down. You can create and edit the scenario in Loadster Workbench, and then run the load test to see how your site performs.

Datasets

A Loadster dataset is a collection of unique data to use in your test. If you’re load testing a dynamic webapp or API, you should probably use dynamic datasets in order to avoid caching or other factors that might lead to inaccurate results. For example, you could simulate hundreds of unique users, by putting a collection of usernames and passwords into a dataset and referencing it in your script.

Importing and Exporting Projects

If you ever need to create a backup of a Loadster project or share it with someone outside your Loadster team, you can export it to a zip file and then reimport it.

To export a project, select it in the repository and choose Export… from the File menu or right-click it for the context menu. The project will be saved to a zip file on your local machine.

To import a project, choose Import… from the File menu and locate the file.

Creating Scripts

Scripts are a series of steps to be executed by a virtual user. You can view and modify them in the Script Editor.

If you’re just getting started, you can record a script from your web browser, or manually create steps.

Creating Steps Manually

You can manually add a step to your script by clicking on the relevant step type.

An HTTP step makes an HTTP request to a server. A Wait step causes the virtual user to wait for a specified amount of time before continuing with the script. We’ll talk more about each of these soon.

Removing Steps

You can also remove one or more steps from your script. Simply click once on the step to highlight it, and then use the Delete key on your keyboard or the corresponding button in the Script Editor.

If you delete a step by mistake, you can bring it back with Undo.

Reordering Steps

Reordering steps is as simple as selecting one or more contiguous steps you want to move, and dragging them to their new location in the script.

When testing dynamic applications, the order of steps is often important.

Copying and Duplicating Steps

Predictably, to copy a step, simply select it and do the usual Copy and Paste function.

You can also duplicate the selected steps with the Duplicate option in the editor.

Wait Steps

Wait Steps are simple. They simply tell the virtual user to pause execution of the script for the specified number of seconds.

Having realistic wait times is important! You should try to make your scripts behave much like a real user would. If your wait times are unrealistic, your load test results will be too.

While working in the Script Editor, however, you can Fast Play your script to bypass the wait times.

HTTP Steps

HTTP Steps are a bit more complicated. They represent an HTTP request to the server, much like your user’s browser or client would make. When many virtual users run your script at the same time, it is these concurrent HTTP requests that put a load on your server.

HTTP Method

The HTTP Method (or verb) is a standard part of every HTTP request. The commonly used methods are GET and POST, but many apps (especially REST APIs) make use of others such as PUT and DELETE also.

If you’ve recorded your script, you will rarely need to change the method of a step. However, if you’re creating steps manually, it is important that they have the correct method.

Request URL

The Request URL is critical to each request. These will look rather familiar.

Edit the URL inline by clicking on it. If you’re editing query string parameters, make sure they are URL encoded!

Request Body

The Request Body is an optional part of certain types of HTTP requests. Traditionally, POST and PUT requests tend to have bodies, while other types do not.

Loadster supports both text and binary request bodies.

If the body is a typical web form, you can edit it as key-value pairs with Edit as form….

If it’s a typical web form or some other text format, you can view and edit the raw body with Edit as text….

For any content type, including binary, you can replace the body directly from a file with Replace from file….

It is important that the Content Type of your request body is correct. Loadster sends this so that the server knows what kind of content to expect. The most common type for a web form is application/x-www-form-urlencoded. However, some forms (such as file upload forms) typically send multipart/form-data. REST APIs frequently expect application/json, application/xml or text/xml.

Last of all, take notice of the Interpolate variables in request body option. This setting determines whether any string in the body that matches Loadster’s ${variable} syntax should be interpolated into a value. Unless you are knowingly using variables in your script, it is safest to disable this option. Learn more about variable interpolation in Variables and Expressions.

Request Headers

Request Headers are sent with every HTTP request. Most of the time, Loadster sends generic default headers for you. In some cases, you may need to specify one yourself.

Page Resources

Loadster can optionally request additional Page Resources with a primary request. If your step includes additional resources, they will be requested in parallel after the main request completes, much like a web browser would load an HTML page and then load the associated images, CSS, JS, etc.

Loadster’s reported response time for this step will include the time it takes to load these resources.

Page Resources only make sense if the request is for a web page. You should not use them when testing an API.

Basic Authentication

One of the ways some websites and APIs authenticate requests is with HTTP Basic Authentication. While you could also use a custom request header to pass this information with each request, Loadster also has a shortcut.

If you make use of Basic Authentication, the username and password will need to be set on all HTTP steps that require authentication.

Actually, Basic Authentication is not all that common nowadays; it is more likely you will be passing the credentials with an HTTP POST form submission.

Validation Rules

Validation Rules are a way to assert that the server responds to a step the way you want it to. You can add one or more of these to any HTTP step.

Read all about them in Validation Rules.

Capturing Rules

Capturing Rules are a way to capture data from a step’s response, so you can reuse that value later in your script.

Read all about them in Capturing Rules.

Recording From Your Web Browser

Recording a script from your web browser is a great way to get started. Just use your website (or app) in exactly the same way your end users do, and Loadster will record and replay every step you take. Don’t worry, you can still customize the script after you’re done recording.

Before You Start Recording

There are two ways to record scripts in Loadster: the Chrome extension and the recording web proxy.

You’ll only need to set up one of these to record a script. If you’ve already done this, read on.

Start Recording

Once you’ve enabled the Loadster Recorder Chrome Extension or pointed your browser at Loadster’s recording proxy, you’re ready to record a script.

Just click the Record button (which looks like a familiar red circle).

Click the red record button to record a script

Recording In Progress

During recording, Loadster Workbench displays a recording log that shows all the events it observes. These events can come from the Loadster Recorder Chrome extension, or from the HTTP(S) recording proxy.

Recorded traffic appears in the recording log

When you’ve recorded all the activities you want in your script, stop the recording.

Loadster’s recorder doesn’t discriminate between traffic on your primary domain and 3rd party domains… it records everything it sees.

Filtering Recorded Events

If the recorder captures events that you don’t want in your script, you can uncheck them. Only the traffic from selected domains will be added to your script.

Filter the recorded traffic to only include resources from the slothereum.net domain

Usually, it’s a good idea to only include the site(s) you are load testing in your script. Hitting 3rd parties (such as Facebook, Google, etc) with a load test generates needless traffic and introduces uncertainty. However, there are some situations when it makes sense to include them in your script (for example, when load testing a site with lots of static content served by a CDN).

Finishing the Recording

Once you’ve selected the domain(s) you want to include in your script, click Done Recording to finalize it.

Loadster automatically analyzes the recorded traffic to add steps to your test script.

After recording, the steps appear in your script

If the requests look good, recording was successful.

Running Tests

Loadster Workbench has a built-in load engine that can run smaller tests up to 50 v-users. For larger tests, it can communicate with any of the 15 Loadster cloud regions (for publicly reachable sites) or your own self-hosted Loadster Engine (for sites on private networks).

Running Tests With Loadster Cloud

In Loadster Workbench, running tests with the cloud regions is very simple. In the scenario editor, just select a cloud region for each of your v-user groups.

Choose a cloud region where your virtual users should run

rkb

When you run the test, Loadster’s cloud services will automatically spin up a cluster of cloud instances in that region. The instances are fully managed by Loadster and you do not need to pay separately for them.

Running Tests With Loadster Engine

Once your engine is installed and running, Loadster Workbench needs to know about it before you can use it for load testing.

In Loadster Workbench, open the Preferences dialog and go to the Load Engines tab.

Adding a load engine to Loadster Workbench

Click the + button to add your engine. Then enter a Name (anything is fine), Hostname (or IP address), and Port for the load engine and then click OK.

Adding a load engine to Loadster Workbench

The engine will now be available to Loadster Workbench and you can select it for any virtual user group in any of your load test scenarios.

Once configured, Loadster Workbench will display the engine in the sidebar, along with a rough indicator of the engine’s resource utilization.

Loadster Workbench indicates how busy your engines are

When running a test, make sure to select an engine that is not over-utilized. Testing with an overworked engine can lead to inaccurate measurements!

Preferences

Loadster Workbench has many preferences that affect script recording, playback, managing your dedicated Loadster Engines, and enabling optional integrations with other apps and services.

You can find the Preferences in the Window menu on Windows, or the Loadster Workbench menu on the Mac.

General

The General tab is where you can change your repository type and location.

Project Repository

The repository is where Loadster stores your scripts, scenarios, test results, etc.

Select Store projects in the cloud to store them in Loadster’s cloud services. When you store them in the cloud, other team members can share your repository after you invite them to your Loadster team.

Select Store projects locally to store them on your local filesystem. You can specify a folder as the Location. There is no automatic project sharing available with this option.

Global System Proxy

If your organization requires that you use an outbound proxy for internet access, you can configure a Global System Proxy in Loadster Workbench.

To set up a proxy, you must first click on the checkbox for the Use a proxy to access the internet option. Then enter the following:

  • Proxy Host - The host name of the proxy server.
  • Proxy Port - The port number used for the proxy.
  • Username - The username for logging into the proxy server.
  • Password - The password for logging into the proxy server.
  • No Proxy For - A list of hosts that should be reached directly, bypassing the proxy.

You might need to get some of this info from your network admin.

If you enable a proxy, Loadster will use it for communicating with the cloud services and remote load engines, including Loadster Cloud. Actual load test traffic is not sent through the proxy.

Recording

Loadster Recording Proxy

Loadster Workbench includes a built-in web proxy for the sole purpose of recording scripts (read more about this in Recording Scripts). By default it listens on port 1997, but you can change the port.

Security

By default, when Loadster’s built-in recording proxy encounters a self-signed or otherwise untrusted SSL/TLS certificate, it raises an error and refuses to connect. If you are recording a script for a site with an untrusted certificate and want to trust it anyway, you can Trust all SSL certificates during recording. This option will not affect script playback or any other feature in Loadster, only the recording proxy.

Playback

Network Defaults

HTTP Version

Just kidding, HTTP Version is not really configurable at this time. In the near future Loadster will support HTTP 2.0.

HTTP User Agent

The HTTP User Agent is what Loadster sends in the User-Agent header with every request. In general, it only matters if you are testing a site or application that inspects this header. For example, some applications inspect this header to serve a different version for Firefox and Internet Explorer, but this has become less common nowadays as browsers have become more standardized.

Resource Timeout (ms)

For our purposes here, a Resource is anything that your script loads from an HTTP URL. It could be the primary request from the step, or any of the included page resources for a step. The Resource Timeout is the maximum number of milliseconds that Loadster will wait to download something.

Page Timeout (ms)

The Page Timeout is the maximum amount of time that Loadster will wait to download a page and its included page resources, if any.

Threads per V-User

The Threads per V-User tells how many parallel threads Loadster will use to download included page resources.

The primary request for a step always happens in the virtual user’s primary thread. After that, if there are page resources, they will be downloaded in parallel by these resource threads. This is similar to how full-featured web browsers download resources.

Security

This section is specifically around security during script playback and when running a test.

Trust all SSL certificates during playback

By default, when a virtual user encounters a self-signed or otherwise untrusted SSL/TLS certificate, it raises an error and refuses to connect. If you want to load test against sites with self-signed or untrusted certificates, you can Trust all SSL certificates during playback.

Error Handling

These options control what Loadster should do when an error is encountered during script playback or when running a test.

Halt script execution on any error

Normally, if a virtual user encounters an error (like an HTTP 4xx/5xx or a socket timeout, for example) it will report the error and then move on to the next step in the script. Halt script execution on any error will change that behavior, so that the iteration ends immediately if any error is encountered, and the virtual user restarts the script in a new iteration.

Log response details on validation errors

The built-in engine in Loadster Workbench and the dedicated Loadster Engine both have the ability to log complete response details to their log file if there’s an error. For this to be useful, you need to have access to the engine’s local filesystem so you can retrieve the logs.

A dedicated Loadster Engine writes its logs to the following locations:

  • Windows: C:\Users\<user>\AppData\Local\Loadster\Loadster Engine\Data\logs
  • Mac: /Users/<user>/Library/Application Support/Loadster/Loadster Engine/logs
  • Linux: ~/.loadster_engine/Data/logs

The built-in engine in Loadster Workbench writes its logs to the following locations:

  • Windows: C:\Users\<user>\AppData\Local\Loadster\Loadster Workbench\engine\logs
  • Mac: /Users/<user>/Library/Application Support/Loadster/Loadster Workbench/engine/logs

Checking this option enables detailed logging every time a validation error happens during a test.

Log response details on HTTP errors

Similar to above, if this feature is enabled, any HTTP 4xx/5xx errors will be logged to the engine’s log file.

Checking this option enables detailed logging every time an HTTP errthoughor code is received during a test.

Load Engines

Dedicated Engines

This section is only important if you are running a self-hosted Loadster Engine. If you are only doing cloud testing with Loadster Cloud, you can ignore it for now.

If you’ve already installed Loadster Engine and want to use it from Loadster Workbench, you can add it here by clicking the + button. Enter a Name (anything is fine), Hostname (or IP address), and Port for the load engine and then click OK.

The engine will now be available to Loadster Workbench and you can select it for any virtual user group in your scenario.

Cloud Engines

Loadster Cloud allows on-demand cloud testing without needing to install any engines. It is enabled for you as part of your Loadster account.

Integrations

Loadster Workbench API

Even though it is a desktop application, Loadster Workbench has its own built-in API. You can use this to trigger a test programmatically, extract test results as JSON, and more.

For example, extracting a JSON test report can be accomplished by:

$ curl -H 'Accept: application/json' \
       'http://localhost:1999/projects/00001/scenarios/00002/tests/00001/report?apiKey=xxx'

The apiKey must match the API Key setting in this dialog. By default, it’s a randomly generated string unique to your environment. To change it, simply enter the new key in Loadster Workbench.

This API is included for the benefit of our customers with particular integration needs, such as test scheduling or continuous delivery. Please drop a line to support@loadster.app so we can help you with what endpoints to use.