Contents ▼

Validation Rules

Load testing is more than just sending a bunch of HTTP requests and measuring response times. To ensure the accuracy of your test, you also need to validate that the responses come back as expected.

Some types of errors are automatically detected by Loadster. Anytime your application (or the server it is running on) returns an HTTP 4xx or 5xx response code, Loadster will automatically treat it as an error. Socket timeouts, connection refused, and other network-layer errors are also automatically detected.

More subtle errors, especially those that are unique to your application, may not be automatically detected by Loadster. You can create Validation Rules to check for these. You can also use validation rules to check for your own specific requirements.

When to Use Validation Rules

Realistic examples of when you might need to use custom validation rules include:

  • Make certain a key page loads in less than 3.0 seconds.
  • Make certain the response includes the word “Welcome” when you submit the registration form.
  • Make certain the words “User not found” do not appear in the response when you log in.
  • Make certain a specific page is always more than 15kb and less than 25kb.

It’s a good idea to add validation rules to eliminate uncertainty in your test results.

Setting Up a Validation Rule

A validation rule can be added to a step by clicking on the “+…” Button. This will bring up the advanced options menu.

Adding validation rules to a step

There are four types of validation rules available for an HTTP step:

  • Content Validator
  • Response Time Validator
  • Response Size Validator
  • JavaScript Validator

You can add multiple validators of each type to an HTTP step, if needed.

Multiple validation rules on the same step

Response Time Validators

This type of rule validates the response time of the primary request, or the entire page (primary request plus resources).

You might have a requirement that a certain key page must always load within a certain time; say, 3 seconds. A response time validation rule is one way to enforce this. To set this type of validation rule for an HTTP step, select “Validate response time” from the step context menu.

Response time validation rules can optionally be applied to just the primary request, or the primary request plus any associated page resources. In the example below, the response time for the primary request must not exceed 3.0 seconds.

Validating response time including resources

If you don’t create specific response time validation rules, the step will only be considered an error if it has a socket timeout or returns an HTTP error code (4xx, 5xx).

Response Size Validator

This type of validator checks that the response body is within a certain size range.

Often, if an error happens on the server side, the response may be incomplete or show a short error page. By making certain the response body falls within an expected size range, you can be more confident that the response is being returned correctly.

To configure a validation rule based on the response size, select “Validate response size” from the step context menu.

Then enter a value for the Minimum and Maximum Size (in bytes). This information will be used to validate that the response is within the specified size range.

Validating response size of the primary request

In the example below, the Minimum Size has been set to 30000 bytes and the Maximum Size has been set to 50000 bytes.

Content Validator

A content validation rule can validate that the response body contains or does not contain a specified string.

To configure this type of validation rule, select “Validate content” from the step context menu.

Here, we make certain the desired string is showing up in the response body for our demo site product page.

Validating the response contains a certain string

Likewise, you might add another validation rule to make certain the body must NOT contain the words “Login Failed”, as shown below:

Validating the response does not contain an error message

Since many web applications return an HTTP success status code (such as 200) even if there is an error on the page, it is often a good idea to validate the actual page content instead of relying solely on HTTP status codes.

JavaScript Validator

Loadster’s JavaScript validator is the most flexible and powerful. It lets you create your own JavaScript function to parse and interpret the response.

To configure a validation rule with JavaScript, select “Validate with JavaScript” from the step context menu.

Now you will be able to write your own validation rule for the step using JavaScript.

Validating the response with a custom JavaScript function

Your validation function must be called validate and return a boolean value. If it returns true, the step passed validation. If it returns false, the step failed validation and an error will be recorded.

Example 1: Validating a Content-Type Header with JavaScript

HTTP response headers are used to convey information about the response. The Content-Type header is a common HTTP header that indicates what type of content is contained in the response.

To create a custom validator that checks for a Content-Type of “application/json”, create a function like this:

function validate(response) {
    var headers = response.getHeaders();
 
    console.log("The response has " + headers.size() + " headers");
 
    for (var i = 0; i < headers.size(); i++) {
        var header = headers.get(i);
 
        if (header.name == "Content-Type") {
            if (header.value == "application/json") {
                return true; // validation succeeds!
            }
        }
    }
 
    return false;
}

Example 2: Validating Body Content with a Regular Expression

Loadster’s Content Validator makes it easy to search for a fixed string in the response, but what about an expression that might be a bit different each time, like a tracking number that was generated by your application?

With a JavaScript validator you can search for a regular expression:

function validate(response) {
    var regex = /Confirmation #AWB(\d{8})Z/gm;
    var matched = regex.test(response.bodyAsString);
 
    return matched; // Validation is successful if the regex matched
}

Example 3: Validating Structure of a JSON or XML Response

Validating a JSON response in Loadster can be done with JavaScript, as follows:

// {"books": [
//      {"id": 4, "name": "Slaughterhouse Five"}, 
//      {"id": 11, "name": "The Old Man and the Sea"}, 
//      {"id": 25, "name": "Don Quixote"}
//  ]}
 
function validate(response) {
    var json = JSON.parse(response.bodyAsString);
 
    return json.books && json.books.length > 0; // Make sure at least one book was returned
}

Similarly, validating an XML document is just as easy:

// <cars>
//     <car id="13"><make>Volkswagen</make><model>Thing</model></car>
//     <car id="80"><make>Land Rover</make><model>Series IIA</model></car>
// </cars>
 
function validate(response) {
    var xml = XML.parse(response.bodyAsString);
    var cars = xml.childNamed('cars');
    
    // Validation is successful if there was at least 1 car
    return cars && cars.childrenNamed('car').length > 0; 
}

Pro tip: Loadster’s XML parser is based on xmldoc, a pure JavaScript XML parser. More documentation on how to use this parser is available on GitHub.

Testing Your Validation Rules

Before starting a real load test, you should run your script in the script editor to make sure it works properly and test the validation rules. Simply play the script as usual and watch the Playback Log located at the bottom of the screen.

Whether your validation rule passes or fails, you should see a message in the log. If validation fails, it will log an error message in red.

Validation errors appear in the Playback Log

You can use the Playback Details tab to find out information such as the response size and time to determine why the validation rule failed.

Drilling down into the response in the Playback Details

Drilling down on the step lets you see the exact response headers and body. If the validation rule fails consistently, you may need to adjust your script.

Viewing the actual response body to figure out why validation failed

Troubleshooting JavaScript Validation Errors

Errors from JavaScript Validators depend on your own custom functions. If you’re having trouble, it is often helpful to use console.log() in your validate function to print debug messages.

Validation Rules in a Load Test

Once you have added validation rules to your script, these same rules will be applied in a load test, for every virtual user running that script. Any validators that fail will raise an error in the test dashboard.

To learn more about error reporting during a load test, please see Running Load Tests.