Contents ▼

Capturing Rules

With Loadster, you can define Capturing Rules to scan an HTTP response and capture string values to use later in your script.

A few examples of when a capturing rule might be necessary include:

  • A unique session ID that the web application/server sets in the URL or Location header
  • A CSRF token (many web frameworks use these to prevent cross-site request forgery)
  • A dynamically generated ID for a resource that your script creates in the application
  • An order confirmation number, receipt, or transaction ID

These values returned from the server may be different each time you play your script, and you have no way of knowing ahead of time what values may be returned.

Creating a Capturing Rule

To create a capturing rule, click on the Plus (+…) in an HTTP step to expand the step context menu.

Creating a capturing rule

Loadster can capture values in several different ways. Select which type of capturing rule you want to create.

Types of Capturing Rules

Loadster has several types of capturing rules: text capturers, regular expression capturers, and JavaScript capturers. A capturing rule can scan the entire response body, or a specific response header. It then stores the captured value in a variable of your choosing.

Text Capturers

Text capturers are the simplest type of capturer. They scan for a string nested between two boundary strings that you specify.

Example #1: Capturing from the Location Header

Many web applications make use of HTTP 301 or 302 redirects. When a status of 301 or 302 is returned, the application also sends a location in the Location header. The web browser (or other HTTP client) is supposed to navigate to the URL specified in the Location header, thus completing the redirect.

To capture text from a response header such as the Location header, select the “Capture text from header” option.

Capturing a redirect URL from the Location header

Then enter a Capture Variable, Header Name, and optional Text Before and/or Text After to create your capturing rule.

Capturing a UserID from the Location header

Capture Variable is the name of the variable that should be assigned the captured value. In this example we are capturing to a variable called UserID, which will be referenced later in the script as ${UserID}.

From Header is the name of the HTTP header you want to scan. In this case we are capturing from the Location header.

Text Before and Text After are optional. If they are specified, only text between them will be captured. In this example, we are capturing the UserID that occurs in a string like /welcome?userId=83829 so we enter /welcome?userId= in Text Before, and leave Text After blank.

Example #2: Capturing a CSRF Token in the Response Body

Many web applications use dynamic CSRF tokens to protect web forms from cross-site request forgery.

In a Loadster script, submitting a web form with a CSRF token will fail unless we capture the CSRF token and submit it with the form. Since the CSRF token changes every time, we must create a rule to capture it from the web form and then include it as a parameter when we submit the form.

To capture text from the response body, select “Capture text from body”.

Then, enter the Capture Variable, Text Before, and Text After within the response body. Here, we are capturing the value of a hidden HTML form input called _csrf into a variable called CSRF.

Capturing a CSRF Token from an HTML body

After capturing the variable, we can then submit the form with the variable ${CSRF} so that the correct value is submitted to the server each time.

Regular Expression Capturers

You can also capture variables using regular expressions.

A regular expression (or regex) is a sequence of characters that forms a search pattern. They are used in many programming languages for pattern matching and string manipulation. Loadster’s regular expressions are basically JavaScript-style regular expressions.

To create a regular expression capturer in Loadster, click on the Plus (+…) within the desired HTTP step.

As with other types of capturers, you can choose to apply the regular expression to the entire response body, or a specific header.

Example #1: Capturing an Order Number After Placing an Order

Let’s say we are scripting an ecommerce checkout flow, and the user has just submitted an order. A unique order number is generated each time, and we need to capture it.

To capture a regular expression from the response body, select the “Capture regular expression from body” option.

Capturing a system-generated order number with a regular expression

Capture Variable is the name of the variable that the captured value will be assigned to. Since we are capturing to a variable called OrderNumber, we will be able to reference it later in our script as ${OrderNumber}.

Regular Expression is the actual expression that will be applied to the response body. The numeric order number is represented by (\d+), which in regular expression parlance is a numeric sequence of one or more consecutive digits.

Output Expression lets us specify exactly which portion of the expression should be captured. Since we specified $1, we will only be capturing the contents of the first “capturing group”, nested in parentheses. In other words, we will only be capturing the part represented by (\d+).

Example #2: Capturing User Info from a Profile Page

Let’s say you want to use a regular expression to capture an email address from an HTML page. It occurs in the HTML in a format like this:

<a href="mailto:jeff@boomhauer.org">Jeff Boomhauer</a>

We can create a regular expression to capture the dynamic parts:

<a href="mailto:(.*?)">(.*?)</a>

Notice that this expression actually has two capturing groups. The first capturing group matches the email address, and the second capturing group matches the name. We can refer to these in the Output Expression as $1 and $2, respectively. If we only care about the email address itself, we would use $1.

Capturing a user's name and email address and rearranging them

If you want, you can use this same regular expression but change the Output Expression to capture different things:

$0                 <a href="jeff@boomhauer.org">Jeff Boomhauer</a>
$1                 jeff@boomhauer.org
$2                 Jeff Boomhauer
$2 <$1>            Jeff Boomhauer <jeff@boomhauer.org>

The value represented by the Output Expression is stored in your Capture Variable (in this case ${EmailAddress}), and can now be referenced later in your script.

More about Regular Expressions

Here are a few common patterns you can use in your regular expressions:

.      any character
\d     a single digit, 0-9
\w     a single "word" character, a-z, A-Z, 0-9
\s     a whitespace character, such as a space or a tab

You can use wildcards to represent multiple consecutive instances of the same thing:

\d*    zero or more consecutive digits
\w+    one or more consecutive characters
\w+?   one or more consecutive characters, non-greedy

Regular expressions are very powerful and we do not have space for an exhaustive primer here. There is an article at Mozilla that provides more background.

JavaScript Capturers

A JavaScript Capturer allows even more precision than the Text Capturer and Regular Expression Capturer. This type of capturing rule allows you to create a custom JavaScript function. Your custom function will then be executed on the HTTP response to capture a value.

To add a JavaScript Capturer to a step, select “Capture with JavaScript” from the step context menu.

To capture a value with JavaScript, you must implement a custom function called capture(). The function takes a single argument (the response object, which encapsulates the HTTP response coming back from the server). The output of your capture function will be assigned to whatever Capture Variable you specify.

Enter a Capture Variable and Capturing Function for the selected HTTP step. In the example below, the output will be assigned to a variable called UserID and can be used later in the script as ${UserID}.

Capturing a UserID with JavaScript

Example 1: The Simplest Capturing Function

In its simplest form, you could implement a “capture” function that ignores the response and returns a constant value every time.

function capture(response) {
    return "abc";
}

Or, instead of a hardcoded value, you could also generate a value on the fly using JavaScript’s Math.random(), a date or timestamp, or whatever you want. The only requirement is that the function returns a string or something that can be coerced to a string.

Example 2: Capturing HTTP Response Headers with JavaScript

JavaScript can also be used to scan the HTTP response headers and extract a value.

One common pattern in web applications is the “create and redirect” flow. After a web form is submitted, the application creates something in the database and redirects the user to a specific location to view the newly created resource. The redirect is accomplished by sending an HTTP 302 with a Location header that contains the URL the browser should visit next. From a tester’s standpoint, it is important to capture the URL from the Location header so it can be reused later in the script.

You can capture the Location header from a response like this:

function capture(response) {
    var headers = response.getHeaders();

    console.log("The response has " + headers.size() + " headers");

    for (var i = 0; i < headers.size(); i++) {
        var header = headers.get(i);

        if (header.name == "Location") {
            return header.value; // this is the one we want!
        }
    }

    return ""; // a Location header was not found
}

If you only need part of the Location header, you could parse it out with JavaScript’s regular expressions or substring functions and only return the part you need.

Capturing a simple header value is also possible using Loadster’s Text Capturer or Regular Expression Capturer.

Example 3: Accessing the HTTP response body with JavaScript

The response object also makes the entire response body available to your JavaScript functions. To capture the entire response body into a variable, it’s as simple as this:

function capture(response) {
    return response.bodyAsString; // capture the entire response body
}

More likely, you will want to perform some kind of string manipulation to extract only what you need from the body:

function capture(response) {
    var regex = /<title>(.*?)<\/title>/gim;
    var match = regex.exec(response.bodyAsString);

    // Return the match that was in parentheses
    return match[1];
}

While much of this is also possible with Loadster’s Text Capturer or Regular Expression Capturer, there are times when the response may contain multiple matches and you need custom logic to select the right one. There may also be occasions when the markup is too complex to parse with a single expression, and you need to write your own parser with JavaScript.

Example 4: Parsing a JSON Response with JavaScript

Many web apps return JSON responses. This is particularly true of RESTful web services and rich client applications that communicate with their back-end using JSON.

While it might be difficult to parse a JSON response with ordinary text or regular expression capturers, it is quite easy to do so using JavaScript:

// {"users": [
//      {"id": 1, "fullName": "Hank Hill"}, 
//      {"id": 2, "fullName": "Peggy Hill"}, 
//      {"id": 3, "fullName": "Bobby Hill"}
//  ]}

function capture(response) {
    var json = JSON.parse(response.bodyAsString);

    return json.users[2].fullName;
}

The example above parses a JSON response and then traverses it to capture a specific value (in this case, the full name of the 3rd user in the list, “Bobby Hill”). Naturally, you could also use loops and other JavaScript control flow if your JSON structure is more complicated.

Example 5: Parsing an XML Response with JavaScript

XML is also a popular response format for web applications and web services. SOAP and XML-RPC applications in particular use a lot of XML, along with some RESTful web services.

Parsing and traversing XML in Loadster is quite easy:

// <users>
//     <user id="1"><fullName>Hank Hill</fullName></user>
//     <user id="2"><fullName>Peggy Hill</fullName></user>
//     <user id="3"><fullName>Bobby Hill</fullName></user>
// </users>

function capture(response) {
    var users = XML.parse(response.bodyAsString);

    return users.childrenNamed("user")[2].childNamed('fullName').val; // returns "Bobby Hill"
}

The example above parses an XML response and then traverses it to capture the full name of the 3rd user in the list, again named “Bobby Hill”. To parse an attribute rather than an element, the @attrib notation is used:

// <users>
//     <user id="1"><fullName>Hank Hill</fullName></user>
//     <user id="2"><fullName>Peggy Hill</fullName></user>
//     <user id="3"><fullName>Bobby Hill</fullName></user>
// </users>
 
function capture(response) {
    var users = XML.parse(response.bodyAsString);
 
    return users.childrenNamed("user")[2].attr["id"];
}

You could also iterate each user using the length() method to find out how many there are:

// <users>
//     <user id="1"><fullName>Hank Hill</fullName></user>
//     <user id="2"><fullName>Peggy Hill</fullName></user>
//     <user id="3"><fullName>Bobby Hill</fullName></user>
// </users>
 
function capture(response) {
    var xml = XML.parse(response.bodyAsString);
    var users = xml.childrenNamed("user");

    for (var i = 0; i < users.length; i++) {
        var user = users[i];

        if (user.childNamed('fullName').val == "Bobby Hill") {
            return user.attr['id']; // That's him!
        }
    }

    return ""; // Bobby Hill not found!
}

Pro tip: Loadster’s XML parser is based on xmldoc, a pure JavaScript XML parser. More documentation on how to use this parser is available on GitHub.

Example 6: Capturing a Multi-line Value with JavaScript

Loadster’s ordinary text capturers only evaluate a single line at a time for efficiency reasons. If you need to capture a value that spans multiple lines, here is one way to do it:

function capture(response) {
	var str = response.bodyAsString;
	var lb = "<textarea name=\"description\">";
	var rb = "</textarea>";
	var startIndex = str.indexOf(lb) + lb.length;
	var endIndex = str.indexOf(rb, startIndex);

	return str.substring(startIndex, endIndex);
}

Using Captured Variables

Once you capture variables, you can reuse them later in your script with the usual ${var} syntax. Read more in Variables & Expressions.