Changelog

Monthly Fuel Plans

We’ve reintroduced Monthly Fuel Plans for our customers who load test fairly regularly and would like to get Loadster Fuel every month at a discount. Monthly subscription fuel works exactly like Pay As You Go Fuel, with the one caveat that if you eventually cancel your subscription and have unused fuel from it, the unused fuel will expire when the subscription ends.

Pay As You Go fuel pricing is staying exactly the same, and we’re pleased to offer Monthly Fuel Plans to lower the cost for our customers with longer-term testing needs.

Other Improvements & Bug Fixes

• Further improved the print stylesheet for test reports, so it prints or saves to a PDF more nicely.
• Added step numbers to the script preview in a load test report.
• Fixed a bug that prevented navigation after re-launching a test from the report view.
• Modified an API endpoint to force JSON output when invoked after running a test from the Loadster CLI.
• Fixed a bug that caused the first 1-2 seconds of stats in a load test to sometimes be omitted from test reports.
• Improved test advice when engines had high CPU utilization during a load test, to point out that this can sometimes cause errors or skew measurements.
• Adjusted the way we compute overall load test response time percentile approximations on the graph, for a more accurate average in most cases.

Project Page Improvements

The project page now shows scripts, datasets, scenarios, and test runs in separate tabs. We’re no longer cramming everything onto a single page, so each artifact now shows as a tile with more information than before.

You can also search, filter, and sort project artifacts so it’s easier to find what you are looking for in a project.

Other Improvements & Bug Fixes

• Fixed a bug related to time series memory compaction.
• Fixed a bug that caused a customer’s Loadster invoices to show in Speedway after switching apps via the app switcher.
• Improved decimal rounding in the Total Time Spent table.
• Added support for 24-hour time on the graph axes.

Loadster CLI 2.0

The Loadster CLI now supports script playing from external JavaScript files, as well as from existing scripts that you’ve edited in your Loadster dashboard.

$ loadster play --file ./examples/my-protocol-script.js
$ loadster play --file ./examples/my-browser-script.js --type=browser

The CLI can also run load tests using a local script.

$ loadster start --script=./my-browser-script \
        --type=browser \
        --bots=100 \
        --location="Europe - London" \
        --ramp-up-minutes=3 \
        --ramp-up-pattern=aggressive \
        --peak-minutes=3 \
        --ramp-down-minutes=3 \
        --ramp-down-pattern=linear

Building your Loadster scripts as external .js files is helpful if you’d like to use your own version control repository to store scripts and keep track of changes outside of Loadster, or if you prefer to edit scripts in your own text editor instead of editing them in your browser.

When you play scripts or launch load tests with the Loadster CLI, the Loadster platform takes care of launching cloud instances and aggregating the results, so you can view test reports and share them with your team just like you do when testing from the dashboard.

The Loadster CLI is also the easiest way to add load testing to your continuous integration process.

Bandwidth Throttling for Browser Bots

Browser Bots now support simulating slower network connections, like Protocol Bots have been able to do all along. Simulating a connection speed is helpful to measure the impact of slower network connections on your site’s response times and scalability.

To simulate a slower network connection, expand the gear menu on a bot group in your scenario, select “Simulate network connections” from the gear menu. Each bot in the load test will have network connections throttled to simulate the connection speed you choose.

Full Page Screenshots

Full page screenshots are now the default when you’re playing a script in the editor. This will make debugging scripts easier in certain situations where something important is on the page but outside the current viewport.

Other Improvements & Bug Fixes

• Fixed a bug that prevented file uploads in a browser script from saving correctly.
• Fixed a recent UI issue that caused script steps to temporarily vanish when you started playing a script in the editor.
• Fixed a bug that prevented account deletion through the dashboard in some cases, when an free Speedway monitoring subscription was active.
• Adjusted a few API rate limits.

Browser Console Logs in Traces

Browser Bot traces now include logs from the bot’s browser console. This can sometimes be helpful if a page didn’t load correctly due to an error on the page.

Other Improvements & Bug Fixes

• Improved cache behavior in the Chrome browsers used by Browser Bots.
• Added a filter to the Total Time Spent table to calculate aggregates (total, average, and request count) based on a subset of data.
• Adjusted the automatic test advice section to reduce confusion between functional and performance errors.
• Addressed a global state issue that sometimes caused old versions of scripts to persist in the UI if someone else on your team had modified them.
• Fixed a bug with trace filtering treating blank values as matches instead of non-matches.
• Improved the progress indicator so it doesn’t block navigation to other pages.
• Cleaned up live test sidebar text wrapping.

Trace Multi-Filtering

You can now filter traces on a live test or test report by multiple dimensions. For example, you can filter where the trace level is ERROR and where the message matches a certain substring.

You can also toggle the match type for each filter between exact match and partial match.

Traces are useful for diagnosing problems with a load test or ensuring that the script was running as planned. Clicking on a trace in the table pops open more details such as the bot logs, full requests and responses, screenshots, resource waterfalls, and error details.

Other Improvements & Bug Fixes

• Fixed a timezone issue in the activity log that sometimes caused events to appear on the wrong day in certain timezones.
• Fixed a bug that sometimes prevented updating or deleting existing integrations.
• Fixed a rare bug that caused slightly fewer bots to be launched than intended, due to a cluster rounding error.
• Added an error message so it’s apparent when a WebSocket connection fails due to network problems, typically during script playback.
• Updated the Slack integration to differentiate between Loadster and Speedway messages.
• Upgraded Chrome browsers to address instability with certain sites.
• Fixed a bug that caused a test’s Terminate button to be unclickable immediately after clicking the Stop button.

Better Slack Messages

Loadster’s Slack integration now posts a message to your destination channel when a load test finishes, with a few high-level metrics about the test (iteration count, errors, average response time, etc). The message also includes a direct link to the report for any of your team members who might not have a Loadster account but want to review the test results.

Previously the integration only posted a message at the start of a load test.

Enable & Disable Browser Geolocation

Browser Bots have location services enabled by default, with each bot assigned a random latitude and longitude somewhere. This is often helpful for testing sites with mapping or geolocation features.

If you need to set a specific latitude and longitude in your script you can now do so in a code block at the beginning of your script, or you can disable location permissions altogether.

// Enable geolocation at a specific latitude and longitude
browser.enableGeolocation(40.7128, -74.0060);
// Disable geolocation
browser.disableGeolocation();

This could be useful if you need to make sure a particular location comes up to make your script consistent from one iteration to the next, or to make sure the locations are within a certain geographical range.

Other Improvements & Bug Fixes

• Fixed a layout issue with the top navigation when names are long and the window is narrow.
• Fixed an annoying bug that caused the Traces table pagination to reset after viewing a trace in a live test.
• Fixed a bug that caused the Percentiles graph to stop displaying the other lines when you used p50 in a graph overlay.
• Added a request retry feature to improve dashboard support for poor connections.
• Fixed a bug with the Team Info form data refreshing periodically.
• Improved performance and reliability of the dashboard’s WebSocket implementation.
• Modified the test report to show the number of bots actually started rather than the peak active bots, which in most cases is the same, but not always.
• Added JavaScript autocomplete in the embedded text editors for scripts.
• Fixed an occasional race condition that made it look like a test was still in progress after it had actually finished, because the progress bar kept incrementing.
• Improved the speed at which 3rd party integrations process events that happen in Loadster.

More Supported Cloud Regions

Loadster now supports 5 additional cloud regions, bringing the total to 29.

You can now run load tests from Australia - Melbourne, India - Hyderabad, Asia - Indonesia, Europe - Spain, and Europe - Italy, along with the 24 previous regions. All of these new regions are powered by AWS.

Bot Logs in Traces

When your bots generate traces in a load test, they now include logs, similar to what you see when running a single bot in the script editor. The logs are all the logs generated by that bot in the current iteration.

These bot logs can make it easier to figure out what went wrong, or confirm what data the bot typed into a form field, or what the bot was most recently doing when the error occurred.

Other Improvements & Bug Fixes

• Fixed a bug with overriding a script variable’s update or selection strategy when overriding dataset variables for a bot group.
• Fixed a potential naming conflict caused by previously deleted scenarios of the same name.
• Made the “Strip URL query parameter values” setting work on # fragments as well as ? query strings, even though they aren’t technically the same thing.
• Fixed a graph issue that caused Network Throughput maximum values to get obscured when the graph was zoomed to a particular time range.
• Clarified aggregated values in the Response Time Percentiles graph, so it shows an overall percentile instead of an average of time-sliced percentiles.
• Improved resilience to browser crashes when a bot’s browser crashed in a load test.
• Fixed a problem that caused silent clicks to still raise errors if the element existed in the DOM but was invisible.
• Fixed a bug that prevented bots from stopping a script on error when the error arose from an included script.
• Clarified the fuel estimate to show how much is needed for tests when fuel is insufficient.
• Fixed a usability problem with dragging text when an inspector tooltip is expanded in the script editor.

Block Browser Resources by URL Substring

Since Browser Bots use real Chrome browsers, they normally load all page resources automatically.

Occasionally you might want to block or disable certain requests that the page would otherwise make. For example, you might want to block a certain 3rd party tracking script so bot traffic isn’t included in your analytics, or you might want to block some CDN resources so that your load test focuses only on your primary site.

browser.blockRequestsIfUrlContains('analytics.js');
browser.blockRequestsIfUrlContains('.jpg');
browser.blockRequestsIfUrlContains('cdn.example.com');

Of course, if you block a critical resource like a script that renders the page, the page might not render properly.

Other Improvements & Bug Fixes

• Improved selector creation in the Loadster Recorder Browser Extension to work better with frameworks that generate dynamic IDs and CSS classes.
• Fixed a bug that caused the engine/region to not always show on a finished test report.
• Improved cluster resilience when instances become overloaded so the test still completes normally.
• Fixed a bug with the Activity log that caused latest renamed test names to not show up correctly.
• Improved usability of the dataset editor around adding columns/rows.
• Fixed a bug with Files steps failing to attach certain files on cloud engines.
• Fixed a bug with crypto functions not accepting non-strings as parameters.

Manual Timers

Loadster automatically tracks timing metrics for any browser navigation (in the case of Browser Bots) or HTTP step (in the case of Protocol Bots). These metrics appear on the Response Times graph in the test report.

You can also add custom transaction timings to the report from a code block. Traditionally, it’s been done like this:

// report custom timing for everything executed in this block
bot.timer("My Custom Transaction", () => {
    browser.navigate('https://example.com');
    bot.wait(3000);
    browser.click('button.join');
});

We’ve now introduced another way, in case you want to calculate the timings yourself.

// report custom timing in milliseconds
bot.timer("My Custom Transaction", 3150);

This might be useful if you have your own way of calculating timing. For example, you might want to combine the timing of multiple steps but subtract the wait times.

Custom Cluster Sizing

Loadster automatically sizes cloud clusters for your test based on the type and number of bots. For the large majority of load tests, the default cluster sizing works fine.

However, certain sites impose an unusually heavy burden on the client’s CPU. This mainly includes sites with lots of video, 3D graphics, or client-side performance bugs. When you’re load testing a site like this, default clusters may become overloaded. You’ll usually see it on the Engine CPU Utilization graph when the CPU utilization approaches 100%.

To help with this we’ve introduced custom cluster sizing, so you can request that your bots be launched on clusters with 2X, 4X, or 8X resources.

For now, the ability to request larger clusters is enabled on a per-account basis, so if you’re experiencing the symptoms of overloaded engines and would like to launch bigger clusters, please email help@loadster.app.

Wildcard Regions

The biggest load tests require thousands of cloud instances, and even the large AWS regions occasionally have intermittent capacity shortages that prevents the test from launching. A few of our customers have had to resort to trial and error – moving bots around and trying again with different regions – in order to get their test to launch successfully.

To avoid the uncertainty of false starts, we’ve introduced a “wildcard” region. When you select the wildcard region, Loadster will automatically choose a cloud region that has capacity for your test. This should make the process of launching unusually large load tests smoother.

Please contact help@loadster.app if you’d like the wildcard region enabled for your account. Note that this is usually only necessary for very large load tests with tens of thousands of Browser Bots, so if you haven’t experienced this you probably don’t need it.

Content Security Policies

Content Security Policies (CSPs) give sites a way to restrict which resources may be loaded onto the page. They are meant to prevent against certain Cross Site Scripting (XSS) and Click Jacking attacks.

As it turns out, some sites now have very strict CSPs that prevented Playwright (the browser automation library used by Browser Bots) from injecting JavaScript onto the page to take screenshots or perform other legitimate testing actions.

Loadster’s browsers will now launch with CSPs disabled by default, since you are load testing your own trusted site with bots. If you want to keep them enabled, you can toggle the “Disable Content Security Policies (CSPs)” setting in the Browser Bots section of Settings.

Additional Features & Improvements

• Added a browser.setUserAgent(userAgentString) method for when you need to override the User-Agent for Browser Bots. Previously it was possible to override the User-Agent header sent to servers, but this didn’t help with client-side JavaScript or analytics that interrogate the user agent.
• Made the “Next Steps” section at the top of a load test report show up for 30 days following the test. Previously it only lasted about 1 day.
• Tidied up the printable stylesheet for test reports a bit, although it still has room for improvement.
• Improved the log messages in the cluster log when a test fails to launch because one or more cloud regions had capacity problems.

Bug Fixes

• Fixed an inconsistency in the Errors by Type and Errors by URL counts for an edge case where errors arise from a code block.
• Fixed a UI issue with our custom dropdown component going offscreen when it contained a large number of values.
• Fixed a bug on the publicly shared test reports that sometimes caused the engine/region and bot type to show up incorrectly.
• Fixed a WebSockets race condition that occasionally caused the script editor to say a script had failed to play when it actually did play.
• Fixed an issue that caused bots to ramp up in a stair-stepping pattern instead of a smooth curve when a high cluster multiplier was used for the bot group. This was only noticeable in tests with Browser Bots using a high cluster multiplier, which was uncommon.
• Fixed an incorrect displayed row count that was displayed after uploading a very large CSV dataset.
• Made the silent option work for select operations, like browser.selectOption('#element', '[0]', { silent: true }). Previously this worked for most other element operations but not selecting an option from a dropdown.
• Adjusted API rate limits to prevent self-hosted engines from getting rate limited.

Host-Specific Custom Headers

Protocol Bots already had the capability to set global default HTTP headers for all requests to a specific domain or host, and we’ve added the capability to Browser Bots.

browser.setHostHeader('example.com', 'X-Load-Test', 'true');
browser.removeHostHeader('example.com', 'X-Load-Test');

This can be useful to bypass rate limiting or other security precautions that can interfere with a load test, or to supply authentication headers without undertaking the full authentication flow in your script. You can read more about custom headers and see a few examples in the Code Blocks section of the manual.

Other Improvements & Bug Fixes

• Corrected test reports to show friendly cloud region names instead of region codes.
• Improved the cloud region map to indicate which regions may have limited capacity or have a history of poor reliability.
• Fixed the test launch screen so you can easily tell which region failed to launch, in the case of cloud capacity issues.
• Improved CSS selectors when recording a browser script against EmberJS SPAs.
• Improved speed and reliability of launching large tests with many cloud clusters in separate regions.
• Improved the explanation text for test scenarios when your fuel or plan is insufficient to launch the test.
• Fixed a bug that caused the wrong icon to appear for ramp down patterns on test reports.
• Added locale-specific number formatting on the tiles at the top of a test report.

Improvements & Bug Fixes

• Fixed pagination and filtering issues with the Traces table.
• Separated trace details onto multiple tabs.
• Fixed an annoying issue with password prompts from the LastPass browser extension.
• Improved the reliability of launching large cloud clusters.

Improvements & Bug Fixes

• Improved scrollbar visibility.
• Enabled scrolling with PageUp/PageDown in each pane of the script editor and elsewhere.
• Added pagination to the Traces table instead of embedded scrolling.
• Added friction when deleting a project, to prevent accidental deletion.
• Fixed a bug that caused extra long bot group names to prevent tests from starting.
• Fixed a bug that prevented your notes from appearing on shared test reports.
• Added a direct link to the Traces section in the test advice section of a test report, to aid in debugging.

Comment Blocks

Many users have requested a way to add comments in a script, so you can remind yourself what’s going on in the surrounding steps or just to improve readability. Comment blocks are just that.

You can also put comments in a code block or eval block with the ordinary JavaScript comment syntax.

Other Improvements & Bug Fixes

• Fixed a memory leak in the script editor that caused slowness when playing a large browser script with many screenshots.
• Increased the internal limit for browser bot file uploads in a code block from 20mb to 60mb.
• Fixed a broken admin toggle on the Team page.

Modifier Keys

Some web applications support clicking with modifier keys, like a spreadsheet that lets you Shift+Click to select multiple rows. You can now do this in a code block by passing an array of modifier key codes:

browser.click('.row:nth-child(2) .cell:nth-child(5)', { modifiers: ['Shift'] });

Similarly, for keyboard actions, you can string the key codes together like this:

browser.keyboard.press('Control+F');

Other Improvements & Bug Fixes

• Fixed a browser memory leak in the script editor, which caused the editor to become sluggish after playing a long browser script with many screenshots.
• Fixed a problem with the right and left cursor errors when editing a wait time.

Report URL Path Simplification

If you’ve load tested an API or web application with dynamic tokens embedded in the paths (like /users/131476/profile or /companies/fcabac8d-8b88-11ed-8b19-12104280a3f7), you might have noticed a lot of similar URLs cluttering your test report.

Loadster can now automatically collapse these dynamic sections to show URLs that are logically the same (like /users/_/profile or /companies/_) so their response times are aggregated together. To enable this, go to the Report section in Settings and select Strip URL path parameter values under Report URL Path Simplification. This setting takes effect for all subsequent tests.

Load Test Report Improvements

We’ve improved the Next Steps section that appears at the top of a load test report. This section is automatically generated after the test finishes. It calls attention to any errors that happened during the test and explains their likely cause and whether each error was likely load-related (the result of an overloaded site) or functional (the result of a scripting problem or a data problem).

The errors are now grouped together more clearly than before, and we summarize your site’s overall response time measurements.

Also, Next Steps now calls attention to any CPU utilization problems in the load test infrastructure, since an overloaded engine can cause inaccurate response time measurements, timeout errors, and other problems.

Duplicate Projects

You can now duplicate a project with all its scripts, scenarios, and datasets with a single click. This can be helpful if you want to load test a different version, instance, or environment of your site and want to use the previous setup as a template or simply keep the results separated by project.

Other Improvements & Bug Fixes

• Fixed a broken link to the dashboard that was sent out in some automated emails.
• Fixed a problem that caused Timeline cards to not immediately appear when steps were playing from an included script.
• When Timeline cards result from steps in a code block, they will now focus the code block when you click the title.
• Prevented exceptionally long URLs from clogging up the test result metrics pipeline.
• Improved error messages for rate limiting and other less common conditions.
• Improved the test report layout for tests with lots of scripts.

Browser Bot URL Instrumentation

You can now add extra instrumentation to measure the performance of background requests made by your Browser Bots. This includes asynchronous requests to your API or a 3rd party API, background scripts, etc.

Previously it was difficult to measure background request timings, because the ordinary Response Times graph only includes navigations when the browser URL changes, but does not include background requests that happen once the page has already loaded.

To capture timings on these background requests, you can now add extra instrumentation for specific URLs or URL patterns. The configuration for this is in the Browser Bots section of Settings.

You can enter one or more URLs or partial URLs (substrings), and any request made by the bot’s browser to a matching URL will trigger a sample with timing.

Instrumented URL timings will then show up in the load test report on a new graph called Instrumented URL Response Times.

Trace Limits & Filtering

Loadster captures traces to give you more visibility into what’s happening with individual bots in a load test.

The Traces table in a load test report now allows you to customize which columns appear, and filter the traces by column values. You can also search traces for a particular value.

This makes it easier to find the traces you’re interested in when you’re trying to figure out what went wrong in a test, or when you’re simply trying to spot-check that everything went right.

Because the trace data is quite heavy, not every bot can take a trace every time, but we’ve increased the limits to 250 info traces and up to 750 error traces per bot group. Previously each bot group could only take 500 traces of any type, which sometimes caused error traces to be omitted, particularly for errors that occurred late in the test.

Other Improvements & Bug Fixes

• The registration flow is now mobile-friendly once again.
• When launching a cloud cluster, Loadster now shows you more frequent updates about the launch process as instances are added to the cluster.
• Subscription cancellations now take effect at the end of the current term, instead of immediately.
• The Loadster Engine Docker image now specifies a high-numbered UID/GID to reduce the chance of UID clashes with the host system.
• Improved the handling of special characters in file names, when using Files steps in browser scripts.
• Fixed an inconsistency in the color of the finished test progress bar for tests with errors.
• Fixed a race condition that occasionally caused launch instability in very large cloud clusters.
• Fixed a regression that prevented file uploads from Browser Bots in a load test.

Refreshed Scenario Editor

The scenario editor got a redesign, to make editing each bot group more straightforward.

You can still access the advanced options by expanding the gear menu to the right of the number of bots, if you want to impose an iteration limit (per bot or per group), simulate a slower network, override DNS entries or variables, etc..

Running Bots by Group

The Running Bots by Group graph is now a stacked area graph instead of a superimposed area graph.

This makes it easier to see each area and also to easily tell the total number of running bots at any moment.

New Methods in Code Blocks

We added a few methods for identifying which bot is running, in case you need to add bot-specific branching logic to your code blocks.

bot.getBotNumber();         // Index of this bot in the group, starting with 0
bot.getBotGroupNumber();    // Index of this bot group in the test, starting with 0
bot.getIdentifier();        // Unique string identifier for the bot

You could also send these to your server if you want to correlate requests to which bot they came from.

Export Full Graph Data

Clicking on the small clipboard icon in the top-right corner of each graph will now copy the full graph data in a tabular format, so you can paste it into a spreadsheet for further analysis. Each row represents a short time window (2-second resolution for short tests, and longer resolution for longer tests). Each series in the graph has its own column with aggregated data points in each time window.

This data was previously only available in the JSON download and required parsing.

Other Improvements & Bug Fixes

• Added an indicator when series are hidden on a graph with many series.
• Fixed a bug with the response size validator that only happened in a load test, not in scripting mode.
• Fixed a connection leak that occasionally happened with long-running engines running lots of browser scripts.
• Improved the speed with which finished tests are cleaned up and test reports are generated. Previously these could get a bit backed up if several load tests were finishing simultaneously on the platform.

Graph Timestamps on Hover

The time series graphs in the load test report now show a hover line with the precise timestamp when you hover the graph. This makes it easier to correlate spikes or errors with data from your APM system or server logs.

Trace Filtering

You can now filter the traces in the Traces section of your load test report, to easily find traces for a particular URL or error message. The table now includes trace error messages (if any) so it’s easier to find the trace you’re interested in without clicking on them.

Request Details in Protocol Traces

Traces from Protocol Bots now include details about the HTTP request, not just the response. Clicking on a trace will reveal all the request headers, and usually the request body as well, as long as the body length didn’t exceed a set limit. This makes it easier to tell what your bots sent that resulted in the errors.

Faster Script Playing from North America and Europe

We’ve deployed dedicated bare metal script players for the North America and Europe regions. If you designate one of these regions as your Script Player in Settings, your scripts will now start within milliseconds in most cases.

Speedway Limit Emails

Speedway now emails you if you are approaching or have exceeded the maximum monitoring requests or robocall and text message notifications for your Speedway plan. This gives you notice that your monitors may stop running you or stop notifying you if the limit is exceeded, so you can decide whether to upgrade your plan or wait until the limits reset next month.

Other Improvements & Bug Fixes

• Fixed a bug in Speedway that prevented deletion of notification policies when in use.
• Fixed a bug with Ctrl-C/Cmd-C copying text from the script logs in the script player.
• Introduced browser.setCookie() as a way to set cookies from outside the browser in code blocks.
• Fixed a regression with browser.setFiles() in a code block. Ordinary Files steps were unaffected.
• Improved font legibility in the Variables dialog.
• Increased precision of sub-second response times to 3 digits.
• Fixed a minor scrolling annoyance that occurred after saving a long script.

Delayed Start for Bot Groups

By popular request, there’s now a way to delay the start of certain bot groups in your load test. This allows stair stepping patterns and other creative patterns for ramping the load.

The delayed start option is hidden under the wrench that appears to the right of each bot group.

Individual Keypress Actions for Browser Bots

Testing certain kinds of web apps (WebGL games, etc) requires generating individual keypress events in the browser window. This is different from typing text into an input field, so we’ve exposed a simple way to do it in code blocks.

browser.keyboard.press('ArrowUp');
browser.keyboard.press('ArrowUp');
browser.keyboard.press('ArrowDown');
browser.keyboard.press('ArrowDown');
browser.keyboard.press('ArrowLeft');
browser.keyboard.press('ArrowRight');
browser.keyboard.press('ArrowLeft');
browser.keyboard.press('ArrowRight');
browser.keyboard.press('B');
browser.keyboard.press('A');
browser.keyboard.press('Enter');

bot.wait(3000);

browser.screenshot();

Any plain character or a value from KeyboardEvent.key is acceptable.

Where the keypress event gets handled depends on what element on the page has focus. Your script might need to click on an element first to focus it, or click somewhere on the body to remove focus from an element if you want the keypress to be handled at the window level instead.

Speedway Updates

We’ve been busy with Loadster’s companion product, Speedway. Speedway lets you use your Loadster scripts, or new scripts, for 24x7 Site & API Monitoring.

New in Speedway this month is the ability to map different Notification Policies to each of your monitors. For example, you might want certain critical monitors to send immediate SMS messages to everyone on your team, and also raise the incident in PagerDuty and email everyone. Less critical monitors (like for your a staging site) might just email a single person.

Notification Policies are now more flexible, too. You can set a policy to delay notifications until the monitor has been in a failing state for a certain amount of time (like 10 minutes), so they don’t wake you up for network blips or issues that quickly took care of themselves.

Watch this space! There are exciting developments in the works for Speedway. It might even get its own Changelog soon instead of bumming off Loadster’s.

Other Improvements & Bug Fixes

• We’ve created a better getting started experience after creating your account. If you’ve been a Loadster user for a while, you’ll never see this. If you’re a new user, you won’t appreciate it either, because you didn’t have to do it the old way.
• Improved performance loading the Traces table in a test report for tests with lots of traces.
• Improved the generation of text selectors in the Loadster Recorder browser extension, for sites where the same text exists in multiple places.
• Links to invoices in emails when you complete a purchase are now permalinks. You can now access the Stripe invoice and receipt months later.
• Fixed the average response time calculation on the tile at the top of the test to be the average of all groups in the test, when there are multiple groups of bots. Tests with only one bot group were unaffected. The values in the Average Response Times graph below were similarly unaffected.
• Fixed an issue with clicking on a title in the Timeline selecting multiple steps in the editor, so it wasn’t clear which step the card correlates to.
• Changed the formats.base64decode() function to return a byte array instead of a string. Returning the decoded value as a string was kind of pointless because Base64 is mainly used to encode things that aren’t strings.

New Features & Improvements

• We’ve simplified the onboarding experience for new accounts, to streamline the path to create and play your first script.
• The script editor now defaults to a layout that shows the Logs and Results/Timeline tabs side by side.
• Login and registration pages now load a lot faster.

Bug Fixes

• Fixed a usability issue with dropdown menu buttons, making it possible to close the menu by clicking outside it.
• Fixed performance issues related to large datasets bogging down the editor.
• Fixed an issue with bot numbers in the Traces table being incorrect for large distributed tests.

New Features & Improvements

• The new “Request Count” column in the “Total Time Spent” table of the test report now indicates how many total requests were made to each URL in your load test.
• There’s now a double-click option for click steps in a browser script. Previously you could only double-click from a code block, not an ordinary click step.
• You can now resize, sort, and hide columns in several of the test report tables.
• The Request Content-Type field in a protocol step is now a combo box, with common defaults.
• The dataset editor once again has undo/redo history.
• Code block HTTP operations like http.get now synchronously return a response object, for easier control flow. Previously you would have to grab it from a validator function and store it in a higher-scoped variable.
• We’ve improved alphanumeric sorting in the projects list and elsewhere.

Bug Fixes

• Fixed an issue with bot reservations sometimes not being released immediately when a cloud provider lacks capacity for a big load test.
• Fixed a problem that prevented errors from Evaluate steps from showing up in the script editor logs.
• Fixed a type coercion issue with typing non-strings from code blocks.
• Fixed some usability quirks with removing request bodies in a protocol script.

Copy/Export Test Data

Need to get your load test results into a spreadsheet or external system for further analysis? It’s now easy to copy tabular data by clicking on the small clipboard icon at the top-right of any graph or table in the test report. The copied data pastes nicely into Google Sheets or your other spreadsheet of choice, and it’s easy to parse as tab-separated values if you want to do something custom with it.

Global Custom Headers

You can now set global custom headers for both protocol scripts and browser scripts. Global custom headers will be sent with all outbound requests, regardless of destination. You can set them for Protocol Bots and Browser Bots in a code block using http.setGlobalHeader(name, value) and browser.setGlobalHeader(name, value), respectively.

Improvements & Bug Fixes

• Completed the migration of the Loadster dashboard to Vue 3.
• Replaced the 3rd party dataset editor with a custom grid component.
• Enabled smart redirect for unauthenticated users, to send you to the login page if you have previously logged in from the same browser.
• Updated the browser script recorder to automatically include wait times if there are more than a few seconds between recorded steps.
• Clarified a help message when viewing details on a trace.
• Made it more straightforward to tab between fields in the script editor.
• Separated Loadster and Speedway invoices so they only appear in their respective apps.
• Fixed an intermittent bug with the test page title being outdated.
• Fixed a bug that caused the Terminate button to be obscured when stopping a test.
• Made the checkout flow default to your country and tax number if you’ve entered them previously.
• Deprecated the X-Loadster-User header in favor of custom user-defined headers.

Better Project Page Test History

The project page has been improved to surface more information about recent load tests, with a map of cloud regions, links to the related scenario and scripts, and high-level stats. It’s no longer necessary to click through to the full test report if you just want to see high-level test metrics. The scripts, scenarios, and datasets are still reachable from the project page in much the same way as before, but they have been pushed to the sidebar.

Revised Activity Feed

We’ve redesigned the Activity Feed widget on the Loadster dashboard, to make it more comprehensive and surface more information about tests. This way it’s easier to find a recent test, regardless of project, and see what your teammates have been up to.

Improvements & Bug Fixes

• Fixed a bug in the Loadster Recorder browser extension that caused trouble recording events in nested frames after a nested frame navigation.
• Improved the Loadster Recorder browser extension to prevent redundant navigations from being recorded after clicking a link.
• Fixed a bug with the test launch logs that showed cluster activity from other tests, if you were running multiple tests at once.
• Fixed a bug affecting the iterations limit in certain situations.
• Eliminated harmless warnings when launching a Loadster Engine in a Docker container.
• Improved filtering of jsessionid parameters from URLs in test results.
• Clarified the USD currency on the billing pages.
• Added support for recording from old-fashioned frame elements to the Loadster Recorder browser extension, in the same way we handled iframe elements previously.
• Added the ability to override the typing delay for browser.type(...) in a code block.
• Added the ability to set custom timeouts for browser.waitFor(...) in a code block.
• Surfaced a read-only historical view of the script steps on a load test.

New Features & Improvements

• Added the ability to open a new browser window or tab from a code block.
• Added a silent option for clicks and other steps when called from a code block. With this option, the step will be carried out if possible, but if it fails the script will continue without error.
• Made it possible to stop a script immediately, even if it’s in the middle of a code block, without waiting for the code block to finish.
• Improved report filtering for tests with many distinct URLs, to prevent the Response Times graph legend from becoming unintelligible.
• Improved the size and layout of the trace modal.
• Improved the display of n-month plans on the Billing page (such as a custom plan that renews every 6 months).

Bug Fixes

• Ironed out apparent data inconsistencies between the Live View and Report View. The data was correct, but inconsistent because each view was showing data from different time windows.
• Made the step number in the timeline always correspond to the step number in the editor after running a test, so you can more easily relate screenshots to the step that took them.
• Fixed a performance issue with accessing very large response bodies in code blocks or JavaScript validators/capturers.
• Fixed a bug that caused Browser Bots to ignore the “Display interpolated URLs in reports” setting in some situations.
• Fixed issues with stripping out URL parameters or truncating the query altogether in the reports.

Graph Overlays

You can now overlay one graph on top of another, both when running a load test and in the test report after it finishes. Mix and match different graphs to show the relationship between different metrics, like the effect of running bot count on response times or errors. You can overlay graphs from the sidebar while a test is running, or at the bottom of the test report after a test finishes.

Improvements & Bug Fixes

• Improved the quality and consistency of automatically generated selectors when recording a browser script.
• Enhanced the notification center to relay broadcast system messages in case there’s a partial outage.
• Changed the engine utilization graph legends to show the full group name instead of just the engine region, so it’s clear which cluster each refers to.
• Fixed a bug with the Loadster Recorder extension for Chrome, now using the browser extension Manifest v3.

Improvements & Bug Fixes

• Fixed a rare issue with shaking tiles in the Timeline.
• Fixed a keyboard focus issue that sometimes caused steps to be deleted inadvertently when editing a URL.
• Fixed an issue with timezones in the Activity Feed.
• Extended the Traces TTL to 30 days.
• Improved messaging for when a script is deleted that’s in use by one or more scenarios.
• Redesigned the Play and Fast Play buttons in the script editor.

Improvements & Bug Fixes

• Made single bot scripts play in AWS Lambda instead of shared engines in most regions, for greater reliability.
• Enabled self-service username changes instead of requiring you to email support - finally!
• Removed the “Launch test with manual start” feature.
• Made Browser Bots send the X-Loadster-User header like Protocol Bots do.
• Fixed a bug with deleted engines showing up in the Script Player dropdown.
• Fixed a scrolling issue with the script editor caused by hitting the space bar while playing a script.
• Improved a misleading error message when timing out after a click action.

Browser Bots: Camera & Microphone Support

Browser Bots now provide a dummy camera and microphone device, and automatically grant permission to them, if the pages you’re testing ask for it. The devices are stubbed out and send uninteresting video and audio streams. You don’t actually get to see the bots looking into their cameras, sadly.

Browser Bots: File Inputs

Browser Bots can now use the Files step to interact with file input elements, like <input type="file"/>.

When you choose a file in your script with this step, the file content becomes part of the script, so you can run the script out in the cloud or on a self-hosted engine without needing access to your local filesystem.

There’s also an equivalent way to do this in a code block, but you’ll have to encode the file’s content as Base64 yourself, like this:

browser.chooseFiles('input[type=file]', [
    {
        name: 'cat',
        contentType: 'image/jpeg',
        contentBase64: 'UklGRuZ/AABXRUJQVlA4INp/AABwrAGdASr0AQACPjkYi0QiIaET...'
    }
]);

Report URL Simplification

Your Loadster reports include a graph with a breakdown of response times by URL. Each unique URL from your load test is a separate graph series. Normally this is super useful, but it gets messy if your test visits too many unique URLs. For example, your bots might get redirected to a dynamic URL that’s different each time, and showing each of these dynamic URLs as a separate graph series is unhelpful.

To tidy up your graphs, we’ve added a new setting to Settings, called Report URL Simplification. Possible values are:

• Leave URLs as-is (in their full unaltered glory)
• Truncate URL query strings (remove the whole query string, starting with the question mark)
• Strip URL parameter values (keep the query params, but replace values with underscores)

Improvements & Bug Fixes

• Fixed a bug that caused iterations per group to be ignored with small numbers of bots.
• Made the Halt script execution on any error setting apply to individual actions within a code block, not just the entire code block.
• Stopped background navigations within nested iframes from showing up in the Response Times.
• Fixed a bug that sometimes caused the Timeline and Logs to get stuck when stopping a script early in the editor.
• Made our home-brewed iframe selectors work with all the CSS attributes.
• Added the step number to traces so you can tell which step generated the trace.

Clobber-Resistant Script Editing

If you and someone else on your team happened to be editing the same script, there was a danger that you might clobber each other’s changes. We’ve added some guardrails to prevent this with a recent update.

You’ll now see a user badge appear at the top of the script editor whenever someone else on your team also has that script open. The editor will also double-check that you have the latest copy when you go to save it, and alert you if someone else saves a copy in the meantime.

Protocol Bots: Global Headers Per Domain

Since time immemorial, Protocol Bots could specify custom headers for each individual request, but that could get pretty tedious if you’re testing an API that requires the same headers on every request. Thankfully there’s now a way to set a default header per host in a code block anywhere in your script, like this:

// Add a header to all future requests to a host
http.addHostHeader('petstore.loadster.app', 'Authentication', 'Bearer 5a830a310b5ca38a');
// Remove host headers matching a host/name/value
http.removeHostHeaders('petstore.loadster.app');

Once you set a host header, all requests your Protocol Bot makes to that host will automatically have the header, until the end of the script or until you remove it.

Improvements & Bug Fixes

• Clarified the error message when trying to choose a non-existent option from an HTML select.
• Added the monthly test hours consumption to the billing dashboard, so if you have a plan with an hours limit you can see how many you’ve used.
• Updated the Loadster Recorder browser extension to ignore _ngcontent-* and similar unreliable properties when crafting selectors.
• Fixed buggy pagination on the Total Time Spent table of the test report.
• Improved the quality and specificity of error messages from Evaluate steps.

Browser Bots: Select Options by Value, Text, or Index

We’ve made it easier to choose the option you want from Browser Bot Select steps. Now, you can specify an option by text, by value, or by index. Consider the following HTML:

<select name="countries">
  <option value="CR">Costa Rica</option>
  <option value="HR">Croatia</option>
  <option value="CU">Cuba</option>
</select>

To choose Croatia, your step can specify CR to select by value, "Croatia" with double quotes to select by text, or [1] with square brackets to select by index.

Browser Bots: Manage Multiple Browser Windows

We used to assume that each Browser Bot would only control a single browser window or tab at a time. Not anymore! Now you can toggle between browser windows, close a window, and list the bot’s browser windows programmatically in a code block.

// List all of the bot's browser windows/tabs
browser.listWindows();
// Get the bot's currently active browser window/tab
browser.getActiveWindow();  // 0, 1, 2…

// Focus a different browser window/tab
browser.setActiveWindow(0); // the bot's original tab
browser.setActiveWindow(1); // the first opened tab
browser.setActiveWindow(2); // and so on…

// Close a browser window/tab
browser.closeWindow(1);

Each Browser Bot starts out with a single window, but if your site opens links in another tab or pops up another window, it will be added to the list.

Speedway Site & API Monitoring

We’re pleased to announce Loadster’s sidekick application, Speedway, for Site & API monitoring!

You probably noticed the monitoring feature in Loadster this past year: run a Loadster script on a schedule and get alerted if anything goes wrong. It’s kind of like Pingdom or Uptime, but with more powerful scripting.

This monitoring approach is often called active monitoring or synthetic monitoring, and it’s an important part of the site reliability stack. Active monitoring is highly recommended if you want to be alerted of site downtime and breakage, especially if you want to test multiple pages or endpoints back to back.

The scripting is the same, but the motivations and methodology are a bit different between load testing and monitoring.

Reusing your load test scripts for active monitoring (and vice versa) saves time on script creation and reduces the effort of maintaining your scripts, while making sure you can react quickly to problems and be proactive about it.

Read more about Speedway »

Improvements & Bug Fixes

• Added the full results from Scrape and Evaluate steps to the result details, which you can see by clicking on the card in the Timeline.
• Clarified the users per team limit on the Team page.
• Clarified the confusing error message when a Wait For step times out.

Traces

Browser Bots now capture traces whenever an error occurs, and the first bot in each group captures traces of every step regardless.

Traces show up on the Traces tab of your live test, and in the Traces section of the test report.

They include:

• A screenshot of what the browser looked like at that moment
• A waterfall chart of resource timings from the page
• Detailed error messages from the browser automation library when available
• The bot number, step number, and URL where the error occurred

Read more about Traces »

Browser Bots: Wait For Element Steps

Web applications often have overlays, progress indicators, or deferred loading of components to improve the user experience, particularly for slower actions that can take a few seconds to complete. This makes sense from a UI perspective, but it can present a challenge for automated testing because the thing you want to interact with might not be there yet, or might be covered up by something else.

With the new Wait For steps, your Browser Bots can wait until the element achieves a certain state before continuing.

Browser Bots: Select Steps

We seem to have overlooked <select/> dropdowns in the initial Browser Bots release, so there was no easy way at first to select an option. That’s fixed with Select steps.

Browser Bots: Hover Steps

Most of the time, building automated tests for a web application doesn’t require explicitly hovering on things before clicking them… but sometimes it does. The most obvious example is a fly-out menu, where you have to hover the menu title and then the contents fly out, exposing things you can click on.

If you need to hover something, you can now do so with the new Hover steps.

Include Scripts in Scripts

Reusability! Composability! Finally, you can include one script in another, so you can keep frequently used sequences of steps in one place.

For example, you might want to have your login flow in a script called Login, and include this at the beginning of all the scripts that require logging in.

Read more about how it works in the Include Script FAQ.

Multiple Team Support

User accounts and Loadster teams are now a many-to-many relationship! Before, your Loadster account was tied to a single team, but you can now join multiple teams and easily switch between them.

This is helpful if you consult with different companies, or just want to use Loadster for personal projects without intermingling them with your company’s account.

If you’re linked to more than one team, you’ll see a team switcher at the top of your dashboard. Just choose another team from the team switcher to switch teams.

If you’re an admin on your Loadster team, you can invite someone else to join on the Team page just like before. If they already have a Loadster account at that email address, they’ll simply be linked to your team upon accepting the invitation. Your friend can then bounce between your team and any other teams they are associated with.

Similarly, if you’re an admin on a team and want to remove someone from the team, you can also do it from the Team page. Once you remove someone, they’ll still have a Loadster account and will still be able to see their own personal projects and those of any other teams they’re associated with… they just won’t have access to yours anymore.

Browser Bots Have Entered the Arena!

Most web load testing tools, including Loadster up until now, require you to automate your load tests at the HTTP protocol layer. Scripts were basically a sequence of HTTP/S requests, chained together, with validators and capturers bolted on when necessary. This approach made testing pretty much any web application possible, but not always easy.

For some web apps, testing at the protocol layer is actually quite difficult, especially when you have to capture parameters from a server response and use them in a subsequent request. Scripting OAuth and SAML flows at the protocol layer can be particularly cumbersome because these flows require a lot of capturing and regurgitation of special tokens.

Our new Browser Bots make scripting such complicated interactions a whole lot easier, because they control real headless Chromium browsers.

Scripting with Browser Bots is made up of user actions like navigate to URL, click on element, type text into an input field, and so on. You don’t have to worry about every individual HTTP request, because the browser figures that out for you. Automation at the browser level especially shines for testing complicated web applications with a lot of client-side logic or convoluted authentication flows.

What about the old v-users or virtual users? They aren’t going away, but from now on we’ll call them Protocol Bots. Protocol Bots are still ideal for load testing HTTP APIs (REST, GraphQL, etc) because they give you precise control of each request. They generally work fine for simple static sites too. And testing with Protocol Bots, when practical, is very affordable: they only consume 1/4th as much Loadster Fuel as Browser Bots.

In short, every time you create a script, you’re free to choose the right bots for the job: Protocol Bots when you want to work at the protocol layer, and Browser Bots when you want to automate real browsers.

Loadster + Slack, Opsgenie, and VictorOps

We’re pleased to announce three new Loadster integrations: Slack, Opsgenie, and VictorOps. Whenever a monitor fails or recovers, you and your team will get notified through the right channels so you can take action quickly.

Read the full announcement about Slack, Opsgenie, and VictorOps »

Loadster + PagerDuty

Loadster now integrates with PagerDuty! If you’re using Loadster’s Site & API Monitoring, and want amped up notifications, escalations, and on-call rotations, look no further than PagerDuty.

Read the full announcement of our PagerDuty integration »

Goodbye, Loadster Workbench

Rumors of Loadster Workbench’s demise have not been exaggerated. It’s officially EOL as of May 1st, 2020. Let’s thank Loadster Workbench for all the good times, and look forward to more good times to come while moving faster on the web platform.

Read the full Loadster Workbench EOL Announcement »