A load test is only as good as the assumptions you make going into it!
While no test can entirely eliminate the risk of a site crashing, you can greatly reduce the risk if you get these basic elements right.
The important things are…
Do you understand how your real users behave on the site?
Certain user actions can be a lot more expensive than others. As far as your server is concerned, serving up static pages might be a breeze, but a shopping cart checkout flow would be more expensive because it requires accessing some kind of datastore, communicating with a third party payment processor, and so on.
Your test should imitate the primary user behaviors on your site as closely as reasonably possible… especially the most expensive ones.
The term requirements has a somewhat looser meaning when it comes to load testing than it does with functional testing. Because performance and scalability are multi-dimensional, it can be a bit tricky to state the requirements as binary pass/fail assertions.
We usually start with rather simplistic requirements from our partners and stakeholders, for example:
The problem with all of these requirements is they leave too many things open to interpretation. Some of them state the acceptable response time with no mention of load; others state the throughput with no mention of acceptable response times.
A useful scalability requirement needs to specifically account for both the load on your site and the acceptable response time.
Let’s try rewriting the vague requirements above to be a bit more specific:
These requirements are better because they now account for both the load and the acceptable response time.
Needless to say, your site’s infrastructure will have a big impact on its performance and ability to handle sustained levels of throughput.
Ideally, the environment you test should be running the same software and hardware (virtual or physical) as your production site. It can even be your production environment – as long as you’re aware of the risk and all the other stakeholders are too.
If you’re not able or willing to load test your actual environment, it’s crucial to test one that’s as similar as possible.
Attempting to extrapolate load test results to a different environment than the one that is tested is risky, and should be avoided if possible! Scalability of complex systems is rarely linear.
A load test, by its very nature, is influenced by many variables. When you share test results with your colleagues and stakeholders, we recommend calling out these variables as possible caveats, because they affect the applicability of your test results.
Important variables/assumptions that you should call out in your report include:
Remember, the validity of your test is only as good as the assumptions that went into it!
A meaningful load test is one where you can state, with a high degree of confidence, that given the assumptions listed above, the site did (or did not) meet the requirements.
Load testing complex systems is no walk in the park, but Loadster helps you operate at a higher level by focusing on what really matters in your testing, instead of the mundane tasks.
If you’ve done load testing in the past, you’ll be pleasantly relieved to know that…
By doing all this for you, Loadster makes your job easier, and you can focus on getting actual meaningful results in as little time as possible.