|Don't Be A Donkey...Load Test Your Cart!|
My Fore Into Ensuring Applications Behave Well...Part 1: What Is Load Testing?!?!!
Hey there friends, a little while ago, I decided that in order to expand my quality perspective, I needed to get involved in some sort of performance testing. I always had a bit of a hunch that it was important, but always focused on automated testing which ensured the application under test behaved correctly. Oddly enough I never really focused on ensuring it ran well. Silly me. Anyway, a few months ago, a few of us on my current QA team went on a performance kick. So, I decided it was time to get involved in ensuring our apps behaved correctly and well! The timing for this couldn't be better, as our team was in the later stages of releasing some externally facing applications, which really needed to behave correctly and well.
What Is Load Testing
After being able to dedicate some time to figuring out what performance testing really was, I started researching. I was able to deduce that performance testing was essentially two things. Figuring out how to measure how applications behave under certain conditions, and then exasperating those conditions. Boy was I excited. I mean, I think one of the best portions of testing for quality, is staging scenarios which stress, bend, twist, prod and poke our apps. So to be able to learn how to do this to the max, seemed like a lot of fun.
Load testing emulates high user load on an application by means of automation. A Load test is basically made up of two components: a performance test, and repetition to achieve load.
The first portion of the load test (the performance test) essentially emulates a single user's actions through HTTP requests. HTTP requests are essentially what browsers use to communicate with websites, and the performance test (after written or recorded) replays captured HTTP requests.
In order to write or record a performance test, one captures the traffic resulting from regular user actions. The traffic is then massaged, to be scaled. Performance testing tools such as SOASTA's Cloud Test, or Microsoft's Visual Studio Performance Testing suite allow us to do this relatively easily. When writing a performance test, you will need to think about what it means to repeat the same actions, but possibly with a different user, or with different security credentials.
Some of the things which we may want to vary per test, could be user login, password, or different links to click. The tools which we work with allow us the ability to parametrize our HTTP requests, so we can replay them with input parameters, such as a list of orders to accept. Our goal for creating a performance test, should be to create a script which can be replayed many times with different parameters.
How To Turn On The Heat (Load)
So after we establish our performance test, and are able to run it in single iterations, we need to scale it. This is where the true power of our load testing tools come into play. Most load testing tools give you the ability to utilize the power of multiple machines to create a great deal of HTTP requests. Modern load testing tools even give us the ability to use cloud infrastructures such as MS Azure, or Amazon Web Services to create extensive load and even control which region of the world it comes from!
This really is magic, as we now are able to setup a load test to not only simulate a user's HTTP traffic, but also simulate where it comes from! We can even set the user agent strings on the HTTP requests to see how our application behaves for different simulated browsers, or devices.
The Load Test, is effectively a scaling of one performance test. And the fact that we can have a script and framework work together to do this at scale is amazing.
The Most Important Piece
We now have a good idea of what a load test is , and what it consists of. But the real question is, what value do we get from this? What should we be watching for? A load test tool will give us a great deal of metrics and graphs and charts and shiny things. But what should we be doing with all this info? From my experience, and based on advice from more experience performance testers, we need to know what we want to see. We need to know what to look for, so we know when see something different! There are a few ways of doing this, and one sure fire way is to establish a baseline. In order to do this, we basically run a test, record some metrics , and then judge against those. From my limited experience, I've noticed that for high load threshold, I tend to look at request send rate vs. request response time, and for performance measurement, I tend to look at page load time. Using these two metrics, I am able to get a good idea if my application is really tanking, when stressed. These two metrics also provide me with a good idea of how well the application will behave for the user. There are definitely other measures which are interesting, but at a high level, I believe these two are a great starting point.
Until next time.