Stress Testing involves using a load testing tool to attempt to break a system. This typically means taking the system out of its comfort zone. In some cases the amount of hardware needed to do so might be cost prohibitive. In other cases, it would take too long to provide the hardware resources needed to fully stress the system. In this situation, cloud computing really shines.
Cloud computing allows you to scale resources economically and provides instantaneous and cost effective access to significant computing resources. In addition, cloud computing allows you to ramp up and ramp down computing resources as needed.
The purpose of stress testing is to stress a system to the extent possible and to see when it breaks. This can often require a large hardware investment. With the advent of cloud computing this is no longer the case. Stress testing in the cloud makes a lot of sense. The cloud is elastic and allows the tester to easily and instantaneouslyscale hardware resources to ensure that stress testing can be properly accomplished. Moreover, if you use those resources only for a few days you will only need to pay for the cost of using the hardware for a few days. Cloud testing changes how we perform stress testing. We no longer have to worry about hitting a hardware resource bottleneck. We can scale the hardware much more easily and not have to worry about factors such as time and cost.
Using our performance testing tool we have tested many websites and helped our customers improve the performance of their websites. In doing so, we have employed many methods that can be used to improve website performance. Here I will present some of the most effectively utilized techniques.
Make sure that you have saved your images in the correct format to optimize size. PNG is usually the best format for solid colors and JPEG is usually the best format for photos. JPEG size is dependent on quality. Find your minimum quality threshold and save your JPEG images based on the minimum quality threshold to minimize file size. Moreover, make sure that your saved image has the same size as the displayed image. If your saved image is larger you are unnecessarily using more bytes to store the image.
Server Side Compression
Gzip and Deflate are the two compression methods that are available to minimize the file size that needs to be downloaded. Web pages are usually compressed quite well by these methods. A 1 MB HTML page could easily be reduced to a 200 KB or lower size by using these compression techniques.
HTTP Requests Minimization
Each HTTP request incurs an overhead. It always amazes me to see how many websites have web pages that are referencing resources (images, stylesheets etc) that do not exist resulting in a 404 error code. Most performance testing tools will immediately uncover such HTTP requests. Combining multiple images into a single image will also reduce the number of HTTP requests. Image Maps and CSS Sprites are some of the many techniques that can be used to combine multiple images into single images.
Redirects are sometimes a simple way to active certain tasks. However, they do incur an overhead and they should be avoided. A very common mistake made by developers is to forget to add a forward slash / to the end of a URL (e.g. http://www.verisium.com/products is used instead of http://www.verisium.com/products/). This results in an additional 301 request that increases the page response time thus degrading website performance.
Regular Website performance Testing & profiling
Performance testing tools are important weapons in your arsenal to test website performance and discover the source of the various bottlenecks. Performance testing tools will reveal the areas that need immediate attention.
We have worked with lots of test systems and had many organizations consult with us about their performance and load testing efforts. Over the years we have encountered many common mistakes. These mistakes often greatly compromise performance and load testing effectiveness. In this post I will discuss some of the more common mistakes that are made during performance and load testing and how to avoid them.
1. Testing on a system that does not resemble the production system: The system used for load testing needs to mimic the production system as closely as possible. If your production system, for example, has 8 GB RAM but your test system only has 2 GB of RAM, it is highly likely that for a given number of virtual users the response times will differ. In addition, the operating system and any installed software on test system should be configured in the same way. This will lower the likelihood that the test system and the production system will behave differently.
2. No page validation in scripts: Load testing tools will report errors when an HTTP request produces a response with an unacceptable response code .e.g. a 404 (page not found), a 500 (server error) etc. However you may have still have errors that will produce perfectly valid HTTP response codes. For example, you may have a web page where you submit an invalid login name or password and the resulting page asks you to attempt to login once again. You might consider this situation an error but there is no way for the load testing tool to know this (since the page will probably produce a valid HTTP response code of 200). You need to specifically direct the load testing tool to invalidate such a web page response. You can do this by using a text checkpoint where you can look for specific text on the page and use that condition to validate the page.
3. Not starting load testing early enough: Performance bottlenecks are best identified as early as possible. If you wait until the system has completed development, you might find performance issues that require a architectural redesign. This would take time and would be likely to result in missed deadlines.
4. Test Cases do not represent real scenarios: This might seem obvious but I have seen many systems that underwent load testing but suffered significant performance problems and downtime after they were released. The most common reason is that the test scenarios that were used for load testing did not fully represent actual real world scenarios. The closer your test cases are to real world scenarios, the greater the likelihood that your production release will not encounter too many issues and will be successful.
There are several other mistakes that are made but the ones listed above tend to be the most common and appear to have the greatest impact on the success of the project.
The terms load testing and stress testing are often used interchangeably. However, there is a fundamental difference between the two. Let’s assume that we have an application that at peak times is subjected to a load of 1000 virtual users. Let’s also assume that a response time of 5 seconds is considered acceptable.
Load Testing attempts to measure how well the application performs at a given load. In the case described above we would attempt to load the application with 1000 virtual users and attempt to measure the mean application response time and find out if it meets our criteria. The load should be as representative as possible of the actual real world situations. This means that the data should be parameterized and different virtual user ramp up scenarios should be modeled. If the application fails to meet the acceptable criteria, then it should be fixed and the load tests should be run until the application satisfies the acceptable criteria.
Stress Testing attempts to break the system. This means taking the application out of the range of acceptable conditions. It could involve using a much higher virtual user load than the specified limit. It could also involve giving the application insufficient RAM or configuring the database with less memory resources than is normal. Under these conditions the application will reach a breaking point. At this point the application should degrade and finally fail gracefully. Stress testing can be related to load testing or can involve stress parameters that may not be directly related to load testing. Stress testing is thus by definition not a subset of load testing but rather a particular form of testing that may encompass load testing. In the application described above, stress testing could involve ramping up to a virtual user load of 5000 virtual users. The application’s performance should degrade gracefully and if it does fail, it needs to do so gracefully.
During load testing design, it is important to establish if the testing process would also encompass stress testing. Moreover, the criteria for stress testing should be clearly established and the development team should come to an agreement as to what should happen when the application is subjected to the specified degree of stress.