CTE Newsletter - Performance/Load Testing

In today's complex web application world, executives, project managers, and developers need to test applications that go beyond simple HTTP-based pages. They need to test Rich Internet Applications that incorporate complex technologies from Microsoft, Oracle, SAP, IBM, Web 2.0, etc.

For example, an application server's default settings for accessing databases, and the default settings of the databases themselves (connection pools, dedicated memory allocation, table indexation...), are not always suited to the application itself and often need to be adjusted. The load test, coupled with server monitoring, allows you to tweak these settings and observe any subsequent improvement or deterioration in performance.

Work According to a Proven Method
CTE recommends following these steps:

Define a realistic test plan scenario.
Build/Record the scenario.
Execute the scenario to simulate concurrent users, while each time simulating more users.
When performance is degraded beyond usability, you have reached the maximum number of users.

Define a Realistic Plan & End-user Scenario
When you build a scenario that is to be executed, there will be potentially hundreds of simulated users playing that scenario concurrently. It is important that the scenario is representative of the actions of your user population. The scenario should perform the same types of operations as your users - this is obvious to most people - but also at the same speed your users would.

Often, testers will carefully research what actions their users perform. Satisfied, they will then build a scenario, but record the actions in quick succession, thinking that it's the only the transactions themselves that are important. Equally important is the think time, the time between operations.

Learn from the Test Results
Your testing efforts will demonstrate the following facts about your system:

number of open, system and idle sessions for the database?
number of calls per second to your server?
percentage of calls to the database that did not use a cache?
percentage of SQL requests using indexes?
number of blocks per second for each of the logical accesses?
number of blocks per second for each of the physical accesses?
wait time in seconds for a particular event?
memory allocated to the various pools and buffers?

To find out the different break points, you want to progressively increase the load of the application. The test results should answer the following main questions:

How many users can the application handle while maintaining an acceptable (predefined in the plan) response time?
What is the load threshold above which the server begins to generate errors and/or refuse connections?

Be prepared to repeat this process and refine the test scenarios when the system fails.

Key Factors for Successful Load Testing

Test Plan, People and Tools. This is obvious but, without the proper plan, people skills, and tools the test results will yield ineffective and frustrate the organization.
Testing at different speeds. It is the only way that allows you to see if slower connections use more resources. This may reduce the number of virtual users who may simultaneously visit a Web site.
Testing on different browsers. You don't generally do load testing to check your browser, but it can offer important insight into the error-free performance of Web-based applications on different browsers.
The ability to do complex scenario to simulate user experiences. To simulate a real user experience, the company that load tests needs to create a scenario where the information to do the test are provided on the testing browsers. This scenario needs to be closely alike the transactions performed by the real users of the Web site.

Our customer feedback indicated that managers and developers wanted clear reports that show them errors, time response, all information that help optimizing the performance of their web site.

Given the growing interest of enterprises for this activity, the report of the testing data should try to be more concise and easily understandable. As for the testing tool, it should be more user friendly and intuitive.

Customers who were the most satisfied with their load-testing tools were those who thought that their products did not only a good job of predicting real-world performance but also provided easy-to-understand reports. Price was not a significant criterion of satisfaction, nor was the total IT budget of the respondent's company. Overall, individuals currently using load-testing tools placed greater importance on the various activities provided by vendors than did individuals who were not presently conducting load tests.

Summary
The first point to check so as to measure the quality of a load testing tool; is its ability to simulate the real user behavior at your Web site. You must be sure the tool can adapt itself to your needs through relevant scenarios, monitoring and other relevant activities. The more information a load testing tool can accumulate about your Web site and the user experiences, the more likely it is to identify load problems that would affect your end-users.

The last point to check so as to evaluate your load testing tool is the information and advice that are provided at the end of the test. Ask for an example of report so as to see if the information provided are useful for you. Of course be careful to choose an enterprise that has experience in load-testing and that can provide you services and help you if necessary in your load testing project. The recommendations of load testing experts help to optimize the use of the load testing tool.

To learn more about CTE's Performance/Load Testing capabilities, please contact Mark Luciw at (443) 994-2701 or e-mail MarkLuciw@CelticTestingExperts.com.