Performance Services & Tool Leasing
SQA has found that most applications fail at 5-10 concurrent users under first time performance tests. Few applications out of the box are optimized to support the largest possible user base and achieve reasonable response times with minimum hardware. SQA's PSP offering leverages commercial and open source tools to get maximum performance from your applications. Our basic 3-step approach:
Baseline Scalability Test
A number of pre-determined tests are created and executed against a real-world model and simulated by running a suite of scripts on a population of concurrent virtual users. The test scenario gradually builds load to the expected thresholds until the system fails or until either front-end or back-end performance criteria thresholds are violated. This provides a benchmark of probable "day one" performance of the system. This test can also be performed on a predecessor system to give an "apples to apples" comparison of performance, or on an earlier release of the same system to ensure that new functionality/defect repairs have not degraded performance.
Baseline Stress Test
This test leverages the scalability test baseline, but in this case the test scenario builds load to the expected thresholds and beyond (typically at least 150% of expected usage), gradually increasing the concurrent user volume until "user transactions" experience error conditions (typically 5% of transactions), the system fails, or either front-end or back-end performance criteria thresholds are violated. This test verifies the robustness of the system as well as providing an indication of probable "headroom" remaining to support usage growth.
Tuning/Optimization Test
Performance metrics are monitored and analyzed to identify the root cause of performance bottlenecks or scalability issues. Development addresses any remedial action to alleviate problems and resubmits the application/system for further testing. These tests may be a repeat execution of earlier scalability/stress tests, or customized tests to demonstrate specific performance "tweaks". These tests can also be run against various software/hardware configurations for network tuning or to conduct "what-if" testing of competing hardware configurations.