Performance test design – Top 5 considerations

When we give top down estimates for performance testing in scope, we normally consider the application type and the type of protocol. Once the application is ready, we discover that the performance testing effort needed is much higher due to various factors like environment availability, instable code, incompatible tool etc. It may even go to the extent of reselecting the tool, rewriting the performance test strategy etc. This might lead to surprises for the customers, as they normally expect results as per their project schedule.

In order to avoid such situations, we can consider the following five points and set the customer expectation from the beginning:

  • Protocol: Even if the application being tested is a web-based application, we may need to consider the kind of protocols used in detail. We cannot assume that Http or Https for a web based application is the default – it could be a combination of protocols. If the application is being tested first time for performance, it is better to include a POC phase in the performance testing lifecycle to freeze the protocol clearly before we proceed further.
  • Testing tool: If there is a tool available with the customer for performance testing, we need to consider that as the first choice. If the application has been tested already using the same tool, we may not require any new tools. If the application is new, sometimes we may end up using some other testing tool due to compatibility issues. So, a POC is highly recommended to ensure this if the application is being developed for the first time.
  • Application type: Web applications could be two major types. A business process might call for one user role if it is a simple online shopping kind of scenario. In the case of workflows, a business process might call for multiple roles to complete all the activities involved in that. So, the scripting, test data preparation, application data preparation, test execution etc., might differ a lot. We need to ask specifically what kind of application is given for testing.
  • Architecture: The kind of architecture might change the overall design of the performance testing. For example, if there is a cluster based architecture, then it will be possible to narrow down the reasons quickly. If it is a normal 2 or 3 tier architecture with a couple of powerful servers, then the test strategy needs to be different to verify the scalability.
  • Load generation: If the requirement is to validate the network latency by injecting loads from multiple locations of the world, we may end up spending higher setup costs and time. If it is verification of different transactions from the client perspective we may not have higher setup costs, travel, multi-location licenses, etc.
We need to consider these points to make the estimation fairly accurate.

 

Suresh Srinivasan

Director - QA, Virtusa. Suresh is an electrical engineer and holds a masters degree in business administration, with 24+ years of experience in software testing and development. He has worked in different industry domains including manufacturing, Telecom, Insurance, Banking and Logistics. Suresh started his career in testing from the industrial automation and industrial robotics programming and also specialized in business system software testing in multiple domains like Banking, Finance, Logistics, Healthcare, Telecom etc. Currently, at Virtusa he is playing the role of QA practice head for the company's various technology centers in India. He is also a hands-on player on specialized testing and heads some of the activities in this area. During his more than two decades career, Suresh has traveled worldwide to implement various testing projects, covering test automation, SOA testing and performance testing, for global enterprises. Suresh has experience in implementing testing methodologies and concepts from ATLM, TPI, TMAP and TMM. He is also working closely with tool vendors like Rational, HP and iTKO. Leveraging his experience, Suresh has also developed process frameworks for executing testing projects end-to-end which reduces the cycle time while improving the test coverage. He has presented papers on testing solutions at various international testing conferences. Apart from working closely with different educational institutions, Suresh conducts seminars related to software testing so as to bring provide and showcase the practical view to students.

More Posts

4 Comments

  • Neha Sharma August 14, 2014

    It is very well said in the above article that we cannot make prior assumptions about the protocol of the application being tested.

  • Bhaskar August 21, 2014

    Very usefull information provided regarding PT.

  • Sundaresan Jayaraman October 6, 2014

    Very good information! I would like to add one more criteria for Performance Test Estimation:
    *Application/System readiness for Performance Testing
    In some of the organizations, the performance testing environment is set for the first time. In that case smoke test and functional tests to be conducted before commencing the performance testing. These tests take considerable amount of time, due to various fact

  • Sundaresan Jayaraman October 6, 2014

    factors like networking to external applications( approvals required), connecting to internal applications, preparation of test data( in some cases 10K records need to be created, transferred to appropriate location and tested for verifying whether processing is done or not.to test with millions of record. Although, we may call it as smoke testing, it ends with a complete functional and readiness testing, in reality

Comments are closed.