Performance testing: Strategy for the cloud based environment

My previous blog in this series was about the role of functional testing in the cloud environment. In this blog, let’s see how performance testing plays a major role in cloud migrations.

Performance testing Strategy for the cloud based environmentPerformance testing in the cloud is fundamentally similar to how it is done in a non-cloud environment. However, there are differences while planning and running performance tests on the cloud – as certain category of performance tests may not be applicable in a cloud environment (depending on the type of cloud adoption – public, private or hybrid cloud). In a public cloud, the performance tests that can impact other customers are not performed (e.g. stress test). Care should be taken to baseline and measure the response times, as the response time on a cloud environment may not be similar to that for a non-cloud based infrastructure. Elasticity testing is applicable in the case of cloud-based performance testing. Typical tests applicable in the context of cloud are:

  • Load test: Start with getting information from the cloud provider on the load statistics of other customers sharing a public cloud platform. This will give us insights in what kind of response time to expect (as load on other applications hosted on the public cloud can impact our application). Run the tests when the load is expected to be high (i.e. real time load situations). Importantly, run these tests regularly to get a range of response times in order to establish “minimum response time”, “maximum response time” and “average response time”.
  • Stress test: As stress test in a public cloud can impact other customers, extreme care should be taken when planning and executing stress tests in a public cloud situation, as compared to a private cloud context where the cloud is in our exclusive ownership and control. However, using the boundary value analysis technique, stress just beyond the agreed range can be applied to verify whether the situation is gracefully handled.
  • Endurance or volume test: Using memory monitoring tools, endurance or volume test is conducted. In the case of IaaS, the test team can monitor the memory by installing memory monitoring tools in the virtual environment itself.
  • Testing elasticity and scalability: The objective of elasticity testing is to determine whether the performance meets the defined SLAs and does scalability happen at the determined point. In a pay-per-use model, elasticity testing is necessary to validate the generation of financial invoice based on the amount of infrastructure really consumed. The approach consists of gradually increasing the load to cross the threshold of the current limit, thereafter check whether the system scales up as the load increases and scales down as the load reduces. In this case, the boundary value analysis testing technique may be useful to determine the edge of different limits.

What has been your experience? How did you go about addressing some of the challenges in doing performance testing in a cloud environment? In my next blog I plan to share my experiences with security and interface testing in a cloud environment.

Venkata Ramana Lanka

Director - QA, Virtusa. Venkata Ramana Lanka (LRV) is an accomplished software test management professional with an extensive background in software testing, test automation framework design, building domain specific testing solution accelerators, leading large software testing teams and supporting presales initiatives. LRV is an hands-on manager with proven ability to direct and improve quality initiatives, reduce defects and improve overall efficiency and productivity. He has in-depth and proven expertise in developing and implementing test strategies and operational procedures. LRV has extensive experience in working with multiple commercial and open source test tools. As a QA thought leader, LRV has written multiple White papers and articles some of which have been published in various media. Further, he has spoken and presented papers at various software testing conferences.

More Posts - Website

One Comments

  • Addex IQ March 4, 2016

    Hi, keep up posting such content.

Comments are closed.