Cloud as a technology is no longer new. Many organizations are openly adopting cloud based services and many others are migrating existing services to cloud. This change or adoption of cloud does raise many questions – “what are the important factors impacting cloud testing?”, “how should we test the success of migrating to cloud?”, etc.
Important factors impacting cloud testing
Cloud does not change the traditional testing process that is adopted by different organizations. The fundamental test process consisting of test planning and control, test analysis and design, test implementation and execution, evaluating existing criteria, and reporting and test closure activities is equally applicable in the cloud context. Before we start, it is important to look at cloud definition and its characteristics.
The National Institute of Standards and Technology (NIST), USA clearly defines the essential characteristics of cloud computing. Cloud testing derives its test strategy based on this definition. According to NIST, “cloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management of effort or service provider interaction.”
The cloud computing model promotes availability and is composed of five essential characteristics (on-demand self-service, broad network access, resource pooling, rapid elasticity and measured service); three service models [Cloud Software as a Service (SaaS), Cloud Platform as a Service (PaaS), and Cloud Infrastructure as a Service (IaaS)]; and four deployment models (private cloud, community cloud, public cloud, and hybrid cloud). Key enabling technologies of this model include: (1) fast wide-area networks, (2) powerful, inexpensive server computers, and (3) high-performance virtualization for commodity hardware.
Test strategy is significantly impacted as a result of one or more of these essential characteristics, service models and deployment models. This essentially means that while the business process testing (or functional test strategy) will more or less remain the same, irrespective of cloud or web based hosting model, non-functional aspects of testing like application performance testing, security testing, interface testing, compatibility testing and test environment management (environment provisioning, environment availability and environment release) are the key areas that will get impacted (will differ from web) while switching over to cloud.
Another important factor determining test approach is the software development model. Generally one of the following three software development models is seen. Each model is unique and depending on the model, test strategy will undergo a change.
- Cloud is used only to host production application: In this case the quantum of testing is very much restricted since any test will directly impact production. Any test that requires interaction with system’s DB (login validation, data entry, data modification, etc.) will not be allowed (though risky and not advisable, in certain situations, test logins are created on the production and shared with a restricted audience to run E2E functional tests).
- Development happens outside cloud environment, while testing and deployment happen on the cloud: In this scenario, the code is built in-house and deployed on a test server hosted on the cloud. Here, it is essential to understand whether all interfaces to the application are deployed on the cloud or whether the test server has to connect to certain applications outside the cloud environment. Interface testing is a critical focus area as defects can reside at the interfacing points.
- Development, testing and deployment are all done on cloud: While this model is yet to be widely embraced by software organizations (data security, privacy and complete dependence on cloud provider are causes of concern), this model works seamlessly. From a testing perspective, this is the best model as the team need not validate in different environments. Full-fledged “Continuous Integration Testing (CIT)” runs through automated tests, and performance testing can be seamlessly executed.
Functionally, there will be no change in the test approach between web based and cloud environment. However, test teams should focus on certain aspects like performance, security, environment availability, integration with interfaces, customer experience and adherence to SLAs when testing in a cloud based environment.
Test strategy undergoes changes depending on the service models and implementation models selected. Choice of tools and techniques will be similar as in the case of web based applications, however, during tool selection the fact that they are available on cloud and they support the pay-as-you-use model should be considered.
If you have already migrated solutions to the cloud, we would like to hear your thoughts and experiences around some of the factors and challenges you faced while testing cloud solutions. Please feel free to share your comments. Come back to this space for my forthcoming blogs on the various test strategies, test environments and types of testing that organizations need to consider for successful cloud adoption.