Software product upgrades are nothing new. But with the spread of continuous development and continuous integration they are becoming more and more frequent. Typically, the upgrade process takes one or more specific components and upgrades them to a new version, for example, to a newer code or configuration version. Whenever there is a need to perform an upgrade, there are several factors to consider to ensure that the product works correctly.
One key consideration is downtime – Is the downtime resulting from the upgrade process acceptable? Upgrades can involve some downtime. If availability is critical, as is the case for most online applications today, then it is important that the upgrade process be seamless and errorless. Even in cases where some downtime is acceptable, it is important that it be capped and kept to a minimum.
There are also resiliency issues that need to be addressed. For example, if a request is issued exactly when the upgrade is in initiated, how is the interrupted request handled? Is it picked up later on? If not, what are implications for the system?
Backward compatibility can also be an issue, for example, if a system exposes some kind of API. It is important to make sure that the API does not “break”, otherwise external users of the API might not be able to continue to use it. This can create problems for system integration.
Data migration is a particularly painful issue. If the upgrade includes changes in the data schema, then the data itself must also undergo migration. Data migration tends to be prone to errors, and the successful migration of data should be carefully verified.
Testing as a tool for smoother migration
The overall functionality and stability need to be verified before upgrading your production environment. If something isn’t quite right with even a single component, the functionality and availability of the entire system might be compromised.
Manual testing can be very time consuming. For this reason, tests should be automated. Upgrade tests should be performed as part of the Continuous Integration process so that any problems are detected as early as possible in the development process.
Best practice is to ensure seamless upgrade with zero downtime is to make sure that all application servers are stateless (instead of stateful). The best way to discover stateful servers is through upgrade tests, purely functional tests do not detect this kind of issue.
For upgrade testing to be reliable, it needs to be run on an environment that closely mirrors the production environment. There are a lot of environment-sensitive components that affect upgrading, for example, load balancing and database configuration.
4 Tips for planning your upgrade testing
In planning for your next upgrade, consider these stages in developing your upgrade test efforts:
- Identify the relevant upgrade scenarios
- Set up the test environment, with simulated data
- Develop the test infrastructure and the actual tests
- Run the test on the test environment as part of continuous integration
Ravello’s ability to replicate your production environment provides a feasible option for creating a cloned environment for testing. In a future post, we will describe how we perform migration tests at Ravello.
Instant Dev/Test Environments – Learn more