For most commerce software engineers, dependency management includes upgrades and best practice suggests upgrading dependencies proactively. In order to ensure a smooth process and one that is maintained properly, it is best to have a well-defined policy and process.
Why you need a policy and process
All software depends on other software to accomplish useful tasks. Even the smallest hello world program depends on the libraries of the language it is written in. Typically, as a piece of software gets more sophisticated it depends on more third-party software or libraries. Leveraging existing third-party libraries is a standard practice for commerce engineers to enable faster development.
However, vulnerabilities and bugs of dependencies become vulnerabilities and bugs of the software the dependencies are included in. As a result, dependencies must be chosen with due consideration, and constantly monitored. Best practice is to not only upgrade dependencies proactively, but to also be on the latest versions of libraries as often as possible.
Why Upgrade Dependencies?
The single biggest reason for upgrading dependencies is security. Most software has vulnerabilities that are gradually fixed over time. Upgrading dependencies, is the simplest way to avoid security issues. The latest versions can introduce new vulnerabilities that are not yet discovered. However, it is still safer to upgrade in order to fix known vulnerabilities.
Most of the changes between versions are bug fixes and performance improvements, that’s another good reason to upgrade. In recent tests, a docker version upgrade gave a significant boost to the performance numbers.
Other reasons to upgrade include being able make use of new features, avoiding giant upgrade steps as it is always easier to update in smaller increments, and it is also easier to upgrade a single dependency if everything else is up to date.
Upgrade Experiences
Commerce software developers should be upgrading dependencies on a regular basis.
Here are a few examples of good, bad and ugly dependency upgrades that inform the upgrade process.
The Good
Overall library upgrades should be smooth or have relatively few issues. Good libraries maintain backward compatibility, and provide suitable replacements for any APIs that are deprecated. As a result, upgrading such libraries should involve only minor code changes. Good migration guides, and well-crafted deprecation warnings make this process relatively painless. Recent examples of these include various Apache Commons libraries, Mockito, and RxJava.
The Bad
Every now and then an upgrade does not go according to plan, and unexpected issues arise due to not having the bugs or changes well documented. Recent example of this occurred with an upgrade to Guava version 25. Guava upgrades are typically seamless and jumping to version 25 was expected to be a simple version bump.
However, intermittent errors occurred in performance tests because of a misbehaving cache. The intermittent nature of the errors meant that they were difficult to track down, resulting in a days-long test to locate the issue. To allow enough time to deal with such surprises, it is best to do upgrades at the beginning of a release cycle.
The Ugly
Sometimes an upgrade can get ugly. This is typically the case when a number of dependencies have to be upgraded in lockstep. This problem will be exacerbated if regular upgrades are not done, and several upgrades of versions need to take place in one go.
As an example, a recent Spring upgrade required upgrading Cucumber. Upgrading Cucumber required upgrading Spring. Also resulting in needing to upgrade Camel, Junit, Servicemix, and Gemini. Attempting to untangle this dependency web was challenging. 
Automation
Upgrading dependencies is not the most exciting work for a commerce engineer. To ensure upgrades are quick and happen often, it is strongly recommended to leverage all automated tools available.
For example, Versions Maven Plugin is best to use for maven projects to generate reports listing the libraries that have new versions available. The plugin can also auto-upgrade the versions directly in source code. Most languages and frameworks have similar tooling available.
It is worth noting to only use the auto-upgrade feature to do revision, and minor version upgrades. These upgrades are typically quick and require minimal code changes. Overall, the plugin is a huge time saver.
A major version upgrade is often a more conscious decision to upgrade with the understanding that such an upgrade will require some work.
New vulnerabilities are discovered every day, and it is hard to keep up with without some automation.
Use a security and license compliance tool, e.g. Black Duck, to identify vulnerabilities and license compliance issues in the open source libraries.
Lastly, it is recommended to create a robust suite of functional and performance tests that ensure library upgrades do not have negative consequences.
The Upgrade Process – what a strong process looks like
- Identify the libraries to be upgraded. Leverage an automated tool in this step. Most languages and frameworks have one. E.g. Use Versions Maven Plugin for Maven projects.
- Ensure version being upgraded to does not have known security vulnerabilities, and license issues. Before changing a single line of code one can verify candidate versions using a security compliance tool e.g. Black Duck.
- Do the bare minimum code change to upgrade the version i.e. fix any compilation errors, and any functional test failures. Don’t try to use any new features at this point.
- Verify performance. Run your whole performance test suite.
- As much as possible, do the upgrades at the beginning of a release cycle to give you time to live with any surprises that might come up.
- As an optional exercise – use the new features. Take a look at release notes of the new versions, check if there is anything that catches the eye and would be helpful. Use your judgement to determine if it might be useful to spend a bit of time on this.
In closing, be sure to continue to learn from experiences and fine tune this process.
The post Dependency Upgrades: why commerce software engineers need a policy and process appeared first on Get Elastic Ecommerce Blog.
from
https://www.getelastic.com/dependency-upgrades-why-commerce-software-engineers-need-a-policy-and-process
No comments:
Post a Comment