Using the Correct Versions in Your Package Manager

This article is aimed at everyone from novice to expert. If you don’t feel like reading the summary is:

When introducing a new package to your application you should find the latest stable version and include it with the major, minor and patch versionwith the tilde operator like: ~1.2.3

Package managers have become common place in most languages. Node uses npm, PHP uses composer, Ruby uses gems… and many more.

A package manager is not just useful to fetch dependencies for us, but also to resolve version requirements between many different packages to try to ensure that everyone gets along as best as possible.

The vast majority of projects that are designed to be used by a package managers follow the semantic versioning rules close enough that it rarely causes problems. This is great and is enough for most projects.

If you prefer a safer and easier process. Or, if you have ever asked yourself the following questions, then this article will dive deeper:

  1. How is semantic versioning supposed to work?
  2. What is the correct way to handle merge conflicts on a lock file?
  3. What is the correct version or operator to choose for my packages?
  4. It is better to have more flexible or stricter versioning requirements?

Semantic Versioning Recap

You probably understand the basics of semantic versioning. There is a more in depth explanation here. I need to highlight a few points:

  1. The semver is broken into three parts — the major, minor and patch version. There is more, but we only care about these ones for the moment.
  2. The rules for major and minor are different when the major version is 0.
  3. The version is supposed to specifically represent the API of the package, not the features or changes.

It’s the last point that causes the confusion when selecting and resolving versions. In a perfect semantic versioned world increasing the minor or patch version should be 100% backwards compatible. Most applications are designed to pull in dependencies with this in mind, but I’m going to explain why this is generally not a good idea and should actually be avoided.

What’s the Difference Between the ^ and ~ Operators?

There is a healthy mix of operators to constrain versions in different ways. The obvious ones like > and < I won’t go into. However, I will explain the difference between ~ (tilde) and ^ caret.

The ^ operator specifies a minimum version and creates an upper bound that must be less than the next major version. This allows minor and patch releases above the version:

  1. ^1.2 means >=1.2.0 and <2.0.0.
  2. ^1.2.3 means >=1.2.3 and <2.0.0.

The ~ operator follows all the same rules as the ^ operator but with one important distinction. It will only allow the last digit to go up. This means that:

  1. ^ and ~ behave the same if patch is not provided. Eg. ~1.2 is the same as ^1.2.0 and the same as ^1.2.
  2. ~1.2 means >=1.2.0 and <2.0.0, however ~1.2.0 means >=1.2.0 and <1.3.0.

How It Applies to Your Application

I say application here because I am talking specifically about an end product (like a web application) and not about a reusable library. Libraries/packages should not follow the rules below.

Consider the following rules:

  1. New packages should strive to use the latest stable version that is available and compatible with your application.
  2. You should always be able to include the latest patch versions (they should only contain bug and security fixes that do not change the API).
  3. You do not want to include new features for existing packages, even though it does not (or at least, should not) break the existing API.

The last point sounds counterintuitive but it makes sense when you also consider:

  1. When your application uses a package you don’t care if that package introduces new features after the you have developed the solution because you are not using them.
  2. Not everyone follows semver closely enough that you can be absolutely sure that it won’t change or break the existing API or functionality in slight ways that can introduce silent (or obvious) bugs.
  3. Anyone (even in a team of one person) should be able to safely update dependencies with impunity to get the latest bug fixes without breaking others parts of the application that would be unrelated.
  4. You should wait until you need the new functionality of the new version of the library in your application. This means that your packages will be updated at the same time you do testing on that area. You do testing, right?

“But I test my/our entire entire application very throughly with every release!”

That’s great but the first point is still true, if you application is not using the new features of the package then why do you need to include them right now?

Handling Merge Conflicts in The Lock File

The lock file (which should be committed with the rest of your application) contains specific versions that the package manager has chosen that fits all the requirements of your application, its dependencies and their descendant dependencies.

Most version control conflicts happen when the same lines (or overlapping lines) are changed by more than one person. This requires intervention to understand the context and resolve the issue (mashing the lines together almost certainly would not work).

You may have noticed that if the package manager file that outlines your dependencies (such as composer.json in PHP) contains two different changes by different people the file does not conflicts but the lock file will be in a conflict. This is actually by design.

The conflict is caused by the hash that represents the whole state of the lock file. The packages resolved by each person may be different from the packages that would be resolved if we updated the packages at the same time. Another way to put it is that we know the application works with the independent version changes but we have to prove that the application works with both version changes considered. The conflict prompts us to perform consider the update with both changes at the same time.

Hopefully, if you agree with the advice in How It Applies to Your Application, the process is simple and safe:

  1. Delete the lock file.
  2. Run update so that all new versions are resolved for the whole application.
  3. Commit the new lock file.

Otherwise the recommended course of action is (to prevent updating packages that are not yours):

  1. Look at the diff of the package manager file (ie. composer.json) and store or remember the changes.
  2. Revert the lock file to it’s previous version.
  3. Perform the diff changes manually with your package manager. The new composer.json should look the same as what the merge originally suggested but you will have a new lock file that only has changes to a specific subset of packages (the ones involved in the merge).

If you notice, the more complicated process really just avoids unrelated packages from changing at all which means you get the pain and don’t even get the lovely security updates and bug fixes!

In conclusion, when introducing a new package to your application you should find the latest stable version and include it with the major, minor and patchversion with the tilde operator like: ~1.2.3

Originally published at on December 20, 2017.

I’m a data nerd and TDD enthusiast originally from Sydney. Currently working for Uber in New York. My thoughts here are my own. 🤓