MSRV and semantic versioning

[dr-orlovsky]

In Edition 2022 Rust has introduced MSRV enforced at the cargo build system level in Cargo.toml. This had a good indent, but ended up badly.

A lot of dependencies bump their MSRV in Cargo.toml in patch or minor releases, breaking compilation of everything below. Thus, regularly, your crates which were released and worked fine become uncompilable in one-two month term since they get some of dependencies bumping their MSRV higher than MSRV of your crate.

There is a different problem, which were present before edition 2022, but now with MSRV just became even more apparent: people do not care about semantic versioning at all, and released crates stop compiling due to breaking changes in dependencies over a course of half year or so.

TLDR of the problem

I believe these are two manifestations of the same problem: semantic versioning doesn't work as was expected. In reality, people think that 1.2.0 and 1.3.0 can be API-breaking (and do that the same way as they freely break APIs between 0.2.0 and 0.3.0). However, in most of cases, from my experience, they never break in between patch versions, i.e. 1.2.1 and 1.2.2 are mostly compatible - except the case of MSRV, which is frequently broken even in patch releases.

Proposed solution

First, abandon specifying MSRV in the Cargo.toml: this will let you still be compilable with the latest stable even if some of dependencies breaks on MSRV. Provide MSRV information in README, manifest, anywhere you'd like - but not in Cargo.toml.

Second, instead of caret-style dependency references, recommended by Cargo book (i.e. version = "^0.5.1", which is equivalent to version = "0.5.1") we should use tilde-style references with just two components always present and the last component being non-zero: ~0.5. If there is no minor non-zero version (like the latest version is 1.0.5) we should use normal garret-style with all three components given.

This will result in the fact that we always ignore patch version, but disallow even minor version updates, which should be done manually after testing.

Hopefully this should prevent released crates from breaking the builds.

Calling into the thread @yanganto, @nicbus and @Kixunil.

[yanganto]

The rust-version in Cargo.toml is important for a crate and possibly much more in the future.

The MSRV is not a definition, it is a status. It is not the thing written down in Cargo.toml, README, or MANIFEST.yml, but it is proof that can build pass in our CI server.

There is some chaos, so Rust gradually manages this. In the future, I believe the crate publishing process possibly checks on this field with Cargo.lock. We are in the Rust ecosystem, we need to follow this, our users will be happy to adapt our crates to their projects without a doubt. For now, it is easy to have a CI check that MANIFEST.yml is the same version as Cargo.toml, or a tool/build_script to generate MANIFEST.yml from Cargo.toml.

Some crates are bad, but we do not have time to rewrite everything. On the other hand, if we have other older MSRV in Readme, and a user runs into the older version issue than the MSRV they said, that will be our mistake. There is nothing bad about having a younger MSRV when we correctly show what the upstream crates' requirement is. Besides, a user wants a legacy version of Rust and special support from us. We have experience and are happy to take care of this.

[dr-orlovsky]

This is all good except it doesn't anyhow solves the problem we are facing today: published crates stop building after a while (weeks) since once of dependencies bumps MSRV against semantic versioning.

[yanganto]

We need to lock our dependency with exactly the full three digits, then use a CI job to check on any update automatically, GitHub already provides a free bot to work on this issue.

Also, it will provide a special summary report(which can be accessed only by the repo admins) if there is an additional security issue on this update, such that we will easily decide whether to bump MSRV for a new version. It will be my pleasure to introduce all CI flow into RGB repos. 😄

[yanganto]

The following are related PRs, @nicbus @Kixunil If you can review only the first one 🙏, and other PRs will follow the first one to implement.

• Refactor ci flow rgb-core#235
• Refactor ci flow rgb-std#181
• Refactor ci flow #161
• Add CI rgb-interfaces#1

TLDR about the implementations

Nix provides a modulized CI we can have RGB-WG/utils and every other repo under RGB-WG can import the CI template and patch the flow as needed. However, current PRs are not doing that way. The main reason is not everyone here familiar with modulized Nix stuff, you will see the repeating pattern in the following PR, but it will be easier for people to modify it. It will also be easier to reproduce any errors seen in the CI server, we can just nix develop .#stable -c command. Nix will prepare the same environment for you to test.

Here is a single command to install nix on any linux or mac
$ sh <(curl -L https://nixos.org/nix/install) --no-daemon

[yanganto]

The dependency updates seem annoying at this moment because we lack unit tests. Such that the bot is removed and will be considered after having a better unit test coverage.
RGB-WG/rgb-core#249

[Kixunil]

Hey, sorry for being late, I've been away for a while.

I think this is a misunderstanding of how dependencies work. Firstly rust-version in Cargo.toml does not prevent one from using a higher version of the compiler. Removing it doesn't help at all.

Second you say

people think that 1.2.0 and 1.3.0 can be API-breaking

Who actually does so? I've never seen this in Rust except for genuine mistakes which can be fixed. I suspect you may think MSRV is a part of API and you consider bumping MSRV breaking. Under Rust rules it's not because the code still compiles fine on a newer version. It's equivalent to upgrading any other dependency. The way to fix it is to use Cargo.lock I believe which should be committed for libraries too. In bitcoin we've committed two lock files to lock down dependencies and avoid breaks. Take a look there.

In case it's not really MSRV but people actually breaking the API and refusing to fix the mistakes then I'd love to know who are they so that I know who to avoid.

[Kixunil]

Looks like this whole issue is basically blocked on Cargo handling MSRV properly. E.g. making sure cargo add picks the right version. This work is currently ongoing, so it shouldn't take super long for it to resolve.

What I meant is "I think we should be thinking about them as API-breaking" since in reality it is so frequent that hard to count.

Over-restricting creates bigger problems. I'm lately even thinking that it should be possible to override major version restriction. If a dependency broke the API multiple times and was unwilling to add cargo-semver-checks to its CI, I'd just run away from it. (fork it)

Serde_with

They do revert breaks though: jonasbb/serde_with#572

[dr-orlovsky]

Looks like this whole issue is basically blocked on Cargo handling MSRV properly. E.g. making sure cargo add picks the right version. This work is currently ongoing, so it shouldn't take super long for it to resolve.

Not sure it is that simple.

I basically have a lot of crates which compile today - and will stop compiling in few weeks since some shitty deep dependency I have never heard of (seventh dependency of some kind of serde family) bumped MSRV in minor or patch - and Cargo.lock doesn't save from that since I can't lock versions of all dependencies of my dependencies (it is equal to set all deps in Cargo.toml to = up to a patch).

Thus, the system just does not fucking work as indented and recommended due to a human factor.

fork crate

It is not just about forking but also about maintaining after. I already maintain > 100 crates with > 4m downloads, so I can't fork everything indefinitely, especially of serde_with kind

[Kixunil]

I can't lock versions of all dependencies of my dependencies

Why? I never had a problem with it.

It is not just about forking but also about maintaining after. I already maintain > 100 crates with > 4m downloads, so I can't fork everything indefinitely, especially of serde_with kind

Yeah, that sucks but I don't really see a better way. :( For a while I was thinking of commercial crates for Rust that would guarantee some kind of quality and support.

[dr-orlovsky]

Why? I never had a problem with it.

Well, if you try to use =xx.xx.xx everywhere you will soon witness how your library will break all compilation for any of its users in just a matter of a few weeks

[Kixunil]

I though by "lock" you mean using Cargo.lock.

Anyway, I was recently thinking of having some out-of-band MSRV index. That would help people resolve issues with crates not having MSRV specified. Maybe it could also work as some kind of test index where a crate is tested with all MSRV-capped versions of its dependencies (not all combinations since that has exponential complexity, unfortunately). That way we could have an alternative to cargo add and cargo update which would take these indices into account.