Cargo Workspace and the Feature Unification Pitfall

In Rust, workspaces help organize a project into smaller packages. The downside is if there are shared dependencies that list different features, then one will most likely see unexpected behavior when compiling a binary from the workspace level. This post shows an example of when this can occur and what solutions exist.

For me, workspaces have worked nicely while developing a project for server, browser, and desktop targeted code.

Except for one vexing situation.

On the server, we need the fastest implementation of gzip compression, so in our toml we have:

[dependencies.flate2]
version = "1.0.20"
features = ["zlib-ng-compat"]
default-features = false

Which adds to the following entry to the Cargo.lock

[[package]]
name = "flate2"
version = "1.0.20"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cd3aec53de10fe96d7d8c565eb17f2c687bb5518a2ec453b5b1252964526abe0"
dependencies = [
 "cfg-if 1.0.0",
 "crc32fast",
 "libc",
 "libz-sys",
 "miniz_oxide",
]

This Cargo.lock is shared with all members of a workspace. Notice that libz-sys is listed as a dependency. Cmake is required to build libz-sys. This is not a problem for the server which is built on a linux machine where cmake is available.

A desktop binary (housed within the desktop package) is the issue, as it also relies on flate2 but doesn’t require the same level of performance, so the desktop only references the default flate2 implementation. And yet, compilation will fail due to cmake not being detected when we try and build this desktop binary:

cargo build --bin run_app

with the following error:

error: failed to run custom build command for `libz-sys v1.1.2`
  --- stderr
  thread 'main' panicked at '
  failed to execute command: The system cannot find the file specified. (os error 2)
  is `cmake` not installed?

The fix seems simple. Since flate2 defaults to a pure rust backend instead of the c backend that is introduced with the zlib-ng-compat feature, we can solve this problem by directing cargo to use the default implementation for the desktop version.

Failed Attempt: Cfg Dependencies

My first thought was to have the server package describe that flate2 should only be compiled with the zlib-ng-compat feature enabled on unix.

[target.'cfg(unix)'.dependencies.flate2]
version = "1.0.20"
features = ["zlib-ng-compat"]
default-features = false

[target.'cfg(not(unix))'.dependencies.flate2]
version = "1.0.20"

But this won’t work – at least not yet, we’ll reference this later.

Perils of Feature Unification

Viewing the dependency tree with features shown should help us understand the problem.

cargo tree -e features

Which returns an important snippet where we see the flate2 feature listed twice:

|-- flate2 feature "default"
|   |-- flate2 v1.0.20 (*)
|   `-- flate2 feature "rust_backend" (*)
|-- flate2 feature "zlib-ng-compat"
|   |-- flate2 v1.0.20 (*)
|   |-- flate2 feature "libz-sys"
|   |   `-- flate2 v1.0.20 (*)
|   |-- flate2 feature "zlib"
|   |   |-- flate2 v1.0.20 (*)
|   |   |-- flate2 feature "any_zlib"
|   |   |   `-- flate2 v1.0.20 (*)
|   |   `-- flate2 feature "libz-sys" (*)
|   `-- libz-sys feature "zlib-ng"

The reason we see flate2 included twice is because it is compiled with both features enabled. To paraphrase the docs on this: when building a binary in a workspace, the enabled features of a dependency of the package are all workspace packages dependencies enabled features unioned together. Bit of a mouthful.

The docs then go on to say:

The resolver runs a second time to determine the actual features used when compiling a crate, based on the features selected on the command-line.

So we shouldn’t see any trace of zlib-ng in the built executable, right? Executing:

nm ./target/release/run_app

Contains the following symbols.

zng_inflate_fast
_ZN11miniz_oxide7inflate

I’m far from an expert but if zng_inflate_fast is from zlib-ng and miniz_oxide::inflate is from the rust implementation then both crates are included in the final executable. And since the flate2 crate gives preference to zlib-ng when detected, not only are we needlessly inflating the binary size but we’re not even getting the implementation we wanted! Not to go on a tangent but this reminds me of the request to add mutually exclusive features to cargo, so that we don’t end up with the possibility of including two implementations and accidentally using the wrong one.

The docs could possibly be made clearer, as my initial understanding when reading the docs was that even when features are unified at the Cargo.lock level (ie: the resolve graph), individual crates are compiled with the features dictated by the package’s Cargo.toml. But this is not the case as one can inspect the verbose output from a cargo build and see that a feature enabled in one package will affect the features enabled in another.

Is this a bug? I don’t think so, but it would be nice if cargo would give a warning when a crate in a top level package is compiled under a different set of features than stated, though even this may have enough nuance to make implementing painstakingly difficult.

Solution #1: Cargo Resolver 2

Cargo 1.50 introduced a new option where one can specify the use of a different resolver version (shown below at the workspace level):

[workspace]
resolver = "2"

Which has an eye catching feature:

Features for target-specific dependencies are not enabled if the target is not currently being built.

Could this be the solution to our failed attempt with cfg dependencies?

It is!

Solution #2: Manifest Path or Package Flag

If leveraging new features to cargo 1.50 is not viable, there is another solution right within the same docs section linked to earlier:

If you have a circumstance where you want to avoid that unification for different workspace members, you will need to build them via separate cargo invocations.

There are two ways to do this, pick your favorite:

cargo build --manifest-path .\src\desktop\Cargo.toml --bin run_app
cargo build -p desktop --bin run_app

Honorable Mention: Sub-workspaces

If the above solutions don’t work, you can change your mental model of the project: from a single workspace with multiple members to a monorepo that contains many workspaces. This is how cargo fuzz is able to embed itself into a project without changing how dependencies are resolved:

# Prevent this from interfering with workspaces
[workspace]
members = ["."]

This mental model may fit better when working with packages that are related yet serve different use cases, like code targeted at desktop, server, and browsers.

So at the end of the day we have what some may call surprising behavior in feature / dependency resolution, but we have several solutions that one can choose from.