Nix Fleet, Part 2: Remote Builders — Never Compile Linux on a Mac Again

2026-06-26T00:00:00+00:00

Nix Fleet, part 2. Part 1</a> laid out the one-flake-every-machine architecture. This part solves the problem that makes a mixed Linux/macOS fleet painful: my laptop can't build Linux. </blockquote>
When you develop NixOS configs on a Mac, you hit a wall fast: an ARM macOS machine cannot build x86_64-linux</code> or aarch64-linux</code> derivations locally. Nothing about Apple Silicon produces Linux binaries. So either you accept that you can only iterate on Linux configs from a Linux machine, or you set up a remote builder and make the problem disappear.
I set up the remote builder. Now my M3 laptop offloads every Linux build to a beefy x86_64 NixOS box, and from where I sit it's transparent — nix build</code> just works, the fans on the laptop stay quiet, and the heavy lifting happens on a machine designed for it.
The shape of the solution</h2> There are two roles:The builder — my always-on x86_64 NixOS box (SSH alias nixos</code>). It has the cores, the RAM, and the right architecture.</li> The controller — my M3 MacBook. It evaluates the flake and orchestrates, but delegates the actual compilation.</li> </ul> The trick that makes this clean: configure the controller so that it does zero local jobs and ships everything to the builder. Builder side: accept work, and emulate ARM</h2> The builder needs to be reachable over SSH and willing to build for the architectures I care about. The one non-obvious piece is ARM: I also run an aarch64-linux</code> machine (a Helios64 NAS</a>), and rather than stand up a separate ARM builder, I let the x86 box build ARM derivations through binfmt/QEMU emulation: # on the x86_64 builder boot.binfmt.emulatedSystems = [ "aarch64-linux" ]; </code></pre> That one line registers a QEMU user-mode handler so the x86 host transparently runs aarch64</code> build steps. It's slower than native ARM (more on that below), but it means one builder covers my entire Linux fleet across both architectures. Controller side: offload everything</h2> On the Mac, I point Nix at the builder and tell it not to build anything itself: # /etc/nix/nix.conf (or the equivalent NixOS/darwin option) builders = @/etc/nix/machines builders-use-substitutes = true max-jobs = 0 </code></pre> # /etc/nix/machines nixos x86_64-linux / - 8 1 kvm,big-parallel </code></pre> The magic value is max-jobs = 0</code>. It tells the local machine to run zero build jobs — so every derivation that isn't already in a binary cache must go to a remote builder. There's no "sometimes it builds locally and melts the laptop" ambiguity; local builds are simply off. builders-use-substitutes = true</code> lets the builder pull from binary caches directly instead of having the controller ferry everything, which saves a lot of pointless data movement. The /etc/nix/machines</code> line reads as: builder nixos</code>, builds x86_64-linux</code>, default SSH key, up to 8</code> parallel jobs, speed factor 1</code>, and it supports the kvm</code> and big-parallel</code> features (so it'll accept jobs that need KVM or heavy parallelism). Verifying it actually offloads</h2> The failure mode here is silent: you think you're offloading and you're not. So verify. First, list what the flake even knows how to build: nix eval .#nixosConfigurations --apply builtins.attrNames </code></pre> Then kick off a real Linux build from the Mac and watch the builder, not the laptop: nix build .#nixosConfigurations.<host>.config.system.build.toplevel </code></pre> If it's working, the laptop sits nearly idle while the x86 box lights up. If max-jobs</code> is misconfigured, you'll feel it — the Mac will either try (and fail, for Linux) or grind. Watching nproc</code>/free -h</code>/build load on the builder during a build is the quickest confirmation. When builds go wrong</h2> Offloading introduces its own failure modes, and I keep a short runbook for them: OOM on the builder, especially during ARM-emulated builds, which are memory-hungry. Mitigations: serialize with --option max-jobs 1 --option cores 2</code>, or add temporary zram</code> swap:{ zramSwap = { enable = true; memoryPercent = 50; }; } </code></pre> </li> Inspecting a failed derivation without re-running the whole build:nix log /nix/store/<hash>-<drv>.drv | less -R </code></pre> </li> Checking effective settings when something isn't offloading as expected:nix show-config | egrep '^(cores|max-jobs|builders|system-features|substituters)' </code></pre> </li> </ul> The honest trade-off</h2> Emulated ARM builds are slow — QEMU user-mode emulation has real overhead, and a big aarch64</code> derivation built on x86 can take noticeably longer than it would native. For my fleet that's an acceptable price: I build ARM rarely (the NAS config doesn't change often), and the alternative is maintaining a second physical ARM builder. If you're iterating heavily on ARM, buy or borrow a native aarch64</code> builder and add it to /etc/nix/machines</code> as a second entry — the controller config doesn't otherwise change. The architecture scales to N builders the same way it handles one. The win is the same one Part 1 was about: I get to live entirely in macOS, drive my whole Linux fleet from there, and never once wait on my laptop to compile something it was never meant to compile. Part 3</a> tackles the next fleet-wide concern — secrets — including how a first-boot install fetches them securely over the same SSH trust I just set up here. The builder/controller config is part of my parallaxisjones/dotfiles</code></a> repo. — Parker Jones, parkerjones.dev</a>

Parker Jones Dev Blog - remote-builders

Nix Fleet, Part 2: Remote Builders — Never Compile Linux on a Mac Again

The shape of the solution</h2> There are two roles:</p> The builder</strong> — my always-on x86_64 NixOS box (SSH alias nixos</code>). It has the cores, the RAM, and the right architecture.</li>