0036: make install installs every tool the build needs, pinned and checksummed, on macOS and Linux¶
Context and Problem Statement¶
Setting up a fresh Mac showed that make setup covered the Python tools but left the rest to be found and installed
by hand: gh for make pr, make ci-logs and make release; vhs with ttyd and ffmpeg for the README demo;
gitleaks, which CI downloaded as a pinned binary with a checksum while the pre-commit hook had prek build it from
source with a Go toolchain it downloaded first. pyright fetched an unpinned Node.js at its first run. External
contributors read in CONTRIBUTING.md that they need "uv and make" and then hit whatever their machine lacks. Which
programs does the build need, and how does a contributor get all of them, the same way on macOS and Linux and in CI?
Inventory, from the make targets, scripts, git hooks, tests and workflows:
| Program | Used by | Before | Now |
|---|---|---|---|
| git, make, bash | everything | prerequisite | prerequisite (macOS: Xcode command line tools) |
| uv | everything; installs Python | prerequisite | prerequisite (make doctor checks required-version) |
| ruff, mypy, pyright, pytest, prek, commitizen, copier, git-cliff, bump-my-version, pip-audit, import-linter, zizmor, typos, actionlint, shellcheck | lint, types, tests, hooks, release | dev dependency group | unchanged |
| Node.js (for pyright) | make typecheck |
downloaded by pyright at its first run, unpinned | pyright[nodejs]: a wheel in uv.lock |
| gitleaks | pre-commit hook; CI history scan | hook: built from source by prek; CI: pinned binary | pinned binary, build group; hook and make secrets |
| gh | make pr, make ci-logs, make release |
by hand | pinned binary, build group |
| vhs, ttyd, ffmpeg, jq | make demo |
by hand | pinned binaries, demo group, on demand; ttyd from PATH on macOS |
| python3 | scripts/template-test.sh |
the system's | uv's (uv run --no-project python) |
| curl, tar, unzip | CI only | runner | runner (unzip checks the wheel); nothing local needs them |
| claude | the Claude review workflow, agents | optional | optional, not part of the build |
Decision Drivers¶
- A fresh clone needs only git, make and uv; nothing needs sudo or a system package manager.
- macOS (arm64, x86_64) and Linux (x86_64, arm64) behave the same, and CI runs what contributors run.
- Every downloaded program is pinned to a version and verified by checksum (supply chain, ADR 0011).
make installstays idempotent and fast when everything is present, and prints one line per tool.- Plugins get the same through the template (ADR 0007).
Considered Options¶
- Python tools stay in the dependency groups; other binaries are pinned per platform with a sha256 in
scripts/tools.tomland installed byscripts/tools.pyinto.tools/bin(chosen) - Homebrew (Brewfile) and apt: differs per OS and distribution, needs sudo on Linux, versions float with the package manager, and CI would install differently from contributors.
- mise, aqua or asdf: pins and verifies binaries well, but is one more program to install before
make setup, and its plugin registries are another supply chain to trust; the set here is five small binaries. - PyPI wrappers for everything: no maintained wheels for gh, gitleaks, vhs or ttyd; where one exists for a binary (imageio-ffmpeg), its file is pinned by checksum without making it a project dependency.
- prek-managed hooks for gitleaks (Go toolchain): works without a system Go, but downloads a toolchain and builds on the first commit, and CI's history scan would still need its own binary: two pins for one tool.
- Containers (dev container): reproducible, but heavy for a Python project, and hooks and
hzwould run inside a container instead of on the contributor's machine.
Decision Outcome¶
Chosen option: dependency groups for Python tools, and pinned, checksummed binaries in .tools/bin for the rest,
because it needs nothing beyond git, make and uv, works identically on all four platforms, and gives each tool exactly
one pin that hooks, make and CI share.
scripts/tools.tomllists each binary: version, group (buildordemo), why it is needed, and per platform the file and its sha256.scripts/tools.py(standard library only, so it runs beforeuv sync) downloads a file, refuses it unless the checksum matches, extracts the binary and links it from.tools/bin/<name>; downloads are cached in~/.cache/heinzel-dev/downloadsby checksum, so more clones and worktrees cost nothing.make installrunsuv sync --all-groupsand installs thebuildgroup;make setupadds the git hooks. Both are idempotent; with everything present they take about a second. Every make target has.tools/binfirst on PATH.- Optional tools install when their target needs them (
demogroup formake demo). A platform without a pinned file (ttyd on macOS) uses the program on PATH and says how to install it (brew install ttyd); the same fallback, with a warning, covers a pinned download that fails. - The gitleaks pre-commit hook and
make secrets(CI's history scan) run the pinned binary throughtools.py run gitleaks. pyright takes Node.js from thenodejs-wheel-binarieswheel (pyright[nodejs]). make doctorlists the prerequisites, the environment, every tool and the gh login, each missing one with the command that fixes it, and fails while something required is missing.- CI's
make setupjob runsmake setupandmake doctoron ubuntu-latest, ubuntu-24.04-arm, macos-latest and macos-15-intel, andmake tools-verifydownloads every pinned file of every platform and checks it. - Bumping a tool: change its version in
tools.tomland runuv run --script scripts/tools.py pin <name>.
Consequences¶
- Good: contributors on a fresh machine run
make setupand are done; CI and the hooks run the same binaries; plugins get it through the template. - Good: one place lists every program the build needs and why.
- Bad: Dependabot doesn't update
tools.toml; the pins are bumped by hand (apincommand makes it one step). The gitleaks hook no longer follows the pre-commitrevthat Dependabot could update. - Bad: vhs still downloads its own Chromium at the first recording, unpinned; acceptable for an optional target.
- Bad: about 60 MB of binaries per machine for the build group, once.
Confirmation¶
tests/unit/tools_test.py checks the manifest (every build tool pinned on all four platforms, checksums, https),
installing, checksum refusal, caching, upgrades, the PATH fallback, run, pin and make doctor, that the Makefile
installs the tools and puts .tools/bin first, and that gitleaks has no second pin in the hooks or workflows. CI's
make setup job proves the bootstrap on each platform and make tools-verify the pins.
More Information¶
CONTRIBUTING.md lists the prerequisites; make doctor explains what is missing.