Skip to content

0082: Plugin documentation is generated from the kernel and recorded by the tooling every plugin shares

Context and Problem Statement

A plugin brings modules, and with them operations, settings, credentials, capabilities, themes, playbooks, jobs and, declared contributions to the web app. Today the core documents its own modules (MODULES.md, CLI_REFERENCE.md, CONFIGURATION.md, generated by scripts/generate_docs.py) and shows one animated CLI intro in the README, recorded by VHS from a tape the modules' demo.tape snippets assemble. A plugin gets none of that: its README is written by hand, nothing shows its screens, and a company's private plugin has no page at all. The maintainer asked for a page per plugin, with animated recordings of its CLI and its UI, its kinds, its CLI reference "and all of that", generated automatically for first-party, community and private plugins alike, with tooling and CI, and for the README's intro to show the web app beside the CLI. Which program produces the pages, where does its knowledge come from, how are the recordings made reproducible, and how does every plugin get all of it without copying code?

Decision Drivers

  • Heads are views on operations (ADR 0003): the documentation is one more view, derived from what the kernel knows, never from parsing source.
  • One copy of the tooling, in the core, that plugins get through the template (ADR 0007, ADR 0017): first-party, community and company plugins are documented the same way, and make update-template brings improvements.
  • Deterministic and offline page generation; recordings use fresh private profiles without inherited machine logins. Scene execution retains the caller's OS permissions, so scenes and transcripts are reviewed before publication.
  • Nothing private leaks: a private repository's pages never land on a public site.
  • A fresh clone needs only git, make and uv; tools are pinned (ADR 0036).
  • Accessible: every animation has a text alternative and a still for people who turn motion off.

Considered Options

  • A system.plugin.docs module whose operations introspect the kernel, Markdown pages per plugin, recordings by a shared script (VHS for the CLI, Playwright for the UI) from declared scenes, MkDocs Material for the site (chosen)
  • A system.docs module of its own: it would need system.plugin's kinds and first-party list, and the system submodules don't import each other (import-linter); documentation of plugins is the plugin module's concern.
  • Grow scripts/generate_docs.py: it imports heads and built-in modules, so it can't run in a plugin's repository, and copying it there would drift. It stays, for what only the core has (CLI reference, badges, README).
  • Parse the plugin's source (AST, docstrings): finds declarations the kernel never loads and misses what a module computes; the kernel already holds every let with its owner, and hz system modules proves it.
  • Hand-written pages from a template: the README the template renders today; it drifts the day the plugin changes, which is the problem.
  • Astro Starlight for the site: a good site, but a Node toolchain in every plugin repository, which has none (plugins ship no web/; the core installs Node only to build its app). MkDocs is a Python package uv locks like any other dev dependency, its input is the GitHub-renderable Markdown already generated, and it deploys to GitHub Pages as static files.
  • Playwright video recordings: a webm per scene whose length and frames differ run to run. A scripted sequence of screenshots with declared hold times gives the same frames for the same app, and far smaller files.

Decision Outcome

Chosen option: documentation is a view on the kernel, hz system plugin docs, which writes a page per plugin from the lets the kernel holds; recordings come from scenes a plugin declares in data, rendered by a script every plugin gets from the template; MkDocs Material turns the pages into a site on GitHub Pages, because the pages can't drift from the code, no plugin writes tooling, and a private plugin gets the same pages without a public site.

  • The shared exposure rule lives in the kernel. heinzel.kernel._exposure projects the actual head restrictions and private-operation policy, reusing the canonical path helpers in heinzel.kernel._paths. Heads retain their compatibility import from heinzel.heads._exposure; documentation reads the same projection. Public REST and HTTP MCP exclude private operations; the scoped private API and authenticated web head use their own columns. input_schema describes the same public parameter types and declared constraints, without a blanket strictness override. Existing exposure and private-head parity tests continue to cover the shared rule.

  • The model. hz system plugin docs show [--plugin NAME] (read-only; the heads of hz system plugin list: CLI, MCP over stdio, scoped private API and authenticated web head, because installed-provider metadata is yours to see, not any local page's) returns Documentation: per distribution (PluginPage) its name, summary, version, licence, kinds (connector, theme, capability, playbook, as PluginInfo.kinds defines them, plus layer), source repository and install command; per module its operations as every head sees them (CLI command and usage, MCP tool name and input schema, separate HTTP MCP/public REST/private API/web exposure, private, read_only, audience, heads), its settings (key, type, default, description, environment variable, from the settings model's JSON schema), credentials (name and purpose, never a value), adapters and the capabilities they serve with their hosts, requirements, themes (every token in light and dark, fonts, radius and the complete checked theme including backdrops and variants), playbooks (asset types, templates, recommended settings), jobs (schedule, catch-up, timeout), layers, setup step and health checks. Every other kind of let, including kinds added later such as UI contributions, is listed by its kind and name, so a new let shows up before the renderer learns about it; services are internal and left out. hz system modules stays the one-line listing of modules and their lets by kind; this is the full account, for people and agents alike.

  • The pages. hz system plugin docs build [--directory OUT] [--plugin NAME] (CLI and MCP over stdio: it writes files) writes OUT/<distribution>/index.md per plugin (default docs/plugins), GitHub-renderable Markdown: the summary, kinds and install line first, then the recordings (an animated image with alt text, a <details> transcript of what it shows, and a still for prefers-reduced-motion), then operations with CLI usage, MCP schema and endpoints, then settings, credentials, capabilities, theme swatches (SVG written beside the page from the colours, light and dark), playbooks, jobs, layers, and the lets the renderer has no section for. A README.md in OUT lists the pages built. One shared writer validates the complete plan, refuses custom files and links, and rechecks generated markers on the actual no-follow descriptors. New files and directories start owner-only before deliberate publication. Recording inputs are bounded regular files; linked transcripts cannot copy outside content into a page. In the core, scripts/generate_docs.py writes that gallery instead, adding the plugins the core knows of but doesn't ship: the reference company and docs/plugins/registry.toml, a list community repositories add themselves to (repository data, so core-only). --check reports the pages that differ from what would be written; make fix writes them, make check checks them, in the core and in every plugin.
  • The scenes. A plugin declares what to record in docs/scenes.toml: [[cli]] scenes (a title, hz command lines, hold times) and [[ui]] scenes (a title and steps: goto a route, click a role and name, hover, fill, key, wait for text, screenshot a caption). A scene needs no code and names roles and labels, never selectors. Without the file, one CLI scene is derived from actual CLI-visible module paths (--help, then read-only operations without required arguments, everyday ones first, the first also as --json); a plugin without CLI operations gets no derived CLI scene. UI scenes are declared only, since which screen shows an operation is the web app's knowledge (web/src/layout/navigation.ts), not the kernel's, and guessing or parsing it is what this record rejects. Commands typed in a CLI scene start with hz and are checked like the README's demo.tape snippets, with a closed stdin-filter flag allowlist. Control characters, shell operators and reader file operands are rejected. Transcript stages receive argument vectors and stdout only; diagnostics remain separate from the next reader's input. Scenes execute reviewed hz operations in the caller's process permissions, with isolated profile state rather than an OS security boundary. A step with optional = true is skipped when its element isn't there. [scenes] plugin = "<distribution>" names whose scenes the file holds (a repository of several plugins needs it); the core's file also has [intro] scenes, the scenes appended to the README's intro. show and build take --scenes PATH (default docs/scenes.toml, when it exists), --plugin NAME and --builtin (the page heinzel of the core's own modules). The modules' demo.tape snippets stay: a snippet is a module's contribution to the core's README intro, a scene is a plugin's own recording. Dependencies are installed before recording; snippets cannot remove or reinstall the tool environment they run from. The recorder renders both through the same tape settings, and merging the two formats is left open below.
  • The recordings. scripts/docs_record.py (PEP 723, shared through the template) builds a sandbox like the README demo's script did (fresh private temporary HOME/XDG and uv tool directories, its own HZ_* directories, a made-up profile, HZ_UPDATE_CHECK=false, and the credential backend environment), installs the core and the plugin editable, then: for each CLI scene writes a VHS tape (settings fixed: theme, font, size, framerate, typing speed) and renders PNG frames; for each UI scene starts hz ui --no-browser --port 0 and drives it with Playwright (the Python package, so a plugin needs no Node) in light and dark mode at a fixed viewport, taking one screenshot per step. Frames become docs/plugins/<distribution>/assets/<scene>.gif, .webp and .mp4 through the pinned ffmpeg (scripts/tools.toml, group demo), each inside the requested budget and a 480 KiB hard cap below the unchanged 500 KiB large-file hook. Every format and still is staged and checked before export; failed encodings preserve previous assets. Descriptor-relative replacement also works across temporary/project filesystems. The first frame is kept as <scene>.png (light and dark). Playwright's Chromium comes from playwright install, pinned by Playwright's version but not checksummed by tools.toml: unpinned in ADR 0036's sense, as VHS's own Chromium already is, and acceptable for an optional target. The core thus carries two Playwrights, the npm one make test-e2e drives the app with and the Python one the recorder uses, because plugins have no Node. The transcript (<scene>.md) lists the steps and the text each frame shows, so the page's <details> and alt text come from the same data. make docs-record renders locally; owned backend startup and shutdown are bounded, including silent startup and descendant cleanup. The Markdown build references only checked local files and never needs the recording tools.
  • The site. mkdocs.yml at the repository root, MkDocs Material, the generated pages and the hand-written docs as its navigation; make docs-site builds site/. The source hook rejects linked and nonregular repository files before rendering or copying the frozen checkout. The template gets the same, with the plugin's own pages.
  • CI. The docs step of make check fails when a page is stale or missing (text only, offline). The Docs workflow (.github/workflows/docs.yml, shared with the template) re-records every scene when the scenes or the demo tape change on main and on demand, and opens a pull request with the changed media (as the Demo workflow did for the README's GIF: rendering isn't byte-identical between machines, so media isn't diffed on every pull request, and a release doesn't re-record, which would grow the history by megabytes each time). On a release release-tag.yml calls it (a tag pushed with the GitHub token triggers nothing): it builds the site from what main holds and deploys it to GitHub Pages when the repository is public, or when the repository variable DOCS_PAGES is true. Otherwise it uploads the site as a workflow artifact and stops: a private repository's pages stay with the repository.
  • The README's intro. docs/scenes.toml of the core declares the intro ([intro] scenes): after isolated installation completes, real first commands in the CLI (the demo tape), then Today, the workflow, a Heinzelmännchen's page, the team graph and a theme switch in the UI, then the CLI equivalents. The recorder concatenates the tape's video and the scenes' frames into one docs/assets/demo.gif (and .mp4, .webp; make demo), so one pipeline makes both the README's intro and every plugin's recordings. The modules' demo.tape snippets stay the CLI's part of it, assembled by scripts/demo_tape.py as today.
  • Public API, additive (ADR 0026): Exposed, exposed, public_parameters, owns_path, collapses, restricts, input_schema, MCP_STDIO and MCP_HTTP in heinzel.kernel.
  • Dependencies. mkdocs-material (MIT) and playwright (Apache-2.0) join the docs dependency group, which only the documentation targets install; neither ships with hz, so ATTRIBUTIONS.md, which lists what hz ships, is unchanged, and .license-allowlist already allows both licences. VHS, ttyd and ffmpeg stay the pinned demo tools.

Consequences

  • Good: a plugin author gets a complete page, a gallery entry and recordings by running make fix and make docs-record; the page can't drift, because it is checked like MODULES.md.
  • Good: a company's private plugins get the same pages in their own repository; nothing is published unless they turn Pages on.
  • Good: coding agents read the same model through MCP (system_plugin_docs_show), so "what does this plugin do" has one answer for people and agents.
  • Bad: the recorder needs VHS, ttyd, ffmpeg and a Chromium; make docs-record installs them, but it is the slowest step of the repository, so recording runs on scene changes or demand; releases build the site from reviewed media.
  • Bad: a scene is a promise about the UI's labels; a renamed button breaks a recording, which the Docs workflow then reports. The scene format names roles and labels, never selectors, to keep that rare.
  • Neutral: scripts/generate_docs.py keeps the core-only outputs (CLI reference, badges, README blocks) and calls the same builder for docs/plugins.

Confirmation

  • tests/unit/modules/plugin_docs_test.py: the model names every let kind of a test plugin (an unknown kind among them), hides injected parameters, names exactly the commands the CLI head exposes, derives the CLI scene, checks the scenes file's grammar (hz commands only, names, a screenshot per UI scene, duplicates), renders every section with the recordings found, and --check reports a stale page without writing.
  • tests/integration/plugin_docs_test.py: hz system plugin docs show --json and build through the CLI head with the Acme plugin from tests/fixtures/layers.
  • tests/unit/heads/exposure_test.py and tests/unit/public_api_test.py: the exposure rule and the names it adds to the public API.
  • Recorder and registry regressions cover shell-free stdin pipelines, control characters, reader operands, private HOME/credential isolation, bounded/reaped backend startup, all-format size limits, failed-export preservation, cross-filesystem export and escaped registry text/HTTPS links. File tests reject symlinks, hard links and custom output in both the CLI builder and the core generator; the site hook rejects linked repository sources.
  • The docs step of make check (scripts/generate_docs.py --check) and the template's make check run the builder in check mode; make template-test renders a plugin and runs its checks, so the generated page of the example plugin is part of CI.
  • tests/unit/documentation_test.py keeps every generated page listed and every link resolving.

More Information

  • Requested by the maintainer on 2026-10-24: a page per plugin with animated CLI and UI, the plugin's type, its CLI reference, generated for first-party, community and private plugins with tooling and CI; the README's intro to show the UI too, automatically. The community weather plugin (heinzel-contrib) is the showcase.
  • UI contributions (footer items, Today widgets, sidebar pages) are being decided in a separate record; this one lists their lets generically until that record names their fields, then the renderer gets a section for them.
  • Open: one grammar for the README intro's CLI part and a plugin's CLI scenes (today demo.tape snippets and [[cli]] scenes).
  • Open: whether a plugin's page should also be served by hz ui (a "Plugins" screen that renders the Markdown), and whether hz system plugin search should show a plugin's documentation URL from its repository metadata.