Add /generate-feature-docs skill for spec-to-docs generation#675
Add /generate-feature-docs skill for spec-to-docs generation#675
Conversation
Two-stage skill that converts implemented engineering specs into publisher-facing docs at docs/guide/, with handle verification against code, augment-in-place behavior for existing pages, and mechanical updates to configuration.md, api-reference.md, and error-reference.md. Establishes a spec-readiness convention (status frontmatter) and a directory split between drafts and implemented specs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sixteen tasks across three phases: prerequisites (VitePress exclusion, directory layout, spec migration, CLAUDE.md update), skill implementation (slash command + SKILL.md built up incrementally), and validation (greenfield, idempotency, augmentation, non-feature, and drift cases). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… SKILL Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds Section 16 documenting three amendments motivated by the JS Asset Auditor greenfield validation: verification-rate threshold (Section 16.1), branch-state heuristic (Section 16.2), and verified_against_commit audit field (Section 16.3). Section 6 updated to list the third optional field and to use chat-native prompt phrasing consistent with SKILL.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…SKILL - Edit 1: add optional frontmatter fields block (implemented_in, last_reviewed, verified_against_commit) to the spec readiness check section; specify that verified_against_commit is surfaced in the Stage 1 outline header - Edit 2: add Step 1.7a with verification-rate computation and a hard prompt when fewer than 50% of handles verify against the codebase - Edit 3: add Step 1.7b with a branch-state heuristic using git log from merge-base; surfaced as an informational note in the Stage 1 outline header - Step 1.9 outline template updated to include Verified against commit metadata and the conditional branch-state note Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirrors the optional field added in design Section 16.3 and SKILL.md, so the documented schema stays in sync with what the skill recognizes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Auto-formatted by prettier --write to satisfy the docs/format CI check. Mechanical changes only: trailing whitespace, table column padding, blank lines around fenced code blocks, italic style normalized to underscores. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ChristianPavilonis
left a comment
There was a problem hiding this comment.
Summary
This PR is directionally useful: the two-stage docs workflow, code-grounded extraction, spec lifecycle convention, and VitePress exclusion are all good additions. My main concern is that the skill reads too much like a policy document and uses many hard-negative rules. That will likely make the agent rigid and less collaborative. I recommend rewriting the skill around defaults, warnings, and ask-before behavior, reserving hard stops for destructive actions like pushing, committing without approval, or editing code when the task is docs-only.
| - One file under `docs/guide/<feature-slug>.md` (created or augmented). | ||
| - Up to three additive updates to `docs/guide/configuration.md`, `docs/guide/api-reference.md`, and `docs/guide/error-reference.md`. | ||
|
|
||
| Writes are confined to the four files listed above. You never write any other file, never open PRs, never push, never deploy, never modify code under `crates/`, and never modify the spec you are reading. |
There was a problem hiding this comment.
Output scope should be a default, not an absolute prohibition: This section uses hard boundary language: writes are confined to exactly four files and the agent never writes anything else. For a skill, I think this should be phrased as a default scope with ask-before behavior. Some docs work may legitimately need a sidebar, index, or related guide update.
Suggestion:
Default write scope is the generated feature page and relevant reference docs. If another docs file appears necessary, explain why and ask before editing it.| - Any other value, or missing `status`: stop. Print: | ||
| > "This spec has `status: <value>` (or no status). The skill operates on `status: implemented` specs. Continue without status: implemented? Reply `y` to proceed." | ||
|
|
||
| Wait for the user's reply. Treat any reply other than a single `y` (case-insensitive) as abort. On `y`, print this warning once before continuing: |
There was a problem hiding this comment.
Avoid exact-response gates for normal collaboration: Requiring a single literal y makes the skill feel brittle. This is useful for irreversible actions like committing, but too strict for proceeding with a draft.
Suggestion: Treat non-implemented status as a warning and continue on ordinary affirmative replies such as yes, proceed, continue, or draft it. Reserve exact confirmation for commits or destructive operations.
|
|
||
| If `verified_against_commit` is present, surface its value in the Stage 1 outline header (alongside the other metadata) so the user can compare it against the current branch state if drift is suspected. | ||
|
|
||
| ## Style rules (apply to ALL output, both chat messages and written files) |
There was a problem hiding this comment.
Style rules should apply to generated docs, not chat: Applying these rules to all chat messages is likely one reason the agent feels stiff. The agent should be able to collaborate naturally in chat while still producing docs that match the site style.
Suggestion:
Style rules apply to generated documentation files. In chat, be concise, clear, and collaborative.I would also remove the duplicate self-check section near the bottom of the skill.
| 1. Compute `verified_count` (handles marked `verified`) and `total_extracted_count` (all extracted handles, regardless of type). | ||
| 2. Compute `verification_rate = verified_count / total_extracted_count`. | ||
| 3. If `total_extracted_count` is zero, this is the "no shipped code" edge case (Section "Edge cases and failure modes"). Use the existing handling for that case. | ||
| 4. If `verification_rate < 0.50`, add a hard prompt to the Stage 1 "Issues" subsection: |
There was a problem hiding this comment.
The verification threshold should be a confidence warning, not a gate: A hard 50 percent verification threshold is likely to create false friction because handle extraction can be noisy. Specs may include examples, pseudocode, planned names, or docs-only concepts that do not grep cleanly.
Suggestion: Summarize handles as verified, unverified, and ambiguous. Ask how to treat unverified handles that would appear in final docs, but do not block generation solely based on a numeric threshold.
|
|
||
| Runs only after the user types `proceed`. Inputs: the spec, the approved outline from stage 1, and the existing docs. Output: files written to disk; nothing is committed until the user approves the diff. | ||
|
|
||
| ### Step 2.1: Branch check (before any writes) |
There was a problem hiding this comment.
Move branch and working-tree checks to commit time: Refusing to proceed before any writes makes the skill harder to use for drafting. Users often want to generate files, inspect the diff, and decide later. Branch hygiene matters most when committing.
Suggestion: Allow draft generation on any branch after user approval. Before committing, check the branch and working tree, stage only the generated docs files, and ask if a docs branch should be created.
|
|
||
| If any are present, rewrite. This includes prompts, status updates, summaries, the extraction outline, and the final diff-review message. | ||
|
|
||
| ## Out of scope |
There was a problem hiding this comment.
Prefer default boundaries over an out-of-scope list: This section is written entirely as negative constraints. In practice, those do not instructions can make the agent stubborn. I would reframe this as default boundaries and ask-before-expanding-scope guidance.
Suggestion:
## Default boundaries
This skill is optimized for publisher-facing docs. By default, focus on guide pages, reference docs, and code search for grounding concrete claims. If the user asks for adjacent work, such as updating narrative docs, changing specs, or preparing a PR, pause and confirm the expanded scope before doing it.
aram356
left a comment
There was a problem hiding this comment.
Summary
The skill design is solid and the two-stage flow (outline → user redirects → diff → explicit commit) with hard stops on main/master and uncommitted-tree state is well-thought-out. CI is green. I built the docs locally and confirmed srcExclude produces 40 HTML files with zero under */superpowers/* in dist/.
One blocker: the migration of existing specs into drafts/ is incomplete, and the body's count claim does not match the diff.
Blocking
🔧 wrench
-
Incomplete migration:
docs/superpowers/specs/2026-04-02-pr10-logging-initialization-design.mdwas not moved intodrafts/. The PR body claims "One-time migration of 12 existing specs intodrafts/," and the implementation plan (docs/superpowers/plans/2026-04-28-generate-feature-docs-skill.md, Task 3) lists 12 files, with Step 1 verifyingls docs/superpowers/specs/*.md | wc -l == 12. The actual diff shows 8 renames + 4 net-new drafts, and at PR HEAD the filedocs/superpowers/specs/2026-04-02-pr10-logging-initialization-design.mdstill sits at the spec root, not underdrafts/.This file landed on
mainin commit73101982(PR #610), which is in the merge base of this branch (c7e1e692). The migration commit17a2207a Move existing specs to drafts/...ran before this file was on the branch and was never re-run. CI didn't catch it becausesrcExclude: 'superpowers/**'masks the entire tree from the public build. The structural intent ("all specs live underdrafts/orimplemented/") is broken onmainonce this lands.Fix:
git mv docs/superpowers/specs/2026-04-02-pr10-logging-initialization-design.md \ docs/superpowers/specs/drafts/2026-04-02-pr10-logging-initialization-design.md # then prepend the same `---\nstatus: draft\n---\n\n` frontmatter the other 8 received(This file is plausibly a candidate for
implemented/since PR #610 has shipped, but the convention is "spec lands indrafts/first, gets promoted in a later body-update PR" — defaulting todrafts/is the safe choice for this PR.)
❓ question
- PR body says 12-spec migration; diff shows 8 renames + 4 net-new drafts. Was "12" intended literally (in which case the wrench above plus three more — but I can only find one stray spec at
specs/), or was it a stale plan reference and the actual count was always 8 renames? Please reconcile the body with what landed once the wrench is fixed.
Non-blocking
🤔 thinking
- SKILL Step 1.7b bash one-liner can fail open (SKILL.md:147). Subshell expands to empty when both
mainandmasterare absent, producing an invalid revision range; gate explicitly. Inline comment with details. - Verification semantics include test-only matches as
verified(SKILL.md:120). A handle that exists only inside#[cfg(test)]/mocks passes the verification-rate gate. Filter or add a third state. Inline comment. - Slug rule is non-deterministic, undermining the idempotency claim (SKILL.md:85). Inline comment with details.
- No prettier run before commit (SKILL.md:333). The skill commits LLM-emitted markdown that is unlikely to match prettier; pushing will fail CI's
format-docscheck. Inline comment.
♻️ refactor
git checkout -bdoesn't handle existing branch; prefergit switch(SKILL.md:239). Inline comment.grep crates/**/*.rsrelies on shell globstar (SKILL.md:120). Inline comment with portable alternatives.
🌱 seedling
- 50% verification-rate threshold is a tunable magic number (SKILL.md:138). Track and revisit after real usage. Inline comment.
- Empty
implemented/at merge time. First-time use of/generate-feature-docshits an empty directory. Consider a follow-up that promotes one shipped spec (e.g., the EC KV work, or the same PR10 logging spec referenced above) so the skill is exercisable end-to-end against this repo.
📌 out of scope
- No automated guard against re-exposing
superpowers/on the public docs site (config.mts:41). Plan documents the manualfindcheck but it isn't in CI; a small post-build assertion would prevent regressions. Inline comment.
CI Status
- cargo fmt: PASS
- clippy (Analyze rust): PASS
- cargo test: PASS
- vitest: PASS
- format-docs: PASS
- format-typescript: PASS
- CodeQL: PASS
- integration tests: PASS
- browser integration tests: PASS
Local verification: cd docs && npm run build succeeds, find docs/.vitepress/dist -path '*/superpowers/*' returns zero results.
| @@ -0,0 +1,408 @@ | |||
| --- | |||
| name: generate-feature-docs | |||
| description: "Use when generating, writing, or updating publisher-facing documentation from an implemented engineering spec. Activates on requests like \"generate docs for spec X\", \"write a guide page for the RSL spec\", \"update docs for the EC KV extension\". Operates on specs under docs/superpowers/specs/implemented/ with status implemented frontmatter." | |||
There was a problem hiding this comment.
⛏ nitpick — The description value is a single ~700-char line. YAML allows folded (>) or literal (|) block scalars that render the same to the harness and edit much more cleanly:
description: >
Use when generating, writing, or updating publisher-facing documentation
from an implemented engineering spec. Activates on requests like
"generate docs for spec X", "write a guide page for the RSL spec",
"update docs for the EC KV extension". Operates on specs under
docs/superpowers/specs/implemented/ with status implemented frontmatter.|
|
||
| ### Step 1.3: Resolve the target page path | ||
|
|
||
| Slug the feature name to kebab-case (e.g., "RSL AI Crawler Licensing" becomes `ai-crawler-licensing`). The target page is `docs/guide/<slug>.md`. |
There was a problem hiding this comment.
🤔 thinking — Slug rule is non-deterministic, which undermines the idempotency requirement.
The example silently drops the leading acronym (RSL AI Crawler Licensing → ai-crawler-licensing). A different invocation could produce rsl-ai-crawler-licensing or rsl-licensing. If the slug differs across runs, the augment-vs-greenfield decision flips and the "zero diff on re-run" guarantee in the Idempotency section breaks.
Either codify a mechanical rule ("lowercase, replace runs of non-alphanumeric with single hyphens, strip leading/trailing hyphens") and treat the example as a stylistic shortening that requires user confirmation, or always confirm the slug with the user on first run and persist it (e.g., via a TODO marker in the spec frontmatter).
|
|
||
| For each handle, search the code: | ||
|
|
||
| - Config keys: grep `crates/**/*.rs` and `trusted-server.toml` for the key name. Capture the file and line number when found. |
There was a problem hiding this comment.
♻️ refactor — grep crates/**/*.rs requires shopt -s globstar in bash and silently expands wrong on plain sh. Recommend modeling the right pattern:
rg --type rust --fixed-strings -- '<key>' crates/
# or
find crates -name '*.rs' -exec grep -nH -F -- '<key>' {} +Claude will likely reach for rg regardless, but the skill should show the pattern that works portably.
🤔 thinking — Verification semantics include test-only matches as verified. A handle that exists only inside #[cfg(test)], mocks, or fixtures will pass the verification-rate gate even though no production code path uses it, hiding the very drift the gate is designed to catch. Either filter */tests/* (and #[cfg(test)] blocks) from the search or add a third state ("verified in tests only") so the user can see it in the outline.
|
|
||
| > "Stage 1 verified `<verified_count>` of `<total_extracted_count>` handles in this codebase (`<rate>%`). Below 50% suggests this spec may not be fully implemented in this branch. Options: (A) generate stubs for unverified handles, (B) abort and check status, (C) override and proceed normally." | ||
|
|
||
| The user must pick A, B, or C before the skill proceeds. The threshold of 50% is initial; tune based on real usage. |
There was a problem hiding this comment.
🌱 seedling — The 50% threshold is a magic number, and the design itself flags it as "initial; tune based on real usage." After a few real runs, capture the actual verification rates (and the false-positive vs false-negative outcomes at 50%) and revise. Worth a follow-up issue so the calibration doesn't get forgotten.
| In addition to handle verification, check whether the current branch has touched code relevant to the feature: | ||
|
|
||
| ```bash | ||
| git log --name-only $(git merge-base HEAD main 2>/dev/null || git merge-base HEAD master 2>/dev/null)..HEAD -- crates/ trusted-server.toml |
There was a problem hiding this comment.
🤔 thinking — If both git merge-base HEAD main and ... master fail, the $(...) subshell expands to empty and the line becomes:
git log --name-only ..HEAD -- crates/ trusted-server.tomlwhich git rejects as a bad revision. The skill text below this block says "skip this check silently and proceed" but the literal command doesn't enforce that. Capture the merge-base into a variable and gate on it:
MB=$(git merge-base HEAD main 2>/dev/null || git merge-base HEAD master 2>/dev/null)
if [ -n "$MB" ]; then
git log --name-only "$MB"..HEAD -- crates/ trusted-server.toml
fi| > "You are on `<branch>`. Create branch `docs/<slug>` and switch to it?" | ||
| - The user can specify a different branch name. | ||
| - The skill refuses to proceed on `main` or `master` under any circumstance, including override attempts. | ||
| - After confirmation, run `git checkout -b <branch-name>`. |
There was a problem hiding this comment.
♻️ refactor — git checkout -b <branch-name> will fail if docs/<slug> already exists (which it will on a re-run of the skill against the same spec on the same day). Modern git also prefers git switch. Suggested:
if git show-ref --verify --quiet "refs/heads/docs/<slug>"; then
git switch "docs/<slug>"
else
git switch -c "docs/<slug>"
fi|
|
||
| ### Step 2.7: Commit | ||
|
|
||
| Stage explicitly via path: |
There was a problem hiding this comment.
🤔 thinking — No prettier run before commit. The repo enforces cd docs && npm run format in CI (the format-docs check). LLM-emitted markdown is unlikely to match prettier's rules exactly, so users who push the skill's commit will hit a CI failure and have to amend.
Suggest running the formatter on just the modified files before the diff-review message in Step 2.5:
(cd docs && npx prettier --write \
guide/<slug>.md \
guide/configuration.md \
guide/api-reference.md \
guide/error-reference.md)or, simpler: instruct the user to run npm run format before pushing as part of the final "Committed as <sha>" message.
| description: | ||
| 'Privacy-preserving edge computing for ad serving and edge cookie (EC) generation', | ||
| base: '/trusted-server', | ||
| srcExclude: ['superpowers/**', '**/node_modules/**'], |
There was a problem hiding this comment.
📌 out of scope — srcExclude works (verified locally: cd docs && npm run build produces 40 HTML files, zero under */superpowers/* in dist/), but there is no automated guard against a future edit silently re-exposing internal specs. The plan's Task 1 Step 4 documents the manual check (find docs/.vitepress/dist -path '*/superpowers/*') but it never made it into a CI assertion.
Suggest a follow-up issue to wire that check into the docs build job — a one-line post-build assertion that exits non-zero on any hit.
Summary
Adds a Claude Code skill that converts implemented engineering specs into publisher-facing documentation pages on the Trusted Server VitePress site. Closes a recurring gap where specs get written and code ships, but the docs at iabtechlab.github.io/trusted-server do not get updated.
The skill is invoked as
/generate-feature-docs <spec-path>and runs in two interactive stages: an extraction pass that produces a structured outline for user review, and a generation pass that writes prose, applies mechanical updates to reference docs, and commits on user approval. Built entirely on Claude Code's existing skill and slash-command primitives. No new runtime infrastructure.What's in this PR
.claude/skills/generate-feature-docs/SKILL.mdand slash command at.claude/commands/generate-feature-docs.md.docs/superpowers/specs/drafts/for brainstorm output,docs/superpowers/specs/implemented/for post-implementation truth, withstatus:frontmatter (draft,in-progress,implemented).drafts/withstatus: draft.srcExcludeforsuperpowers/**so internal specs no longer render on the public docs site.CLAUDE.md.docs/superpowers/specs/drafts/2026-04-28-generate-feature-docs-skill-design.mdand implementation plan atdocs/superpowers/plans/2026-04-28-generate-feature-docs-skill.md.What is NOT in this PR
srcExcludeconfig; the only runtime impact is on the public docs site (which removes internal spec pages, the desired outcome).Risk profile
Low. The skill is markdown-only, no compilation, no execution path through the runtime. The most material change is to the public docs site (internal specs no longer indexed). External links to those spec URLs will break, but they were never intended to be public.
Test plan
/generate-feature-docs docs/superpowers/specs/drafts/2026-04-28-generate-feature-docs-skill-design.mdto confirm the slash command is auto-discovered. Replyabortat the readiness prompt; expect a clean exit with no files written./superpowers/.cargo fmt,cargo clippy,vitest,npm run formatindocs/,npm run buildindocs/. Thecargo test --workspaceViceroy failure is pre-existing and unrelated to this branch.Follow-up issues
How to use the skill
After this lands, any team member who pulls
maincan invoke/generate-feature-docs <spec-path>against any spec underdocs/superpowers/specs/implemented/(withstatus: implementedfrontmatter) to generate or augment its publisher-facing guide page. The skill verifies handles (config keys, endpoints, headers, error variants) againstcrates/, surfaces drift before writing prose, and commits the result on user approval. See the design spec for the full behavior contract.