Skip to content

Add /generate-feature-docs skill for spec-to-docs generation#675

Open
jevansnyc wants to merge 17 commits intomainfrom
feature/generate-feature-docs-skill
Open

Add /generate-feature-docs skill for spec-to-docs generation#675
jevansnyc wants to merge 17 commits intomainfrom
feature/generate-feature-docs-skill

Conversation

@jevansnyc
Copy link
Copy Markdown
Collaborator

Summary

Adds a Claude Code skill that converts implemented engineering specs into publisher-facing documentation pages on the Trusted Server VitePress site. Closes a recurring gap where specs get written and code ships, but the docs at iabtechlab.github.io/trusted-server do not get updated.

The skill is invoked as /generate-feature-docs <spec-path> and runs in two interactive stages: an extraction pass that produces a structured outline for user review, and a generation pass that writes prose, applies mechanical updates to reference docs, and commits on user approval. Built entirely on Claude Code's existing skill and slash-command primitives. No new runtime infrastructure.

What's in this PR

  • Skill at .claude/skills/generate-feature-docs/SKILL.md and slash command at .claude/commands/generate-feature-docs.md.
  • Spec lifecycle convention: docs/superpowers/specs/drafts/ for brainstorm output, docs/superpowers/specs/implemented/ for post-implementation truth, with status: frontmatter (draft, in-progress, implemented).
  • One-time migration of 12 existing specs into drafts/ with status: draft.
  • VitePress srcExclude for superpowers/** so internal specs no longer render on the public docs site.
  • Documentation of the convention in CLAUDE.md.
  • Design spec at docs/superpowers/specs/drafts/2026-04-28-generate-feature-docs-skill-design.md and implementation plan at docs/superpowers/plans/2026-04-28-generate-feature-docs-skill.md.

What is NOT in this PR

  • No Rust application code changes.
  • No JavaScript application code changes.
  • No skill behavior changes to the trusted-server runtime.
  • The only build-time change is the VitePress srcExclude config; the only runtime impact is on the public docs site (which removes internal spec pages, the desired outcome).

Risk profile

Low. The skill is markdown-only, no compilation, no execution path through the runtime. The most material change is to the public docs site (internal specs no longer indexed). External links to those spec URLs will break, but they were never intended to be public.

Test plan

  • Pull this branch, open Claude Code in the repo, run /generate-feature-docs docs/superpowers/specs/drafts/2026-04-28-generate-feature-docs-skill-design.md to confirm the slash command is auto-discovered. Reply abort at the readiness prompt; expect a clean exit with no files written.
  • After merge, deploy the docs site and confirm iabtechlab.github.io/trusted-server no longer serves any URL under /superpowers/.
  • Verify CI passes: cargo fmt, cargo clippy, vitest, npm run format in docs/, npm run build in docs/. The cargo test --workspace Viceroy failure is pre-existing and unrelated to this branch.
  • (Optional) Run a real spec through the skill end-to-end on a feature branch. Validation runs documented in the design spec Section 11.

Follow-up issues

How to use the skill

After this lands, any team member who pulls main can invoke /generate-feature-docs <spec-path> against any spec under docs/superpowers/specs/implemented/ (with status: implemented frontmatter) to generate or augment its publisher-facing guide page. The skill verifies handles (config keys, endpoints, headers, error variants) against crates/, surfaces drift before writing prose, and commits the result on user approval. See the design spec for the full behavior contract.

jevansnyc and others added 17 commits April 30, 2026 10:57
Two-stage skill that converts implemented engineering specs into publisher-facing
docs at docs/guide/, with handle verification against code, augment-in-place
behavior for existing pages, and mechanical updates to configuration.md,
api-reference.md, and error-reference.md. Establishes a spec-readiness convention
(status frontmatter) and a directory split between drafts and implemented specs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sixteen tasks across three phases: prerequisites (VitePress exclusion,
directory layout, spec migration, CLAUDE.md update), skill implementation
(slash command + SKILL.md built up incrementally), and validation
(greenfield, idempotency, augmentation, non-feature, and drift cases).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… SKILL

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds Section 16 documenting three amendments motivated by the JS Asset
Auditor greenfield validation: verification-rate threshold (Section 16.1),
branch-state heuristic (Section 16.2), and verified_against_commit audit
field (Section 16.3). Section 6 updated to list the third optional field
and to use chat-native prompt phrasing consistent with SKILL.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…SKILL

- Edit 1: add optional frontmatter fields block (implemented_in, last_reviewed,
  verified_against_commit) to the spec readiness check section; specify that
  verified_against_commit is surfaced in the Stage 1 outline header
- Edit 2: add Step 1.7a with verification-rate computation and a hard prompt
  when fewer than 50% of handles verify against the codebase
- Edit 3: add Step 1.7b with a branch-state heuristic using git log from
  merge-base; surfaced as an informational note in the Stage 1 outline header
- Step 1.9 outline template updated to include Verified against commit metadata
  and the conditional branch-state note

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirrors the optional field added in design Section 16.3 and SKILL.md, so
the documented schema stays in sync with what the skill recognizes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Auto-formatted by prettier --write to satisfy the docs/format CI check.
Mechanical changes only: trailing whitespace, table column padding, blank
lines around fenced code blocks, italic style normalized to underscores.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown
Collaborator

@ChristianPavilonis ChristianPavilonis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

This PR is directionally useful: the two-stage docs workflow, code-grounded extraction, spec lifecycle convention, and VitePress exclusion are all good additions. My main concern is that the skill reads too much like a policy document and uses many hard-negative rules. That will likely make the agent rigid and less collaborative. I recommend rewriting the skill around defaults, warnings, and ask-before behavior, reserving hard stops for destructive actions like pushing, committing without approval, or editing code when the task is docs-only.

- One file under `docs/guide/<feature-slug>.md` (created or augmented).
- Up to three additive updates to `docs/guide/configuration.md`, `docs/guide/api-reference.md`, and `docs/guide/error-reference.md`.

Writes are confined to the four files listed above. You never write any other file, never open PRs, never push, never deploy, never modify code under `crates/`, and never modify the spec you are reading.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Output scope should be a default, not an absolute prohibition: This section uses hard boundary language: writes are confined to exactly four files and the agent never writes anything else. For a skill, I think this should be phrased as a default scope with ask-before behavior. Some docs work may legitimately need a sidebar, index, or related guide update.

Suggestion:

Default write scope is the generated feature page and relevant reference docs. If another docs file appears necessary, explain why and ask before editing it.

- Any other value, or missing `status`: stop. Print:
> "This spec has `status: <value>` (or no status). The skill operates on `status: implemented` specs. Continue without status: implemented? Reply `y` to proceed."

Wait for the user's reply. Treat any reply other than a single `y` (case-insensitive) as abort. On `y`, print this warning once before continuing:
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoid exact-response gates for normal collaboration: Requiring a single literal y makes the skill feel brittle. This is useful for irreversible actions like committing, but too strict for proceeding with a draft.

Suggestion: Treat non-implemented status as a warning and continue on ordinary affirmative replies such as yes, proceed, continue, or draft it. Reserve exact confirmation for commits or destructive operations.


If `verified_against_commit` is present, surface its value in the Stage 1 outline header (alongside the other metadata) so the user can compare it against the current branch state if drift is suspected.

## Style rules (apply to ALL output, both chat messages and written files)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style rules should apply to generated docs, not chat: Applying these rules to all chat messages is likely one reason the agent feels stiff. The agent should be able to collaborate naturally in chat while still producing docs that match the site style.

Suggestion:

Style rules apply to generated documentation files. In chat, be concise, clear, and collaborative.

I would also remove the duplicate self-check section near the bottom of the skill.

1. Compute `verified_count` (handles marked `verified`) and `total_extracted_count` (all extracted handles, regardless of type).
2. Compute `verification_rate = verified_count / total_extracted_count`.
3. If `total_extracted_count` is zero, this is the "no shipped code" edge case (Section "Edge cases and failure modes"). Use the existing handling for that case.
4. If `verification_rate < 0.50`, add a hard prompt to the Stage 1 "Issues" subsection:
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The verification threshold should be a confidence warning, not a gate: A hard 50 percent verification threshold is likely to create false friction because handle extraction can be noisy. Specs may include examples, pseudocode, planned names, or docs-only concepts that do not grep cleanly.

Suggestion: Summarize handles as verified, unverified, and ambiguous. Ask how to treat unverified handles that would appear in final docs, but do not block generation solely based on a numeric threshold.


Runs only after the user types `proceed`. Inputs: the spec, the approved outline from stage 1, and the existing docs. Output: files written to disk; nothing is committed until the user approves the diff.

### Step 2.1: Branch check (before any writes)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move branch and working-tree checks to commit time: Refusing to proceed before any writes makes the skill harder to use for drafting. Users often want to generate files, inspect the diff, and decide later. Branch hygiene matters most when committing.

Suggestion: Allow draft generation on any branch after user approval. Before committing, check the branch and working tree, stage only the generated docs files, and ask if a docs branch should be created.


If any are present, rewrite. This includes prompts, status updates, summaries, the extraction outline, and the final diff-review message.

## Out of scope
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Prefer default boundaries over an out-of-scope list: This section is written entirely as negative constraints. In practice, those do not instructions can make the agent stubborn. I would reframe this as default boundaries and ask-before-expanding-scope guidance.

Suggestion:

## Default boundaries

This skill is optimized for publisher-facing docs. By default, focus on guide pages, reference docs, and code search for grounding concrete claims. If the user asks for adjacent work, such as updating narrative docs, changing specs, or preparing a PR, pause and confirm the expanded scope before doing it.

Copy link
Copy Markdown
Collaborator

@aram356 aram356 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

The skill design is solid and the two-stage flow (outline → user redirects → diff → explicit commit) with hard stops on main/master and uncommitted-tree state is well-thought-out. CI is green. I built the docs locally and confirmed srcExclude produces 40 HTML files with zero under */superpowers/* in dist/.

One blocker: the migration of existing specs into drafts/ is incomplete, and the body's count claim does not match the diff.

Blocking

🔧 wrench

  • Incomplete migration: docs/superpowers/specs/2026-04-02-pr10-logging-initialization-design.md was not moved into drafts/. The PR body claims "One-time migration of 12 existing specs into drafts/," and the implementation plan (docs/superpowers/plans/2026-04-28-generate-feature-docs-skill.md, Task 3) lists 12 files, with Step 1 verifying ls docs/superpowers/specs/*.md | wc -l == 12. The actual diff shows 8 renames + 4 net-new drafts, and at PR HEAD the file docs/superpowers/specs/2026-04-02-pr10-logging-initialization-design.md still sits at the spec root, not under drafts/.

    This file landed on main in commit 73101982 (PR #610), which is in the merge base of this branch (c7e1e692). The migration commit 17a2207a Move existing specs to drafts/... ran before this file was on the branch and was never re-run. CI didn't catch it because srcExclude: 'superpowers/**' masks the entire tree from the public build. The structural intent ("all specs live under drafts/ or implemented/") is broken on main once this lands.

    Fix:

    git mv docs/superpowers/specs/2026-04-02-pr10-logging-initialization-design.md \
           docs/superpowers/specs/drafts/2026-04-02-pr10-logging-initialization-design.md
    # then prepend the same `---\nstatus: draft\n---\n\n` frontmatter the other 8 received

    (This file is plausibly a candidate for implemented/ since PR #610 has shipped, but the convention is "spec lands in drafts/ first, gets promoted in a later body-update PR" — defaulting to drafts/ is the safe choice for this PR.)

❓ question

  • PR body says 12-spec migration; diff shows 8 renames + 4 net-new drafts. Was "12" intended literally (in which case the wrench above plus three more — but I can only find one stray spec at specs/), or was it a stale plan reference and the actual count was always 8 renames? Please reconcile the body with what landed once the wrench is fixed.

Non-blocking

🤔 thinking

  • SKILL Step 1.7b bash one-liner can fail open (SKILL.md:147). Subshell expands to empty when both main and master are absent, producing an invalid revision range; gate explicitly. Inline comment with details.
  • Verification semantics include test-only matches as verified (SKILL.md:120). A handle that exists only inside #[cfg(test)]/mocks passes the verification-rate gate. Filter or add a third state. Inline comment.
  • Slug rule is non-deterministic, undermining the idempotency claim (SKILL.md:85). Inline comment with details.
  • No prettier run before commit (SKILL.md:333). The skill commits LLM-emitted markdown that is unlikely to match prettier; pushing will fail CI's format-docs check. Inline comment.

♻️ refactor

  • git checkout -b doesn't handle existing branch; prefer git switch (SKILL.md:239). Inline comment.
  • grep crates/**/*.rs relies on shell globstar (SKILL.md:120). Inline comment with portable alternatives.

🌱 seedling

  • 50% verification-rate threshold is a tunable magic number (SKILL.md:138). Track and revisit after real usage. Inline comment.
  • Empty implemented/ at merge time. First-time use of /generate-feature-docs hits an empty directory. Consider a follow-up that promotes one shipped spec (e.g., the EC KV work, or the same PR10 logging spec referenced above) so the skill is exercisable end-to-end against this repo.

📌 out of scope

  • No automated guard against re-exposing superpowers/ on the public docs site (config.mts:41). Plan documents the manual find check but it isn't in CI; a small post-build assertion would prevent regressions. Inline comment.

CI Status

  • cargo fmt: PASS
  • clippy (Analyze rust): PASS
  • cargo test: PASS
  • vitest: PASS
  • format-docs: PASS
  • format-typescript: PASS
  • CodeQL: PASS
  • integration tests: PASS
  • browser integration tests: PASS

Local verification: cd docs && npm run build succeeds, find docs/.vitepress/dist -path '*/superpowers/*' returns zero results.

@@ -0,0 +1,408 @@
---
name: generate-feature-docs
description: "Use when generating, writing, or updating publisher-facing documentation from an implemented engineering spec. Activates on requests like \"generate docs for spec X\", \"write a guide page for the RSL spec\", \"update docs for the EC KV extension\". Operates on specs under docs/superpowers/specs/implemented/ with status implemented frontmatter."
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick — The description value is a single ~700-char line. YAML allows folded (>) or literal (|) block scalars that render the same to the harness and edit much more cleanly:

description: >
  Use when generating, writing, or updating publisher-facing documentation
  from an implemented engineering spec. Activates on requests like
  "generate docs for spec X", "write a guide page for the RSL spec",
  "update docs for the EC KV extension". Operates on specs under
  docs/superpowers/specs/implemented/ with status implemented frontmatter.


### Step 1.3: Resolve the target page path

Slug the feature name to kebab-case (e.g., "RSL AI Crawler Licensing" becomes `ai-crawler-licensing`). The target page is `docs/guide/<slug>.md`.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 thinking — Slug rule is non-deterministic, which undermines the idempotency requirement.

The example silently drops the leading acronym (RSL AI Crawler Licensingai-crawler-licensing). A different invocation could produce rsl-ai-crawler-licensing or rsl-licensing. If the slug differs across runs, the augment-vs-greenfield decision flips and the "zero diff on re-run" guarantee in the Idempotency section breaks.

Either codify a mechanical rule ("lowercase, replace runs of non-alphanumeric with single hyphens, strip leading/trailing hyphens") and treat the example as a stylistic shortening that requires user confirmation, or always confirm the slug with the user on first run and persist it (e.g., via a TODO marker in the spec frontmatter).


For each handle, search the code:

- Config keys: grep `crates/**/*.rs` and `trusted-server.toml` for the key name. Capture the file and line number when found.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ refactorgrep crates/**/*.rs requires shopt -s globstar in bash and silently expands wrong on plain sh. Recommend modeling the right pattern:

rg --type rust --fixed-strings -- '<key>' crates/
# or
find crates -name '*.rs' -exec grep -nH -F -- '<key>' {} +

Claude will likely reach for rg regardless, but the skill should show the pattern that works portably.

🤔 thinking — Verification semantics include test-only matches as verified. A handle that exists only inside #[cfg(test)], mocks, or fixtures will pass the verification-rate gate even though no production code path uses it, hiding the very drift the gate is designed to catch. Either filter */tests/* (and #[cfg(test)] blocks) from the search or add a third state ("verified in tests only") so the user can see it in the outline.


> "Stage 1 verified `<verified_count>` of `<total_extracted_count>` handles in this codebase (`<rate>%`). Below 50% suggests this spec may not be fully implemented in this branch. Options: (A) generate stubs for unverified handles, (B) abort and check status, (C) override and proceed normally."

The user must pick A, B, or C before the skill proceeds. The threshold of 50% is initial; tune based on real usage.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🌱 seedling — The 50% threshold is a magic number, and the design itself flags it as "initial; tune based on real usage." After a few real runs, capture the actual verification rates (and the false-positive vs false-negative outcomes at 50%) and revise. Worth a follow-up issue so the calibration doesn't get forgotten.

In addition to handle verification, check whether the current branch has touched code relevant to the feature:

```bash
git log --name-only $(git merge-base HEAD main 2>/dev/null || git merge-base HEAD master 2>/dev/null)..HEAD -- crates/ trusted-server.toml
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 thinking — If both git merge-base HEAD main and ... master fail, the $(...) subshell expands to empty and the line becomes:

git log --name-only ..HEAD -- crates/ trusted-server.toml

which git rejects as a bad revision. The skill text below this block says "skip this check silently and proceed" but the literal command doesn't enforce that. Capture the merge-base into a variable and gate on it:

MB=$(git merge-base HEAD main 2>/dev/null || git merge-base HEAD master 2>/dev/null)
if [ -n "$MB" ]; then
  git log --name-only "$MB"..HEAD -- crates/ trusted-server.toml
fi

> "You are on `<branch>`. Create branch `docs/<slug>` and switch to it?"
- The user can specify a different branch name.
- The skill refuses to proceed on `main` or `master` under any circumstance, including override attempts.
- After confirmation, run `git checkout -b <branch-name>`.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ refactorgit checkout -b <branch-name> will fail if docs/<slug> already exists (which it will on a re-run of the skill against the same spec on the same day). Modern git also prefers git switch. Suggested:

if git show-ref --verify --quiet "refs/heads/docs/<slug>"; then
  git switch "docs/<slug>"
else
  git switch -c "docs/<slug>"
fi


### Step 2.7: Commit

Stage explicitly via path:
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 thinking — No prettier run before commit. The repo enforces cd docs && npm run format in CI (the format-docs check). LLM-emitted markdown is unlikely to match prettier's rules exactly, so users who push the skill's commit will hit a CI failure and have to amend.

Suggest running the formatter on just the modified files before the diff-review message in Step 2.5:

(cd docs && npx prettier --write \
  guide/<slug>.md \
  guide/configuration.md \
  guide/api-reference.md \
  guide/error-reference.md)

or, simpler: instruct the user to run npm run format before pushing as part of the final "Committed as <sha>" message.

description:
'Privacy-preserving edge computing for ad serving and edge cookie (EC) generation',
base: '/trusted-server',
srcExclude: ['superpowers/**', '**/node_modules/**'],
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📌 out of scopesrcExclude works (verified locally: cd docs && npm run build produces 40 HTML files, zero under */superpowers/* in dist/), but there is no automated guard against a future edit silently re-exposing internal specs. The plan's Task 1 Step 4 documents the manual check (find docs/.vitepress/dist -path '*/superpowers/*') but it never made it into a CI assertion.

Suggest a follow-up issue to wire that check into the docs build job — a one-line post-build assertion that exits non-zero on any hit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants