Skip to content

feat(cache): split system prompt into stable/dynamic blocks for independent caching#27377

Open
martinffx wants to merge 3 commits into
anomalyco:devfrom
martinffx:feat/system-prompt-split
Open

feat(cache): split system prompt into stable/dynamic blocks for independent caching#27377
martinffx wants to merge 3 commits into
anomalyco:devfrom
martinffx:feat/system-prompt-split

Conversation

@martinffx
Copy link
Copy Markdown

Issue for this PR

Closes #5416, #5224
Related: #14065, #5422, #14203, #14743

Part of the stacked PR decomposition of the prompt caching fix (2 of 4). Adds cache token audit logging as observability foundation, the metrics logged here are what we use to verify cache hit improvements in subsequent PRs.

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Splits the system prompt into stable (global instructions + global skills) and dynamic (environment + project instructions + project skills) blocks. The LLM adapter already supports this split structure for independent cache-key reuse, so stable blocks can persist across turns without re-encoding.

Gated behind OPENCODE_EXPERIMENTAL_SYSTEM_PROMPT_SPLIT (env var or OPENCODE_EXPERIMENTAL).

  • Instruction.system() returns { global, project } instead of a flat array, separating global config dir instructions from repo-local ones.
  • SystemPrompt.skills() returns { global, project } by scope.
  • When the flag is on, the assembled prompt is { stable: [...globalInstructions, ...globalSkills], dynamic: [...env, ...projectSkills, ...projectInstructions] } instead of a flat array.
  • Added OPENCODE_EXPERIMENTAL_CUSTOMIZE_SKILL flag and withStatics import to fix typecheck.

How did you verify your code works?

bun run typecheck — all 14 packages pass.

Screenshots / recordings

N/A — no UI change.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

martinffx and others added 3 commits May 13, 2026 21:09
…endent caching

Splits Instruction.system() into { global, project } scopes. When
OPENCODE_EXPERIMENTAL_SYSTEM_PROMPT_SPLIT is enabled, the LLM receives
two system messages: stable (global instructions) and dynamic
(env + project instructions + skills). This allows Anthropic prompt
cache to independently cache the stable prefix across sessions.

Co-authored-by: Bhagirathsinh Vaghela <bhagirathsinh.vaghela.001@gmail.com>
- Add missing withStatics import and remove non-existent zod helper
- Add missing OPENCODE_EXPERIMENTAL_CUSTOMIZE_SKILL flag
- Remove stray extra closing brace
@github-actions
Copy link
Copy Markdown
Contributor

Hey! Your PR title Feat/system prompt split doesn't follow conventional commit format.

Please update it to start with one of:

  • feat: or feat(scope): new feature
  • fix: or fix(scope): bug fix
  • docs: or docs(scope): documentation changes
  • chore: or chore(scope): maintenance tasks
  • refactor: or refactor(scope): code refactoring
  • test: or test(scope): adding or updating tests

Where scope is the package name (e.g., app, desktop, opencode).

See CONTRIBUTING.md for details.

@github-actions
Copy link
Copy Markdown
Contributor

The following comment was made by an LLM, it may be inaccurate:

Based on my search, here are the related PRs found (excluding the current PR #27377):

Related PRs:

  1. fix(cache): improve Anthropic prompt cache hit rate with system split and tool stability #14743 - "fix(cache): improve Anthropic prompt cache hit rate with system split and tool stability"

  2. feat(cache): add cache token audit logging behind OPENCODE_EXPERIMENTAL_CACHE_AUDIT #27007 - "feat(cache): add cache token audit logging behind OPENCODE_EXPERIMENTAL_CACHE_AUDIT"

  3. chore(cache): rebase prompt caching updates for #14743 #26875 - "chore(cache): rebase prompt caching updates for fix(cache): improve Anthropic prompt cache hit rate with system split and tool stability #14743"

    • Related: This is a rebase/maintenance PR for the prompt caching updates.
  4. fix(session): cache messages across prompt loop to preserve prompt cache byte-identity #24842 and fix(session): cache messages across prompt loop to preserve prompt cache byte-identity #25367 - "fix(session): cache messages across prompt loop to preserve prompt cache byte-identity"

    • Related: These PRs address prompt caching improvements across sessions.

These PRs are part of the same stacked PR decomposition effort for the prompt caching fix, as mentioned in the current PR's description. They're not duplicates but rather complementary PRs in the same feature set.

@martinffx martinffx changed the title Feat/system prompt split feat(cache): split system prompt into stable/dynamic blocks for independent caching May 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE]: Anthropic (and others) caching improvement

1 participant