feat(cache): split system prompt into stable/dynamic blocks for independent caching#27377
feat(cache): split system prompt into stable/dynamic blocks for independent caching#27377martinffx wants to merge 3 commits into
Conversation
…endent caching
Splits Instruction.system() into { global, project } scopes. When
OPENCODE_EXPERIMENTAL_SYSTEM_PROMPT_SPLIT is enabled, the LLM receives
two system messages: stable (global instructions) and dynamic
(env + project instructions + skills). This allows Anthropic prompt
cache to independently cache the stable prefix across sessions.
Co-authored-by: Bhagirathsinh Vaghela <bhagirathsinh.vaghela.001@gmail.com>
- Add missing withStatics import and remove non-existent zod helper - Add missing OPENCODE_EXPERIMENTAL_CUSTOMIZE_SKILL flag - Remove stray extra closing brace
|
Hey! Your PR title Please update it to start with one of:
Where See CONTRIBUTING.md for details. |
|
The following comment was made by an LLM, it may be inaccurate: Based on my search, here are the related PRs found (excluding the current PR #27377): Related PRs:
These PRs are part of the same stacked PR decomposition effort for the prompt caching fix, as mentioned in the current PR's description. They're not duplicates but rather complementary PRs in the same feature set. |
Issue for this PR
Closes #5416, #5224
Related: #14065, #5422, #14203, #14743
Part of the stacked PR decomposition of the prompt caching fix (2 of 4). Adds cache token audit logging as observability foundation, the metrics logged here are what we use to verify cache hit improvements in subsequent PRs.
Type of change
What does this PR do?
Splits the system prompt into stable (global instructions + global skills) and dynamic (environment + project instructions + project skills) blocks. The LLM adapter already supports this split structure for independent cache-key reuse, so stable blocks can persist across turns without re-encoding.
Gated behind
OPENCODE_EXPERIMENTAL_SYSTEM_PROMPT_SPLIT(env var orOPENCODE_EXPERIMENTAL).Instruction.system()returns{ global, project }instead of a flat array, separating global config dir instructions from repo-local ones.SystemPrompt.skills()returns{ global, project }by scope.{ stable: [...globalInstructions, ...globalSkills], dynamic: [...env, ...projectSkills, ...projectInstructions] }instead of a flat array.OPENCODE_EXPERIMENTAL_CUSTOMIZE_SKILLflag andwithStaticsimport to fix typecheck.How did you verify your code works?
bun run typecheck— all 14 packages pass.Screenshots / recordings
N/A — no UI change.
Checklist