Cache hit rate is low / zero

[surapuramakhil]

Description

I am not sure if this is openCode issues or Open work

deep infra dashboard report - you can see hit rate is zero.
Dashboard - DeepInfra.pdf

Plugins

No response

OpenCode version

No response

Steps to reproduce

1. create a worker - have few chat conversations
2. see metrics

Screenshot and/or share link

No response

Operating System

macOS

Terminal

via OpenWork

[github-actions]

This issue doesn't fully meet our contributing guidelines.

What needs to be fixed:

• Steps to reproduce: Please provide detailed steps showing how to reproduce the cache hit rate issue
• System information: Please fill in your Operating System and Terminal environment
• Additional context: Explain what cache hit rate you're expecting vs what you're seeing, and any relevant configuration details

Please edit this issue to address the above within 2 hours, or it will be automatically closed.

---

This issue might be a duplicate of existing issues. Please check:

• Due to the lack of userId and sessionId, some models will not hit the cache. #8623: Due to the lack of userId and sessionId, some models will not hit the cache (directly related to cache hit problems)
• /compact doesn't utilize prompt caching #10342: /compact doesn't utilize prompt caching (related cache utilization issue)
• applyCaching called too freely, causes LiteLLM to fail #6473: applyCaching called too freely, causes LiteLLM to fail (related caching issue)

If you believe this was flagged incorrectly, please let a maintainer know.

[bhagirathsinh-vaghela]

PR #14743 addresses the root causes of Anthropic prefix cache misses: system prompt was a single merged block (dynamic content invalidated stable content), bash tool schema included the absolute cwd (changed per-repo), and skill tool ordering was nondeterministic. If you're on Anthropic, cross-repo first-prompt hit rate goes from 0% → 97.6% in my setup. The prefix stability fixes may also help providers with automatic prefix caching (OpenAI, DeepSeek, Gemini, etc.) but I can't confirm for DeepInfra specifically.

[surapuramakhil]

for now I have switched to gemini Api's - As i am able to see chache hits for it.

[TomLucidor]

Does this problem apply to other LLM APIs? #5224

[surapuramakhil]

@TomLucidor

#5224 -> i have seen this in deep infra

but suprisingly gemini Api's are getting cache chits but not deep infra, both uses byte idential prefix (implicit caching)

[surapuramakhil]

@thdxr @adamdotdevin @rekram1-node

request to have a look it, I have inspected OpenCode SQLite DB -> it confirms gemini "byte identical prefix cache is working for gemini but its not working for deepInfra"

[surapuramakhil]

note: looks like this is only happening to moonshoot/ kimi 2.5 model