opencode_rlm_env

[snimu]

Description

opencode_rlm_env

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Test improvement

Testing

• All existing tests pass when running uv run pytest locally.
• New tests have been added to cover the changes

Checklist

• My code follows the style guidelines of this project as outlined in AGENTS.md
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• Any dependent changes have been merged and published

---

Note

Medium Risk
Adds a new sandboxed agent environment that handles concurrent sub-LLM requests and changes interception/config plumbing; concurrency and request-routing changes could affect rollout stability and token accounting.

Overview
Introduces OpenCodeRLMEnv, extending OpenCodeEnv to support Recursive Language Model sub-agent calls via the snimu/oc plugin, including sandbox bootstrapping (bun + plugin install), header-based sub-LLM detection, and concurrent handling of sub-requests with semaphore-limited parallelism and optional trajectory logging.

Refactors CliAgentEnv request polling into _poll_next_request() to support the new concurrent routing, adjusts first-turn prompt capture to ignore sub-LLM steps, and updates OpenCodeEnv’s generated OpenCode config to use a fixed intercepted/model provider mapping.

Enhances the interception layer to record incoming HTTP headers per request (used for sub-LLM role detection), adds a monitor rubric for main vs sub-LLM token/turn metrics, adds comprehensive tests for config rendering and metrics, and updates environment docs to list OpenCodeEnv/OpenCodeRLMEnv.

^{Written by Cursor Bugbot for commit 8bf7d0a. This will update automatically on new commits. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 3 potential issues.

Autofix Details

Bugbot Autofix prepared fixes for all 3 issues found in the latest run.

• ✅ Fixed: Fire-and-forget task may be garbage collected
  • Added _sub_llm_tasks set to track task references and prevent garbage collection, following the pattern used in parent class.
• ✅ Fixed: New environment not listed in environments README
  • Added opencode_rlm_test to environments/README.md under experimental environments section and pattern reference section.
• ✅ Fixed: Swallowing CancelledError prevents proper task cancellation
  • Added raise statement after exception handling in _handle_sub_llm_request to properly propagate CancelledError and other exceptions.

Or push these changes by commenting:

@cursor push 362a510ff0

Preview (362a510ff0)

diff --git a/environments/README.md b/environments/README.md
--- a/environments/README.md
+++ b/environments/README.md
@@ -45,6 +45,9 @@
 - **RLMEnv (Recursive Language Model)**
   - **rlm_secrets**: Puzzle environment testing RLM functionality including root-level tools, sub-LLM tool use, and file operations.
 
+- **OpenCodeRLMEnv (OpenCode with RLM plugin)**
+  - **opencode_rlm_test**: Smoke-test environment for `OpenCodeRLMEnv` demonstrating concurrent sub-LLM handling with the RLM plugin.
+
 - **HarborEnv / CliAgentEnv (CLI agent sandboxes)**
   - **opencode_harbor**: Runs the OpenCode CLI agent on Harbor tasks with API interception via Prime Tunnel.
   - **terminus_harbor**: Runs the Terminus agent on Harbor tasks with API interception via Prime Tunnel.
@@ -75,6 +78,7 @@
 - **CLI agent sandboxes**: `opencode_harbor`, `terminus_harbor`
 - **MCP integration**: `mcp_search_env`
 - **RLM (recursive LLM)**: `rlm_secrets`
+- **OpenCode RLM integration**: `opencode_rlm_test`
 - **Environment and rubric composition**: `math_group`, `math_python`, `wiki_search`
 - **Procedural datasets**: `reasoning_gym_env`
 - **Multimodal**: `mmmu`

diff --git a/verifiers/envs/experimental/opencode_rlm_env.py b/verifiers/envs/experimental/opencode_rlm_env.py
--- a/verifiers/envs/experimental/opencode_rlm_env.py
+++ b/verifiers/envs/experimental/opencode_rlm_env.py
@@ -167,6 +167,7 @@
         self.sub_timeout_ms = sub_timeout_ms
         self.include_sub_llm_in_trajectory = include_sub_llm_in_trajectory
         self._sub_llm_semaphore = asyncio.Semaphore(max_sub_llm_parallelism)
+        self._sub_llm_tasks: set[asyncio.Task[None]] = set()
 
         kwargs.setdefault("run_command_template", RLM_RUN_COMMAND_TEMPLATE)
 
@@ -304,9 +305,11 @@
 
             if self._is_sub_llm_request(intercept):
                 # Fire-and-forget: handled concurrently outside the loop
-                asyncio.create_task(
+                task = asyncio.create_task(
                     self._handle_sub_llm_request(state, request_id, intercept)
                 )
+                self._sub_llm_tasks.add(task)
+                task.add_done_callback(self._sub_llm_tasks.discard)
                 continue
 
             # Main-agent request → return to rollout loop
@@ -349,6 +352,7 @@
             except BaseException as e:
                 error = e
                 logger.warning("Sub-LLM request %s failed: %s", request_id, e)
+                raise
             finally:
                 if intercept.get("stream"):
                     await synthesize_stream(intercept, response, error)

_{This Bugbot Autofix run was free. To enable autofix for future PRs, go to the Cursor dashboard.}

[mikasenghaas]

nice, lgtm!

[mikasenghaas]

ah, i actually like this as a pattern to distinguish env args from this env or parent envs

[mikasenghaas]

i dont like all these section dividers

[mikasenghaas]

maybe put the first part of this method (which i think is shared with cli agent env) in a shared util so we dont dup code here?

[mikasenghaas]

what's the reason to compute metrics at rollout runtime as opposed to in the rubric reward fn?

[snimu]

Copied it from the RLMEnv where some time ago I think I was planning to live-update metrics bc the rollouts were so long, but then didn't; I can undo it.

[snimu]

oh wait there is a reason: the sub-LLM calls aren't guaranteed to be in the trajectory (and by default, they aren't), so we need to collect those metrics incrementally. I could do it afterward for the root-LLM but it save ~1 line of code, so it's probably not worth it.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}