Chenyu Yang*, Zixuan Wang*, Tianchen Zhao*, Jin Wang*, Yuntao Chen, 2026

*Work done during internship at MiroMind.AI.

<aside> 🔖

RC3 reduces the context processed during training rollouts. Rollout Chunking with Context Compression (RC3) divides a long agent trajectory into shorter chunks and carries the query, initial plan, and recent model outputs across chunk boundaries. Evaluation uses the full accumulated interaction history. The reported rollout generation times are approximately 580 seconds per step with 32k chunks and 1,200 seconds with a 128k context, a 2.1× speedup. The benchmark table reports GAIA-103 scores of 74.2 for RC3 and 74.9 for full-context RL.

</aside>

Figure 1. A ReAct agent alternates reasoning, tool calls, and observations within an RL training loop. [1] The diagram illustrates synchronous rollout and update phases. The implementation uses MiroRL.

Introduction

On-policy reinforcement learning trains deep-research agents from the outcomes of their own tool-use trajectories. A ReAct policy alternates reasoning and actions with observations from the environment. [1] Group Relative Policy Optimization (GRPO) uses rewards from multiple rollouts of the same query to construct a group-relative learning signal. [2] Our training uses verified final answers as outcome rewards.

Longer interaction horizons increase the cost of collecting training trajectories. The MiroThinker report studies interactive scaling, in which agents improve research performance through more extensive interaction with their tools and environment. [20] Our motivating tasks include GAIA-103 and BrowseComp-en, which require agents to locate and combine evidence across multiple sources. [3] [4]

Figure 2. Reinforcement learning shifts the distribution of tool-use turns and improves the reported scores of MiroThinker-v1.0-30B across BrowseComp, BrowseComp-ZH, HLE, and GAIA. The figure is reproduced from the MiroThinker technical report. [20]

Long-context rollout decoding repeatedly attends to a growing history of model outputs and tool observations. KV-cache methods such as H₂O reduce the retained cache and its memory cost during generation. [29] FlashAttention-2 and FlashAttention-3 improve attention execution through kernel design. [5] [6] RC3 targets the amount of interaction history that the policy processes during training rollouts.

<aside> 👉🏻

RC3 tests whether shorter training contexts can retain the accuracy of full-context RL while reducing rollout time. The method keeps the existing ReAct inference loop and applies deterministic context compression during training.

</aside>

Preliminary

Inference-time context retention has model-dependent effects on the GAIA-103 scores in Table 1. The comparison includes the standard ReAct loop, recency-based tool-output retention, and the IterResearch framework. [1] [3] [7] The evaluated model families are described in the Qwen3 and MiroThinker reports. [21] [20]

Table 1. Reported GAIA-103 scores (%). Keep-last-5 denotes recency-based retention of tool outputs, with earlier tool outputs omitted. IterResearch reconstructs a compact workspace around an evolving report. [7] The label n/r denotes an unreported entry.

The reported retention results support testing compression separately for each policy. Keep-last-5 changes the Qwen3-30B-A3B score from 31.0 to 29.6 and the MiroThinker-30B-SFT score from 72.0 to 73.5. IterResearch changes the three reported Qwen3 scores by −1.0, +5.9, and −4.4 percentage points, respectively.

Rollout generation accounts for 95.8% of a training step in the profiled 128k configuration. We profiled a 30B-A3B model across context windows from 8k to 128k. Figure 3 reports an increase in LLM inference time from 160 to 1,330 seconds and an increase in tool execution time from 40 to 70 seconds. LLM inference therefore accounts for approximately 95% of rollout time at 128k.