LongCat-2.5 Released

Timeline: Preview to GA

The preview drew attention for a non-technical reason too: OpenCode listed LongCat-2.5-Preview as a free model for two weeks with zero data retention, which gave Western developers a low-risk way to test a Chinese frontier model inside an existing agent workflow.

2. Scaling Agentic RL for Long-Horizon Tasks

The team frames RL scaling as the key lever for pushing intelligence further, and they scale the pipeline on four axes simultaneously: data, environments, algorithms, and harnesses.

Large-scale data synthesis and environment construction

For coding tasks the model must learn from real environment feedback, so the pipeline builds environments, not just prompts. Filtered GitHub pull requests and linked issues are annotated by agents, environments are synthesized, and each task is validated with fail-to-pass (F2P) and pass-to-pass (P2P) tests. Real internal coding tasks with their own acceptance criteria broaden coverage of ordinary engineering work.

Verification is treated as a moving target. Live rollout pass rates drive relabeling and filtering, while offline analysis of false-positive and false-negative trajectories continually corrects and extends F2P tests — improving coverage of the task description rather than merely matching the diff that happened to merge.

Asynchronous RL and stability

Long-horizon tasks produce a heavy-tailed rollout distribution. LongCat-2.5 uses asynchronous partial rollouts: each RL step maintains an oversampling ratio, and when a batch is filled, in-flight trajectories are interrupted, saved, and resumed in the next sampling phase. Oversampling increases with precomputed pass rates, and zero-gradient groups — where every rollout succeeds or every rollout fails — are filtered out so the effective gradient signal stays meaningful.

The cost of asynchrony is staleness: tokens in one trajectory may originate from different historical checkpoints. The fix is twofold — clip the discrepancy between sampling-policy and current-policy log-probs, and rescale each token's gradient contribution by its accumulated historical discrepancy, bounding the gradient bias staleness would otherwise introduce. Numerical differences at equal weights are handled by the train–inference alignment described below.

Reward hacking and output behavior

Task environments are hardened during training: network access disabled, git history and cached files removed, and an LLM judge auditing trajectories for exploits. Behaviorally, the model is penalized for repetitive reasoning and repeated tool calls during multi-turn interaction; overthinking and rapid response-length growth are counteracted by redistributing reward among positive samples rather than applying a blunt length penalty.

Mixed training and generalization

The model adapts to similar tasks and harnesses during RL. Only a small number of scaffold adaptation examples are needed in earlier training stages. During RL, tasks aligned with different benchmarks are mixed and their scaffolds scaled in varying proportions — mixed training across tasks and scaffolds improves generalization, and gains on one task reinforce gains on others.

4. Infrastructure: Under 80 GB Per Card

LongCat-2.5 uses long-sequence training in both mid-training and supervised fine-tuning. The binding constraint is that domestic accelerators have less than 80 GB of memory per card, so the training system is designed around memory first and topology second.

Supernode-aware parallelism

Parallel groups are configured to match the supernode topology: FSDP and EP degrees equal the card count per supernode, and CP never exceeds it. That keeps parameter all-gather, expert-token all-to-all, and KV all-gather inside the high-bandwidth interconnect. Hybrid vocabulary parallelism shards the base embedding, LM head, and N-gram embeddings across the DP × CP domain for further memory savings.

Expert-parallel load imbalance is handled by threshold-triggered dynamic expert replication built on MoonEP. Replication only activates when imbalance crosses a configured threshold, which avoids the trap where added communication costs more than the balance gains — and avoids the two OOM modes: routed tokens overflowing the receiver, comm operations overflowing the sender.

The discrete image-token design pays off here as well: with the vision encoder out of the training graph and image token counts known at packing time, modality load ratios are predictable, so modality proportions stay approximately uniform across data-parallel ranks.

Dual-shard FSDP

Parameters are sharded within each supernode's FSDP domain, while FP32 master weights and optimizer states are sharded further across FSDP domains through HSDP replicas — preserving parameter communication efficiency while cutting resident memory. Parameters, gradients, and optimizer states are organized into contiguous flattened buffers that support non-uniform sharding and Muon optimizer sharding. Gradients are written directly into weights and temporary buffers are reused, reducing intermediate copies and peak memory.

At the communication layer the system uses zero-copy AllGather/ReduceScatter to eliminate the intermediate copy in default collective algorithms; EP packs tokens together with routing probabilities and uses an AIV-based path to relieve AICPU task-expansion bottlenecks; parameter prefetch overlaps AllGather with compute while managing contention with CP; and ScMoE overlaps EP communication with dense-branch compute. Host-side overhead is trimmed by batching gradient operations, merging metric reductions, CPU core pinning, and NUMA-aware scheduling.

Combined, the FSDP configuration delivers 86% higher per-card throughput than a DDP baseline on long-sequence SFT.

Production-grade train–inference consistency

Building on IsoExec, GLM-5, and XoRL, the team built a consistency scheme where training, prefill, and decode each pick their own parallel configuration while per-token log-probs remain bitwise identical. Both sides use efficient batch-invariant operators with aligned framework math and complete backward support. It is enabled by default in experiments and production, and serves as a shared numerical regression baseline for operator and framework iteration.

Two extensions go beyond community solutions, both targeting cases where differing parallelism breaks alignment:

• TP: instead of split-K partial sums requiring a reduce-scatter, compute the full K dimension. Training keeps weights sharded and uses weight all-gather plus input all-to-all so each rank completes the full-K GEMM. Prefill follows the same approach; decode converts o_proj/fc2 to column-parallel, using input all-gather, full-K GEMM, and output all-to-all — avoiding both per-step full-weight aggregation and cross-card numerical summation
• DCP: training forward and prefill adopt matching sparse-attention operators that reproduce decode's KV chunking and reduction order under the given DCP configuration, aligning precision across all three modes

PD disaggregation for long-context rollouts

For long-context multi-turn rollouts with controllable load, LongCat-2.5 separates prefill and decode instances and optimizes three things together:

• Larger effective cache: prefill uses QCP to allocate compute per token and KVP to store KV by page, avoiding the KV redundancy MLA incurs under TP. Once KV is migrated back to device memory, HiCache releases main-memory copies that nothing else references. Weight management and actor offload are rebuilt on Mooncake to free more memory for HiCache
• Cross-layer prefetch to hide transfers: LSA's odd layers reuse the previous even layer's top-k indices, so odd-layer KV can live entirely in host memory with only a small working cache on device. Once an even layer produces its indices, the next odd layer's KV is loaded ahead and overlapped with the current layer's remaining compute — nearly doubling decode KV capacity at no TPOT cost
• Capacity-aware scheduling: the router admits requests based on L1+L2 cache capacity and total request tokens, prioritizing groups close to completion so batches fill and caches release early. This prevents the recompute avalanche where multiple long trajectories alternately evict each other's prefixes

A 3:2 prefill-to-decode instance ratio was optimal for the tested long-context workload.

W4A4 inference

For a 1.6T MoE, the inference solution quantizes expert weights and activations to 4 bits and runs matmuls on native INT4 units. Accuracy builds on FlatQuant's learnable equivalent transformations, further tuned for MoE, and matches the unquantized model on every evaluated downstream task.

• Data path: a shared input transformation across experts quantizes before token replication and dispatch, so one INT4 representation is reused through expert communication and computation
• Kernels: transform, scale computation, quantization, and packing fused into dedicated operators; weight layout and transposed access optimized to reduce movement and conversion
• System: compute–communication pipeline overlap tuned separately for prefill and decode according to data dependencies and hardware constraints

6. Showcases

Eight end-to-end demos accompany the release, all generated by the model in single sessions. Each targets a different failure mode: sustained physical reasoning, multi-source reconciliation, 3D composition, and long HTML output that has to actually run.

Related

• LongCat-2.5 model page — full architecture breakdown
• Complete benchmark table — 33 benchmarks, 9 models
• LongCat-2.0 — predecessor, open weights (MIT)
• LongCat-Next — where the discrete visual token idea came from
• LongCat-Flash-Thinking-2601 — the agentic RL pipeline 2.5 inherits
• Training and serving on domestic accelerators
• Which LongCat model for which agent workflow
• All news and releases