To enhance the coding capabilities of Nemotron 3 Ultra, its 550-billion-parameter open model released in June, Nvidia worked with Factory to generate approximately 20,000 rows of post-training data using the Droid coding-agent harness.

Capturing reproducible agent failures

Factory unveiled a case study on Wednesday describing how Nvidia deployed Droid within an isolated network environment to identify and capture reproducible failures across instruction following, tool calling, and error recovery mechanisms for model refinement.

The Droid system depends on Nemotron to preserve a structured task plan during execution. Factory identified specific issues involving tool call formatting and argument construction, along with cases where the model failed to recover from errors before proceeding.

Factory also ran Nemotron against proprietary evaluation datasets, including Legacy Bench, to pinpoint the specific decisions that led to task execution failures.

Factory collaborated with us to analyze those traces to improve the model's planning, tool use, and error recovery.

Jian Zhang, Nvidia's director of applied research for Nemotron post-training

What PivotRL changes

Nvidia employed PivotRL, a reinforcement learning technique unveiled in March, which trains models on critical decision points within existing supervised fine-tuning trajectories while avoiding the computational overhead of complete multi-turn rollouts. This approach evaluates whether an action produces the intended result rather than whether it matches a reference trajectory.

On agentic coding tasks, PivotRL achieved accuracy comparable to end-to-end reinforcement learning with one-quarter as many rollout turns.

When tested against identical datasets, PivotRL surpassed standard supervised fine-tuning by an average of 4.17% across four agentic domains and 10.04% on out-of-domain non-agentic tasks. Nvidia had previously integrated PivotRL into production-scale post-training for Nemotron 3 Super, and for Ultra, the company sent updated model checkpoints back to Factory for additional testing.

Guarding against harness overfitting

To mitigate the risk of overfitting to Droid-specific characteristics, Nvidia modified the harness configuration and incorporated training data from multiple agent environments.

Nvidia's NeMo RL documentation outlines a multistage pipeline that merges reinforcement learning with verifiable rewards, human feedback and multi-teacher on-policy distillation. Production configurations include OpenCode, Codex and an environment called agent006 alongside Droid-specific PivotRL data.

Through distillation, capabilities from teacher models trained across different technical domains transfer into the student model. Nvidia has made its reinforcement learning blends publicly available along with reproduction instructions for GB200 NVL72 hardware.

Nemotron 3 Ultra's Terminal-Bench 2.0 score increased from 38.5% to 49%…

According to Factory, Nemotron 3 Ultra's Terminal-Bench 2.0 score climbed from 38.5% to 49%, while an early version of Nemotron 3.5 Super VL improved from 52.11% to 60.4% on Terminal-Bench 2.1.

Factory has not released an ablation study that isolates its specific contribution to these benchmark improvements or shared before-and-after error rates for individual tool-use failures.

Harness-dependent performance gains also appeared in testing of Google's Gemini 4 Neon, which showed notably stronger results when evaluated within Google's Antigravity harness.

Training harness-specific adapters

Hugging Face's Harness Hub team pursued an alternative strategy outlined in a November 6 technical post, using reinforcement learning to train separate LoRA adapters for Pi, OpenCode and Hermes without altering the harnesses or underlying model weights.

On Tuesday, Hugging Face CEO Clem Delangue announced on X that a Pi-optimized variant of Qwen 3.8 27B improved its Terminal-Bench 2.1 score from 39.5% to 55.8% and reduced tool calls per task by 55% to 62%.

The team also created safeagents, a rule-based proxy that inspects tool calls to prevent reward hacking during the training process.

Neither Nvidia nor Hugging Face has released comparable evaluations that measure the performance and computational trade-offs between harness-specific adaptation and multi-harness post-training approaches.