Real-world usage patterns expose model weaknesses that laboratory benchmarks often miss. Understanding which edge cases matter in production and what users actually need requires capturing that signal and deciding whether the model itself requires adjustment or whether the issue lies elsewhere—in retrieval systems, context handling, prompts, tooling, or the serving layer itself.
Modifying model weights cannot resolve problems originating outside the model. When intervention at the model level is appropriate, evidence from production deployments can simultaneously enhance quality and reduce costs. A lighter model might match a heavier one's performance; superior training examples might boost reliability; outcome-based rewards might align better with the task. Iterating on an existing capable model can also sidestep the expense and complexity of migrating to a new model generation.
Selecting the right training approach based on failure patterns
Model distillation applies when a stronger model already exhibits the desired behavior and you want a smaller, more cost-effective model to replicate it. Supervised fine-tuning (SFT) applies when high-quality labeled examples can instill more reliable behavior. Reinforcement learning (RL) applies when outcomes can be reliably measured but the ideal response cannot be easily specified beforehand.
Changing weights won't fix a failure elsewhere in the system. When the model is the right place to intervene, production evidence can improve both quality and economics.
All three methods may be deployed across a model's lifetime. Base decisions on the evidence your workload provides, even if that means adopting a technique requiring more initial setup.
Transforming production traces into training data demands compute resources and orchestration infrastructure. The newly trained model must undergo evaluation before returning to inference. Closing this feedback loop represents the core challenge. In reinforcement learning scenarios, inference generates rollouts during training, and updated weights must continuously flow back to the serving system throughout execution.
At Fully Connected 2026, CoreWeave unveiled CoreWeave Model Distillation, CoreWeave RL Rollouts on CoreWeave Inference, and a programmable training API. These capabilities integrate within CoreWeave Forge. The programmable API is currently in limited preview, with broader availability targeted for later in 2026.
Refreshing model behavior without replacing the base model
CoreWeave Model Distillation operates as a managed workflow. CoreWeave Agent Lens monitors inference activity as the application executes. The system converts this traffic into training data, refines weak predictions, evaluates different base models and parameter configurations, deploys candidate versions to Serverless Inference, and benchmarks them against the currently running model.
When a model already possesses the necessary capabilities for your use case, recent production data can refine its performance without altering the serving characteristics you depend on. Test the updated version against your existing production model before shifting traffic.
Method employed this approach to iterate on a previously deployed model. During 2024, the company used OpenPipe to fine-tune Llama 3.1 8B for routing calls through bank interactive voice response (IVR) systems, leveraging GPT-4o outputs as training targets. In August 2026, Method incorporated fresh production data, re-labeled it using GPT-5.6 Sol, and trained a new candidate. In direct comparisons, the new version prevailed in 57.5% of evaluations against the deployed model.
The underlying base model remained unchanged. Model Distillation within Forge, combined with newly collected production data, enabled another iteration cycle in less time. The 57.5% figure represents the proportion of head-to-head matchups the newer version won.
Designing custom training workflows in Python
Teams seeking to architect their own training process can use the programmable training API for Serverless Training, which decouples training logic from infrastructure concerns. You define a Python loop; CoreWeave handles compute provisioning and underlying training infrastructure.
Below is one approach combining RL updates with periodic supervised updates on verified examples. The code instantiates a learner and serving adapter, configures an Adam optimizer with a learning rate of 1e-5, and pushes updated weights to the sampler following each iteration.
learner = await training.create_learner(model=model, lora=lora)
sampler = await serving.create_adapter(name="sampler")
optimizer = AdamConfig(learning_rate=1e-5)
for step in range(num_steps):
trajectories = await run_agent(sampler, tasks)
update = learner.batch()
update.forward_backward(
build_rl_batch(trajectories), loss="importance_sampling"
)
update.optim_step(optimizer=optimizer)
if (step + 1) % sft_interval == 0:
update.forward_backward(next_demo_batch(), loss="cross_entropy")
update.optim_step(optimizer=optimizer)
update.publish_weights(to=sampler)
await update.result()
await (await learner.save_state("final")).result()
await learner.close()
The rollout and batch helpers shown are illustrative examples. Your agent logic, reward computations, and data handling belong in those helper functions. The loop applies importance-sampling loss for RL and cross-entropy loss for supervised examples, then awaits the update completion. It persists the final learner state and terminates the learner.
You retain control over how frequently supervised examples feed into the loop, how rewards are computed, and how the training recipe adapts. SFT, RL, and distillation all leverage the same training primitives, while the platform manages the distributed systems infrastructure.
Willow started with supervised learning, then incorporated RL reward models to minimize filler language, enhance readability, and eliminate self-corrected passages. These reward models merged into a unified reward function. Lawrence Liu, Willow's chief technology officer and co-founder, noted improvements in text formatting, emoji processing, and responsiveness to user commands compared to the production model.
Eliminating checkpoint synchronization overhead in RL training
RL serving requires sufficient rollout generation, alignment with the policy under training, and rapid weight propagation. The trainer, reward computation, and environment can remain in place while CoreWeave Inference manages the serving component of the loop.
CoreWeave RL Rollouts builds on Nvidia Dynamo with vLLM integration. When the trainer writes a checkpoint, the service loads it into an active deployment without interrupting in-flight requests or endpoint operations. It supplies training metadata: token log probabilities (logprobs), token identifiers, and expert routing information for mixture-of-experts (MoE) architectures.
When the trainer writes a checkpoint, the service hot-loads it into a live deployment while preserving in-flight requests and endpoint availability. Serving uses the trainer's current policy weights, removing staleness and redeployment delays that would otherwise hold up the next batch of rollouts.
Checkpoints can be full snapshots or incremental updates, minimizing synchronization overhead after the initial transfer. The serving layer operates with the trainer's latest policy weights, eliminating staleness and redeployment latency that would otherwise delay subsequent rollout batches.
Evaluating quality gains and infrastructure efficiency independently
CoreWeave validated the workflow in collaboration with Nvidia and you.com by post-training Nvidia Nemotron 3.5 Lightning for web search functionality. The training run combined SFT and RL, incorporating you.com's search capabilities directly into the training loop.
BrowseComp accuracy rose from a baseline of 36.97% to 45.45%, representing a 22.94% relative gain. Average tool invocations decreased by 30.24%. Independently, hot-loading checkpoints into an active deployment operated roughly 15 times faster than conventional redeployment procedures.
Improved search integration, post-training optimization, and harness refinements collectively contributed to accuracy gains and reduced tool usage. Hot-loading eliminated time spent waiting for checkpoint synchronization. For teams attempting to make a smaller or more efficient model competitive with models significantly larger, that additional training velocity becomes critical.
Maintaining traceability between models and experiments
CoreWeave Forge reconnects production inference, training, and serving. Model Distillation provides a managed pathway. The programmable API enables researchers to direct the training loop without managing infrastructure. RL Rollouts allows teams to maintain their own trainer while delegating rollout inference to CoreWeave.
Models generated by serverless training emerge as tagged artifacts that transition into Serverless Inference without requiring a separate registry step. Experiments remain documented alongside them, preserving the connection between the training run, resulting model, and eventual deployment.
Choose a failure you can reproduce, use it to guide the training method, and compare the candidate with the model serving users.
Identify a reproducible failure, let it inform your choice of training technique, and evaluate the candidate against the model currently in production. Preserve the production examples, evaluation results, and resulting artifact together so your organization can reference them in future iterations.
Additional details appear in the CoreWeave Forge blog and announcement press release from Fully Connected 2026. CoreWeave intends to publish further technical documentation covering Model Distillation, the programmable training API, and RL Rollouts.