DeepSeek's open-source coding agent is taking a performance hit on a major benchmark due to one of its built-in behaviors: terminating background processes after completing a task. While this cleanup approach is generally considered good practice in software development, it's causing measurable problems in specific evaluation scenarios.
The issue emerged when Hugging Face harness optimization researcher Joël Niklaus incorporated DeepSeek Harness into his coding-agent leaderboard, which evaluates ten different harnesses running identical underlying models. Across five scored evaluations, DeepSeek Harness ranks statistically tied for first place. However, on Terminal-Bench specifically, it falls to sixth position—a drop driven almost entirely by this single design choice.
Process termination breaks persistent tasks
The root cause: DeepSeek Harness shuts down background processes when a task concludes, but three Terminal-Bench tasks require those processes to remain active to succeed. Testing with DeepSeek-V4.1-Flash and GLM-5.3-Flash models revealed that DeepSeek Harness completed only two of 36 attempts on these problematic tasks. By contrast, OpenCode, Pi, and Nous Research's Hermes Agent each passed all 36 attempts.
DeepSeek Harness passed just two of 36 attempts … while OpenCode, Pi and Nous Research's Hermes Agent passed all 36 on those same tasks.
Joël Niklaus
This behavior cost DeepSeek Harness between 1.5 and 1.7 percentage points in overall scoring. On DeepSeek-V4.1-Flash with high reasoning effort, the penalty exceeded the entire 1.3-point gap separating it from the leaderboard leader. DeepSeek's own documentation characterizes its local job registry as intentionally ephemeral, stating that it should be used "when jobs should live in the harness process and die with it." Workloads requiring persistence across restarts or multiple processes need a different backend configuration.
Harness choice matters as much as model selection
Niklaus launched his Harness Leaderboard on November 5 after running 28,849 trials using nine harnesses and three open-source models: DeepSeek-V4.1-Flash, GLM-5.3-Flash, and Flint. By keeping the model, provider, and reasoning effort constant while varying only the harness, he demonstrated that orchestration layer choices produce substantial performance differences.
Mini-swe-agent, a relatively straightforward bash-based harness, achieved a 51.4% task completion rate, while OpenClaw managed only 27.3%—though several other harnesses were statistically equivalent to the leader. Cost efficiency diverged even more sharply: Pi delivered a 49.4% solve rate at $0.29 per solved task, whereas OpenCode cost $3.35 per solved task during initial testing.
Hugging Face co-founder Thomas Wolf summarized the findings on X as "same open model, different harness => very different results."
Proprietary models show similar harness sensitivity
Y Combinator President and CEO Garry Tan ran parallel evaluations using Claude Opus 5.5 and GPT-6.1 Sol, finding that harness selection influenced cost more dramatically than task success rates. When running Opus 5.5, OpenClaw's per-solved-task cost was roughly double that of competing Opus harnesses, primarily because it repeatedly transmitted context with each request.
Testing of Tan's gstack workflow skills revealed another harness-related issue: across 46 sessions, Opus 5.5 read the instructions but never executed the prescribed workflow steps, and these attempts cost approximately twice as much as expected. Researchers acknowledged that the tasks were relatively straightforward, leaving open the question of whether instruction-following would have improved outcomes on harder problems.
The results make a case for testing the harness just as carefully as the model.
The New Stack reached out to DeepSeek for comment but had not received a response by the time of publication.