The on-call engineer at Nitric used to receive three or four pages per week, many of them false alarms. Caution at 2 a.m. seemed justified, but the team eventually realized that most interruptions stemmed from recurring issues that could be prevented. By refining their incident workflow and deploying specialized AI agents for distinct tasks, they reduced both unnecessary alerts and the underlying problems causing them.

The breakthrough came from letting engineers review and correct agent assessments, then using those corrections to improve future incident handling. As the team fixed recurring issues, confidence grew about when to actually escalate. This experience prompted a rethink: if a single capable agent could investigate, ticket, and prepare a fix, why split the work at all? The answer revealed itself through practice.

When an agent's work needs correction, can you identify the responsibility involved and make a change you can evaluate independently?

A shared workspace for people and agents

Nitric built Sugabots, an open-source self-hosted workspace where teams collaborate with multiple agents on the same task. Engineers ask questions, contribute evidence, review findings, and bring in additional agents as needed, with all exchanges visible to participants. Work organizes into shared environments called pods, with external system connections through MCP. Each agent can run its own model and instructions, while the pod's connections and tool policies define shared access. Teams configure agents for specific responsibilities and refine instructions based on engineer review.

How we run incident triage in Sugabots

The incident workflow depends on each engineer or agent being able to continue from recorded evidence, decisions, and corrections. The Sentry triage agent investigates new, regressed, or escalating issues and decides whether to alert the on-call engineer, request an in-hours review, hand a small fix to an implementation agent, or keep the issue in Sentry for monitoring. Follow-up work gets recorded in Linear, where a backlog agent organizes tickets and the implementation agent prepares a GitHub pull request for review.

Engineers participate throughout, reviewing findings, challenging urgency assessments, correcting assumptions, and deciding whether to ship proposed changes. The investigation remains available as work moves between people and agents. Safeguards include instructions for urgent escalation when an issue could plausibly be a production outage with uncertain urgency. Issues classified as noise get archived until escalating, and Sentry retains a direct spike-alert route to incident.io as backup.

Workflow diagram of incident triage in Sugabots
Our incident workflow: agents handle distinct responsibilities while engineers contribute to the shared investigation. Engineers can take on any of the contributions shown.

Follow-up work matters as much as initial assessment. Previously, the on-call engineer might investigate an alert, determine it could wait until office hours, and move on. Recording the underlying problem in the backlog and prioritizing it gave the team a chance to resolve it before the next on-call engineer encountered the same alert. Having the backlog agent record follow-up work in Linear provides a consistent way to prioritize fixes for recurring issues behind the interruptions.

Specialize around a decision

Each agent gets configured for a particular responsibility, with model and instructions chosen to suit the work and a clear definition of what success looks like. Nitric chose Opus 5.5 for Sentry triage, with instructions covering evidence gathering, classification, escalation, handoffs, and recovery from tool failures. The triage agent makes these decisions:

  • Urgent: active customer harm requiring immediate attention; alert the on-call engineer through incident.io.
  • In-hours review: a defect requiring engineering judgment that can wait; notify the on-call engineer at low urgency.
  • Auto-fix: a small, non-urgent code change outside restricted areas such as authentication, billing, and migrations; brief the implementation agent to prepare a PR.
  • No action: expected or transient noise; archive until escalating so a spike brings it back.

The triage agent states the evidence and uncertainty behind each decision. Implementation briefs must stand alone and include relevant findings and links. The backlog manager's instructions reflect decisions involved in organizing follow-up work: checking for duplicates, looking up the current Linear cycle, and distinguishing actionable work from issues needing engineer clarification or design decisions. Those rules provide defaults while preserving the team's explicit choices about priority, assignment, and scheduling.

Triage and implementation make a useful comparison. Triage asks whether available evidence justifies interrupting an engineer. Implementation asks what change would resolve the problem and how to verify it. Those decisions have different failure modes: a missed urgent incident, an unnecessary interruption, or a plausible fix that introduces another bug.

Triage asks whether the available evidence justifies interrupting an engineer. Implementation asks what change would resolve the problem and how to verify it. Those decisions have different failure modes.

Maintaining those responsibilities separately allows refining how incidents are classified while keeping implementation instructions stable. If an investigation leads to an ineffective fix, the team can examine the assessment's accuracy, the evidence preserved in the ticket, and how the implementation addressed the recorded problem. Each gives a specific part of the workflow to improve.

Each responsibility yields concrete evaluation questions:

  • Triage: Did it escalate genuinely urgent incidents? Did it unnecessarily interrupt an engineer?
  • Backlog management: Did it record the follow-up work, recognize existing tickets, and preserve enough evidence to act on it?
  • Implementation: Did the proposed change address the problem, and what did an engineer need to correct before accepting it?

Evaluation must consider the reduction in wake-up calls alongside how genuinely urgent incidents are handled, including any that triage missed. Fewer interruptions could reflect both better classification and fixes that reduce incident volume. Those outcomes help judge workflow performance and identify improvement areas.

Match permissions to responsibilities

Separating responsibilities also helps decide which tool actions each agent needs and when an engineer should approve them. Triage instructions include specific write actions: creating Linear issues, sending alerts through incident.io, and archiving Sentry issues until escalating. Tool access must reflect those responsibilities, including which actions require engineer approval.

Agent-specific instructions define responsibilities within the access available to the pod; they do not enforce separate permissions for agents sharing that pod. In Sugabots, teams configure tool-level Off, Ask, and Allow policies on a pod's connections, and every agent in that pod inherits those policies. Configuring them once gives a consistent set of rules for automatic actions, engineer approvals, and disabled tools, including when another agent joins the work.

Triage needs access to evidence and a way to escalate, while implementation needs enough access to prepare a change. Genuinely different access boundaries require separate enforcement, potentially through different pods with different tool policies or appropriately scoped connections. Merge and deployment authority also need explicit policies and, where available, appropriately scoped credentials.

Separate the working context from the investigation

Three distinct elements have different useful lifetimes:

  • Configuration: the maintained model, instructions, tools, and expectations for a responsibility.
  • Execution context: the messages and tool results available to the model while it works on a particular task.
  • Shared history: the record of evidence, hypotheses, corrections, and decisions that people and agents accumulate.

Consider an investigation where the agent rules out an outage and concludes an issue can wait until office hours. After enough tool calls, its context may need compacting. A summary might preserve the conclusion while omitting evidence that ruled out the outage. That detail becomes important if an engineer challenges the assessment. Keeping the original history gives them a way to inspect the reasoning and evidence behind the summary. The agent can work with a compacted context while the fuller record remains available for inspection or retrieval.

To continue the investigation, the next agent needs relevant evidence and corrections selected from that record and passed along in a brief, or retrieved as it works. Preserving the conversation gives a source to draw from; selecting the working context remains an explicit part of the handover.

Keep people in the investigation

The same distinction matters when work moves between engineers. An engineer may recognize a pattern the agent missed or explain why an apparently sensible fix would introduce a regression elsewhere in the codebase. That contribution needs to remain available to the engineer or agent continuing the investigation or preparing the fix.

Diagram showing the reviewer and author relaying findings between their separate chats.
The reviewer and author relay findings and corrections between their separate chats.
Diagram showing engineers and agents contributing to a shared discussion.
Engineers and agents contribute to a shared discussion that preserves their findings and corrections.

In separate chats, people become responsible for carrying those corrections across. Even when both agents can inspect the same repository and tickets, each conversation contains a different account of what the team has learned. In Sugabots, engineers can contribute to the shared discussion, leaving their corrections alongside the findings that prompted them. The next engineer can see how the investigation changed direction, and an agent continuing the work can receive that correction with its brief. MCP gives the agent access to evidence, while a shared workspace gives the team access to the investigation.

Screenshot of a staged Sentry triage conversation in the Sugabots interface.
A staged Sentry triage conversation rendered in the actual Sugabots interface. The messages, event counts, issue reference, and handoff are scripted demonstration data, showing how an engineer’s correction can remain alongside the assessment and implementation brief.

When is another specialist worth it?

Each additional specialist brings configuration to maintain and a handover whose context needs care. Splitting agents by language, service, or deployment environment is useful when that boundary helps the team maintain and evaluate the work. Nitric's current divisions follow responsibilities that already existed: the on-call engineer assessed urgency, the team organized follow-up work in the backlog, and engineers prepared and reviewed fixes. Giving agents distinct configurations for those responsibilities lets the team improve each part of the workflow independently.

For a team trying this approach, start with one recurring responsibility where you can describe a useful result and recognize a bad one. Write down the success criteria, including the errors that matter: for triage, that means examining missed urgent incidents as well as unnecessary escalations. Define the evidence the agent needs, choose a configuration that enforces the required tool access, and specify when an engineer should review its findings or approve an action.

Keep the investigation history, including the evidence behind decisions and the corrections engineers make, so the next engineer or agent can continue from it. Review those corrections to decide whether the instructions, available evidence, permissions, or handover need to change. Earlier investigations can then provide cases to revisit alongside new work, checking whether a change addresses the original mistake without introducing another.

Increasingly, I'm interested in whether we can define an engineering responsibility clearly enough that every investigation, correction, and reviewed fix helps us improve how we handle the next one.

That may justify a dedicated agent, or it may be enough to load a reusable configuration when the job appears. The question is whether the separation makes the work easier to maintain and evaluate. Nitric started by wondering whether to specialize agents. The team increasingly focuses on whether an engineering responsibility can be defined clearly enough that every investigation, correction, and reviewed fix helps improve how the next one is handled.

Try Sugabots

Sugabots is open source and self-hosted. Start with one recurring engineering task and a colleague who can review the agent's work, then use the shared discussion and corrections to refine how it handles that responsibility. If the project looks useful, give Sugabots a star on GitHub and share it with a team that might benefit. Stars and recommendations help an open-source project reach the engineers who could use it or contribute to it.