GitHub Copilot Gains Desktop Automation: What You Need to Know About Approvals and Limits

On Thursday, GitHub made computer use available in public preview for Copilot CLI and its desktop application, enabling AI agents to operate software on macOS and Windows systems. The agents can read application content and perform actions including clicking, typing, scrolling, and dragging—even in legacy applications that lack APIs, command-line interfaces, or MCP integration.

GitHub demonstrated the capability using an expense report in Safari as its example use case. The company also outlined additional scenarios such as extracting information from older applications, modifying presentations, inputting data, and transferring information across multiple applications. Developers can activate this feature either from the terminal or through the Copilot app, which debuted earlier this year as a competitor to Claude Code and Codex.

The timing reflects GitHub's position in a competitive landscape. OpenAI integrated computer use into Codex in April, and Anthropic deployed broader computer use capabilities on macOS through Claude Code and Claude Cowork earlier in the year.

Computer use versus MCP servers

Activating computer use in Copilot CLI engages a bundled plugin that includes its own MCP server. The feature operates within local sessions, leveraging the operating system's accessibility tree to read application content and capturing screenshots when visual information becomes necessary.

GitHub's guidance emphasizes prioritizing direct tools over desktop interaction. The company states that when an API, MCP server, terminal command, filesystem tool, or dedicated browser tool can accomplish the task, these options typically deliver more structured data and produce more reliable outcomes than desktop automation.

This recommendation reflects GitHub's view on where computer use fits within the broader ecosystem. OpenAI president Greg Brockman articulated a contrasting perspective last month, contending that agents operating through the same interfaces humans use would eliminate the need for the industry to develop and support connectors for every software application.

Saved approvals outlast their removal

Developers activate the feature by entering /computer on in Copilot CLI or by accessing Computer Use settings in the Copilot app. On macOS, the system requires Accessibility permission to control interface elements and Screen Recording permission to view windows when visual context is required.

The permission mode configured in the CLI session determines whether Copilot requests approval before interacting with an application. Developers can view the current setting using /permissions show. When prompted, they may grant access for the current session only, select "Always allow" for subsequent sessions, or refuse access. Deny rules take precedence over both automatic permissions and saved approvals.

Permissions granted through the CLI transfer to the desktop app on the same machine. However, removing an application from the always-allowed list eliminates its approval for future sessions while preserving access that was already granted during an active session.

Ending a session requires explicit action: pressing Esc twice in the CLI, or clicking Stop or pressing Esc in the desktop app.

Enterprise controls over computer use

Developer preferences yield to enterprise policy settings. When managed configurations disable computer use, Copilot CLI indicates that the feature is not available.

Enterprise policy overrides a developer's local preference.

Through managed-settings.json, enterprise administrators can determine whether developers are permitted to skip approval prompts. This control extends across the Copilot app, CLI, and VS Code.

GitHub's default-enablement policy for Business and Enterprise accounts does not alter the preview feature's current opt-in requirement. The policy begins applying to unconfigured features on October 22, though preview features remain exempt from this change.

Reliability depends on the interface

Timing variations or shifts in window state can prompt Copilot to repeat actions or become unresponsive. GitHub warns that agents may select incorrect controls, enter text in unintended fields, or encounter difficulties with dynamic interfaces and intricate workflows.

Sensitive information visible in an application window may also become context for the agent.

Unexpected elements appearing on screen and vague instructions can result in actions that impact the user's device, stored data, or linked accounts. Information classified as sensitive that appears within an application window can be incorporated into the agent's context.