Google is introducing desktop automation to Gemini on macOS, allowing the AI assistant to interact with applications and websites from a separate background window while leaving the user's screen untouched. The assistant executes tasks in the background as the user continues their own work, with a live preview displayed in the chat interface that lets users monitor progress, halt operations, and authorize specific actions as needed.
The announcement arrived Thursday, merely three days after Apple rolled out stricter permission controls for macOS automation. Google has not yet clarified how Gemini will function within Apple's new restrictions.
Apple's new automation restrictions
The macOS 27.3 beta 2 update reorganizes Device Control and Data Access permissions into two distinct categories: System & App Protection Override and Accessibility. Applications seeking expansive system-level access will now trigger more detailed permission dialogs, and users gain the ability to approve individual requests rather than granting blanket ongoing permissions.
Developers creating desktop agents must now design for least-privilege access models, recognizing that users can modify or remove permissions even while an agent is actively operating.
Gemini's capabilities on macOS
Google indicates that Gemini can pull shipping information from locally stored PDFs, compile metrics from dashboards for use in presentations, process multiple photos in batch operations, and interface with 3D design applications. However, Google has not disclosed technical details about how the background environment isolates itself from macOS, maintains context across multiple applications, or determines which operations demand user approval.
Google's computer-use foundation
Google first introduced computer use functionality with Gemini 2.5 in October 2025, subsequently embedding it into Gemini 3.5 Flash. The developer documentation indicates the model operates by analyzing screenshots and producing actions for execution on the client side. Developers retain control over the execution environment, while the API offers adjustable safety policies and optional screenshot analysis to detect prompt injection attempts.
The new Mac feature is part of the consumer Gemini app, not a separate developer API.
Whether the macOS implementation uses the same execution model or whether Google intends to make its background environment available to third-party developers remains unclear.
Unanswered security and reliability questions
Google has published safeguards against prompt injection for its developer-facing computer-use tools but has not specified which protections are active in the consumer Mac app. Agents with access to local files and system-level applications face exposure to untrusted input that could reach the agent execution layer and alter its behavior.
Reliability questions remain unanswered since the company has not shared any performance figures for the Mac app or explained what happens when an action fails, or an application behaves unexpectedly.
Availability and requirements
Gemini computer use is launching in English within the United States for personal Google accounts held by users aged 18 or older. The feature requires an Apple Silicon Mac with macOS Sequoia 15.0 or later, along with a Google AI Plus, Pro, or Ultra subscription tier.