Published

Earlier on this story17 Nov: Google makes it easier to use Gemini without training AI on your data

Technology and Scienceupdated 26m ago2 min read

13 outlets compared

Google Upgrades Gemini for Mac With Computer Use To Click, Type, and Fill Forms

AppleInsider
“but the convenience comes with security risks serious enough to make handing over that control a bad idea.”
Crypto Briefing
“Control is the selling point”
5 Western Alternative4 Other2 West Asian2 Other source types

This article is an automated aggregation of coverage from news outlets. Our editorials offer in-depth analysis curated by the NewsCord team.

Gemini for Mac can control apps, files, and websites to complete tasks. It clicks, types, and fills forms in a background/isolated window.

Gemini takes over Mac tasks

Google is upgrading the Gemini app on Mac with a computer use feature that lets the assistant click, type, navigate menus, and fill out forms across local desktop apps and the web. Gemini runs in an “isolated background window with a live preview right in your chat,” while the main screen and cursor remain free for other work.

“Gemini works inside an isolated background window with a live preview right in your chat.”
blog.google

Google says users can stop Gemini at any time with one click, and Google also says users choose what apps Gemini uses. Google is rolling out computer use in US English starting today for Google AI Plus, Pro, and Ultra subscribers, and the feature is described as handling multi-step tasks that span desktop apps and websites that do not connect to each other.

Reporting for this sectionblog.google9to5Google

Permissions, previews, and pauses

Google says Gemini can perform actions instead of simply helping users prepare them, and the assistant will navigate menus, click buttons, enter information, and move between apps to finish a task. Google also says Gemini works in a separate virtual screen, leaving the main display and cursor available for other work, and Google says users can stop the task at any time.

AppleInsider highlights that Google requires users to enable macOS Accessibility and Screen & System Audio Recording permissions before Gemini can operate Mac apps, and AppleInsider also notes that Gemini requests permission before controlling individual apps. Apple is tightening Mac permissions in macOS 27.3 developer beta 2 on November 16, and Apple’s changes include options to require approval each time and display an indicator when one app controls another.

Reporting for this sectionAppleInsider

Use cases and safety warnings

Google’s examples for computer use include speeding up creative workflows by batching color-grade photos in design software or setting up complex 3D scenes, and Google also points to skipping manual form entry by extracting weights, dimensions, and addresses from local PDFs. Google also describes turning live data into ready-to-present decks by pulling metrics and screenshots from live dashboards across multiple regions and building a presentation in Google Slides.

Google warns that Gemini can access active sessions in native Mac apps and read or interact with multifactor authentication codes visible in desktop or messaging apps, and Google warns that untrusted content in one app could redirect Gemini into taking actions in another app it has permission to use. Google also warns that Gemini can attempt to create temporary backups when its on-demand backup setting is enabled, but Google warns that some apps do not support backups and certain changes cannot be undone, with backups expiring after 24 hours or when the user submits another prompt, whichever comes first.

Reporting for this section9to5GoogleAppleInsider