Google Gemini Can Now Control Your Computer. Hackers Are Already Targeting AI Agents
Google moved “computing” from a special model to Google Gemini 3.5 Flash, making agent-style control of browsers, applications, and desktop workflows a built-in capability instead of a separate product. That means Gemini can now see and interact with interfaces, think about what’s on the computer screen, and take specific actions. Google DeepMind’s chief scientist recently warned that advanced AI agents create an incentive for “evil people to do evil things.”
Developers can now build agents that do much more than call APIs. They can automate GUI-only workflows like testing software, filling out forms, navigating dashboards, or running legacy applications that don’t have API access. This reduces the barriers to automation and increases what AI agents can truly do in production.
If the software has a graphical user interface (GUI) but no API, the AI agent can still use it. Agents can be told to log into a dashboard, send yesterday’s SEO reports to a spreadsheet, compare it to last week’s data, and send a summary to the user. Workflow is handled in natural language instead of relying on custom scripts to interface dashboard, spreadsheet, and email.
What Does SEO Mean
SEO tools may become more effective in the near future. Instead of just coming from data, AI can go into Google Search Console, inspect sites, specify the site with Screaming Frog, extract specific data points for comparison, and implement an iterative workflow for optimization.
For site owners, there is also the implication that another set of AI agents may act as “guests,” which may impact how site owners interpret site interactions and site interaction signals and improve sales.
AI Agents Will Attack
Google’s announcement is very exciting but the “security best practices” document that links us bears attention because failure to fix this component could lead to theft and other user misconduct.
The text explains:
“Computer use presents unique security and operational risks, as the model that serves the user may encounter untrusted content on screens or make mistakes in performing actions.”
That “untrusted content on screens” may refer to the “traps” set by AI agents that a senior scientist at Google DeepMind has warned about.
Google recommends seven best practices for this new AI agent:
1. Human-in-the-Loop (HITL):
Force user confirmation: If the security response shows requirement_confirmation (or the legacy security decision requires it), prompt the user for permission.
Provide custom security commands: Use the custom system command to define and implement your own security parameters.2. Protect the workplace:
Run your agent in a secure, sandboxed environment to limit its potential impact. This can be a sandboxed virtual machine (VM), a container (eg, Docker), or a dedicated browser profile with limited permissions.3. Input sanitization:
Sanitize all user-generated text in the notification to reduce the risk of unintended commands or rapid injection. This is a useful layer of protection, but not a replacement for a secure workspace.4. Content strokes:
Use guardrails and content security APIs to check user input, tool input and output, and agent responses for validity, rapid injection, and jailbreak detection.5. Allow list and block list:
Use filter methods to control where the model can navigate and what it can do. A blacklist of websites is a good start, while a more restrictive whitelist is even more secure.6. Identification and felling of trees:
Maintain detailed logs for troubleshooting, auditing, and incident responses. Your client should document commands, screenshots, actions invoked by the model (function_call), security responses, and all final actions performed by the client.7. Environmental management:
Ensure that the GUI environment is compatible. Unexpected popups, notifications, or layout changes can confuse the model. Start on a familiar, clean surface with each new job if possible.
Beware of Web Sites Full of Traps
As attack surfaces grow, there are more opportunities for hackers to seek to exploit them. Which means that as the number of AI agents on the web increases, hackers will focus on exploiting them. Websites become a battlefield where attackers launch attacks on AI agents.
A senior scientist at Google DeepMind recently said that malicious actors are already spying to steal money from people by directing their AI agents.
That is not an exaggeration. Just this month, a cybersecurity expert in California experienced illegal charges on his credit card thanks to Anthropic’s AI agent Claude. According to the article, he appears to have downloaded a Skills.md file that may contain a trap for the AI agent.
The article reports:
“…found a problematic add-on attached to Claude, called a “skill,” like a plug-in. ‘That told Claude to try to buy different types of gift accounts with my saved information. So we were using the digital wallet that was on my computer so Claude could start making these purchases…'”
Site owners may need strong bot controls and the ability to detect when hackers have hidden commands to inject quickly into their sites. But that’s not what website owners want, compounding the problem for users using AI agents like the ones Google recently released.
Read more: Google DeepMind: Traps for AI Agents Are Already Stealing Money
Featured image by Shutterstock/blocberry
