Following the launch of the Comet AI browser by Perplexity, experts began examining its security features. Evaluations by Brave confirmed that these types of browsers are susceptible to malicious requests from fraudsters, which can endanger users' personal information. This was further corroborated by OpenAI.
OpenAI, which recently released the ChatGPT Atlas browser, published a new blog detailing a discovered vulnerability and measures for its remediation. They emphasized that the implementation of malicious requests is an ongoing security issue in artificial intelligence, necessitating regular enhancements in product defenses.
Prompt injection represents a type of attack on AI agents in browsers, where harmful instructions are deliberately embedded into the content. These attacks can be hidden on websites, in emails, PDF files, or other materials processed by the AI. The goal of such attacks is to manipulate the model's behavior and execute commands from the attacker instead of the user's requests.
These attacks are particularly dangerous as they often do not require human involvement. Users may remain unaware that the AI agent is secretly transmitting their personal information to fraudsters or performing other actions set by malicious actors, such as sending spam emails.
To counter these attacks, OpenAI developed an "automated LLM-based attacker" — essentially, an AI bot that simulates hacker actions and attempts to implement prompt injection. Initially, this AI tests attacks in a separate simulator to observe how browser agents respond. By analyzing results, the system continuously modifies and improves its attacks to better detect them in real-world scenarios. The data obtained is later integrated into protection mechanisms.
OpenAI also showcased a demonstration of prompt injection that its AI detected and used to enhance the protections of ChatGPT Atlas. In this scenario, an attacker sent an email containing a hidden instruction for the AI agent — effectively a template for a resignation letter addressed to the CEO. Later, when a user requested to draft a message about absence from work, the agent could have used this instruction to send a resignation email. However, due to its training, the system recognized that the instruction was a harmful prompt injection and did not execute it without explicit confirmation from the user.
"The nature of prompt injection makes deterministic security guarantees challenging, but through scaling our automated security research, competitive testing, and reinforcing our rapid response cycle, we can enhance the model's resilience and protection before facing a real attack," the company stated in its blog.
Despite the introduction of new tools and security measures, prompt injection remains a serious threat to AI-based browsers. This raises concerns among some industry experts regarding the advisability of using such agent-based browsers given the risks to personal data.
