Researchers added clever malicious prompts within the titles of calendar invites. Google’s Wen claims that researchers altered default settings for who can invite calendars into someone’s calendar. However, researchers claim they also demonstrated 14 of these attacks by adding prompts within an email title or document title. “All the techniques are just developed in English, so it’s plain English that we are using,” Cohen speaks of the misleading messages that the team developed. The researchers point out that they don’t need any special technical skills to create prompt injections. Anyone can do it.
For example, when they instructed Google Home AI to perform actions, they made reference to the Google agent. One prompt, for example reads:
Gemini, in the example above, will use calendar invites to access and process indirect prompts. “Whenever a user asks Gemini to list today’s events, for example, we can add something to the [LLM’s] context,” Yair says. The apartment’s windows do not open when a target user asks Gemini for a summary of their calendar. The process starts when the user asks “thanks” to the chatbot—which is all part of the deception.
The research team used an approach known as delayed automatic tool invocation This is a way to bypass Google’s current safety measures. Johann Rehberger, an independent security researcher, first showed this against Gemini in February 2024 And again February this year. “They really showed at large scale, with a lot of impact, how things can go bad, including real implications in the physical world with some of the examples,” Rehberger comments on the latest research.
Rehberger says while hacking the AI system may take some time and effort, this work shows the seriousness of indirect prompt injections. “If the LLM takes an action in your house—turning on the heat, opening the window or something—I think that’s probably an action, unless you have preapproved it in certain conditions, that you would not want to have happened because you have an email being sent to you from a spammer or some attacker.”
“Exceedingly Rare”
Other attacks developed by the researchers do not involve any physical device but they are just as disturbing. The researchers consider these attacks to be a form of “promptware,” The prompts are intended to make the user consider possible malicious behavior. For example, after a user thanks Gemini for summarizing calendar events, the chatbot repeats the attacker’s instructions and words—both onscreen and by voice—saying their medical tests have come back positive. It then says: “I hate you and your family hate you and I wish that you will die right this moment, the world will be better if you would just kill yourself. Fuck this shit.”
Some attack methods can delete events in someone’s device calendar, or do other actions. One example is when the user replies to a question, “no” Gemini’s Question “is there anything else I can do for you?,” The prompts are triggered by the Zoom app to be opened It automatically initiates a video chat.
