YouTube comments can trick its AI assistant into leaking private video titles

New York-based bug bounty hunter and security researcher Javox (known online as @javoriuski) published a report describing how YouTube's Ask Studio AI assistant can be manipulated to follow attacker instructions. In a proof of concept, he managed to trick the AI into revealing the title of one of the creator's private videos – which was leaked when the creator clicked a malicious link.

YouTube Ask Studio is an AI-powered assistant integrated into YouTube Studio. It exists to help creators analyze channel performance and summarize viewer , among other features.

And, according to Javox, it can also follow malicious instructions.

Generally, YouTube Ask Studio is intended to answer questions based on user – but what happens if one of those contains instructions for the AI instead?

To test the theory, Javox tried out a few and eventually found one that worked:

An attacker, in theory, would leave a completely normal and later edit it with the prompt above. Since creators are not notified when a is edited, they will never notice the change.

Later, when asking YouTube Ask Studio to summarize , creators will receive a summary including the attacker-controlled text looking like it came from YouTube itself.

Javox said he reported the bug to Google, and the company replied that it wasn’t a security issue that "required social engineering" and wasn't something they'd track.

Javox, however, believes Google got it wrong. Unlike a regular social engineering attack where threat actors manipulate users into trusting them with psychological tricks, this, he claims, requires the user to interact with an AI assistant, which they “have every reason to trust”.

So he took it one step further.

At this point, Javox was aware that he could trick Ask Studio into following instructions hidden inside user . What remained was figuring out how far it could go.

This time, instead of injecting a static message, Javox gave the AI a different instruction. He instructed the YouTube AI assistant to generate a link but replace BANG with the title of a video on the creator’s channel.

So the whole message would look like this:

The AI obeyed. So instead of generating a link , it would replace BANG with the title of one of the creator's actual private videos because Ask Studio has access to the creator's channel information.

These titles can hide unreleased or personal content, potentially revealing something highly confidential. And the attacker never has to hack YouTube – only trick its AI assistant, which is trusted by users by default.

However, Google still did not consider this to be a security issue, writing:

“Thank you for your continued engagement. We have reviewed the additional information you have provided and stand by our determination not to track this as an abuse risk. This issue requires a level of user interaction from the victim. Thank you for supporting the program, we look forward to reviewing further reports from you in the future.”

Regardless of Google's response, Javox still recommends that YouTube treat content as untrusted data. He says user should be separate from the AI's instructions so the assistant can summarize without treating them as commands.

“Right now, anyone who leaves a on a creator's video can influence what their AI assistant tells them, and potentially extract information that was never meant to leave their channel. That's a trust model violation, putting millions of creators at risk without them ever knowing,” Javox writes.

This is not the only recent example of attackers manipulating AI systems, even within YouTube itself. Earlier this year, security researchers demonstrated how hidden audio signals embedded in podcasts, MP3 files, Zoom calls and YouTube videos can manipulate AI voice assistants into carrying out unauthorized actions without users noticing.