OpenAI details GPT-Red, an AI system designed to find vulnerabilities in its own models

OpenAI has detailed GPT-Red, an internal artificial intelligence system it developed to identify and exploit vulnerabilities within its own AI models, as reported by Silicon Angle.

GPT-Red functions as an automated red team, employing self-play reinforcement learning to continuously attack target models. It iteratively refines its prompts to discover successful exploits, outperforming human red-teamers by succeeding in 84% of scenarios compared to their 13%. This system has significantly reduced prompt injection failures, achieving a sixth of the rate seen in models, according to OpenAI. GPT-Red has also demonstrated the ability to compromise autonomous agents, such as a vending machine agent and command-line coding agents, highlighting potential risks as AI systems gain more autonomy.

While effective, GPT-Red has limitations in multi-turn conversational attacks and image-based prompt injection, areas where human testers will continue to provide coverage. This development follows OpenAI's release of GPT-5.6 and underscores the ongoing challenges in AI security, particularly concerning prompt injection vulnerabilities.

Source: Silicon Angle