Image input gave GPT-4 a new prompt-injection surface

Vision and voice added to an existing model

OpenAI announced on 25 September 2023 that ChatGPT could see, hear and speak, rolling out image input and a voice interface to Plus and Enterprise users. The same day, OpenAI published the GPT-4V(ision) System Card, which states that GPT-4V uses the same underlying model and training as GPT-4, completed in 2022, with image understanding added on top; the announcement introduced a new input modality and a paired text-to-speech and speech-recognition pipeline built partly on Whisper, not a new base model.

What the system card measured

The system card describes quantitative evaluations built specifically for image input: refusal rates for requests to identify people in photographs, accuracy and refusal parity across demographic groups for age, gender and race recognition, and performance on CAPTCHA-solving and geolocation tasks that the authors flag as capabilities with clear misuse potential alongside their usefulness. It reports that targeted safety training pushed the refusal rate on one internal test of person-identification requests above 98 percent, and separately that combining model-level mitigations with a refusal classifier reached a 100 percent refusal rate on a bank of known text-based jailbreak prompts converted into image form.

Why image input is a new injection surface

That last evaluation exists because of a specific finding: the system card documents that placing jailbreak instructions inside a screenshot, rather than as plain text, made them harder to catch, since text-based filters cannot scan an image before the vision model processes it. This is the same underlying weakness an earlier paper on indirect prompt injection had already named in text-only systems: an application that treats untrusted input as data can still have that input carry instructions the model follows. Image input extends the same problem to a channel that heuristic text scanners cannot inspect at all, which is why the system card describes relying on the vision model's own judgement, rather than a pre-filter, to catch this class of attack.

Questions to carry into your own evaluation

• Does an application that accepts image uploads screen them for embedded text or visual instructions before passing them to the model?
• Is a reported refusal rate measured against the same kind of attack the deployment actually faces, or against a narrower internal test set?
• Does a capability such as geolocation or CAPTCHA-solving introduce a misuse risk that the system card's own evaluations were designed around?

Adding vision to GPT-4 did not just add a capability; it added a channel that existing text-based safety tooling could not see into, which the system card treats as a distinct problem requiring its own evaluations rather than an extension of text-only mitigations.