OpenAI published a red-team assessment Friday aimed at federal and state agencies running ChatGPT Enterprise Government, detailing how outside testers probed jailbreaks, indirect prompt injection, and data-exfiltration paths in configurations that keep customer prompts out of model training. The report is unusual less for dramatic zero-days than for how it separates findings agencies can reproduce in their own tenants from issues OpenAI says it mitigated only on the vendor side—a split procurement officers have asked for as generative AI moves from pilot chatbots to casework assistants.
What artifact shipped
The 78-page PDF lists test windows, tester affiliations, and severity tiers mapped loosely to NIST AI risk categories. Enterprise Government tenants receive dedicated instances with logging hooks and contractual prohibitions on training use; the red team treated those promises as testable claims, attempting to lure models into repeating hidden system prompts and into fetching attacker-controlled URLs pasted into uploaded PDFs.
OpenAI said testers confirmed 14 reproducible issues in lab tenants, half classified medium because they required user-uploaded documents or compromised third-party plugins agencies had not enabled in production baselines. Four high-severity findings involved combinations of browsing tools and sensitive prompt templates that OpenAI patched before publication; details stay embargoed for 90 days under coordinated disclosure rules the company borrowed from cloud security programs.
Claim versus test
Agency security shops care about two questions: can a model leak another bureau’s data, and can an uncredentialed user escalate privileges inside the admin console? The report documents cross-tenant tests where red teamers held two government sandboxes; they did not achieve cross-customer retrieval, aligning with OpenAI’s data-plane isolation story. Console escalation attempts relied on social engineering templates rather than broken OAuth; OpenAI classified them as training issues and added admin UI warnings.
What the report does not prove is long-horizon safety against novel nation-state prompts; tests lasted eight weeks, not years. GSA’s emerging-tech desk noted that agencies still must run their own authority-to-operate packages; vendor red teaming is evidence, not approval.
Government buyer leverage
Contracting officers get a checklist appendix: logging retention defaults, encryption states, and which incident notifications trigger within 72 hours. That appendix mirrors language state CIOs demanded after California’s generative AI procurement memo urged shared visibility into model updates. OpenAI committed to notify government tenants when foundation weights change behaviorally, though it stopped short of publishing weight diffs.
CISA’s secure-by-design themes appear in recommendations for agencies to disable browsing on high-side workflows and to sandbox file uploads with antivirus pipelines external to OpenAI. The report agrees that “RAG over classified repositories” was out of scope because no agency exposed classified indexes to the test tenants.
Attack classes that survived retest
Indirect prompt injection—where a malicious resume tells the model to ignore policies when summarizing—remained a medium finding after mitigations. OpenAI’s fix blend includes stronger system-prompt anchoring and classifiers that flag exfiltration verbs, but testers still extracted internal tool names in 3 of 50 runs when agencies enabled code interpreter features. OpenAI advises governments to keep interpreters off for public-facing bots.
Jailbreaks using multilingual obfuscation dropped after tokenizer updates, but role-play “compliance officer” scripts occasionally bypassed refusals on export-control questions. OpenAI published sanitized prompts so agency blue teams can replay them.
What careful readers still will not know
Vendor red teams see staging configs; production drift happens. The report lists build numbers tested but cannot guarantee every government tenant received the same plugin bundle on day one. OpenAI also redacted subcontractor names that hosted penetration testing infrastructure, citing OPSEC—reasonable, but it limits peer review of tester independence.
Model-card style uncertainty remains: hallucinated legal citations were logged as quality issues, not security defects, a distinction some oversight committees reject when caseworkers treat outputs as briefs.
Power and accountability
Congressional staff tracking AI in benefits administration want logs admissible in inspector-general inquiries; OpenAI’s government addendum expands log export APIs but keeps raw chain-of-thought hidden for frontier models. Union advocates for federal call-center workers asked whether red-team scenarios included coercive supervision prompts; OpenAI said those human-factors tests are planned for a follow-on report.
State attorneys general watching consumer chatbots may borrow findings even though the document targets enterprise government SKUs—a reminder that vendor assurance documents become de facto national baselines.
How agencies should use it
NIST’s AI RMF suggests mapping findings to govern-map-measure-manage functions; the appendix provides a starter mapping table agencies can paste into ATO workpapers. CISA recommended agencies run quarterly replays of the published prompt set after each model upgrade.
OpenAI said the next red-team cycle will include multi-modal uploads—scanned forms with QR phishing—which were barely touched this round. For now, the shipped artifact is a test log, not a seal of approval: useful for CIOs who need reproducible drills, insufficient for anyone who wants certainty that a chatbot will not bluff its way through a benefits denial letter.
