None of the 9 Biggest AI Companies Can Prove You Got the Model You Paid For

Published: August 22, 2026 Impact: High. Affects paying subscribers across ChatGPT, and developers on Gemini's CLI tool; the underlying transparency gap applies industry-wide.

The Scorecard Nobody Passed

Pebblous, an AI data-quality company, published a scorecard on August 14 testing nine major AI model makers (OpenAI, Anthropic, Google, Meta, xAI, Mistral, DeepSeek, Cohere, AI21) on one question: can an outsider verify the model you're told you're using is the model that actually answers you?

None of the nine companies scored well enough to make that verification possible, and the best of them still failed most of the checks. The average score was 16.4 out of 37 points, 44.4%. OpenAI scored highest, at 23 out of 37, 62.2%: the best result in the group, and still a majority failure rate.

Pebblous names five ways a model can change behavior without its version number ever moving: fine-tuning, safety-classifier updates, system-prompt edits, retrieval changes, and routing requests to a different sub-model. Any of the five can happen silently. One caveat matters here: Pebblous itself describes this as "single rater analysis... preliminary," not an audited industry standard. Treat the numbers as a data point, not a verdict.

What "Routing" Means for You

"Model routing" is a company sending your request to a cheaper, smaller model instead of the one you selected, without telling you. Pay for a specific model because it reasons better, and get a lesser one instead, and you're paying premium price for a discount answer. Most people would never know; the only trace is a field in a browser's network traffic, a tab almost no one opens. That's exactly how ChatGPT users found it.

The ChatGPT Case: A Month of Reports, No Confirmation

Since GPT-5.6 launched July 9, OpenAI's developer forum has carried a growing thread of reports. Users on the Pro and "Sol" tier found selecting GPT-5.6 returned answers actually generated by GPT-5.5-mini, confirmed by a field called resolved_model_slug in the network response, which names the model that actually ran. A July 23 thread is still active into August; an August 12 thread extends the complaint to Plus-tier subscribers.

On August 21, a ChatGPT Business admin escalated further, filing a support case with request IDs and HAR files after finding the bug is browser-dependent: Chrome routed to GPT-5.5-mini, Edge's private mode did not, with no confirmation OpenAI compared the two logs. A fresh thread today, August 22, gathers reports from more users, and the same bug has separately been filed against OpenAI's Codex desktop and iOS apps.

OpenAI's own GPT-5.6 update page, published August 6 and corrected August 19, addresses an unrelated evaluation-score error, nothing about routing. As of publication, OpenAI has not acknowledged the issue anywhere, and no major tech outlet has covered it.

A Second Example, Not the Same Story

A comparable bug hit Google's gemini-cli, a developer tool, on August 17: requesting any gemini-<version>-flash model, including version numbers that don't exist, silently returns gemini-3.5-flash instead. A fix has been proposed by an outside contributor, not a Google employee; no Google staffer has commented on the issue itself. No evidence this reaches Google's consumer Gemini app. It's confined to the CLI.

Consumer Protection Q&A

Q: Is OpenAI stealing my money? A: Nobody has proven intent. What's documented: a routing bug, unacknowledged for over a month, while paying users get a cheaper model than they selected.

Q: Does this affect Gemini's consumer app too? A: No evidence of that. The bug is confirmed only in the developer CLI tool, not the app most people use.

Q: How would I check this myself? A: Mostly, you can't, and that's the point of the story. resolved_model_slug only shows up in a browser's network tab, a tool built for developers, not subscribers.

What to Do Next

• ChatGPT Plus/Pro/Business subscribers: an unusually short or shallow response for a model you're paying for is now a documented possibility, not paranoia.
• Developers on any of the 9 rated vendors: treat routing and system-prompt changes as unlogged by default; pin model versions where the API allows it.
• Gemini CLI users: avoid the generic -flash version pattern until PR #28893 merges; request the full explicit model ID instead.

Sources

• Pebblous, "Silent Model Updates: The Disclosure Gap" (August 14, 2026)
• OpenAI Developer Community: July 23 thread, August 12 thread, August 21 case report, August 22 thread
• openai/codex issue #34677 (opened July 22, 2026)
• OpenAI GPT-5.6 August update page (published August 6, corrected August 19, 2026)
• google-gemini/gemini-cli issue #28859 (opened August 17, 2026) and PR #28893

Disclosure: This article's sourcing is OpenAI Developer Community forum posts and GitHub issues, user-submitted reports not independently verified by OpenAI or Google, plus one industry report its own authors call preliminary. No press outlet has corroborated any of it as of publication. Read it as a documented pattern worth watching, not a settled scandal.

Verified by OneHuman · August 22, 2026

Independent. AI-assisted. Human-verified. No ads. No affiliates. No investors.