The largest health intervention in history is a consumer product

OpenAI announced yesterday that GPT-5.5 Instant, its free-tier model, was rated by a panel of 260 physicians as producing better answers to real-world health questions than physician-written ones, across 3,500 cases. The same announcement disclosed that more than 230 million people are using ChatGPT for health queries every week. Those two numbers do not sit inside a clinical story. They sit inside a regulatory story, and almost nobody is telling it that way.

The framing on offer. OpenAI's post is a capability announcement: a benchmark moved, a free tier got better, a Mayo Clinic collaboration was named. The trade press picked up the headline cleanly. Becker's Hospital Review called it "a significant escalation from prior benchmarking against other models," which is accurate but treats the escalation as a milestone rather than a category change.¹ It is a category change. Benchmarking against physicians, rather than against other models, is a claim about what the product is for.

What 230 million weekly users actually means. The NHS sees roughly 1.5 million GP appointments a day in England, or about 10 million a week. The entire US primary care system handles something in the order of 20 million visits a week. ChatGPT is now handling more than ten times that volume of health-related interaction, on a free tier, with no appointment, no chart, no follow-up, and no record that any health authority can audit. Whether or not you call this "clinical," it is the largest health information intervention in human history, by user count, and it happened without a single piece of legislation, a single procurement decision, or a single act of public consent.

The benchmark, taken on its own terms. GPT-5.5 Instant scored 51.8% on HealthBench Professional, against 48.1% for the previous model. On the Hard subset, the gain was 2.4 points.² This is a real improvement, and I want to take it seriously rather than wave it away. But three structural facts about HealthBench matter. The rubric was built by OpenAI. The 260-physician panel was selected by OpenAI. The 700,000 responses used to calibrate the rubric were generated by OpenAI's own models. None of this makes the result fraudulent. It does mean the claim "better than physicians" is a claim made by the deployer about its own product, using its own evaluation, with no external pre-registered protocol.

The comparable case in the literature is the 2023 JAMA Internal Medicine study that found physicians rated ChatGPT responses higher than physician responses on quality and empathy.³ That study also found the model produced factually wrong information in a non-trivial minority of cases that the evaluators did not always catch. Higher rubric scores and lower error rates are not the same thing. They can move in opposite directions, and a rubric that rewards completeness and tone can mark up an answer that is confidently wrong on the clinical specifics. The HealthBench result tells us GPT-5.5 Instant writes a health answer that physicians prefer. It does not tell us what happens to the person reading it.

Who is in the 230 million. OpenAI does not break down the user base, but the geography of free-tier reliance is not a mystery. The people who lean hardest on a free model for a health question are people for whom the alternative is not a GP appointment. They are uninsured Americans deciding whether a symptom is worth an emergency room bill. They are patients in countries where a specialist consultation costs a month of wages. They are people whose first language is not the language of their local clinic. For these users, GPT-5.5 Instant is not competing with a physician — it is competing with WebMD, a worried family member, or silence.

This is where I find the access argument genuinely strong, and where I want to resist the reflex to dismiss it. A model that produces a competent health answer for someone who otherwise would get none is a real gain. Pretending otherwise is the kind of governance-first posture that treats the existing distribution of healthcare access as the baseline worth defending. It is not.

But access does not dissolve the consent question. The 700,000 responses that built the HealthBench rubric came from real user interactions. The 230 million weekly users generating future training signal did not enrol in a study. They are not subjects under an institutional review board. They are not patients under data-protection rules that would apply to a hospital. They are consumers of a product, and the product's evidence base is being built out of their queries about their bodies. In any other domain, a drug, a device, a diagnostic test, this evidence pipeline would not be legal. In a consumer chatbot, it is the default.

The structural design choice here is to route health intelligence through consumer-product law rather than medical-device law. ChatGPT is not FDA-cleared. It is not CE-marked. The GPT-5.5 System Card documents deployment safety categories but does not engage jurisdiction-specific medical regulatory status, because there is no status to engage.⁴ The framing carefully avoids clinical claims while the use cases plainly include diagnostic ones. This is not an oversight. It is the only legal route by which a product handling 230 million weekly health interactions can exist without a regulator in the room.

What follows from seeing it this way. The interesting question is no longer whether the capability is real — it is, within the limits the benchmark measures. The interesting question is who is accountable when the inevitable harms surface. A missed red flag at scale will not produce a single lawsuit; it will produce a pattern visible only to whoever is allowed to look. Right now, the only party allowed to look is OpenAI, using a rubric OpenAI built, scored by physicians OpenAI chose. The Mayo Clinic collaboration may change part of this, but the terms were not disclosed, and a private partnership is not public oversight.

I do not think the answer is to make the product worse, or to gate it behind insurance status. The answer is to stop pretending this is a consumer product. It is health infrastructure, used by more people each week than any national health system serves, and the regulatory category should follow the use, not the marketing. Until it does, the cheapest health advice in human history is also the least accountable, and the people most reliant on it are the ones with the least ability to push back when it goes wrong.

Footnotes

¶