Most software domains have an evidence bar that bends. The product manager wants the feature; the engineering manager pushes back; a compromise ships; users complain or they don’t. Healthcare is not most domains. Clinical-grade software has an evidence bar that was set by the FDA, by HIPAA, by twenty years of harm-avoidance case law, and by the working memory of every clinician who has ever watched a tool fail at the wrong moment. The bar does not bend.
Which is why the AI-builder revolution is going to look strange in healthcare for a while. The marketing manager who can ship a partner-portal microsite cannot, in a regulated environment, ship an intake form that touches PHI. The clinician who can prototype an appointment-routing tool cannot ship the tool to actual patients without somebody licensed and auditable in the loop. The same speed that compresses a quarter into a Tuesday in marketing compresses nothing in healthcare, because the slow part is not the writing. The slow part is the assurance.
In healthcare, the question isn’t can the AI build it. The question is who is willing to sign that the build is safe to use.
What healthcare-grade actually requires
Three things, in rough order of how often we see them missed by AI-built first drafts.
Provenance. Healthcare audit trails are not log files. They are evidence in a possible future investigation. Every change to a patient record, every decision support recommendation, every identity verification, every consent capture, the system has to be able to reconstruct what happened, who saw it, what they did, and why. AI-built software is bad at provenance by default; it treats logs as a debugging tool, not a legal artifact. Closing that gap is most of what a Relay engineer does on a healthcare press.
Boundary discipline.PHI does not belong on every server. Healthcare-grade architecture pushes PHI to the smallest perimeter possible and treats every crossing of that perimeter as a controlled act. AI-built first drafts often cheerfully copy identifiers across services in ways that would never pass a compliance review. The fix is rarely a rewrite. It’s a refactor at the boundaries: a tokenization layer here, a redaction proxy there, a deliberate set of read-only views for the AI to operate on.
Failure semantics.When the system can’t respond, what does it do? In a marketing tool, the answer is “show an error and try again later.” In a clinical tool, the answer depends on the clinical context: sometimes degrade gracefully, sometimes refuse, sometimes escalate to a human. AI-built first drafts collapse all of these into the same retry loop. Picking the right failure mode for each clinical pathway is, again, a software engineer’s job, not a generation problem.
What we’ve seen, in our healthcare presses
We won’t name customers; HIPAA constraints are tighter than the marketing-team constraints we wrote about elsewhere. But the patterns are stable across the dozens of healthcare presses we’ve run.
Most healthcare presses are not, in their first sentence, about healthcare. They are about a deployment problem, an integration problem, an authentication problem. The PHI dimension surfaces in minute four when the engineer asks what data the form will capture, and the customer says oh, names and DOB and the insurance member ID.At that moment the press becomes a different kind of press. The engineer’s job is no longer to ship the form. The engineer’s job is to either move the form into the regulated perimeter, with the right architecture, or to tell the customer the form cannot ship in this shape. We say the second one more often than people expect.
What this means for builders inside health systems
Two things. First: build prototypes. The AI tools that have changed marketing have changed clinical operations too, and a clinical-operations lead with Lovable can ship internal-facing tools faster than the IT department ever could. The boundary, as always, is when the prototype meets a patient. Build to that line; press at that line.
Second: do not let the prototype meet the patient without a senior engineer who has shipped clinical software before. A marketing prototype that ships in a broken state is embarrassing. A clinical prototype that ships in a broken state can hurt a patient and end a career. The press exists, in healthcare, primarily to enforce that line.
We are publishing a longer white paper on the specific infrastructure of HIPAA-compliant Relay sessions, how we train, how we segment the bench, what BAAs cover, how PHI is handled inside the session itself. This essay is the why; the white paper is the how.
Industry essay. Reviewed by our compliance counsel for accuracy on the regulatory points; the opinions are ours.