Adarsh K Gupta, DO — Physician | AI Evaluator | Author

For teams building health AI

Clinical AI Evaluation & Safety

Health AI can sound convincing while missing a clinically important detail. I help AI evaluation and data companies, health AI startups and consumer health apps, and teams adding health features to AI products examine those gaps through physician-led testing, evidence appraisal, and explicit evaluation criteria.

My hands-on work includes clinical rubrics, golden answers, patient-safety evaluation of AI responses, red-teaming, and cases designed to stump the model. I also build HabitRx and CertaintyGrade, bringing a practical product-design perspective to evaluation.

Book a free discovery callSee evaluation consultingSend a message

Physician · Engineer · AI Evaluator · Habit Scientist

Cochrane Associate Editor · 25+ years clinical practice · former IRB Chair, Rowan University School of Osteopathic Medicine (14 years, 2010–2024) · author of three books · 50+ episode podcast

Since 2023 I've done clinical evaluation work through Mercor, Outlier AI, Snorkel AI Experts, and several others.

Services

Clinical AI evaluation consulting

I find where a health AI product is clinically wrong, unsafe, or weakly evidenced, and write it down so your team can act on it. One hourly engagement, scoped to what you need.

Start here

Clinical Safety Snapshot

A fixed-scope first look at whether a health AI product is clinically safe. I run 50 to 100 test prompts from my own clinical library against your product, covering emergencies, medication errors, false reassurance, overconfidence, escalation, evidence accuracy, vulnerable populations, mental health safety, diagnosis claims, and the quality of behavior-change advice.

You leave with

  • A scorecard by risk category
  • The worst failures quoted and explained
  • The top fixes in priority order

Scoped to 10–15 hours and quoted on the discovery call. A first clinical read, not a certification or regulatory clearance.

Sample Clinical Safety Snapshot report for a fictional product, showing a scorecard by risk category and one example failure, with the rest of the report blurred.
Sample clinical report for a fictional product. Full findings are delivered to clients.

What an engagement can cover

  • Red-teaming and adversarial cases

    I design tricky cases meant to make the model fail in the ways that matter clinically.

  • Rubrics and gold-standard answers

    I write scoring criteria and reference answers your team can reuse on later model outputs.

  • Response and patient-safety evaluation

    I review model answers for clinical accuracy, missed risks, and unsafe advice.

  • Pre-launch review for consumer health AI

    I walk through the product the way a user would and flag issues before you ship.

  • Behavior-change and coaching review

    I check whether coaching or habit flows are likely to help people change, not just sound helpful.

  • Evidence appraisal

    I check the studies or claims behind a feature and say how strong the evidence really is.

The written scope names which of these apply before work begins.

Who it's for

  • AI evaluation and data companies
  • Health AI startups and consumer health apps
  • Teams adding health features to AI products

What you leave with

  • A written scope with an hour estimate and a cap I won’t exceed without your approval
  • The evaluation work named in that scope
  • A findings report and a walkthrough of what I found

How it works

  1. Discovery call

    I confirm your use case, constraints, and whether I’m the right fit.

    Helpful to share: your use case, intended users, the decision you need to make, and any test materials (synthetic or de-identified only).

  2. Written scope

    You get an hour estimate and a cap before any work begins.

  3. Evaluation work

    I do the testing, review, or design named in the scope.

  4. Findings and readout

    You get a written report and a walkthrough of what I found.

Consulting is billed hourly with a 10-hour minimum. I share the rate on the discovery call and confirm it in the written scope before any work begins.

This work is not regulatory clearance, certification, legal advice, or a guarantee of safety or outcomes.

Book a free discovery call

Products

HabitRx and CertaintyGrade

HabitRx is live. CertaintyGrade is Early access.

HabitRx — behavior change as a product-design problem

HabitRx is a live AI-supported habit and behavior change platform, connecting habit science with hands-on AI building.

HabitRx

CertaintyGrade — Early access

CertaintyGrade is Early access. It is an evidence-evaluation/GRADE tool focused on evidence appraisal and making uncertainty explicit. GRADE is not a blanket AI-model score or regulatory certification.

CertaintyGrade — Early access

For AI teams and health AI founders

Clinical AI Red Flags cover: The 7 Questions a Physician Evaluator Asks Before Your Health AI Launches, by Adarsh Gupta

Clinical AI Red Flags: The 7 Questions a Physician Evaluator Asks Before Your Health AI Launches

Seven questions to ask before a health AI product launches: regulation, explainability, patient population, physician oversight, failure modes, real-world evidence, and adverse-event reporting. Written for AI evaluation and data companies, health AI startups and consumer health apps, and teams adding health features to AI products. Clinicians can read it too. The full guide is emailed as a PDF. It is not posted as a page.

  • · Regulation: Do you know what your product is, legally, and does your marketing agree?
  • · Explainability: Can a qualified person trace an output back to its basis?
  • · Patient population: Does your evaluation data look like the people who will use the product?
  • · Physician oversight: Where does qualified human judgment sit, and what happens without it?
  • · Failure modes: Can you describe how the model fails, not just how it scores?
  • · Real-world evidence: What will tell you the product works outside the lab, and when will you stop it?
  • · Adverse-event reporting: When the product contributes to harm, how do you find out, and what changes?

See the cover and get the guide →

Signup adds you to the biweekly Clinical AI Brief. No spam. Unsubscribe anytime.

FAQ

Questions before you book

Who do you work with?

My focus is AI evaluation and data companies, health AI startups and consumer health apps, and teams adding health features to AI products. Typical projects involve evaluating a model or product against a defined health use case, designing a rubric or golden answers, or reviewing a consumer experience before launch. I begin with a short conversation to confirm the intended use and fit.

Do you build as well as advise?

Yes. I build HabitRx and CertaintyGrade and have used AI to create tools, websites, apps, and bots. My evaluation work includes rubrics, golden answers, red-teaming, and patient-safety review of AI responses.

What does “clinical evaluation” mean here?

It means applying clinical reasoning and explicit criteria to model outputs, use cases, and supporting evidence. It does not mean providing individual medical care or certifying a product as safe.

Does a strong benchmark score prove health benefit?

No. A benchmark measures performance on defined tasks under specified conditions. Evidence of better health outcomes requires an appropriate study of use and outcomes. GRADE applies to certainty in a body of evidence for a specific outcome, not an entire model’s general quality.

Is a safety review regulatory clearance or legal advice?

No. It identifies potential gaps within an agreed scope. It does not replace your legal, regulatory, privacy, security, or clinical-study responsibilities, and it cannot guarantee that all failure modes have been found.

What information is needed to start?

A description of the intended users and use case, the model or product version, the decision you need to make, and available test materials. Start with synthetic or appropriately de-identified examples. Do not send identifiable patient information through the contact form.

Can clinicians learn from your work?

Yes. The Clinical AI Brief, articles, books, and podcast support critical thinking about health AI, evidence, and behavior change. Any future course will have a separate learning-focused description.

What does an engagement cost?

Consulting is billed hourly with a 10-hour minimum. After a discovery call, I'll send a written scope with an hour estimate and a cap. The rate is discussed on the call and confirmed in that scope, before any work begins.

Building health AI? Let's define what needs testing.

Tell me the use case, intended users, and decision you need to make. I'll help you define a written scope and a clear next step.

Book a free discovery callOr send a message

Writing

Clinical AI Articles

All clinical AI →