How to evaluate dermatology AI
Dermatology AI includes tools that assess a suspicious lesion from a photo, dermoscopy image or device reading, consumer apps that patients use on their own, and software that sorts teledermatology referrals. A scribe used in a dermatology clinic is a documentation tool, not a lesion classifier.
Pick what the tool does below. Each section is a checklist: what physicians and administrators look for, the evidence to ask for, how to test it on your own patients, red flags, and questions to put to the vendor.
Last reviewed November 2026 · Independent and vendor-neutral
What does the tool you’re evaluating do?
Jump to the matching checklist. Some answers lead to a different guide.
Lesion assessment for clinicians
Applies if a clinician uses the tool to assess a skin lesion from a photo, dermoscopy image or device measurement, for example to support a skin-cancer biopsy or referral decision.
Physician looks for
- Who the intended user is: dermatologists, primary care physicians, or both. Check that the FDA authorization (if any) matches how you plan to use it.
- Sensitivity and specificity reported separately for each skin type (Fitzpatrick I to VI), not one overall number.
- Whether a result would change your biopsy or referral decision, or only confirm what you already planned.
- Image-capture quality control: lighting, focus, distance, hair and glare, and whether the tool rejects a poor image or gives a result anyway.
- For tools that add AI-written explanations: how wrong or overconfident text is caught.
Administrator looks for
- Extra referrals and biopsies. A tool tuned to miss few cancers flags many benign lesions. Estimate the monthly volume and whether dermatology and pathology can absorb it.
- Effect on dermatology wait times, including for patients the tool did not flag.
- Device or subscription cost per lesion or per clinician, including disposables.
- Whether any payer reimburses the AI assessment itself. If not, budget it as an operating cost.
- Training time for primary care staff to capture usable images.
IT looks for
- How images are captured (clinic devices, personal phones or a dedicated device) and whether they land in your EHR or image archive.
- Device management for any phones or tablets used to take photos.
Compliance looks for
- The intended users and lesion types in the authorization, and how use beyond them is prevented.
- Patient consent for photographs, and whether the vendor keeps images or uses them for training.
Evidence to ask for
- A study in the intended user group, for example primary care physicians, not only dermatologists.
- Results by skin type with enough darker-skin cases to mean something. Ask how many.
- Outcomes confirmed by biopsy, not only agreement with a dermatologist’s opinion.
- Positive predictive value at the skin-cancer prevalence you expect in your clinics.
Test it locally
- Run it on 100 to 200 of your own previously biopsied lesions, including darker skin types, and compare results with pathology.
- Count how many of your benign lesions it would have sent for biopsy or referral.
Red flags
- No performance data by skin type, or very few darker-skin cases.
- Results shown without an image-quality check.
- Intended use broader than the authorization, for example use on every lesion when it is authorized only for lesions already judged suspicious.
- Only accuracy or AUC reported, without sensitivity and specificity.
Questions to ask the vendor
- “What are sensitivity and specificity for each Fitzpatrick skin type, and how many cases of each did you test?”
- “Per 100 lesions assessed in a setting like ours, how many extra referrals or biopsies should we expect?”
- “What happens when the image is poor quality?”
- “Is the tool authorized for our intended users and patient ages?”
Tools patients use directly
Applies if patients photograph and check their own skin with the tool, without a clinician reviewing each result first.
Physician looks for
- Whether it is FDA-authorized for this use, or sold as a wellness or information app.
- What a low-risk result tells the patient. False reassurance can delay care for a melanoma.
- Whether every result tells patients to see a clinician if they are worried, whatever the app says.
- How high-risk results reach your clinicians, and how quickly.
Administrator looks for
- New visit and message volume from app-triggered concerns, and who staffs the responses.
- What photos and data the vendor keeps, and whether it can use them to train its products.
- Liability if a patient relies on a low-risk result.
IT looks for
- Whether app data or high-risk results reach your systems at all, and how.
- The app’s security and privacy terms, and where photos are stored.
Compliance looks for
- Whether the app is a regulated device or a wellness product, and that how you promote it matches.
- Data collected outside HIPAA may fall under state consumer health-data laws or the FTC Health Breach Notification Rule.
Evidence to ask for
- A study using photos patients took themselves, not clinic dermoscopy images.
- How often confirmed melanomas received a low-risk result.
Red flags
- Consumer marketing with diagnostic claims and no clearance.
- No path from a high-risk result to a clinician.
- Patient photos reused for training by default.
Questions to ask the vendor
- “In your studies, how often did a confirmed melanoma get a low-risk result?”
- “Who sees a high-risk result, and within what time?”
- “Do you keep patient photos, for how long, and for what?”
Teledermatology triage and routing
Applies if the tool sorts or prioritizes referrals and store-and-forward images rather than giving a diagnosis.
Physician looks for
- How it decides what is urgent, and whether a dermatologist can override it.
- How often it under-prioritizes a case later found to be cancer.
Administrator looks for
- Fit with your referral system and EHR, or a separate portal staff must check.
- Teledermatology throughput: consults reviewed per dermatologist hour.
- Wait times for routine referrals after go-live.
- Cost per consult or per seat.
IT looks for
- Integration with your referral system and EHR (HL7 or FHIR), and where images are stored.
- Single sign-on for dermatologists reviewing consults.
Compliance looks for
- A log of the priority given to each referral and any override.
- Scheduled reviews of under-triaged cancers, with a named owner.
Test it locally
- Re-run it on 3 to 6 months of past referrals and compare its priority with the final diagnosis.
- Track time from referral to first dermatology review, by priority level, before and after go-live.
Red flags
- No under-triage rate for cancers
- Urgency rules you cannot see or adjust
Questions to ask the vendor
- “What is your under-triage rate for cancers?”
- “Can we view and adjust the urgency rules ourselves?”
What published studies show
Read the authorization closely. One FDA De Novo device for skin-cancer evaluation (January 2024) is authorized for non-dermatologist physicians, as an adjunct on lesions already judged suspicious, in patients 40 and older. Its pivotal study reported sensitivity of 95.5% and specificity of 20.7%, so many benign lesions also test positive. Buyers should plan for the referral volume that follows.
FDA De Novo decision summary DEN230008Further reading
Browse dermatology tools
Every listing shows regulatory status, published evidence and named health-system users where public.