JAMA Dermatology Study Fundamentally Flawed: SDPA & AAPA Respond

April 18, 2018

The American Academy of PAs (AAPA) and the Society of Dermatology PAs (SDPA) today criticized an article published in JAMA Dermatology as fundamentally flawed with too small a sample size to be representative of the PA profession.

The article, entitled “Accuracy of Skin Cancer Diagnosis by Physician Assistants Compared with Dermatologists in a Large Health Care System,” drew overly broad conclusions by looking at retrospective data from a mere 15 PAs and 15 dermatologists working in dermatology offices affiliated with the University of Pittsburgh School of Medicine.

“The PA profession is disappointed that JAMA Dermatology would publish so thin a study that is not representative of the quality of care provided by PAs,” said L. Gail Curtis, PA-C, MPAS, DFAAPA, president and Chair of the Board of the American Academy of PAs. “The article is rife with methodology flaws, looks at far too few clinicians to draw any serious conclusions, and cherry picks clinical decisions in ways clearly designed to put PAs in a bad light.”

The article claims that, compared with dermatologists, PAs performed more biopsies per case of skin cancer diagnosed, and that they diagnosed fewer melanomas in situ. Based on these findings, the study suggests that the “diagnostic accuracy of PAs may be lower than that of dermatologists.” However, the study also found that there was no difference in the number of invasive melanoma and nonmelanoma skin cancers diagnosed by PAs compared to dermatologists.

Using the number needed to biopsy (NNB) to measure diagnostic accuracy is controversial even among dermatologists, as indicated in this 2016 letter to the editor of JAMA Dermatology from Michael A. Marchetti, MD, Stephen W. Dusza, Dr PH, and Allan C. Halpern, MD, who wrote: “…considering NNB without attention to the sensitivity applied to the detection of skin cancer and the prevalence of skin cancer could be misleading.” More specifically, they write, “2 clinicians with identical diagnostic accuracy but with different thresholds for sensitivity for their examinations will have discordant NNBs… Similarly, clinicians with identical diagnostic accuracy and sensitivity thresholds examining patient populations that differ in disease prevalence will have discordant NNBs.”

Other limitations of the research methodology and interpretation of results include the following:

Small Clinician Sample Size. The study only reviewed diagnoses of 15 PAs and 15 dermatologists, limiting the generalizability of the study. If even one of those clinicians has a tendency to over or under biopsy, the statistical outcome would be unjustly magnified in a particular direction. This would not happen with a large sample size of clinicians, and it is not compensated for by the large number of patient records reviewed.

Years of Experience. The PA group had an average of 6.6 years less clinical work experience than the dermatologist group and this disparity was not statistically controlled for in the analysis. While it would be expected that practitioners of any title with more experience would have an advantage over their less experienced counterparts, it’s worth noting that the less experienced PAs found rates of non-melanoma skin cancers and invasive melanomas comparable to that of the dermatologists.

Patient Preference. While the article states that the 15 PAs saw a higher percentage of people with a history of skin cancer, the dermatologists saw more patients with a history of melanoma. We would expect a higher rate of melanoma diagnosis in this patient population. Also, these patients with a history of melanoma could be choosing to see the dermatologist over the PA. In the same vein, people who seek removal of a non-cancerous skin lesion for cosmetic or convenience reasons (e.g. a mole irritated by clothing) may choose to see the PA (for ease of getting an appointment), rather than the dermatologist; and the universal standard of care is to send the sample for biopsy, even if there is no concern that skin cancer exists.

Understanding of the PA-Physician Relationship. PA autonomy is determined by the PA’s education and experience, state law, and policies of employers and facilities. In Pennsylvania, a PA must have a supervisory agreement with a specific physician, which is approved by the medical board, in order to practice. In fact, the physician is liable for the care that the PA provides. Therefore, in the data, there is no way to fully separate the decision making of the PA from the decision making of the physician.

“PAs value sustained partnerships with physicians, have great respect for the depth of physician training, and rely on the PA/physician team in clinical practice,” said SDPA President Jane Mast, PA-C, MPAS. “There is a wealth of research that highlights improved patient outcomes from the high-quality care that PAs provide, and the SDPA welcomes further research into PAs in dermatology. But this flawed study, which would have benefited from PA involvement, does the practice of dermatology—and dermatology patients—a disservice.”

For more information, please contact Carrie Munk, vice president of communications, cmunk@aapa.org.