Dr. Matthew Zirwas Responds to JAMA Dermatology Study
April 25, 2018
Dr. Matthew Zirwas gives his reaction to the recent study published in JAMA Dermatology by Anderson, et. al. titled “Accuracy of Skin Cancer Diagnosis by Physician Assistants Compared with Dermatologists in a Large Health Care System”:
“The article by Anderson et al that recently published in JAMA Dermatology shocked me. It has not been my experience that PAs are worse than dermatologists at diagnosing skin cancer. However, as I read the title and abstract, it seemed to show exactly that. When I read the actual article, however, it painted a far different picture. Let me explain.
In the results section, they note that the dermatologists had a statistically significantly greater amount of experience in dermatology (13.5 years) compared to the PAs (6.9 years). However, even this is misleading, as they state that they counted experience for the dermatologists as starting after they finished residency, but they counted experience for the PAs as starting on their first day in dermatology. That seems odd – if dermatologists aren’t gaining clinical experience during residency, then what exactly are they doing? Obviously residency should count, meaning the difference in experience between the two groups is actually close to 10 years (6.9 years for the PAs vs 16.5 years for the dermatologists).
Once I saw that difference, and knowing that it has already been proven that dermatologists with 10 years or more experience are better at diagnosing thin melanomas compared to dermatologists with less than 10 years experience, the interpretation of the actual results of the article changed dramatically.(1) In fact, what the title should be is: “Physician Assistants with Less than 7 Years Experience Diagnose NMSC as well as Dermatologists with Over 16 Years Experience”.
That made me start thinking more about the article and the intent behind it. That the dramatic difference in experience was the cause of the difference in diagnostic accuracy is an obvious hypothesis that already has data to back it up.(1) It is also extremely easy to confirm or disprove using the data the authors collected. I can imagine that it just didn’t occur to the authors to run this analysis, but that it didn’t occur to the authors, the reviewers, or the journal editors? Highly unlikely – if it didn’t occur to anybody, then they aren’t very smart. If it occurred to someone but they didn’t do it, they aren’t very ethical.
I’ve written and submitted a letter to the editor of JAMA Dermatology pointing the above out and asking to have the authors run the analysis to control for years of experience. I hope we all get to see the results – maybe we will be surprised and clinical experience won’t be the determining factor of diagnostic accuracy – maybe dermatologists are better or maybe PAs are better. Either way, it is important that we know the truth.”
(1) Morton CA, Mackie RM. Clinical accuracy of the diagnosis of cutaneous malignant melanoma. Br J Dermatol. 1998 Feb;138(2):283-7.
For more information about the study read: JAMA Dermatology Study Fundamentally Flawed: SDPA & AAPA Respond.