Observer Performance Studies

Experience-Related Differences in Diagnosis from Medical Images Displayed on Monitors

Previous studies have noted anecdotally that clinicians with more years of experience tended to perform worse using monitor-displayed images.  The goal of this study was to directly compare diagnostic accuracy using conventional versus monitor-displayed medical images to determine if years of experience as a board certified clinician influences diagnostic performance in either modality.  Two studies were performed, one in radiology and one in pathology [9].

Methods

Images

  • Radiology Study 
    -100 computed radiography chest images 
    -50 with subtle pneumothorax 
    -50 lesion-free 
    -Each image printed to film & saved as soft-copy on optical disk 
  • Pathology Study
    -115 permanent frozen breast biopsy sections 
    -68 malignant specimens 
    -47 benign specimens 
    -Each specimen saved on a microscope slide for viewing directly on the   microscope & via video microscopy

-Display

  • Radiology
    -Viewed film images on a standard viewbox
    -Viewed soft-copy on a 1024 x 1538 monitor
  • Pathology
    -Viewed specimens once via regular microscopy
    -Viewed once via video microscopy off a 950 line color monitor

Procedure

-6 staff radiologists read all chest the images in a randomized order -Years of experience were : 1,5,18,21,28,30 6 staff pathologists read all  the frozen specimen images in a randomized order. Years of  experience were : 2,5,7,11,13,14.

Task

-Report pneumothorax present/absent for radiology
-Report malignant or benign for pathology
-Report confidence in decision using 6-level scale
-Answer questions regarding experience & use of computers and video systems in general

Results

ROC Az Results - Radiology 
Az differences tested using a univariate z-score test Film : correlation between experience & performance r = +0.37 Monitor : correlation between experience & performance r = -0.94

 
Experience Film Monitor p
30 .878 .785 .28
28 .859 .839 .76
21 .737 .847 .22
18 .841 .877 .61
5 .884 .921 .49
1 .732 .918 .03

ROC Az Results - Pathology 
Az differences tested using a univariate z-score test Light Microscopy : correlation between experience & performance r = +0.87 Video Microscopy : correlation between experience & performance r = -0.91

 
Experience Light Video p
14 .996 .981 .10
13 .998 .990 .38
11 .998 .997 .68
7 .979 .980 .95
5 .989 .996 .49
2 .977 .996 .16

Survey Question Results 
Performance using the monitor displays correlated highly with general experience using monitor displays such as computers and digital/video systems in general.

Conclusions

  • There is an experience-related difference in reading medical images from monitors
  • Those with more clinical experience tend to have higher performance reading from the traditional modality (film or light microscopy)
  • Those with less clinical experience tend to have higher performance reading from video monitors
  • More exposure to and use of monitor displays, such as recreational video games and computers, may influence positively the clinician-monitor display system interaction and improve diagnostic performance

Return to Observer Performance Studies page