Medical Image Perception & Performance

Visual Search Workshop SPIE 1997

Visual Search : Mechanisms & Issues In Radiology 
Elizabeth Krupinski, Ph.D. 
Department of Radiology, University of Arizona 
SPIE Medical Imaging Workshop 26 Feb 1997 Newport Beach, CA 

1) Reader Error Studies

  • Post-WWII mass screening of chest images : 4 roentgenographic & photofluorographic techniques - which is better?

  • Board of Roentgenology - 3 chest rads, 2 general rads, 1 biostatistician

  • 9 -24% interobserver variation

  • 3 - 31% intraobserver variation 

Why study visual search? Studies on mass screening accuracy after WWII showed large inter & intra observer variations. Out of the 4 techniques studied could not show that any one yielded better diagnostic accuracy because of the large variations. Were not previously aware of the magnitude of these errors.

2) CRN Program

  • Repeated the study with the same results

  • Possibly just semantics or terminology

  • Have readers describe findings without prognostication

  • Still get large inter- & intra-observer variation

Because of unexpected results in first study, it was repeated (CRN = initials of the radiologists). Got the same results so repeated trying to use only descriptors rather than diagnostic categories & still got wide variations. Not just a matter of semantics.

3) Figure from Garland (1949).

The scientific evaluation of diagnostic procedures. Radiology 52:309-327. Variations in descriptions of roentgen shadows. Corners refer to apparent characteristics of roentgen shadow. Apex = uniform density, left corner = mottling, right corner = honeycombed shadow. Shows individual tendencies of readers to describe shadows in different terms. Reader R (filled in circles) tends to see lesions as mottled or spotty, reader N as more uniform, and reader C as more honeycombed. 

4) GMZ Studies

  • See the lesion : 16% inter-observer, 8% intra-observer

  • Describe the lesion :

  •   density - 53% 67%

  •   sharpness - 51% 72%

  •   uniformity - 57% 76%

  •   shape - 66% 78%

  •   orientation - 66% 83%

  •  Evaluate the lesion - "attitude" affects evaluation, criterion differences 

Repeated the study again trying to break down diagnostic process into 3 phases: see, describe & evaluate the lesion. In all 3 categories still got wide variations. These results tend to suggest that differences may lie in perceptual & cognitive factors rather than looking for explanations like bad technique, poor processing etc. 

5) Reasons For Errors

  • Objective : 

  •    Poor exposure

  •    Poor processing

  •    Poor viewing arrangement

  •    Fatigue etc.

  • Subjective : "human equation"

  • "To behold is not necessarily to observe" 

From Garland (1959). Again suggesting it is worth looking at perceptual & cognitive factors associated with diagnosing radiographic images.

6) Slide of hidden figure. 

7) More Reasons Tuddenham (1960)

  • Gestalt approach : it's a figure - ground problem

  • Failure to organize the visual field

  • Study of film scanning systems (Buswell - difference between looking at a picture & looking for specific findings)  Harrison (1965)

  • Background of the familiar - normal fades to BG & abnormal stands out

Gestalt psychology provided some interesting possibilities for failures of perception. Most prominent being the concept of figure-ground relations. In radiology the normal appearance of the image generally becomes the background against which the figure abnormality stands out. Subtlety of the lesion, camouflage of lesion by normal features etc. can disrupt figure ground organization, making the figure harder to see.

8) Old/young woman figure. Once you organize the figure-ground one way it can be difficult to switch. The same can happen in radiology.

9) Fox in trees figure. Example of how camouflaging of certain features/figures forces you to organize field one way, making it difficult to see the hidden figures.

10) Illusory triangle figure. Certain arrangements of features can cause you to perceive things that are not there. same with FPs in radiology.

11) More Reasons 
Tuddenham (1962)

  • Satisfy quest for meaning & break off search (SOS)

  • Incomplete coverage because of un-patterned search

  • Suggestions

  •    Try to characterize expert scanning strategies

  •    Look into automatic scanners

One of the first papers to suggest looking specifically at visual search processes. Looked at Gestalt principles etc. plus read a book by Buswell (1935) "How people look at pictures" which examined scanning patterns while looking at general pictures (artworks). If can do the same thing in radiology, maybe some answers can be found to see why errors are made. possibly the first person to suggest the SOS phenomenon.

12) More Suggestions

  • Dual / multiple reading

  • Logical flow charts

  • Checklists 

  • Heuristics for perceptual organization & search based on image characteristics (e.g., symmetry)

  • Directed scanning / search 

These suggestions come from a number of investigators at the time of the error studies. Many of them are still researched today. Dual : Yerushalmy (1950), Hessel et al. (1978), Beam et al. (1996). Flow charts : Tuddenham (1968, 1969), many radiology textbooks. Checklists : Getty et al. (1988). Heuristics : Wackenheim (1984, 1986, 1987). Directed scanning : Tuddenham (1962). Within directed scanning could probably put CAD & other aids that point things out to radiologists.

13) Why Visual Search?
"Search is the first step toward directed discovery" - Thomas 1969

  • Psychology - Neisser, Gibson, Treisman (bottom-up & top-down)

  • Photointerpretation - Enoch

  • Military search problems

  • Eye - position recording - Buswell, Brandt, Yarbus

  • Physiology & psychophysics

Why concentrate on visual search? Could focus only on improving the image - better film, processing, computer displays etc. However, the radiologist is still the final answer - the one that has to read the images and reach a diagnostic decision. If we can find ways to understand the processes involved in perception & decision making, maybe we can improve error rates. 

14) Figure of rod & cone distributions. There are a number of factors that make it necessary for humans to search images. Physiologically we are limited by the fact that high-resolution foveal vision has a limited range. The fovea subtends an angle of only about 2 deg (size of a dime at arm's length). To see fine details the stimulus information must fall on the cones. Rods have very limited fine detail ability.

15) Visual Acuity fall-off figure. Visual acuity is greatest in the fovea region of the retina. By 5 deg from center of fovea acuity is down by about 70%.

16) Visual acuity on a chest image. Kundel (1975). Concentric circles show fall-off in acuity with fovea at center of the chest (14" x 17" image).

17) T-Scope Studies 
Kundel & Nodine (1975); Carmody et al. (1981); Mugglestone et al. (1995,1996)

  • Mainly descriptive statements + some location

  • TP lower than free search, FP same or slightly higher

  • Stimulus complexity affects detection

  • Nodule contrast affects detection

  • Detection falls off with displacement 

Exactly what can be seen is when we restrict visual search? These studies show 1) how important visual search is & why it should be studied, & 2) how much can actually be perceived in a very brief flash of information (200 msec). In a brief flash a number of things can actually be seen in an image. Kundel et al. with chest images, Mugglestone et al. with mammograms. Similar findings in both types of images.

18) Figure from Kundel & Nodine (1975). Shows ROC curves for flash vs free search on chest radiographs. d' about 2.5 for free search vs. about 1.2 for flash presentation.

19) Figure from Mugglestone et al. (1996). Shows ROC curves for flash vs free viewing of mammograms. Az about 0.52 for flash, 0.68 for free search.

20) Figure from Carmody et al. (1980). 360 msec flash presentation of chest image with nodules at various displacements from the center. Progressive drop-off in probability of detection as eccentricity increases. Can find a lot in a flash, but peripheral vision (& corresponding drop in acuity) comes into play very quickly.

21) Peripheral Vision
Kundel, Nodine & Toto (1984,1991)

  • Interactive display controls available information in central & peripheral fields (1.75, 3.5, 5.25, 7.0, 8.75, full)

  • % nodules hit 100% by 5.25

  • % detected - 54% at 1.75 to 71% at full

  • ROC Az 0.37 at 1.75 to 0.75 at full

  • Field of at least 5 deg sufficient to examine lungs for nodules

  • Beyond 5 deg peripheral adds little

What about peripheral vision & detection of radiographic lesions? T-scope studies show can detect some lesions peripherally - how much of peripheral vision is used during visual search? Kundel et al. developed an interactive display that either made nodule visible only when foveally hit or only in periphery using various window sizes. Too small of a window degrades performance & it takes longer to hit the nodule. Beyond a 5 deg window get no more improvement. Seems as if we have a useful visual field of about 5 deg which incorporates foveal & some parafoveal vision & we move this around a display during search. Probably want to include parafoveal because need to differentiate normal from abnormal & is more efficient if both are included in the useful field. 

22) Kundel et al. (1991) figure. Shows drop off in time to hit nodule with different window sizes. By 5 deg no change.

23) Kundel et al. (1991) figure. Schematic of probability of detecting an inconspicuous nodule during a single fixation as a function of distance from axis of gaze. Cylinder shows roughly the useful visual field.

24) More on Complexity

  • Noise is noise? NO!!!

  • Stochastic (random) vs Structured Noise

  • Structured noise affects lesion conspicuity (k) = contrast / complexity where detection is proportional to log k

  • Parameters such as size & edge gradient will affect conspicuity as well & hence detection probability

  • Both random & structured noise lead to decreases in detection rates

In radiology can differentiate between random & structured noise. Both affect detection, but structured noise may more directly affect visual search. Revesz et al. (1974, 1977), Kundel (1975), Kundel & Revesz (1976), Kundel et al. (1984, 1985). A number of studies looking at effect of structured noise on probability of nodule detection. Over the years refined definition of nodule conspicuity as it relates to the background complexity. In general, as level of structured noise is increased, probability of detection decreases.

25) Figure from Kundel & Revesz (1976). Shows probability of detecting a nodule in a chest image vs conspicuity.

26) Figure from Kundel et al. (1979). Shows how nodule properties affect detection probability. Detection is therefore a combination of figure (nodule) properties & background (chest anatomy) properties.

27) Other Factors Affecting Detection

  • Ambient light

  • Viewing distance

  • Lesion size, brightness, shape, contrast

  • Lesion location (camouflage)

  • Disease prevalence

  • Context, history, time

  • Viewer experience, attention, memory

Are many other factors affecting lesion detection & hence visual search. Just a few are mentioned here.

28) 1st Visual Search Experiment
Tuddenham & Calvert (1961)

  • paper roentgenograms

  • adjustable spotlight

  • record patterns & times

  • wide variations in patterns (inter & intra)

  • consistent areas of interest for each reader

  • non-uniformity in coverage

  • tend to return to reported areas

  • "comfortable" zone > macular vision 

Probably the first visual search experiment in radiology. Not a very precise study, but many of the findings have been substantiated in later visual search studies.

29) 1st Eye-Position Study
Thomas & Lansdown (1963)

  • patterns unique to subject not display

  • some areas ignored, some repeated

  • each subject prefers certain areas

  • tendency to fixate edges not masses

  • mean fixation = 0.346 sec

  • mean interfixation distance = 2.402 in

  • if add up all subjects' patterns get full coverage (value 2nd opinions)

Probably the first eye-position recording study in radiology. Noted that there are 2 stages in perception of detail : 1) object must be focused on fovea (hence need for search) & 2) information must be translated by brain to meaning. Experiment done with 5 readers just entering training. 30 sec search time.

30) Why Search Only Certain Areas?
Kundel 1974

  • correlation between verbal estimates of information locations & fixations

  • rad lay

  • normal 0.64 0.79

  • abnormal 0.80 0.77

  • rad : search based on probability that region will be abnormal (lateral part lungs, lesions)

  • lay : lack probability info so search based on picture properties (edges, heart)

Why do readers only fixate certain areas? Kundel divided chest into 64 squares & asked radiologists & lay people to 1) create 8 basic areas & rate them as having highest to lowest information content, 2) recorded eye position to see if fixating correlates with subjective estimates of information content. Correlations were fairly high. Differences between radiologists & lay due to knowledge about nodule probabilities & general experience.

31) Specific Aspects of Search

  • Dwell times & decisions

  • Error classification

  • Search (scanning) patterns

  • Search & circadian variation

A number of studies have followed these early studies, looking into more and more specific aspects of the search process. Eye-position recording techniques have been very useful tools. Some of the more important aspects will be discussed in detail.

32) Dwell & Decisions

  • Nodules Mammo Bone

  • TP 2283 2090 1160

  • FN 1283 1240 969

  • FP 2100 1861 923

  • TN 547 887 450

These are general dwell times associated with the 4 possible decision categories (TP, FN, FP & TN) as derived from a variety of eye-position recording studies (Kundel, Nodine, Krupinski, Hu, Gale, Barrett, Papin, deValk) with 3 types of images. Of interest is to note the similarity of findings across image types. In general, TP & FP decisions are associated with the longest dwell times, suggesting that extensive information processing is associated with reaching these decisions. TN decisions are typically associated with the shortest dwell times suggesting little information processing is needed to reach these decisions. FN decisions have intermediate dwell times, suggesting that at some level some feature extraction/recognition is taking place, but that observers either actively reject these areas (not enough information?) or do not recognize the features as belonging to an abnormality. 

33-35) Survival curve figures for chest, bone & mammography. Survival analysis has proven to be a useful tool for characterizing the distributions of fixation dwells associated with the various decisions.

36) Error Classification
Kundel, Nodine & Carmody (1978)

  • 2.8 deg cluster radius => 90% TPs; UVF

  • TP initial dwell = 0.56 sec

  • 2 SD = 0.48 sec = minimum time for TP

  • use 2.8 deg & 0.48 sec to classify FNs

  • 30% nodules not fixated = scanning error

  • 25% fixated <0.48 sec="recognition" error

  • 45% fixated > 0.48 sec = decision error

  • using total dwell times about 1 sec is cutoff

One of the goals of studying visual search is to try and determine why errors are made. Kundel et al. used eye-position recording and decision dwell times to devise an error classification system. 3 types of errors were classified based on dwell : scanning, recognition & decision. The cut offs for these decisions may off course vary for different images & tasks. 

37) Scanning Patterns

  • Chest (nodules) : circumferential, localized, complex

  • Bone (fracture & trauma) : circular, radial, zigzag, complex

  • Mammo : circumferential, localized, complex; bright areas high density very attractive

It was obvious from the early studies that search patterns differed among observers. Is there a way to characterize scanning patterns? General pattern types have been observed. They depend on the task & image to a very large degree. Kundel & Wright (1969) told readers to either do a general search or nodule search & saw distinct differences - general look at most areas, nodule look at lungs much more so. Image type affects pattern as well - anatomic layout guides search to some extent - in chest see more circumferential, in bone more circular & radial.

38) Figure from Kundel & Wright (1969). Shows 3 basic search patterns on chests. In nodule search get the circumferential pattern mostly & in general search get the complex pattern mostly.

39-41) Figures of general scanning patterns for chest, bone & mammography images.

42) Comparison Scans
Carmody, Kundel & Toto (1984)

  • 78% were taught to search systematically

  • 88% said use a set strategy

  • in reality 83% have complex or circumferential patterns

  • only 17% left-right comparison scans

  • decrease lesion conspicuity => increase frequency of comparisons

  • comparisons resolve ambiguities 

A number of texts/articles do suggest that radiologists adopt specific search strategies. Do they? One common recommendation is to use comparison scans for reading chest images. Carmody et al. (1984) found that radiologists are indeed taught (& do teach others) to use comparison scans, but that in practice they rarely do. Most patterns are circumferential or complex & rarely integrate left-right comparisons. 

43) Figure from Carmody et al. (1981). One thing that does promote comparison scans, however, is nodule visibility. As visibility decreases the probability of making a comparison scan increases. If not sure of what you are seeing need to check out similar areas to try and resolve the ambiguities.

44) Comparisons & Task

  • Mammography find more comparison scans (Krupinski, 1996)

  • 54% between corresponding areas left - right breast or CC - MLO views

  • 46% between areas similar density

  • Beard et al. (1997) - most comparisons are between same images new vs. old; fewer between left - right

Task affects probability of comparisons. In mammography find many more comparison scans because you have more images and need to confirm presence of lesions on different views. Beard et al. looked at comparison scan rates to devise workstation for mammography and determine which pairs of images should be pre-stored for easy access.

45) Circadian Variations
Gale et al. (1984)

  • 8:30 am 1 pm 5 pm

  • ROC : lunch - 20% decrease in TP from am to lunch

  • fixation times decrease generally from morning to evening

  • total viewing time decreases generally from morning to evening

Does time of day affect search? To some degree yes. The period right after lunch differs from morning & evening. 

46) Search & Viewing Time
Christensen et al. (1981)

  • tend to stop search while TP rate still higher than FP rate

  • large # TPs early in search then levels off

  • search has 2 components - fast & slow

  • this components are more prevalent with more experience

  • Oestmann et al., 1988

  • subtlety lesion affects detection time

Up to this point most of the studies discussed have limited search time to some extent. others have looked at the entire search process & tried to characterize viewing time in general. One important finding that corresponds nicely with the T-scope studies is that some nodules can be found very quickly (in a glance) and some take longer to find. This suggests that there are 2 components to search - a fast & slow one. Some lesions need very little search - can be found quickly & very often peripherally. Some require search & take more time. This leads to various models of the search process. 

47) Figure from Christensen et al. (1981). Viewing time differs for staff vs residents. Time at which TPs & FPs made also differ. Staff tend to make more TPs early in search, residents more spread out into later search. Staff & residents make FPs throughout search, but staff tend to stop search before FP rate gets too high compared to TPs.

48) Figure from Christensen et al. (1981). Shows how search has a rapid & slow component.

49) Figure from Oestmann et al. (1988). Search time is affected by lesion subtlety. The more obvious the lesion the more likely it is to be detected at a variety of viewing times (0.254, 1, 4 sec & unlimited).

50) Modelling Search

  • Perceptual & cognitive factors

  • What is learned from eye-position studies

  • Can approach mathematically - e.g., Barrett (1995) modelled search as a Markov process; Kundel et al. (1987) compare human with various models

  • Can model from an ergonomic / design perspective - e.g., Beard et al. (1997) for mammography WS

Much of the search data has led to development of various models. Can approach from a number of ways (cognitive, perceptual, mathematical) & use for a number of purposes (general visual models, task oriented - WS design). 

51) Figure from Kundel et al. (1987). Kundel et al. compared human search to two models - random search & systematic search (covers whole image systematically). Found that human search is much more like a random search process, but is not totally random. So what is it?

52) Figure of Neisser's general search model from psychology. Shows interaction between the stimulus to be searched, exploration processes & a general cognitive schema that helps guide search & is modified by search. Many later models incorporate some general aspects of this model. Swensson used this basic model to suggest a basic 2-stage model of search : perceptual recognition & decision-making.

53) Figure of Blesser's model for the radiologic process. Breaks the process down into psychophysical, psychological & nosological parts. Visual search falls into psychological & nosological parts. Parts are not sequential because can get feedback. This is a very basic model but does represent 3 very important aspects of the radiologic process in general. the psychological phase does bring in some aspects discussed in early papers - Gestalt principles, gaining meaning out of patterns etc.

54) Early model by Nodine & Kundel (1987). Specifically on the search & decision process. Global : peripheral vision has a big role, immediate impression, overall characteristics perceived. Discovery : systematic sampling with useful visual field, if find lesion use Foveal Verification to identify & confirm or can return to it after subsequent search & then decide (Reflective Search). If for some reason time is limited may use Post-Search Recall to identify lesions. After all this reach a decision. Can alternate between global & discovery & verification search. Prior knowledge obviously affects Global part but also impacts in all other areas & other areas add to base of knowledge as more images are searched.

55) Figure from Kundel & Nodine (1983). Shows how 1) prior knowledge & 2) recognition of features leads to increase in fixation time (foveal verification) on image features.

56) Figure from Nodine et al. (1987). A more advanced model of the search process. Still incorporates the same basic features - a global phase of basic characteristic recognition, scanning processes & focal attention to specific details to get a plausible perception interpretation, and a final decision phase. Again is a cycle not a serial process.

57) Figure from Gale et al. (1994). Similar model of search specifically for mammography, incorporating same basic global & focal aspects.

58) Influence of Experience

  • Provides awareness of probabilities (locations, disease prevalence etc.), affecting patterns

  • Global analyses more efficient

  • Find lesions earlier

  • Use multiple views more efficiently

  • Have longer FN dwells

  • Can train to adopt advocated scan patterns & if made automatic will help performance

All of the models have had a component called experience, prior knowledge etc. & implied that it has some effect not only on diagnostic accuracy/performance, but on search as well. Listed are some aspects of the search process where experience or prior knowledge has some effect of search. These findings have been seen for chest, bone, mammography & dental images. Possible to train people to adopt patterns (Gale et al. 1983), but it takes a lot of training & a good deal of feedback so it can become automatic. 

59) Figure from Krupinski (1996). Scanning patterns of experienced vs inexperienced reader on a mammogram. Experienced find lesions earlier & tend to spend less time searching lesion-free areas. Could be that global view is much more efficient at identifying areas that are suspicious vs those that are not.

60) Figure from Nodine et al. (1996). In mammography, experienced readers fixate nodules earlier than less experienced & then fixate them in another view (i.e., CC vs MLO or right vs left) much more quickly than less experienced. Again, prior knowledge & experience guide search patterns - know to confirm or validate initially detected lesion in the other view.

61) History & Prompting

  • Clinically case history often provided - does this affect search & performance?

  • Nature of task / instructions

  • scan patterns change

  • areas searched change

Another aspect of prior knowledge is clinical history or some other type of prompting. Especially now with advances in CAD schemes, the effects of prompting on search & decision-making need to be examined. Kundel & Wright (1969) already showed that prompting for a given task (general vs nodule search) does influence scanning patterns. What else does it affect?

62) Global vs Segmented
Carmody et al. (1980)

  • whole chest vs 6 piecemeal segments

  • segmented eliminates search so should improve performance

  • TP same; FP increase in segmented

This studying looked at prompting in the sense that it limited search to only specific segments of a chest image, so all observer had to do really was decide nodule present/absent. Basically performance deteriorated in segmented search because TPs were the same as with whole viewing but FPs went up. Possibly 1) need whole image available for global perception of general characteristics of the film & 2) perhaps need some amount of comparison scans between segments. Not exactly prompting, but might suggest that limiting search could be detrimental. 

63) Figure from Carmody et al. (1980). Graph showing effect of global vs segmented search for staff vs residents.

64) "Superiority of Search"

  • Swensson et al. - history does not help, lesions are not overlooked without history, readers just have stringent reporting criteria, history promotes a criteria change

  • Cooperstein (1990) - history has no effect for film or PACS images

A debate arose as to whether prompting/history (limiting search) was an effective means of improving perception. Swensson et al. took the position that there was a "superiority of search" - prompting did not improve performance, it merely resulted in a criteria shift that gave more TPs & more FPs. 

65) "Superiority of Search"
Swensson et al. (1977)

  • free vs focused search (anatomic area specified broadly or narrowly)

  • 84 focused regions identified by committee

  • ROC curves identical for the 3 conditions

  • replications with different cases & analyses continued to show same effect or free search better than nonsearch (FPs increase)

  • history bypasses initial stage of perception (preattentive filtering) hampering ability to separate normal & abnormal features

Swensson et al. carried out a number of experiments (various chest lesions) limiting search by giving various types of histories - broad vs specific vs extremely specific. Overall, it was found that free search generally is the same as search with history and was often better (FPs would increase without search). Different cases were used, different analyses etc. but the result was always the same. However, other groups showed that history does help & argued about methodology 1) how the areas were chosen for non-search evaluation & 2) how these areas were given default negative ratings for the search condition, possibly biasing the results. 

66) Figure from Swensson et al. (1982). One of many similar figures showing the "superiority of search" result - free search vs focused search (broad area defined) vs non-search (specific location & description given).

67) History Helps

  • Schreiber (1963) - history increases TP not FPs

  • Parker et al. (1982) - directing to quadrant improves Az (0.81 vs 0.92)

  • Rhea et al. (1979) - freq. & significance errors increases with inadequate history

  • McNeil et al. (1983) - detection improves for CT intracranial disease with history

A lot of other studies were showing that providing history does indeed improve overall performance - TP increases without FP increase.

68) Berbaum et al. Studies

  • 1986 & 1988 studies : chest with diverse lesions (1988 re-analysis with jackknife)

  • no prompt; plausible / correct prompt; plausible / incorrect prompt

  • prompts add 2% to location accuracy & 12.5% to classification accuracy

  • FP at 7% in all conditions

  • due to improved perception

  • diverse chest lesion diagnosis improved more than nodules

Main opponent of the Swensson et al. results. Berbaum et al. studies have generally shown that history does improve performance. Used a variety of chest lesions & found that history helped in all cases but not too much for nodules. TP increased with history while FP did not change. Concluded that history actually results in an improvement in perception, not just a criterion shift or decision-making. Why not nodules? Attentional & perceptual processes may differ for nodules. Nodules may be distorted more by interactions with the background, so typically small, round nodules may not look so small & round when interact with background anatomy (camouflaged). If history primes the reader for specific nodule features & those features are camouflaged, then detection/recognition may be less likely to take place. 

69) Figure from Berbaum et al. (1986). ROC curves for prompted & unprompted lesions - diverse & nodules. Diverse improves performance, nodules not really. 70) Berbaum et al. Studies

  • 1988 - clinical history for fractures

  • same results - history improves performance

  • later did with orthopedists and found same results

  • 1993 - effect history prior to view > after

  • 1994 & 1995 - old radiographs as well as reports improve confidence in decisions

A number of other Berbaum et al. studies have confirmed the positive influence of history, of films etc. on detection of lesions in a variety of film types. 

71-72) Figures from Hutt et al. (1994) & Mugglestone et al. (1996). Demonstrate effects of history/prompts in mammographic images (CAD). Mugglestone results show one possible negative result from prompting in mammography - readers tend to make fewer comparisons between images with prompting.

73) History & Detection Times
Berbaum et al. (1993)

  • native lesions, added nodules (SOS)

  • native lesion or mets history

  • normal (no native) & abnormal cases

  • prompt => detect earlier in search

  • detection unprompted not affected by prompting other abnormalities - history decreases SOS

Berbaum et al. looked at effect of history on time course of detection (also an SOS study, dealt with later). Used native lesions &/or nodules with native or metastatic disease history or no history. Giving a history for a native lesion with & without distracter nodule present improves accuracy. When a distracter is present with a distracter history do get decease in detection of native lesion (SOS). No history in general worse than history. History for both nodule & native increases the speed with which the prompted lesion is found. With appropriate history the prompted lesion is found prior to detection of other lesions. 

74) Figure from Berbaum et al. (1993). Detection vs seconds of search for normal (left) & abnormal images (right). Normal = no native lesion. A & B : no nodules, native history. C & D : nodules, native history = native found prior to nodule. E & F : nodules, mets history = nodules found prior to native & native lower than when no nodules present.

75) Perceptual Feedback
Kundel et al. (1990)

  • FNs have dwells intermediate to TN & TPs

  • 1000 msec threshold feeds back 65% FN & only 20% TN

  • Feedback vs 2nd look control yields 16% improvement (AFROC) - TP increases, FP decreases

Perceptual feedback (using eye-position dwell data to feed back areas of prolonged dwell) can be seen as a form of prompting. In this case perceptual feedback resulted in a 16% improvement (increase in TP, decrease in FP) in performance compared to a second look without feedback. Initial view while eye-position recorded provides areas for feedback. second view seen with or without feedback. 

76) Figure of chest image with feedback circles superimposed.

77) Mechanisms

  • History, prompts, CAD, feedback : all direct attention to specific area, focusing perceptual resources & leaving less up to peripheral vision & chance landing in lesion areas

  • Physical prompts (feedback, CAD) :

  •    block distracting information outside of ROI

  •    increase likelihood fixate lesion

  •    decrease dispersion of fixations - possibly acts as a fiducial marker?

Why do prompts work to improve performance? Probably do affect basic perceptual & attentional mechanisms. Later studies by Krupinski et al. (1993) & Kundel et al. (1995) showed that prompts do affect search parameters such as likelihood of fixating lesions & how many times. Possibly prompts act as some sort of fiducial marker, telling eyes where to fixate in reference to where just looked - keep the eyes & attention in the appropriate place, increasing likelihood of fixating lesion.

78) Satisfaction of Search

  • Once a lesion is found, others tend not to be found

  • Is it premature halting of search?

  • Is it capture of attention & perceptual resources?

  • Is search affected?

Tuddenham noted in 60s that SOS - breaking off search once quest for meaning has been satisfied - could be one source of errors. Have we made any progress in documenting & explaining SOS?

79) Does it Exist?
Berbaum et al. (1990)

  • chest images with & without diverse native lesions

  • presented with & without nodules

  • SOS = miss native if nodule present

  • Az : nodule absent = 0.73

  • nodule present = 0.66 (p = .025)

  • decrease in TPs not FPs

  • found with fractures as well (1994)

  • found with contrast studies of abdomen, but with decrease in FPs as well as TPs

Yes SOS exists. Berbaum et al. have done most of the studies. For fractures & chest images SOS results with a decrease in TPs but not FPs. In contrast studies of the abdomen get decrease in FPs as well. Possibly different mechanisms operate for different images/tasks. In these cases (chest) nodule was more conspicuous than subtle native lesions. 

80) Figure from Berbaum et al. (1990). Effect of nodules present on detection of native lesion (left). Right : FROC curves showing same effect.

81) Time Course of SOS
Berbaum et al. (1991)

  • time to report native not affected by presence nodule

  • time to report nodule not affected by presence native

  • nodules reported earlier than native lesions, FPs last

  • time to FP not different for normal / abnormal or for nodule / no nodule

  • total time not different for normal / abnormal or for nodules / no nodules

  • not premature halt of search

Early experiment to try and determine why SOS occurs - what is its effect on search time? Used an interruption technique to study - observer stops search each time abnormality detected, reports while image gone, then can resume search. Time to native (25.5 vs 23.9 sec with & without nodule) or nodule (18.2 vs 17.a sec with & without native) not affected by presence of the other. Will find nodules earlier in search than natives. FPs found later in search than nodules or natives (around 33 sec). Total search about 45 sec in all cases. Conclusion : is not early termination of search. More like stopping search before FPs get to high (as in Christensen). What is it? We have feature detectors which are not independent of each other & which are mediated by attention. If are activated by one set of features (nodules) this may inhibit or mediate attention for other features (native lesion). It makes it less likely that attentional & perceptual processing resources will be available to allocate to the native lesion, so it remains undetected. Is a perceptual set or bias (goes back to Gestalt from earlier - seeing figure-ground one way inhibits seeing it another way).

82) Figure from Berbaum et al. (1991). Time course of SOS as described above.

83) Figure from Berbaum et al. (1991). Shows the same effect in terms of decision confidence. 1 = definite 4 = possible but probably normal.

84) SOS & Eye-Position 
Samuel et al. (1995)

  • looked at effect of native lesions on nodules & nodules on native lesions

  • recorded eye-position to try and determine if SOS due to premature halt or capture of attention

Eye-position recording may provide a more accurate measure of why SOS occurs. 

85) Figure from Samuel et al. (1995). Native lesions were more obvious than nodules so found a reverse SOS effect here - native affects detection of nodules. If image has no native lesion nodules detected more than if native present (SOS) - left. Native lesions & secondary subtle lesion features not affected by nodules (right).

86) Figure from Samuel et al. (1995). Dwell on FNs (nodules) greater on native-free than native lesions. Suggests native does detract from fixating nodules to some degree but FNs are fixated so not as if lesions were not fixated at all.

87) More Results

  • total search time : single native - in native 100% premature halts = FN; in native - free 75% = FN

  • in native 90% FN fixated & 75% corresponding TN areas fixated

  • scan recog dec

  • native 8% 57% 35%

  • free 15% 37% 48%

Readers had 30 total but could terminate search early if chose to. In general terminated earlier for single native lesions than for multiple natives or those with nodules - SOS not really due to early halt if nodule image search lasted longer than native search. When there was premature halt there were FNs. But FNs were fixated just as often as TN areas. Based on eye-position data errors ( n = 37 in native, 27 in native-free) could be classified. Get fewer scanning errors when native present & more decision errors in native free. Not due to lack of fixating. 

88) Mechanisn

  • Early termination & decreased FN dwell data suggest that SOS caused by native lesions being detected early in search (global) which decreases vigilance for other unexpected & unrelated abnormalities = perceptual capture of attention

  • Image features pertinent to interpretation of more salient abnormality receive priority to satisfy observer's "quest for meaning"

Obvious native lesions detected early in search decreasing attention & perceptual resources later on (capture of attention). Missed lesions are still fixated but not as much as when native-free. More salient features satisfy quest for meaning & may search around more but perceptual set makes feature activation less likely for other types of features contributing to early termination.

89) Can We Disregard Inadequate Scanning?
Berbaum et al. (1996)

  • plain-film (PF) & contrast (C) exams of abdomen

  • not premature halt : time on ROIs same on C & PF studies when PF lesion present

  • on C spend > time on contrast areas than non-contrast areas with & without PF abnormalities

  • if PF abnormality is detected (C or PF image) received prolonged dwell

  • if missed in either C or PF received shorter dwell

Samuel et al. showed that for nodules they are scanned but inadequately and recognition does not take place. Berbaum et al. suggest contrast studies of abdomen may have a different SOS explanation since in these SOS studies saw a decrease in FPs as well as TPs. Recorded eye-position on contrast studies of the abdomen (plain-film then with contrast). SOS = see lesion on plain film then miss in contrast study. Without PF abnormality time on contrast > on plain (97 vs 82). With PF abnormality time about same (100 vs 95). Therefore not premature halting of search. Basically, on contrast film spend much more time on contrast areas & less on the plain film areas than spent looking at plain film areas on the plain films. Lesions in either case that are detected get prolonged dwell, while lesions in either case that are missed get less dwell.

90) Error Types & Mechanism
scan recog dec

  • PF 14% 14% 71%

  • C 47% 23% 30%

  • reduced dwell time on non-contrast areas C images leads to an increase in scanning errors = a kind of visual neglect or a "decision not to scan"

Look at error types by dwell times - in contrast films get a huge increase in scanning errors to the plain film abnormality areas. This can be seen as a decision not to scan/visual neglect. Differs from nodule search where more FNs were actually fixated but not reported. Is a definite case of capture of attention. 

91) Figure from Berbaum et al. 1996. Time course of SOS from eye-position. On right shows that still fixate PF area a little during search - right at beginning & right at end, but contrast area quickly grabs attention. Fixating on PF area later in search may not matter because readers tend to be reporting what's in the contrast area - may just be idling.

92) Workstation Design

  • Study visual search patterns to aid in WS design for mammography (Beard et al., 1991,1997)

  • Observe which image display, zoom in & roam operations are used & how often

  • Observe how many & what type of image comparisons are made

  • Increasing # monitors will decrease time

  • Preloading most used comparison pairs will save time & increase efficiency

We can study visual search in other areas as well. One area has been characterizing search patterns to design workstations. If we know how people scan films than we can devise pre-load patterns and decide how many monitors are really needed. Beard et al. (1991, 1997) conducted such a study. Comparison scans were important for deciding on pre-load patterns.

93) Assess Effects of Modality Differences

  • Why does it take longer to view images on a CRT than on film? (Krupinski & Lund, 1997)

  • Calculate decision dwell times

  • Determine when (if) lesions first detected

  • Examine search patterns & areas fixated

  • Decide which factors contribute most to prolonged search times, try to fix

Can look at if & why scan patterns are different for different ways of displaying the same images (e.g., monitor vs film). Does the presence of a menu affect scanning? Are scan patterns & dwell times different? Do these factors affect diagnostic performance? Will they affect use of the new imaging modalities?

94) CAD & Visual Search
Mugglestone et al. (1996)

  •  3 prompts / film (CAD) : TP, FN & FPs

  • prompts did not increase TP but did increase FP

  • eye goes to prompted areas, increasing time on prompt and decreasing time on unprompted areas

  • fewer left - right comparison scans

Goes back a little to effect of prompts, but is specific to CAD. Does CAD affect the people search images? Yes it does. It may give readers a false sense of security. May be more harmful for less experienced readers who 1) may depend on CAD since their global perceptions are less efficient & 2) may not easily disregard the multiple FP areas prompted.

95) Conclusions

  • Visual search studies have elucidated a number of issues that arose in the early error studies

  • Mechanisms for search processes have been detailed

  • Techniques based on search & search mechanisms have been shown to explain why errors occur and what can be done to reduce error rates

  • Search techniques and studies are evolving as technology evolves

  • Where do we stand? Visual search has taught us a lot & it can be seen that there are many areas still to be investigated. 

BIBLIOGRAPHY

Reader Error & Variabilty
Garland LH (1949). The scientific evaluation of diagnostic procedures. Radiology 52:309-327.

Birkelo CC et al. (1947). Tuberculosis case finding: a comparison of roentgenographic and photofluorographic methods. JAMA 133:359-366.

Garland LH (1959). Studies on the accuracy of diagnostic procedures. AJR 82:25-38.

Cochrane AL & Garland LH (1952). Observer error in interpretation of chest films: international investigation. Lancet 2:505-509.

Garland LH (1950). On reliability of roentgen survey procedures. AJR 64:32-41.

Groth-Petersen E et al. (1952). On reliability of reading of photofluorograms and value of dual reading. Acta Tuberc Scandanavia 26:13-37.

Johnson ML (1955). Observer error: its bearing on teaching. Lancet 2:422-424. 

Newell et al. (1954). Descriptive classification of pulmonary shadows. Revelation of unreliability in roentgenographic diagnosis of TB. Am Rev Tuberc 69:566-584.

Yerushalmy J (1955). Reliability of chest radiography in diagnosis of pulmonary lesions. Am J Surgery 89:231-240.

Zwerling et al. (1951). Clinical importance of lesions undetected in mass radiographic survey of chest. Am rev Tuberc 64:249-255.

Kundel HL (1989). Perception errors in chest radiography. Seminars in Resp Med 10:203-210.

Herman PG et al. (1975). Disagreements in chest roentgen interpretation. Chest 68:278-282.

General Visual Search
Riebel FA (1958). Use of the eyes in x-ray diagnosis. Radiology 70:252-258. 

Brandt HF (1945). The psychology of seeing. NY: Philosophical Library, Inc. 

Buswell G (1935). How people look at pictures. Chicago, IL, Chicago University Press.

Tuddenham WJ & Calvert WP (1961). Visual search patterns in roentgen diagnosis. Radiology 76:255-256.

Tuddenham WJ (1962). Visual search, image organization, & reader error in roentgen diagnosis. Radiology 78:694-704.

Tuddenham WJ (1960). Visual problems of roentgen interpretation. NY State J of Med, April, 1234-1238.

Kundel HL et al. (1991). Searching for lung nodules: the guidance of visual search. Invest Rad 26:777-781.

Carmody DP et al. (1980). An analysis of perceptual & cognitive factors in radiographic interpretation. Perception 9:339-344.

General Eye-Position Recording Studies
Thomas EL & Lansdown EL (1963). Visual search patterns of radiologists in training. Radiology 81:288-291.

Potsaid MS (1965). Cine & TV methods of analyzing search in roentgen diagnosis. J of the SMPTE 74:731-736.

Thomas EL (1968). Movements of the eye. Sci Amer 219:88-95.

Thomas EL (1969). Search behavior. Rad Cl of North Am 7:403-417.

Kundel HL et al. (1978). Visual scanning, pattern recognition & decision-making in pulmonary nodule detection. Invest Rad 13:175-181.

Kundel HL et al (1984). Eye movements & the detection of lung tumors in chest images. In AG Gale & F Johnson (eds) Theoretical & Applied Aspects of Eye Movement research. NY, Elsevier Science, 297-304.

deValk JPJ & Eijkman EGJ (1984). Analysis of eye fixations during the diagnostic interpretation of chest radiographs. Med & Biol Eng & Computing July:353-360.

Nodine CF et al. (1987). Using eye movements to study decision making of radiologists. In G Luer et al (eds). Eye Movement Research: Physiological & Psychological Aspects. NY, CJ Hogrefe, 349-363.

Nodine CF & Kundel HL (1987). Using eye movements to study visual search & to improve tumor detection. Radiographics 7:1241-1250.

Kundel HL et al (1989). Searching for lung nodules: visual dwell indicates locations of FP & FN decisions. Invest Rad 24:472-478.

Barrett JR et al. (1994). Unobtrusively tracking eye gaze direction & pupil diameter of mammographers. Acad Rad 1:40-45.

Hu CH et al. (1994). Searching for bone fractures: a comparison with pulmonary nodule search. Acad Rad 1:25-32.

Krupinski EA (1996). Visual scanning patterns of radiologists searching mammograms. Acad Rad 3:137-144.

Gale AG & Worthington BS (1983). The utility of scanning strategies in radiology. In R Groner et al. (Eds). Eye Movements & Psychological Functions: International Views. Hillsdale, NJ, Lawrence Erlbaum, 169-191.

Papin JP et al. (1984). Use of NAC Eye Mark by radiologists. In AG Gale & F Johnson (eds). Theoretical & Applied Aspects of Eye Movement Research. Amsterdam, Netherlands, Elsevier Science, 323-330.

Kundel HL et al. (1984). Eye movements & the detection of lung tumors in chest images. In AG Gale & F Johnson (eds). Theoretical & Applied Aspects of Eye Movement Research. Amsterdam, Netherlands, Elsevier Science, 297-304.

Kundel HL & Nodine CF (1978). Studies of eye movements & visual search in radiology. In JW Senders et al. (eds). Eye Movements & the Higher Psychological Functions. Hillsdale, NJ, Erlbaum, 317-328.

Kundel Hl (1974). Visual sampling & estimates of the location of information on chest films. Invest Rad 9:87-93.

Krupinski EA & Nodine CF (1994). Gaze duration predicts the locations of missed lesions in mammography. In AG Gale et al. (eds). Digital Mammography. NY, Elsevier Science, 399-403.

Krupinski EA (1995). Searching mammograms - what recording eye position reveals. 6th Far West Image Perception Conference, 13-15 Oct, Philadelphia, PA.

Mugglestone MC (1995). Breast cancers missed at screening: an eye movement perspective. 6th Far West Image Perception Conference, 13-15 Oct, Philadelphia, PA.

van der Stelt-Schouten et al. (1995). Eye movements on dental images. 6th Far West Image Perception Conference, 13-15 Oct, Philadelphia, PA.

Mugglestone MD et al. (1996). Defining the perceptual processes involved with mammographic diagnostic errors. SPIE Med Imaging 2712:71-77.

Gale AG et al. (1994). Breast screening: visual search & observer performance. SPIE Med Imaging 2166:66-75.

Perceptual Feedback
Nodine CF & Kundel HL (1990). A visual dwell algorithm can aid search & recognition of missed lung nodules in chest radiographs. In D Brogdan (ed). First International Conference on Visual Search. London, England, Taylor & Francis, 399-406.

Nodine CF et al. (1990). A perceptually-based algorithm provides effective visual feedback to radiologists searching for lung nodules. Proc of the First Conf on Visualization in Biomedical Computing. Los Alamitos, CA, IEEE Computer Society, 202-207.

Nodine CF & Kundel HL (1994). Computer-assisted perception aids pulmonary nodule detection. SPIE Med Imaging 2166:55-58.

Krupinski EA (1994). Perceptual enhancement of pulmonary nodule recognition in chest radiographs. SPIE Med Imaging 2166:59-65.

Kundel HL et al (1990). Computer-displayed eye position as a visual aid to pulmonary nodule interpretation. Invest Rad 8:890-896.

Krupinski EA et al. (1993). A perceptually based method for enhancing pulmonary nodule recognition. Invest Rad 28:289-294.

Krupinski EA et al. (1993). Perceptual enhancement of tumor targets in chest x-ray images. P&P 53:519-526.

Eye-Position Recording Methods
Nodine CF et al. (1992). Recording & analyzing eye-position data using a microcomputer workstation. Behav Res Meth, Instr & Comp 24:475-485.

Carmody DP et al. (1980). Performance of a computer system for recording eye fixations using limbus reflection Behav Res Meth, Instr & Comp 12:63-66.

Barrett J et al. (1994). Unobtrusively tracking mammographers' eye gaze direction & pupil diameter. SPIE Med Imaging 2166:46-54.

Comparison Scanning Behaviors
Carmody DP et al. (1981). Finding lung nodules with & without comparative scanning. P&P 29:594-598.

Carmody DP (1984). Lung tumor identification: decision-making & comparison scanning.

Gale AG et al. (1984). Circadian variation in radiology. In AG Gale & F Johnson (eds). Theoretical & Applied Aspects of Eye Movement Research. Amsterdam, Netherlands, Elsevier Science, 305-312.

Carmody DP et al. (1984). Comparison scans while reading chest images: taught, but not practiced. Invest Rad 19:462-466.

Viewing Time
Christensen EE et al. (1981). The effect of search time on perception. Radiology 138:361-365.

Oestmann JW et al. (1988). Lung lesions: correlation between viewing time & detection. Radiology 166:451-453.

Dual Reading
Yerushalmy J et al. (1950). Role of dual reading in mass radiography. Am Rev Tuberc 61:443-464. 

Beam CA et al. (1996). Effect of human variability on independent double reading in screening mammography. Acad Rad 3:891-897.

The Role of Experience
Davies IR et al. (1994). Expertise in categorizing mammograms: a perceptual or conceptual skill? SPIE Med Imaging 2166:86-104. Harrison CP (1965). The radiograph as a phenomenon. Canad Med Assoc J 93:110-112.

Krupinski EA (1996). Influence of experience on scanning strategies in mammography. SPIE Med Imaging 2712:95-101.

Nodine CF (1995). Naive & experienced viewers: searching mammograms. 6th Far West Image Perception Conference, 13-15 Oct, Philadelphia, PA.

Nodine CF et al. (1996). The nature of expertise in searching mammograms for breast masses. SPIE Med Imaging 2712:89-94.

Krupinski EA (1996). Influence of experience on scanning strategies in mammography. SPIE Med Imaging 2712:95-101.

Herman PG & Hessel SJ (1975). Accuracy & its relationship to experience in the interpretation of chest radiographs. Invest Rad 10:62-67.

Nodine CF et al. (1996). Nature of expertise in searching mammograms for breast masses. Acad Rad 3:1000-1006. Kundel HL & LaFollotte (1972). Visual search patterns & experience with radiological images. Radiology 103:523-528.

Teaching & Visual Search
Tuddenham WJ (1968). The use of logical flow charts as an aid in teaching roentgen diagnosis. Radiology 102:797-803.

Tuddenham WJ et al. (1969). Preliminary evaluation of effectiveness of logical flow charts in teaching roentgen diagnosis. Radiology 93:17-24.

Squire LF (1969). Perception related to learning radiology in medical school. Rad Cl of North Am 7:485-497.

Wackenheim A (1984). La polarite de l'image radiologique. J de Radiologie 65:781-783.

Wackenheim A (1984). La segregation d'units figurales par la proximite et la resemblance de leurs composants. J de Radiologie 65:437-441.

Wackenheim A (1984). La lecture structuraliste de l'image radiologique. J de Radiologie 65:61-69.

Wackenheim A & Zollner G (1987). Symetrie, asymetrie et dissymetrie. Annales de Radiologie 30:60-64.

Wackenheim A (1986). L'heuristique des systemes experts en radiodiagnostique. J de Radiologie 67:853-861.

Wackenheim A (1986). La pensee laterale, pilote de le pensee verticale. J de Radiolgie 67:55-58.

Sowden P & Davies I (1995). The nature of perceptual skills in screening mammography: implications for training. SPIE Med Imaging 2436:143-155.

Psychophysics
Hebb DO & Favreau O (1969). The mechanism of perception. Rad Cl of North Am 7:393-401.

Kundel HL. (1975). Peripheral vision, structural noise and film reader error. Radiology 114:269-273.

Kundel HL & Revesz G (1977). The influence of film density on the radiologic detection of lung lesions. Invest Rad 12:199-200.

Revesz G et al. (1974). The influence of structured noise on the detection of radiologic abnormalities. Invest Rad 9:479-486.

Kundel HL & Revesz G (1976). Lesion conspicuity, structural noise, & film reader error. AJR 126:1233-1238.

Kundel HL et al. (1979). Contrast gradient & the detection of lung nodules. Invest Rad 14:18-22.

Alter AJ et al. (1982). The influence of ambient & viewbox light upon visual detection of low-contrast targets in a radiograph. Invest Rad 17:402-406.

Revesz G & Kundel HL (1977). Psychophysical studies & detection errors in chest radiology. Radiology 123:559-562.

Kundel HL (1981). Predictive value & threshold detectability of lung tumors. Radiology 139:25-29.

Kundel HL et al. (1985). Nodule detection with & without a chest image. Invest Rad 20:94-99.

General
Tuddenham WJ (1969). Roentgen image perception - a personal survey of the problem. Rad Cl of North Am 7:499-501.

Gale AG et al. (1984). Circadian variation in radiology. In AG Gale & F Johnson (eds). Theoretical & Applied Aspects of Eye Movement Research. Amsterdam, Netherlands, Elsevier Science, 313-321.

Gale AG et al. (1989). Vigilance decrement & radiological reporting. In Megaw (ed). Contemporary Ergonomics. London, England, Taylor & Francis, 461-467. 

Kundel HL (1975). Radiological image perception. Applied Radiology Mar/Apr:27-96.

Kundel HL (1982). Disease prevalence & radiological decision making. Invest Rad 17:107-109.

Brogdon BG et al. (1983). Factors affecting perception of pulmonary nodules. Rad Cl of North Am 21:633-654.

Models of Visual Search & the Diagnostic Process
Nodine CF & Kundel HL (1987). The cognitive side of visual search in radiology. IN JK O'Regan & A Levy-Schoen (eds). Eye Movements : from Physiology to Cognition. NY, Elsevier Science, 573-582.

Swensson RG (1980). A two-stage detection model applied to skilled visual search by radiologists. P&P 27:11-16.

Kundel HL et al. (1987). Searching for lung nodules: a comparison of human performance with random & systematic scanning models. Invest Rad 22:417-422.

Kundel HL & Nodine CF (1983). A visual concept shapes image perception. Radiology 146:363-368.

Pauli R & Hammond SM (1994). Modeling two levels of radiological skill: an example from breast cancer screening. SPIE Med Imaging 2166:76-85.

Barrett JR & Dwyer SJ (1995). Modeling the visual search patterns of mammographers as a Markov process. SPIE Med Imaging 2436:116-123.

Blesser B & Ozonoff D (1972). A model for the radiologic process. Radiology 103:515-521.

Ergonomics & Computer Applications
Beard DV et al. (1991). A radiology workstation for mammography: preliminary observations, eyetracker studies, & design. SPIE Medical Imaging 1446:289-296.

Beard DV et al. (1997). A pilot study of eye movement during mammography interpretation: eyetracker results & workstation design implications. J Dig Imaging, In Press.

Krupinski EA & Lund PJ (1996). Comparison of film vs monitor viewing of CR films using eye-position recording. In RF Kilcoyne et al. (eds). SCAR '96, Carlsbad, CA, Symposia Foundation, 269-274.

Krupinski EA & Nishikawa RM (1997). Comparison of eye-position vs computer identified microcalcifications on mammograms. Med Phys 24:17-23. 

Krupinski EA & Lund PJ (1997). Differences in time to interpretation for evaluation of bone radiographs on monitor & film. Acad Rad, In Press.

Dunn SM et al. (1995). Expert eye movements as a sample space for image processing. 6th Far West Image Perception Conference, 13-15 Oct, Philadelphia, PA.

McGrath SP et al. (1995). Image processing with gaze selected sample space. 6th Far West Image Perception Conference, 13-15 Oct, Philadelphia, PA.

Satisfaction of Search
Samuels S et al. (1995). Mechanism of SOS: eye position recordings in the reading of chest radiographs. Radiology 194:895-902.

Berbaum KS et al. (1995). Causes of SOS effects in abdominal contrast studies. 6th Far West Image Perception Conference, 13-15 Oct, Philadelphia, PA.

Berbaum et al. (1996). Cause of SOS effects in contrast studies of the abdomen. Acad Rad 3:815-826.

Berbaum KS et al. (1991). Time course of SOS. Invest Rad 7:640-648.

Berbaum et al. (1990). SOS in diagnostic radiology. Invest Rad 25:133-140.

Clinical History Effects
Berbaum KS et al. (1993). Influence of clinical history on visual search with single & multiple abnormalities. Invest Rad 28:191-201.

Berbaum KS et al. (1986). Tentative diagnoses facilitate the detection of diverse lesions in chest radiographs. Invest Rad 7:532-539.

Parker TW et al. (1982). Directed vs free search for nodules in chest radiographs. Invest Rad 2:152-155.

Swensson RG et al. (1982). Radiographic interpretations with & without search : visual search aids the recognition of chest pathology. Invest Rad 17:145-153.

Swensson RG & Theodore GH (1990). Search & nonsearch protocols for radiographic consultation. Radiology 177:851-856.

Mugglestone MD et al. (1996). The effect of prompting mammographic abnormalities on the human observer. In K Doi et al. (eds). Digital Mammography '96. NY, Elsevier Science, 87-96.

Hutt IW et al. (1994). Prompting as an aid to diagnosis in mammography. In AG Gale et al. (eds). Digital Mammography. NY, Elsevier Science, 389-398. Berbaum et al. (1993). Clinical history affects perception. 5th Far West Image Perception Conference, 7-10 Oct, Newport, RI.

Swensson RG et al. (1985). The value of searching films without specific preconceptions. Invest Rad 20:100-107.

Berbaum KS et al. (1988). Influence of clinical history upon detection of nodules & other lesions. Invest Rad 23:48-55.

Swensson RG et al. (1977). Omission in radiology: faulty search or stringent reporting criteria? Radiology 123:563-567.

Berbaum KS et al. (1988). Impact of clinical history on fracture detection with radiography. Radiology 168:507-511.

Cooperstein LA et al. (1990). The effect of clinical history on chest radiograph interpretation in a PACS environment. Invest Rad 25:670-674.

Good BC et al. (1990). Does knowledge of the clinical history affect the accuracy of chest radiograph interpretation? AJR 154:709-712.

Berbaum KS et al. (1989). Impact of clinical history on radiographic detection of fractures: a comparison of radiologists and orthopedists. AJR 153:1221-1224.

Egglin TKP & Feinstein AR (1996). Context bias: a problem in diagnostic radiology. JAMA 276:1752-1755.

Aideyan VO et al. (1995). Influence of prior radiologic information on the interpretation of radiographic examinations. Acad Rad 2:205-208.

Kundel HL & Wright DJ (1969). The influence of prior knowledge on visual search strategies during the viewing of chest radiographs. Radiology 93:315-320.

Carmody DP (1985). Free search, restricted search, & the need for context in radiologic image perception. In R Groner et al. (Eds). Eye Movements & Human Information Processing. Amsterdam, Netherlands, Elsevier Science, 341-355.

Carmody DP et al. (1980). Global & segmented search for lung nodules of different edge gradients. Invest Rad 15:224-233.

Kundel HL et al. (1995). Further observations on the k effect. 6th Far West Image Perception Conference, 13-15 Oct, Philadelphia, PA.

T-Scope Studies
Kundel HL & Nodine CF (1975). Interpreting chest radiographs without visual search. Radiology 116:527-532.

Mugglestone MD et al. (1995). Diagnostic performance on briefly presented mammographic images. SPIE Med Imaging 2436:106-115.

Return to Eye-Position Recording page