Discourse, vision, and cognition /

Table Of Contents:
Preface ix

Segmentation of spoken discourse 1(18)

Characteristics of spoken discourse 1(3)

Transcribing spoken discourse 3(1)

Segmentation of spoken discourse and `cognitive rhythm' 4(6)

Verbal focus and verbal superfocus 6(4)

Segmentation rules 10(2)

Perception of discourse boundaries 12(4)

Focus and superfocus 13(2)

Other discourse units 15(1)

Conclusion 16(3)

Structure and content of spoken picture descriptions 19(20)

Taxonomy of foci 19(9)

Presentational function 21(1)

Substantive Foci 21(1)

Substantive Foci with categorisation difficulties 21(1)

Substantive List of items 22(1)

Summarising Foci 22(1)

Localising Foci 23(1)

Orientational function 24(1)

Evaluative Foci 24(1)

Expert Foci 24(1)

Organisational function 25(1)

Interactive Foci 25(1)

Introspective Foci & Metacomments 26(2)

How general are these foci in other settings? 28(9)

Presentational function 30(1)

Substantive Foci 30(1)

Substantive Foci with categorisation difficulties 31(1)

Substantive List of items 31(1)

Summarising Foci 31(1)

Localising Foci 32(1)

Orientational function 33(1)

Evaluative Foci 33(1)

Expert Foci 33(1)

Organisational function 34(1)

Interactive Foci 34(1)

Introspective Foci & Metacomments 34(1)

Length of description 35(2)

Conclusion 37(2)

Description coherence & connection between foci 39(16)

Picture description: Transitions between foci 39(7)

Means of bridging the foci 42(1)

Discourse markers 43(2)

Loudness and voice quality 45(1)

Localising expressions 46(1)

Illustrations of description coherence: Spontaneous description & drawing 46(7)

Transitions between foci 50(1)

Semantic, rhetorical and sequential aspects of discourse 51(1)

Means of bridging the foci 52(1)

Conclusion 53(2)

Variations in picture description 55(24)

Two different styles 55(11)

The static description style 57(2)

The dynamic description style 59(5)

Cognitive, experiential and contextual factors 64(2)

Verbal or visual thinkers? 66(3)

Effects of narrative and spatial priming on the two styles 69(7)

The effect of (indirect) spatial priming 70(3)

The effect of narrative priming 73(3)

Conclusion 76(3)

Multimodal sequential method and analytic tool 79(20)

Picture viewing and picture description: Two windows to the mind 80(5)

Synchronising verbal and visual data 81(2)

Focusing attention: The spotlight metaphor 83(1)

How it all began . . . 84(1)

Simultaneous description with eye tracking 85(9)

Characteristics of the picture: Complex depicted scene 86(1)

Characteristics of the spoken picture description: Discourse level 87(1)

Multimodal score sheets 88(1)

Picture viewing 89(2)

Picture description 91(3)

Multimodal sequential method 94(4)

Conclusion 98(1)

Temporal correspondence between verbal and visual data 99(26)

Multimodal configurations and units of verbal and visual data 100(15)

Configurations within a focus 101(1)

Perfect temporal and semantic match 101(1)

Delay between the visual and the verbal part 101(1)

Triangle configuration 102(2)

N-to-1 mappings 104(1)

N-to-1 mappings, during pauses 104(1)

N-to-1 mappings, rhythmic re-examination pattern 105(1)

Configurations within a superfocus 106(1)

Series of perfect matches 106(1)

Series of delays 107(2)

Series of triangles 109(1)

Series of N-to-1 mappings 109(4)

Series of N-to-N mappings 113(1)

Frequency distribution of configurations 114(1)

Functional distribution of multimodal patterns 115(5)

Discussion: Comparison with psycholinguistic studies 120(1)

Conclusion 121(4)

Semantic correspondence between verbal and visual data 125(26)

Semantic correspondence 126(4)

Object-location relation 128(1)

Object-path relation 128(1)

Object-attribute relation 129(1)

Object-activity relation 129(1)

Levels of specificity and categorisation 130(1)

Spatial, semantic and mental groupings 131(6)

Grouping concrete objects on the basis of spatial proximity 131(1)

Grouping multiple concrete objects on the basis of categorical proximity 132(1)

Grouping multiple concrete objects on the basis of the composition 133(1)

Mental zooming out, recategorising the scene 133(1)

Mental grouping of concrete objects on the basis of similar traits and activities 134(1)

Mental grouping of concrete objects on the basis of an abstract scenario 135(2)

Discussion 137(11)

The priming study 138(4)

Dimensions of picture viewing 142(1)

Functions of eye movements 143(1)

The role of eye fixation patterns 143(3)

Language production and language planning 146(2)

Conclusion 148(3)

Picture viewing, picture description and mental imagery 151(20)

Visualisation in discourse production and discourse comprehension 152(5)

Mental imagery and descriptive discourse 157(10)

Listening and retelling 159(2)

Measuring spatial and temporal correspondence 161(1)

Eye voice latencies 161(3)

Picture viewing and picture description 164(3)

Discussion: Users' interaction with multiple external representations 167(1)

Conclusion 168(3)

Concluding chapter 171(8)
References 179(14)
Author index 193(4)
Subject index 197