Week 3 (February 21-27)
Co-speech Acts in Face-to-Face Dialogue
Introduction
This week focuses on the findings of contemporary research showing that face-to-face dialogue is multi-modal. That is, the participants integrate their audible communication (words and intonation) with their visible communication (specifically, gestures and gaze), and they do so on a second-by-second time scale.
While learning this week's material, put aside what you may have learned from textbooks and the media about “body language." That is a popular term, not a scientific term, and it implies (a) that some visible actions function independently of what the person is saying and (b) that an expert can "read" nonverbal actions to know how a person is “really” feeling. The research literature does not support either of these assumptions: Crossing your arms doesn't mean that you're "closed" or hostile; it is more likely to mean that you're cold or just more comfortable that way. Similarly, facial expressions of specific emotions (as illustrated in posed photographs) are relatively rare in face-to-face dialogue, where the face serves many other functions.
Co-speech acts--specifically, hand gestures, facial gestures, and gaze-- are tightly synchronized with speech; they occur rapidly and precisely with words, and they complement the meaning of those words. That is, they form integrated messages with the words. For example,
In this course, we are going to follow contemporary research by staying "on the surface," where the observable dialogue is taking place--just as you practiced in Week 2. We will avoid attributions about what the individuals might be thinking or feeling; neither we as observers nor the other participant in the dialogue can read minds! So throughout this course, we'll remind you to focus outward (on what one person is observably conveying to the other) and not inward (on what you hypothesize that person might be intending, feeling, or thinking).
The analysis this week is one of the most demanding you will do. It is often second-by-second, so when you're finished, you will be a master of ELAN! We hope you will ask for help when you need it and also that the work will be worth it because you are seeing new things.
Learning Objectives
Bavelas, J., Gerwing, J., Healing, S., & Tomori, C. (2016). Microanalysis of face-to-face Dialogue: an inductive approach.
This week: read the sections entitled “Theoretical framework” and “Integrated Audible and Visible Acts,” pp. 133- 136, including Figures 9.3 – 9.5. Also read “Using Video Annotation Rather Than a Transcript,” pp. 138-139, including Figure 9.6. (Note: You downloaded this document for Week 1, so there is no need to download it again.)
Bavelas, J. B., Coates, L., & Johnson, T. (2002). Listener responses as a collaborative process: The role of gaze. Journal of Communication, 52, 566-580.
Demonstration Videos
The five demonstration videos this week ("Co-speech acts 1" etc.) should be considered part of the readings. To understand co-speech acts, it is necessary to see them, not just read about them. Each video explains the hand gestures, facial gestures, addressee responses, and gaze windows in the "Burning Pillow" story. Each also includes a demonstration of how to do the exercises.
Exercise
This exercise has four parts:
1. Download the ELAN eaf and pfsx files entitled "Burning Pillow for Week 3" from the Download, IMA Online Course Dropbox, Week 3. Save your copy into your own Week 3 folder with your initials added. Also copy the video of the "Burning Pillow story.MPG” from Week 1 (or link your new ELAN back to it in the Week 1 sub-folder in the course folder on your computer).
Tier 2 shows the addressee's words.
Tier 3 shows where the speaker used a hand gesture and the exact words that occurred at the same time.
Tier 4 is a paraphrase of what this hand gesture was conveying to the addressee.
Tier 5 shows where the speaker used a facial gesture and the exact words that occurred at the same time.
Tier 6 is a paraphrase of what this facial gesture was conveying to the addressee.
Tier 7 shows all addressee responses to the story, with the visible responses added to the words.
Tier 8 shows whether the speaker was looking at the addressee or not, that is, whether the gaze window was on or off.
All of Tiers 1 and 2 are annotated for you. In tiers 3 - 8, the majority has been annotated, leaving some blank spaces for you to fill in. Watching the demonstration videos and studying these annotations is the best way to learn about these co- speech acts. Then you will follow the same format and do your own analysis in the assigned (blank) sections of each tier.
2. Hand gestures: Watch the demonstration video and learn about each of the annotations of hand gestures on tiers 3 and 4. Focus on how each annotation captures the exact gesture, its relationship to the exact words the speaker is saying at that moment, and what it is portraying for the addressee.
Then watch the speaker from 38.155 to 46.495 (from "But--it was facing down ..." to "...away from the light"). You are to look for any hand gestures the speaker makes in this section. Select the exact time each gesture is made, then annotate your selection on tier 3 with the exact words she is saying. Make the same selection on tier 4, then annotate it with what this hand gesture portrays to the addressee.
3. Facial gestures: Watch the demonstration video which explains each the annotations of facial gestures on tiers 5 and 6. Again, focus on how each annotation captures the exact facial and/or head movement, its relationship to the words the speaker is saying at that moment, and what it is portraying for the addressee.
Then go to three places where there are no annotations on tiers 5 and 6:
33.42 - 36.290 seconds, beginning where she pauses
40.595 - 43.282 seconds, beginning where she says "started burning a hole"
51.74 - 52.86 seconds, as she says " I st--I-I’m still"
Look for any facial gestures the speaker makes in this section. Select the exact time a facial gesture is made, then use this exact time on tier 5 to annotate the words with the facial gesture and use tier 6 to annotate what the facial gesture conveys to the addressee.
4. Addressee responses:
Watch the demonstration video, which shows how to annotate all of the addressee responses, both visible and audible, on tier 7. One section (23.600 to 51.77 seconds) is blank for you to complete in the exercise. Follow the directions in the demonstration video.
5. Gaze windows
Keep in mind what you have read in the assigned article on gaze. Watch the demonstration video, which shows when the speaker is looking at the addressee and when she is not--that is when the gaze window was on or off.
Then in the same blank section (23.600 to 51.77 seconds), watch the speaker's head and face closely to select and annotate where the gaze window was on or off.
Posting
Upload your ELAN for this exercise to Week 3 in the “Uploads, IMA Online Course” Dropbox by midnight Saturday evening, February 25,
Questions
Also post your answers to these questions by midnight Saturday evening, February 25
Go to the Week 3 sub-folder in the Dropbox folder, “Uploads, IMA Online Course.” Download two or three completed ELAN analyses of other course members for this week to the Week 3 sub-folder of the course folder you created on your desktop (download both their .eaf and pfsx files). Look over these ELANs, comparing their analyses to your own. Pick one to discuss for your second post this week and then write about where your ELAN analysis agreed with that class member’s as well as where it disagreed. Comment on what you learned from this comparison. (The second post is due by midnight Monday evening, February 27)
Congratulations on finishing your first intensive microanalysis with ELAN and your immersion into the research literature. We can assure you that this is as hard as it gets.
PDF of Week 3 instructions
This week focuses on the findings of contemporary research showing that face-to-face dialogue is multi-modal. That is, the participants integrate their audible communication (words and intonation) with their visible communication (specifically, gestures and gaze), and they do so on a second-by-second time scale.
While learning this week's material, put aside what you may have learned from textbooks and the media about “body language." That is a popular term, not a scientific term, and it implies (a) that some visible actions function independently of what the person is saying and (b) that an expert can "read" nonverbal actions to know how a person is “really” feeling. The research literature does not support either of these assumptions: Crossing your arms doesn't mean that you're "closed" or hostile; it is more likely to mean that you're cold or just more comfortable that way. Similarly, facial expressions of specific emotions (as illustrated in posed photographs) are relatively rare in face-to-face dialogue, where the face serves many other functions.
Co-speech acts--specifically, hand gestures, facial gestures, and gaze-- are tightly synchronized with speech; they occur rapidly and precisely with words, and they complement the meaning of those words. That is, they form integrated messages with the words. For example,
- Hand gestures are not signs of agitation or lack of fluency; they often convey essential information closely related to the words they are timed with.
- Faces in dialogue rarely reveal inner emotional states. Instead, like hand gestures, facial gestures (e.g., looking skeptical, alarmed, serious, pleased) are timed with speech and portray information that is directly relevant to the immediate dialogue.
- Western cultures use gaze (especially eye contact, which is when they are looking at the other person) to coordinate the roles of speaker and addressee. That is, the addressee looks fairly constantly at the speaker, who does not look steadily at the addressee. When the speaker does look back at the addressee, the addressee is very likely to provide feedback (e.g., a nod, “mhm,” etc.).
In this course, we are going to follow contemporary research by staying "on the surface," where the observable dialogue is taking place--just as you practiced in Week 2. We will avoid attributions about what the individuals might be thinking or feeling; neither we as observers nor the other participant in the dialogue can read minds! So throughout this course, we'll remind you to focus outward (on what one person is observably conveying to the other) and not inward (on what you hypothesize that person might be intending, feeling, or thinking).
The analysis this week is one of the most demanding you will do. It is often second-by-second, so when you're finished, you will be a master of ELAN! We hope you will ask for help when you need it and also that the work will be worth it because you are seeing new things.
Learning Objectives
- To learn about and identify specific visible acts (hand and facial gestures, gaze), which are an integral part of the ongoing dialogue.
- To learn about and analyze how gaze functions in face-to-face dialogue.
Bavelas, J., Gerwing, J., Healing, S., & Tomori, C. (2016). Microanalysis of face-to-face Dialogue: an inductive approach.
This week: read the sections entitled “Theoretical framework” and “Integrated Audible and Visible Acts,” pp. 133- 136, including Figures 9.3 – 9.5. Also read “Using Video Annotation Rather Than a Transcript,” pp. 138-139, including Figure 9.6. (Note: You downloaded this document for Week 1, so there is no need to download it again.)
Bavelas, J. B., Coates, L., & Johnson, T. (2002). Listener responses as a collaborative process: The role of gaze. Journal of Communication, 52, 566-580.
Demonstration Videos
The five demonstration videos this week ("Co-speech acts 1" etc.) should be considered part of the readings. To understand co-speech acts, it is necessary to see them, not just read about them. Each video explains the hand gestures, facial gestures, addressee responses, and gaze windows in the "Burning Pillow" story. Each also includes a demonstration of how to do the exercises.
Exercise
This exercise has four parts:
- Learning about the precise relationship of a hand gesture to words by studying annotated examples in the ELAN. Then locating and annotating some examples yourself.
- Learning about the precise relationship of a facial gesture to words by studying annotated examples in the ELAN. Then locating and annotating some examples yourself.
- Learning to see the wide variety of communicative responses (both audible and visible)
- Learning to see the gaze pattern described in the assigned article. Then locating and annotating several gaze windows.
1. Download the ELAN eaf and pfsx files entitled "Burning Pillow for Week 3" from the Download, IMA Online Course Dropbox, Week 3. Save your copy into your own Week 3 folder with your initials added. Also copy the video of the "Burning Pillow story.MPG” from Week 1 (or link your new ELAN back to it in the Week 1 sub-folder in the course folder on your computer).
- First, watch the whole video again to get familiar with the details of her story
- Then notice that this week's ELAN file has four pairs of tiers:
Tier 2 shows the addressee's words.
Tier 3 shows where the speaker used a hand gesture and the exact words that occurred at the same time.
Tier 4 is a paraphrase of what this hand gesture was conveying to the addressee.
Tier 5 shows where the speaker used a facial gesture and the exact words that occurred at the same time.
Tier 6 is a paraphrase of what this facial gesture was conveying to the addressee.
Tier 7 shows all addressee responses to the story, with the visible responses added to the words.
Tier 8 shows whether the speaker was looking at the addressee or not, that is, whether the gaze window was on or off.
All of Tiers 1 and 2 are annotated for you. In tiers 3 - 8, the majority has been annotated, leaving some blank spaces for you to fill in. Watching the demonstration videos and studying these annotations is the best way to learn about these co- speech acts. Then you will follow the same format and do your own analysis in the assigned (blank) sections of each tier.
2. Hand gestures: Watch the demonstration video and learn about each of the annotations of hand gestures on tiers 3 and 4. Focus on how each annotation captures the exact gesture, its relationship to the exact words the speaker is saying at that moment, and what it is portraying for the addressee.
Then watch the speaker from 38.155 to 46.495 (from "But--it was facing down ..." to "...away from the light"). You are to look for any hand gestures the speaker makes in this section. Select the exact time each gesture is made, then annotate your selection on tier 3 with the exact words she is saying. Make the same selection on tier 4, then annotate it with what this hand gesture portrays to the addressee.
3. Facial gestures: Watch the demonstration video which explains each the annotations of facial gestures on tiers 5 and 6. Again, focus on how each annotation captures the exact facial and/or head movement, its relationship to the words the speaker is saying at that moment, and what it is portraying for the addressee.
Then go to three places where there are no annotations on tiers 5 and 6:
33.42 - 36.290 seconds, beginning where she pauses
40.595 - 43.282 seconds, beginning where she says "started burning a hole"
51.74 - 52.86 seconds, as she says " I st--I-I’m still"
Look for any facial gestures the speaker makes in this section. Select the exact time a facial gesture is made, then use this exact time on tier 5 to annotate the words with the facial gesture and use tier 6 to annotate what the facial gesture conveys to the addressee.
4. Addressee responses:
Watch the demonstration video, which shows how to annotate all of the addressee responses, both visible and audible, on tier 7. One section (23.600 to 51.77 seconds) is blank for you to complete in the exercise. Follow the directions in the demonstration video.
5. Gaze windows
Keep in mind what you have read in the assigned article on gaze. Watch the demonstration video, which shows when the speaker is looking at the addressee and when she is not--that is when the gaze window was on or off.
Then in the same blank section (23.600 to 51.77 seconds), watch the speaker's head and face closely to select and annotate where the gaze window was on or off.
Posting
Upload your ELAN for this exercise to Week 3 in the “Uploads, IMA Online Course” Dropbox by midnight Saturday evening, February 25,
Questions
Also post your answers to these questions by midnight Saturday evening, February 25
- Were the speaker’s hand gestures coordinated with her exact words? Did they add any relevant additional information? Give an example.
- Were the speaker’s facial gestures coordinated with what she was saying (or not saying) at the same moment? Did they add any relevant additional information? Give an example.
- Suppose you only heard this story on an audio recording or read it on a transcript. Would the addressee have seemed very responsive? Explain your answer.
- How well did the addressee's responses follow the pattern described in the assigned article? That is, were they related to the gaze windows? Explain and give one example.
Go to the Week 3 sub-folder in the Dropbox folder, “Uploads, IMA Online Course.” Download two or three completed ELAN analyses of other course members for this week to the Week 3 sub-folder of the course folder you created on your desktop (download both their .eaf and pfsx files). Look over these ELANs, comparing their analyses to your own. Pick one to discuss for your second post this week and then write about where your ELAN analysis agreed with that class member’s as well as where it disagreed. Comment on what you learned from this comparison. (The second post is due by midnight Monday evening, February 27)
Congratulations on finishing your first intensive microanalysis with ELAN and your immersion into the research literature. We can assure you that this is as hard as it gets.
PDF of Week 3 instructions
© International Microanalysis Associates
