Week 13 - Glosser & Wiener, Mayberry & Jacques, McNeill (optional)

This article is directed at explaning the 4 proposals for the interaction between gesture and speech.

1. Gesture and Speech are unrelated communication channels

2. Gesture and Speech are parts of the same psychological structure

3. Gesture and Speech both rely upon common motor sytems for production, so they are related

4. Gesture and Speech arise independently, Specifically gesture occurs when speech is disrupted.

Each of these proposals provide evidence from aphasic speakers. For hypotheses above, the authors say:

1. There appears to be unanimous agreement that gestural communication is never completely normal in aphasics. This explanation, therefore, does not seem to warrant further consideration.

-- I agree with the authors. There seems to be too much evidence to the contrary that speech and gesture are somehow intertwined.

2. "gesture and speech exhibit the same configurational properties" pp. 262. "These (beats) appear to serve structural functions in the discourse" pp. 262. "The results of these descriptive studies are consistent with the view that gestures and speech are functionally related in communication." pp. 263. "Semantic-modifying and relational gestures were found to be the most vulnerable to disruption within the aphasic groups" pp. 263. All of these factors "provide strong evidence for the claim that speech and gesture arise as interacting elements of the same system"

--This hypothesis is the most strongly supported by the psycholinguistic literature. I am inclined to agree.

3. It is incedental that gesture and speech interact.

"It would appear that the data from aphasia are not consistent with a claim that shared motor control systems form the sole basis for the relationship between gesture and speech." pp. 268

4. "The hypothesis that speech related movements are primarily a function of effortful, disrupted speech encoding is not supported by the available , but limited data, from aphaisa." pp. 271.

--This hypothesis seems the most implausible to me.

The author then brings up 3 issues that must be considered when evaluating evidence from aphasia. I feel that these issues are germane to this discussion and warrant consideration.

1. There is not yet a general consensus with respect to the nature of the underlying general/linguistic impairment in different aphasic groups.

2. Aphasic patients cannot be dealt with as a homogeneous group.

3. The fact that aphasics are impaired in numerous non-linguistic domains may contribute to alterations in their gestural communication.

I leave this article with a very clear message: There are multiple facets of the communication exchange matrix, so too, the realtionships of speech and gesture must be evaluated within a broader perspective of communication as social action. Thus, Aphasic data is highly limited and we should look at other indicators for the relation between speech and gesture. This is exactly what Mayberry did. I will now turn to her article.

MAYBERRY & JAQUES

I found this article brilliant. The authors write in a very clear and convincing style and address a new area in the research on speech/gesture relations.

Again, their article provides overwhelming evidence for the "Integrated systems Hypothesis". Not only did they provide support for this hypothesis, they also ruled out a motor-shutdown for this phenomenon.

The only criticizm I have of this article is that the authors lost me in the Neuromotor Coordination of Gesture and Speech section on pp.15. This is probably due to my own ignorance of her term "ocillatory processes", however.

Other than this, I don't have much to say. Again, I thought that the evidence provided by analysing stuttering is quite convincing.

The introductory section of this article is good summary of and segue from the discussion this past week on the semiotics of gesture. Many of the same researchers and theories were presented and reviewed.

Of the four proposed explanations of the relationship between verbal and gestural communication reviewed, the first was obviously discounted by the authors. Of the other three, I am still not clear on which the authors espouse. Is it a combination of the remaining explanations?

On page 260, under the first explanations, the authors write that "Gestures,...., are thought to be driven by primary processes and convey unintended, connotative, emotional information." Though this explanation was discounted, I am interested in this notion of "unintended ....information." This came up in last class, as well, when the following question came up: Can a gesture have meaning if it is not intentional or motivated? What does it mean to produce an unintentional or unmotivated gesture?

Mayberry, R. and Jaques, J. Gesture production during stuttering speech: Insights into the nature of gesture-speech integration.

This was a fascinating study!

On page 7, the authors list three categories of hand movement. They say that the first was not analyzed further and on page 12, they write parenthetically that deictic and beat gestures were excluded from their analysis. What might their findings have shown if they had counted the second category of non-representational beats? It has been a long time since I studied stuttering, so I have a question: Do stutterers learn strategies that might interfere with gesture (like using beats or rhythm or something like that)?

This paper is clearly a confirmation of the integrated-system hypothesis concerning speech and gesture. I am wondering, however, if any work has been done to look at the problem from the reverse perspective, namely manual stuttering. It would be interesting to test someone whose handicap extends to the use of their forearms and hands rather than to their speech. Then we could confirm the opposite side of the same hypothesis, expecting the speech to slow down, repeat itself, or even stop while maintaining synchrony with the manual gesturing. I realize, of course, that the argument can be easily made that speech does not depend on gesture but that gesture does depend on speech; therefore, if someone were not able to gesture correctly to be able to synchronize it with their speech, they may simply choose to stop gesturing, and thus no expirment could be made. Anyhow, I think it is worth considering at least on a theoretical level.

I found one of the concluding statements in the Glosser and Weiner article to be a breath of fresh air: ""a concern may be raised that the whole endeavor of looking for an explanation of a relationship between speech and gesture is so narrowly drawn that we fail to explore other components that co-occur in a communication matrix (Hymes, 1974) and which may bear on the apparent relationships of speech and gesture" (pg. 274). This statement may seem on the face of it to be a cop-out since Glosser and Weiner really do not commit themselves to any view; yet looked at from a broader perspective I think it is correct to say that this whole business of deciding what the relationship of speech is to gesture can not be done with such a pared down approach. For example, semiotically speaking, the link between the object of the speech and the object of the gesture must be established before we can look at timing and other factors; otherwise, one might do the same for gestural praxis and speech, which of course makes no sense since they are not semiotically related.

In the McNeill article, the last experiment between English and Japanese provides confusing evidence that does not support McNeill's general claim that gesture is not merely illustrated speech, but rather both gesture and speech are two types of expression of the same underlying image-schema. If this is the case, then why would Japanese gesture differ from English gesture when they are describing the same visual experience, such as a cartoon? If gesture is not speech illustration and if it expresses the same information as speech but in a very different (holistic) way, then we need to ask why Japanese gesture is so different from English gesture. Surely synchrony does not explain it.

These are interesting studies. I liked what they said about interacting elements of a single system on page 264. Makes me think of the gestures of signers, which I think about a lot!

I was particularly interested in #3 discussion on motor control program. This could be part of the explanation of what is happening with the gesture of signers. If gesture and speech interact at multiple loci, the expression of a spontaneous gesure or a sign might have a correlation with motoric schemas.

Mayberry & Jaques

I found similar things happening in some of the gestures of the participants in my research. In particular, one participant while narrating a story raises her hands and starts the gesture stroke (2-handed gesture). She changes her mind about how to describe the scene in mid-speech. Her hands immediately stop (freeze in mid-air) and stay in that position until she starts talking again at which time she produces a different gesture. Also, the preparation for the gestures and signs of all my participants preceded the related speech.

Itās fun to read studies where the results match mine. SMILE!!!!

Recently, in our Evolution of Communication class, we wre discussing a theory of Hockett's that speech and sign are part of the same system (nothing new), and that what has changed over the millenia is the proportional *mix* of speech and gesture:

|\ /Speech

| \ /

| \ /

| X

| / \

| / \

|/ \Gesture

+-----------------> Time

Could it be that aphasiacs, by using gesture are merely returning to a more basic (in the purest sense of the word) mode of communication when their speech apparatus fails them?

The paper on stuttering and gestures is fascinating. My first bet would be that there would be frantic gesticulation trying to get one's point across, but that just shows how much I know :-). The fact that speech-accompanying gestures are affected by stuttering is -- and I'm using a technical term, here -- sweet.

"Speech, on the other hand [sic], appears to be better specialized for analytic, explanatory, conceptual content." Indeed.