Multimodal Interaction Analysis

Through multimodal interaction analysis, we investigate how students on the autism spectrum (AS) and typically developing students use diverse interaction resources (speech, gaze, gestures, body postures) to participate in the peer group interaction during lessons. The data has been collected by videotaping children’s activities during group work with the help of both video cameras and eye-tracking glasses.

Multimodal interaction analysis is used to investigate the participation of students on the AS and typically developing classmates in social interaction. Among other things, we investigate what kind of initiatives students take and whether the initiatives of students on the AS differ from those of typically developing children. We also look at how students on the AS and classmates respond to initiatives and whether responses differ from each other.

According to studies that have typically looked at the initiatives of developing children, initiatives that trigger joint work are often proposals that invite peers to accept what has been proposed, and thus participate in the group work. In addition to verbal actions, participants use verbal actions, such as gaze, body, and material resources (e.g., a textbook or a computer) to initiate and maintain joint activities (see, e.g., Kämäräinen, Eronen, Björn & Kärnä, 2021; Stevanovic & Monzoni, 2016). Children on the AS who produce little speech, on the other hand, have been found to use various non-linguistic resources, such as long-term eye contact, to solve problem situations (Dindar, Korkiakangas, Laitila &; Kärnä, 2016).

Research data is needed on what kind of interaction resources children on the AS use in group work in educational settings and whether there are differences in the use of interaction resources between children on the AS and typically developing children. Research data is also needed on what kinds of interaction practices promote opportunities for children on the AS to participate in interaction with classmates.

Currently, two peer-reviewed articles have been published about the research conducted with multimodal interaction analysis. In addition, three manuscripts are being prepared. One manuscript focuses on identifying unresponsiveness in peer interaction and two articles intvestigates how students manage collaborative work through directives and embodied resources. Peer interaction in inclusive classrooms has also been examined in the five master’s theses.

Publications:

Kilpiä, A., Dindar, K., Kärnä, E., Räty, H., Kämäräinen, A. & Suero Monter, C. (Hyväksytty julkaistavaksi). Using conversation analysis to identify unresponsiveness in peer interactions in inclusive groups. 

Kilpiä, A., Kärnä, E., Dindar, K., & Räty, H. (2022). Koulunkäynninohjaajan vastaamattomuus autismikirjon oppilaan sosiaalisiin aloitteisiin. Kasvatus, 53(4), 375–390. [Special needs assistants’ unresponsiveness to the social initiations of pupils on the autism spectrum]