Using multimodal learning analytics to study learning mechanisms

Author: Worsley, M. & Blikstein, P.
Year: 2014
Project: Multimodal Learning Analytics
Type: Refereed Conference Paper/Poster/Demo (with Proceedings)
Conference/Journal: EDM 2014

Worsley, M. and Blikstein, P. (2014). Using Multimodal Learning Analytics
to Study Learning Mechanisms. In Proceedings of the 2014 Educational Data
Mining Conference.


In this paper, we propose multimodal learning analytics as a new approach for studying the intricacies of different learning mechanisms. More specifically, we conduct two analyses of a hands-on, engineering design study (N=20) in which students received different treatments. In the first analysis, we used machine learning to analyze hand-labeled video data. The findings of this analysis suggest that one of the treatments resulted in students initially engaging in more planning, while the other resulted in students initially engaging in more building. In accordance with prior literature, beginning with dedicated planning tends to be associated with improved success and improved learning. In the second analysis we introduce a completely automated multimodal analysis of speech, actions and stress. This automated analysis uses multimodal states to show that students in the two conditions engaged in different amounts of speech and building during the second half of the activity. These findings mirror prior work on teamwork, expertise and engineering education. They also represent two novel approaches for studying complex, non-computer mediated learning environments and provide new ways to understand learning.