OpenGesture: a low-cost, easy-to-author application framework for collaborative, gesture-, and speech-based learning applications

Abstract

In this paper, we present an application framework for enabling the development of gesture and speech based applications for collaborative learning environments. More specifically, we are concerned with combining the affordances of natural interfaces with educational theories concerning embodied cognition to develop an application framework that enables education researchers and practitioners to create enriching multi-modal learning experiences. Furthermore, this paper highlights a user study that explores how students interacted in a multi-user, collaborative space. Our initial findings indicate that these applications are certainly feasible for well-defined learning tasks, but may require machine learning based training in order to be successful in contexts where the vocabulary is more diverse.