Using machine learning to examine learner’s engineering expertise using speech, text, and sketch analysis

Abstract

There continues to be a call for learning approaches that promote collaboration, creativity and innovation, as well as culturally-aware, constructivist approaches to STEM learning. Unfortunately, these skills tend to lie in direct opposition to forms of the most commonly used forms of assessment – national standardized tests. Though the education research field has recognized this discontinuity, we do not currently have the technology needed to holistically assess learning which is customized, and well-adapted to the learners’ culture. Accordingly, this study endeavors to fill that gap by presenting results from a multi-modal analysis of naturally derived student data. More specifically, we used student dialogue, and student drawing – two common artifacts in project-based, constructivist learning environments – to develop predictors for student expertise in the area of engineering design. By leveraging the tools of machine learning, natural language processing, speech analysis and sentiment extraction, we were able to identify a number of distinguishing factors of learners at different levels of expertise. As such, this study motivates continued work in this space, and the development of a new paradigm for assessing student knowledge construction.