Science Messaging System: SMS+ & Seeing Science

Project Dates: 2020 – present

The Need for Low-cost, Mobile Inquiry Science

Participation in authentic inquiry-learning activities is essential to STEM education, but the realities of remote learning raise new challenges to providing and supporting students in inquiry-based work. The COVID-19 pandemic has underscored the limitations of existing educational infrastructure to deliver high quality remote instruction to children across the socio-economic spectrum.

Of particular concern are issues of equity for low-income children who often have limited or no access to high-speed broadband internet or to dedicated computers/devices for long term use at home. In remote instruction, the high technological and logistical demands of practice-based STEM education reforms are transferred from teachers to individual families, further aggravating existing socio-economic inequities.

Increasingly frequent disruptions to in-person schooling are expected beyond the COVID-19 pandemic due to a causes ranging from the effects of climate change (e.g., fires, floods) to increased political instability and unprecedented levels of displacement worldwide, among other drivers. Thus, investment in and development of robust, high quality, low cost, pedagogically sound, scalable alternatives are imperative.

Phase 1, 2020-2022: SMS+

Funded by Columbia’s Technology Innovations for Urban Living in the Face of COVID-19 program

The goal of the SMS+ Science Messaging System program is to systematically examine and develop a low-cost, mobile phone-based approach to at-home, inquiry-driven science learning. SMS+ will support real-time, interactive, message-based STEM activities based entirely on a mobile phone. 

Students, either working with an adult or by themselves, work through inquiry-based, hands-on STEM activities through text messages and receive prompts to articulate and reflect on their thinking. They use a family cell phone to take pictures, record audio and video, and share the partial and final results of their investigations with their classmates and teachers. The educator-facing SMS+ interface will include a scripting tool to create inquiry-based curriculum units as well as dashboards to collect responses, track student activity, and monitor learning progress. 

Research will examine students’, parents’, and teachers’ capabilities while working with the novel text message and inquiry-based pedagogy. Additionally, we will study teachers’ curriculum redesign process to identify design principles and develop a generalized framework to support the translation of traditional content and pedagogy into text-message applicable formats. 

Phase 2, 2022-2024: Seeing Science

Funded by the National Science Foundation

Students must understand science to make sense of the world around them, make informed decisions, and participate in civic society and in the workforce. However, many youth see science as a mysterious body of knowledge that feels disconnected from their lives. We aim to bring science into middle school students’ homes and lives by allowing them to see the science behind everyday objects and transforming lived environments into engaging learning spaces. Students will work on inquiry-based learning units on mobile phones that explore STEM phenomena topics like diffusion, electricity, and simple machines that are present in their kitchens, bedrooms, and local parks. They will take photos and videos of their homes and neighborhoods, and computer vision algorithms will augment these images with diagrams, models, and simulations illustrating the underlying principles and mechanisms of STEM phenomena. Using these overlays, students will predict, experiment, and observe phenomena such as tea diffusing in water or heat traveling through walls. The project will capitalize on existing technological devices, such as camera phones, to create “lenses” that enable students to see science all around them.

Operating in diverse, low-resource environments motivates fundamental advances in computer vision. First, algorithms must automatically build up a 3D and temporal representation of a scene of a given physical phenomenon. Second, the system must expose hooks for educators to decide which graphics should be overlaid at which time and in which place atop this scene. Further, the project will engage in human-centered design to bring cutting-edge technologies to youth in ways that are accessible, easy to use, and achieve educational goals. Investigators will conduct extensive interviews with parents, students, and teachers about the aspects of students’ out-of-school lives that they would be willing to share with researchers, peers, and teachers. These data will enable the team to realize the benefits of equitable science education that builds on students’ lives and cultures. This research will help foster the development of a more agentic, inclusive way of engaging in science inquiry at home, encouraging students to have a personal connection with science from a young age. This is particularly important for students most at risk to perceive science as disconnected from their lives, and whom can benefit most from seeing science at work in their lives and community.

Tea Diffusion

In the SMS+ diffusion unit, the AI chatbot walks students through a diffusion experiment using materials from their own kitchens: hot water, cold water, and tea bags. The chatbot prompts students to make predictions ahead of time, capture and share photos every 60 seconds of the experiment, and reflect on their predictions in light of their experimental results. The chatbot also shares text and visual information about the process of diffusion. Students’ images and text responses are saved in a central location as a resource for later discussions with their teacher and classmates.

In the Seeing Science extension to the diffusion unit, we use computer vision to analyze students’ tea diffusion experiments in real time. Using water color to gauge the extent of diffusion, we create a responsive overlay to help students visualize the molecular interactions underlying the observable diffusion phenomenon.

Team Members

Paulo Blikstein (PI)

Lydia Chilton (co-PI), Computer Science Department, Columbia University

Tamar Fuhrmann (co-PI)

Carl Vondrck (co-PI), Computer Science Department, Columbia University

Jonathan Pang

Livia Macedo

Diana Garcia


Alumni and special thanks

Richard Davis

Engin Bumbacher

Elena Cristina Duran Lopez



2020-2022 support is provided through the Technology Innovations for Urban Living in the Face of COVID-19 program made possible by Columbia University Trustee and Columbia Engineering Board of Visitors Member, Dean Dakolias ’89.

2022-2024 support is provided by the National Science Foundation grant # 2202579 with Prof. Lydia Chilton and Carl Vondrick: “Seeing Science: Using Computer Vision to Explore the Scientific Principles Behind Everyday Objects”



For more information, please contact Tamar Fuhrmann (