Given that all university classes are currently being run on-line due to the Covid-19 pandemic, our science teaching journal club topics for the term are mostly focused on remote instruction. Last week’s article [1] was a 2019 study comparing different methods for preparing video-based short lectures. It was especially interesting for reasons that can be summarized in two graphs. The authors, at UCLA, made instructional videos covering the same format but with several different methods — a video of a “classic classroom,” a “talking head” superimposed on slides, a “learning glass” video — writing on a large a transparent glass screen, an “interview” style with no slides, and more. They subjected groups of student volunteers to the videos and then assessed how they liked the videos and how well they did on a test covering the content. The paper is clear and readable, and the study is especially worthwhile given current events.
For student perceptions there were clear outliers. Here’s the graph. (Yes, the violin plot is silly and masks the granularity of the data.)
Students really liked the learning glass. Here’s a picture of it, featuring its inventor (and Univ. of Oregon alumnus) Matt Anderson.
It’s a commercial product, but there are plenty of DIY instructions on the internet [2].
The really striking observation is that none of this matters for how much students learned — the assessments are very similar regardless of what video the students watched:
The paper slightly buries this lede — most of the article is on student fondness for the different video types with relatively little about the assessment of student learning. The latter, however, is definitely the most important point, especially since the different video production schemes differ a lot in how much work and expertise they require. Why agonize over how your video looks, if students learn similarly regardless? I would expect this to be even more important in a real course rather than an artificial assessment of videos, since even more of the learning is done on students’ own time through homework and reading.
Of course, it would be good to see more studies of this, especially with longer videos or more complex topics.
I suppose one could argue that making videos that students like is important regardless of what they learn, just from the perspective of keeping students happy. This may be especially true in a world of remote learning, since students could more easily be poached by schools with more appealing materials. If we take this approach, however, we need to acknowledge that it’s solely for marketing purposes. Also, video production is hard, and the present situation in which universities avoid providing concrete support, while ok in an emergency, is absurd as a general policy. This was discussed a lot at the journal club: why are students paying expensive tuition fees to see us in our garages or at our dining tables, with less visual appeal than 20th century public-access TV shows?
Rather than devoting effort to superficial style, however, the better approach is to explain to students that what actually matters is their learning, and that what’s easily palatable may not be what best helps them learn. We already deal with this in promoting active learning: most lower-level students prefer passively sitting in a lecture hall rather than being spurred to engage with material, and we explain (or we should) our reasons for pushing less comfortable practices. The same principles should apply on-line.
Today’s Illustration
A drawing of insulin, just a colored pencil copy of the ribbon visualization of the crystal structure. I might make a better version for my book, since I discuss how bacteria were engineered to produce the protein.
— Raghuveer Parthasarathy, May 21, 2020
[1] Choe, R. C., Scuric, Z., Eshkol, E., Cruser, S., Arndt, A., Cox, R., … & Crosbie, R. H. (2019). Student Satisfaction and Learning Outcomes in Asynchronous Online Lecture Videos. CBE�Life Sciences Education, 18(4), ar55. Link
[2] Instructions for making your own lightboard are, for example, at https://flippedlearning.org/how_to/how-to-make-a-lightboard-for-less-than-100/ and https://www.youtube.com/watch?v=agsUU9i__hc&feature=emb_logo
The Eighteenth Elephant 