Why is science communication difficult? For subjects like vaccination, climate change, genetic modification, and more, there are rifts between the views of scientists and the views of large segments of the US population, rifts that seem driven not only by a lack of understanding of scientific concepts, but also by a lack of understanding of how science works. One could argue that scientists should improve their communication skills. I’ve been doing two things recently that very directly relate to this, and that also illustrate its pitfalls.
First, the University of Oregon’s wonderful Science Literacy Program (disclaimer: I’m co-director) is affiliated with the Alan Alda Center for Communicating Science, which trained UO theater and communications people to run a really excellent communication workshop for UO science faculty a few months ago. Its focus was mainly on methods for interacting with small groups of people, as in a media interview or a class. It involved a lot of improv comedy exercises, which isn’t the sort of thing I’d think I’d enjoy, but it was really great – fun, thought-provoking, and useful. The relevant thing for this post, however, is its connection to the Alda Center, which will come up later.
Second, the science teaching journal club I attend has been reading “Houston, We Have a Narrative: Why Science Needs Story” by Randy Olson, a book by a tenured professor turned Hollywood writer, whose main point seems to be that scientists needs to make use of narratives to connect to the ways in which people actually think, feel, and make decisions. We want, it says, stories in which a human protagonist makes a transformative journey, and we should shape our stories of science into this mold. (At least I think that’s what it says – we’re only about a third of the way through the book. The topic is great. The book is not. It could be condensed by at least 50%, and the writing style is annoying, though this may be deliberate.)
A danger of the storytelling approach is that style may win over substance – that we’ll judge the importance of scientific claims by how compelling their stories are. Or even if we don’t judge science in this way the attention we give it will be driven by superficial concerns. Olson realizes this of course, and implies that it’s up to us to use the tools of communication for good ends. (I’m hoping this is elaborated on later in the book.)
This danger is made more concrete by a recent example:
1. Power poses
Perhaps the poster child for shoddy science in recent years has been the “power pose,” the claim by psychology researcher Amy Cuddy and colleagues that adopting a confident stance for a few minutes is enough to change one’s physiology, increasing testosterone levels and decreasing levels of the stress hormone cortisol. It provides, therefore, a quick and easy route to self-improvement, backed by quantitative science. This led to lots of attention — Cuddy’s TED talk on the topic is the second most seen ever (37 million views), for example.
It turns out that the power pose is total nonsense. The story is nicely told in a recent article in The Chronicle of Higher Education, “Power Poser: When big ideas go bad” (Tom Bartlett, Dec. 4, 2016). This comes as a shock to no one who has been paying attention to discussions of poor scientific practices in the past few years. The work, published in real, peer-reviewed scientific journals, is based on the weak “signals,” noisy measurements, low numbers of samples, considerable degrees-of-freedom in the analysis, and desperate searches for “significant” effects that plague many fields, of which psychology is perhaps the worst hit. Very notably, the first author of the work, presently faculty at the business school at UC Berkeley, posted a public denunciation of everything having to do with power poses. With admirably brutal honesty, she points out the flaws of the research design and interpretation. It’s worth stressing that nothing about the experiments or their writeup was fraudulent; the methodology was more like this. It’s also worth stressing that power posing might make one feel better, as will all sorts of relaxation or concentration exercises; the claim, however, was of actual connections to physiology.
2. White lies?
(An aside on the consequences of all this — feel free to skip to the next section, which is more interesting.)
Not long ago, on one of Andrew Gelman’s many excellent blog posts on this and other related topics (esp. [1] and [2]), this question came up: What’s the harm in leaving these sorts of studies unchallenged? More personally: a magazine for teenagers wrote an article on the wonderful benefits of power posing. I sent them a copy of the Chronicle of Higher Education article noted above and got a very nice reply that, among other things, asked if there’s any harm in leaving the power pose advice alone.
Here are a few responses. You can probably think of more, and Andrew Gelman has a whole post about this, with lots of comments:
- Leaving bad science alone lets it draw attention away from methods that do work, or the search for methods that work. In the context of power posing, for example, why bother teaching study skills, for example, or addressing real barriers to learning, if instead I could tell my students “one simple trick” to make them more successful?
- It encourages bad scientific practices, and tangibly harms researchers who do good work. Competition in science is stiff. Every faculty search I’ve been part of has at least 100 applicants for each position. Every grant funding panel has success rates of at best 1 in 6. Publications count for a lot, and every scientist who refuses to publish mediocre or overblown work knows that this carries the potential cost of being ignored. The publicity that is given to “flashy” but flawed studies is to some an incentive to cut corners, and to others it’s simply demoralizing.
- It conveys a wrong idea of how the scientific process works (or should work). Shockingly, this seems to affect actual scientists, not just members of the public — see the note below on Cornell business professor Brian Wansink. Why wouldn’t it, since the methods on display clearly lead to “results?”
- It encourages pseudoscientific storytelling, which can be used for bad ends just as well as good ones.
- It’s just wrong, and that should bother us, beyond any narrow utilitarian accounting of consequences.
3. Who communicates with the communicators?
As mentioned, the power pose has attracted a lot of publicity, and not just TED talks and best-selling books. An interview with Amy Cuddy was featured as the inaugural episode of the “Clear + Vivid” web show from … The Alan Alda Center for Communicating Science! (Link.) If people whose job is communicating science can’t sort wheat from chaff, what hope is there? If an organization that does excellent work on science communication, and that certainly has links to many accomplished and solid scientists, doesn’t think critically about its message, what does that mean for communications via “normal” media? Perhaps the Alda Center realizes the issues with Cuddy, but there’s no mention of this on the Clear + Vivid web site. On the other hand, it looks like no other episodes of this show were made!
The task of critical thinking, I suppose, falls to the rest of us — scientists, educators, or thoughtful consumers of science. We can’t assume that the media, even science media, will separate hype from substance; their role is that of storytellers, which may be effective in terms of reaching people, but which doesn’t provide a critical perspective on what stories deserve to be told. Moreover, it takes time and effort to dig into the substance of science, and I have little faith that science-themed media are up to the task. (To be fair, they have plenty of other challenging and important tasks to deal with.) Of course, I’ve only given one example here, and a particularly egregious one at that. Still, I think many of the science stories I see, especially short ones made for social media, are superficial and don’t give any sense that that authors have actually looked into their subject.
Returning to our role as critics: pointing out the flaws of power poses isn’t a good way to win friends or advance one’s career. Maybe the solution is again in storytelling, casting issues of p-hacking and noise mining as gripping tales with heroes and villains. I’ll leave that as an exercise for the reader.
4. Epilogue: The flavor of the month
In case you’re wondering what’s the next big thing after power poses, my present favorite is Cornell food psychology researcher Brian Wansink’s work, or at least some of it, which suppresses Cuddy’s in both scale and cluelessness. (This has been covered in various blogs; the relevant Gelman blog posts 1, 2.)
I gave her [a Ph.D. student] a data set of a self-funded, failed study which had null results… I said, “…There’s got to be something here we can salvage because it’s a cool (rich & unique) data set.” I had three ideas for potential Plan B, C, & D directions (since Plan A had failed). … Every day she came back with puzzling new results, and every day we would scratch our heads, ask “Why,” and come up with another way to reanalyze the data with yet another set of plausible hypotheses. Eventually we started discovering solutions…
In other words, eventually they found a jelly bean color for which the noise nicely mimicked an “effect.”
Admirably, Wansink allows comments to his post, many of which point out that his is a textbook illustration of p-hacking, which is essentially guaranteed to find false signals amid noise. (“False” means not real, and very unlikely to appear again if one repeats the experiment. Mining the data for potential effects is fine, by the way, if one then tests them with new experiments rather than seeing them as a conclusion in themselves!) Wansink’s replies are surreal and almost hilarious; he seems not to understand the issues involved at all.
Wansink is quite famous, and his early work on, for example, the impact of serving sizes on how much people eat is fascinating and influential. (I especially like his experiments on “never-ending” soup bowls, cleverly replenished using hidden tubing.) I have no idea how solid that work is. I’d like to think that it’s great, but that the low-hanging fruit in food psychology was picked long ago, thanks to him and others, leaving only small effects and noise to be harvested now, sliced and diced into cute but unsatisfying meals.
Today’s illustration…
A rhinoceros. It’s larger than my usual paintings, and was too big to easily scan, hence the photo of a painting based on a photo! I think the book, visible at the lower left, was this one, but I’m not sure. (I did this over a month ago.) Note the rhino’s powerful stance.
The Eighteenth Elephant 