What’s the big idea? Science!

The U.S. National Science Foundation ran an interesting call for proposals recently called the “Idea Machine,” aiming to gather “Big Ideas” to shape the future of research. It was open not just to scientists, but to anyone interested in potentially identifying grand challenges and new directions.

I expect that most of the submissions will be on fairly obvious (though important) topics — developing new energy sources, curing cancer, etc. It occurred to me, though, that:

(i) There are non-obvious, or unpopular, ideas that are important. I’ll perhaps discuss this in a later post. (What might you come up with?)

(ii) There is a very big idea, perhaps bigger than all the others, that I’d bet isn’t one of the ~1000 other submissions: fixing science itself.

I therefore wrote up my proposal, titled “A Sustainable Scientific Enterprise.” I’ve copied it below, adding a little commentary. I will stress that I think it has close to zero chance of making it to the second round of the contest, let alone winning — it’s not uplifting or photogenic enough (despite the illustration below) for a contest like this. Nonetheless, it was fun to write, if only for the exercise of succinctly expressing some thoughts on this topic.

Proposal: A Sustainable Scientific Enterprise

What is the compelling question or challenge?

The scientific enterprise has never been larger, or more precarious. Can we reshape publicly funded science, matching trainees to viable careers, fostering reproducibility, and encouraging risk?

[194 characters / 200 character limit]

What do we know now about this Big Idea and what are the key research questions we need to address?

Science has transformed civilization. This statement is so obviously true that it can come as a shock to learn of the gloomy view that many scientists have of the institutions, framework, and organizational structure of contemporary scientific research. Issues of reproducibility plague many fields, fueled in part by structural incentives for eye-catching but fragile results. We train vast numbers of graduate students, in many cases empowering new researchers to expand the frontiers of knowledge or pursue transformative technologies, but also often frustrating their aspirations with preparation for careers that don’t exist or with halfhearted alternative training mashed into an incommensurate educational structure. Funding remains precarious; increased spending on science over the past few decades has been more than matched by an increased number of scientists, leading to a Malthusian competition for resources that wastes time and energy and also hinders long range planning. Relatedly, over 2 million scientific papers are published each year, about one-sixth of which are from the United States, representing both a steady increase in our understanding of the universe and a barrage of noise driven by pressures to generate output. All of these issues together limit the ability of scientists and of science to tackle important questions that humanity faces. A grand challenge for science, therefore, is to restructure the scientific enterprise to make it more sustainable, productive, and capable of driving innovation.

The stresses noted above are driven in large part by vast growth in the number of scientists and a lack of change in how we train, reward, and organize researchers. About 50,000 Ph.D. degrees per year are granted in the U.S., up from 40,000 at the turn of the century and 30,000 forty years ago. Globally, the numbers and their rate of increase are higher. In itself this is wonderful; 90% of all the scientists who have ever lived are alive today, a consequence of exponential growth noted by Derek de Solla Price decades ago, providing an amazing resource for civilization. However, it puts pressure on the scientific enterprise. Methods of scholarly communication that indicate progress in small communities can easily become simple tick-boxes of activity in large, impersonal systems. Continual training of students as new researchers, who then train more students, is very effective for exponentially expanding a small community, as was the goal in the U.S. after World War II, but is clearly incompatible with a sustainable, stable population. The present configuration is so well-suited to expansion, and so ill-suited to stability, that doubling the budget of the U. S. National Institutes of Health between 1998 and 2003 led to a reduction in the success rates of grant proposals, as universities keen on tapping into this bounty swelled their research workforce.

Much has been written on potential solutions to these issues. Ideas abound. Lacking, however, is a concerted effort on the part of major funding agencies to seek and implement large-scale solutions. Notably, this must involve universities; about half of the doctorate holders in the U.S. are employed in academia. Less obviously, it must also involve the non-elite schools at which the bulk of researchers work, which have very different perspectives and experiences than the increasingly disproportionately well-off schools. As solutions, for example, one can imagine, long-term grants to universities that explicitly fund research scientists and not students, shifting incentives towards appealing, stable research careers that that a (smaller) number of doctoral trainees can aspire to. One can also conceive of restructuring indirect costs to remove strong incentives for profligacy. Finally, we can envision universities opting to serve as test cases for new funding models. With serious recognition of the need for structural changes, transformative solutions will arise.

[3984 characters / 4000 character limit]

Comments: Note the severe length limits! 4000 characters obviously doesn’t allow much room for subtlety, so in this and all the sections, my arguments are rather blunt. Still, the central point doesn’t take too long to state: our modern scientific system arose in an era of exponential growth; we don’t acknowledge this, and it leads to lots of problems. I wonder how many scientists are aware of how many scientists there are, and what their growth rate has been?

Why does it matter? What scientific discoveries, innovations, and desired societal outcomes might result from investment in this area?

Improving the structure of science matters because science matters. It is hard to understate the impact of science on society: every mobile phone, DNA test, detection of a distant planet, material phase transition, airborne jetliner, radio-tracked wolf, and in-vitro fertilized baby is a testament to the stunning ability of our species to explore, understand, and engineer the natural world. There are many challenges that remain unsolved, involving such “big” topics as linking genetic information to disease and treatment, imagining and designing new materials, understanding mental health, predicting geological changes, inventing new food sources, helping wildlife flourish, developing sustainable energy sources, and more, and countless “small” topics that add up to a rich, deep understanding of nature. Better science leads to better insights into these topics, and a better world.

“Better” science does not simply mean more of it, but rather an increase in quality. A watershed moment for contemporary science was the publication in 2005 of John Ioannidis’ article, “Why Most Published Research Findings Are False” (PLoS Med. 2, e124 (2005)), cited over 6000 times in the years since. One may quibble about the exact fraction, and its variance across fields, but there is widespread agreement that a disturbingly high proportion of reported findings are simply wrong, driven by poor methodology, inappropriate statistical methods, overstated conclusions, and other flaws. The inability of many research results to replicate, especially in medicine and psychology, is also well known, and is especially disturbing as replication and predictive power are among the hallmarks of science. Again, causes for this state of affairs are often discussed, along with their roots in the incentive structures and structural frameworks noted above. This proliferation of error matters not only because it doesn’t advance human understanding, but because it actively detracts from it, making real insights harder to convey amid a large and expanding sea of noise.

A further reason that reforming the scientific enterprise matters is the retention of talented potential new scientists. Disillusionment with the nature of graduate training is widespread, given the challenges of finding stable research careers in academia, or careers outside academia that make use of one’s scientific skills or that are accessible given the skills one typically acquires in graduate school. While Adam Ruben’s darkly humorous book, “Surviving Your Stupid, Stupid Decision to Go to Grad School” is intentionally shockingly titled, its sentiments are familiar to many, and perceptions that the system is a pyramid scheme rather than a well thought out program of professional development serve to dissuade many potentially promising researchers from pursuing careers in science. Given the complex challenges society faces, this is a loss we cannot afford.

[2931 / 3000 characters]

Comments: In some fields, a lot of what’s published is wrong. More commonly, much of what’s published is correct but minor and unimportant. If I had more space, I would have noted this as well. Of course, most people don’t want to do boring work; the issue is one of structures and incentives.

If we invest in this area, what would success look like?

The benefits of reshaping the scientific enterprise will be reflected in characteristics of the scientific workforce and in progress towards solving grand challenges. Most simply, though most subjectively, success would take the form of increased confidence on the part of scientists – stronger and more widely held convictions that, as much as humanly possible, we are gaining robust and reliable insights into how nature works, and are responsibly training talented people to further advance these insights. More quantitative measures will also improve. For example, the fraction of non-replicable scientific findings should decrease. One might also expect a reduction in the rate of production of scientific publications, but an increase in their citations and impact, as pressures to produce minor output in an over-competitive landscape are replaced by incentives to perform longer-term, deeper studies. The fraction of graduate students leaving science and technology for different fields, in or out of academia, will drop as training better matches careers. Similarly, metrics of underemployment among science doctorates will decline.

During the course of reform, we can also expect to see the pursuit of a diversity of approaches, as universities and funding agencies work together to creatively design, implement, and assess new policies. Beyond its utility, these large-scale experiments promise to provide insights into the sociology, economics, and perhaps even philosophy of science itself.

Ultimately, the real test of scientific reforms is the progress we make on “big questions.” We will hopefully look back on the post-reform era as the one in which challenges related to health, energy and the environment, materials, and more were tackled with unprecedented success.

[1783 / 1800 characters]

Comments: This was the hardest section to write. I don’t really believe in or care about metrics like citations, but I felt I had to write something beyond “we’ll learn more, accomplish more, and also be happier.”

Why is this the right time to invest in this area?

The strains to which the scientific enterprise is presently subjected are in large part driven by its enormous growth over the past several decades. Ignoring them will not make them disappear. Furthermore, science is expanding globally, giving the United States the opportunity to demonstrate a sustainable science policy that can serve as a model for other large countries.

Despite the gloom, it is presently a golden age of science. Our tools, whether directed towards the far reaches of the universe, the intricate machinery of cells, the building blocks of matter, or other targets, are stunning in their capabilities, and with them we have gained insights and capabilities beyond the imagination of previous generations. However, science thrives by self-criticism and skepticism, which should be applied to the institutions of science as well as its subject matter if we are to maximize our chances of successfully tackling the many complex challenges our society, and our planet, face.

[990 / 1000 characters]

Comments: I felt it important to note somewhere, at least briefly, that not all is gloom! We could, however, be doing much better, especially with respect to the human capital of new researchers.

Image

When submitting the proposal, I realized that it requires an image to go along with it. How does one illustrate reshaping the scientific enterprise? I don’t know, and I lacked time to think about it, so I uploaded the rhinoceros painting I’ve posted here earlier. We’ll see what the judges make of it…

Final comments

Though the odds of my advancing in this context are close to zero — how can I compete against high school students who want to save the planet, even if fixing science will make saving the planet more likely? — this was still an enjoyable activity. It could make a good writing exercise for senior-level undergraduates — though see the next paragraph’s note on small ideas — or graduate students. I was surprised at how close my first drafts of all of the sections were to their length limits, perhaps a sign that I’ve spent too much time writing abstracts and other short blurbs!

There’s more to write about big ideas that I think won’t be proposed and (also) won’t be selected. I’ll save that for a later post, and perhaps elaborate also on the great importance of small ideas, which can not only add up to big things, but which have often turned out to be surprisingly deep and important.

Update, March 1, 2019:

My proposal “was not judged strong enough to advance to the video pitch stage of the competition.” I am not surprised.

Today’s illustration

A cherry, a bacterium, and something in between. I could claim this is some sort of metaphor for the transformative power of science, but really I just thought it was fun to draw.

— Raghuveer Parthasarathy. October 30, 2018