Too Much Research
By Peter Klein • Thursday July 8, 2010 11:54 AM PDT • 9 Comments
A controversial piece in last month’s Chronicle of Higher Education,, “We Must Stop the Avalanche of Low-Quality Research,” argues that “the amount of redundant, inconsequential, and outright poor research has swelled in recent decades, filling countless pages in journals and monographs.” The five authors, representing a variety of academic disciplines, point to increases in the numbers of journals, journal pages, and authors and decreases in average citation rates.
[I]nstead of contributing to knowledge in various disciplines, the increasing number of low-cited publications only adds to the bulk of words and numbers to be reviewed. Even if read, many articles that are not cited by anyone would seem to contain little useful information. The avalanche of ignored research has a profoundly damaging effect on the enterprise as a whole. Not only does the uncited work itself require years of field and library or laboratory research. It also requires colleagues to read it and provide feedback, as well as reviewers to evaluate it formally for publication. Then, once it is published, it joins the multitudes of other, related publications that researchers must read and evaluate for relevance to their own work. Reviewer time and energy requirements multiply by the year. The impact strikes at the heart of academe.
I think this assessment is generally on target, for my own field at least. What percentage of the articles in the typical academic journal does anybody read, let alone remember? How much of the research in any scientific field really adds value? Of course, search tools make it easier to find relevant information, so I’m not sure the point about writing literature reviews is all that compelling. Still, it does seem increasingly difficult to sort wheat from chaff.
I’m less impressed with the authors’ proposed solutions — limiting the number of publications that can be considered for promotion and tenure, making greater use of impact factors, and enforce tighter page restrictions. These strike me as superficial fixes. The main problem is the vast increase in the scale and scope of the “scientific” enterprise itself, almost all of it due to public funding. There are simply too many universities and institutes, too many research faculty, too many granting agencies, too much research money. It’s a self-perpetuating process, almost exclusively driven by supply-side considerations (who on earth “demands” the output of most English departments?). Some academic readers will be shocked by the claim that there’s “too much” research money, particularly in today’s austere climate. But I mean too much relative to some social optimum, not too much relative to what university professors want.
Why would we expect this kind of system to produce high-quality research? Perhaps it’s a miracle that any good work gets done at all.
Tags: Corruption, Education, Government subsidies, Science 
Amen, Peter. Digging the diamonds out of this vast garbage dump has become well-nigh impossible. Who knows what valuable pieces go unread? Everyone knows, however, that nearly everything published — not only in the lower-tier journals, but also in the highest-rated ones — is crap. Anyone can satisfy himself in this regard simply by perusing a journal selected at random — or not at random; it scarcely matters.
Robert Higgs | Jul 8, 2010 | Reply
Speaking as an engineering graduate student at an elite university, I agree that there is an avalanche of redundant and inconsequential chaff. What is to be done? I have noticed that some scholars are, frankly, much more creative than others; they are seen as leaders, as innovators; they are atlas. Very little of the public money allocated to them results in chaff production. Their ideas, inventions, and explanations for natural phenomena are the ones that lead to tangible benefits for society.
Sydney Brenner, a Nobel laureate, is of the opinion that a lot of the high-throughput biology being done today is chaff. I agree, and I bet Peter does as well. Sydney proposes, for the same reasons I do, that what the biomedical enterprise needs is theoreticians who can provide frameworks for data organization and provide actual explanations for the observed biology.
So Peter, re your posing the question of what amount of research money is optimum for society, my answer: there should be as much money available as is needed by those who are intellectually able to produce value. In a great society, $100B might be reasonable. In a society with stupider people, maybe only $10B could be soaked up by virtuosos.
In any case, Sydney and I agree that it is the intellectual firepower to theorize that produces for society a return on the money it invests in research. I leave you with ideas from computing: incremental improvements on existing paradigms are done by company’s basic research divisions; new/emergent paradigms, such as quantum computing, almost always get their start in the academy. This again shows that the academy is indispensable, and that theory is its product of value.
Victor Zilinskas | Jul 8, 2010 | Reply
Also, the Howard Hughes Medical institute does not fund the project, but the person. They have a huge impact. It is all about the person, the creator, as we Rand fans know.
Victor Zilinskas | Jul 8, 2010 | Reply
Victor,
I would suggest that there should be no government funding of research of any kind, regardless of the caliber of the recipient. Indeed, such funding is a major reason for the politicization, poor quality and outright fraud we have seen in so many fields. Here are some references in this regard:
“Government and Science: A Dangerous Liaison?”, by William N. Butos and Thomas J. McQuade (The Independent Review, Fall 2006)
“Science as a Market Process,” by Allan M. Walstad (The Independent Review, Fall 2002)
“Peer Review and Scientific Consensus,” by Robert Higgs (Nature Magazine: “Peer to Peer”, September 17, 2007)
As for your remark about Ayn Rand’s defense of creators (i.e., designers), I would also suggest that while her aim was admirable, her rooting this defense in a naturalistic worldview is self-defeating. Such a worldview necessarily means that all human action is mechanically determined solely by natural forces (i.e., the laws of physics) in which no free will, rational agency, objective epistemology and morality, or scientific enterprise have any meaning or are possible. A better framework is provided by the philosopher Alvin Plantinga and others who have shown that naturalism is incoherent and self-refuting and that the natural-law tradition of “substance dualism” explains reality. And indeed, this is how the ideas of liberty and science were discovered in the first place, as Rodney Stark shows in his book, The Victory of Reason. Please also see the following:
“The Argument from Reason,” by Victor Reppert (PhiLo vol. 2, no. 1, Spring-Summer 1999)
“Economic Science and the Poverty of Naturalism: C. S. Lewis’s ‘Argument from Reason’,” by David J. Theroux (Journal of Private Enterprise, Spring 2008)
Warrant and Proper Function, by Alvin Plantinga
C.S. Lewis’s Dangerous Idea: In Defense of the Argument from Reason, by Victor Reppert
David Theroux | Jul 9, 2010 | Reply
So Peter, re your posing the question of what amount of research money is optimum for society, my answer: there should be as much money available as is needed by those who are intellectually able to produce value. In a great society, $100B might be reasonable. In a society with stupider people, maybe only $10B could be soaked up by virtuosos.
Perhaps, but the problem with public funding, as David’s links point out, is that there’s no way to calculate this number. Absent market-based signals of benefit and cost, how can anyone know?
Peter G. Klein | Jul 11, 2010 | Reply
Subsidize something and you just get more of the same. The only solution to the over-production of useless “research” is to remove the government subsidy.
The human element here is that countless years of lost energy go into working on projects that have perceived, not real, value. The subsidy sends a false signal of worth and causes people to invest their limited time in activities that have no market value. Thus we get thousands of credentialed scientists and academics who have few skills that transfer to the world outside academia. It’s a damned shame.
mikehell | Jul 11, 2010 | Reply