One Number to Judge Them All: The Librarian's Tool That Took Over the Tenure File
A Metric Born in a Library Stacks Room
The journal impact factor entered the world not as a measure of scientific quality but as a circulation problem. In the early 1960s, Eugene Garfield developed the metric through the Institute for Scientific Information to help librarians decide which journals their limited budgets could justify. Which titles were cited most often? Those were the ones to purchase. The calculation was deliberately simple: average citations received in a given year by articles published in the journal over the prior two years.
Nothing in that formula was designed to evaluate a scientist, a department, or a body of work. Yet walk into virtually any tenure review at an American research university today, and the impact factor of a candidate's publication venues will surface—sometimes explicitly, often as an unspoken shorthand. The number meant for subscription decisions has become the de facto currency of academic careers.
The Statistical Problem Hiding in Plain Sight
The methodological objections to this practice are neither new nor obscure. The impact factor is a mean applied to a distribution that citation researchers have repeatedly shown to be wildly skewed. A small fraction of papers in any journal accumulates the overwhelming majority of citations, while the median article in a high-impact venue may receive no more attention than one published in a modest outlet. Averages, in this context, describe almost no individual paper at all.
The metric also conflates the journal with the article—an ecological fallacy that would be rejected in any introductory statistics course. Judging a paper by its journal's impact factor is, as critics within the field of scientometrics have long argued, a category error. Add to this the well-documented susceptibility of the number to gaming: journals can inflate their scores through editorial strategies such as encouraging authors to cite recent issues, or publishing review articles that harvest citations at scale.
In 2007, the editors of several leading journals—including, notably, the Journal of Cell Biology—signed a joint statement calling the impact factor an inappropriate tool for assessing individual research. Nearly two decades later, the number still sits in promotion dossiers from Cambridge, Massachusetts, to Berkeley, California.
How the Number Hardened Into Orthodoxy
If the flaws are so well understood, why does the metric persist? The answer lies less in statistics than in administrative convenience. Tenure committees, search firms, and federal study sections face thousands of dossiers and grant applications. Reading each candidate's work closely is expensive; a scalar attached to a journal name is not. The impact factor functions as a triage device, allowing evaluators to sort candidates quickly under conditions of chronic scarcity—of time, of money, of attention.
The result is a self-reinforcing loop. Aspiring faculty publish in high-impact journals because committees reward it; committees reward it because that is where ambitious researchers publish. The loop distorts behavior along the way. Researchers chase fashionable questions over important but unglamorous ones. Incremental, safe studies that fit the two-year citation window crowd out slow, risky work. In fields from ecology to materials science, scholars report shaping manuscripts around citability rather than scientific merit—a quiet misalignment between what science rewards and what it needs.
The Pushback Takes Institutional Form
Change, however, is no longer confined to editorial op-eds. The movement known as the San Francisco Declaration on Research Assessment, or DORA, launched in 2012, now counts thousands of signatories including major US research universities, the National Institutes of Health-adjacent research community, and prominent scientific societies. DORA does not merely criticize the impact factor; it asks institutions to commit to assessing research on its own content—through narrative descriptions of contributions, qualitative review of a small number of significant papers, and attention to the full range of scholarly output.
Some institutions have moved further. A growing number of American universities have rewritten promotion guidelines to instruct committees against using journal-level metrics as proxies for article quality. The National Science Foundation's broader shift toward valuing societal impacts and mentoring alongside publications signals a parallel loosening at the federal level. Meanwhile, a younger generation of researchers, raised on open-science norms, increasingly treats the metric with open skepticism—posting preprints, emphasizing replication, and questioning why a 1960s librarian's tool should govern a twenty-first-century career.
What Comes After the Number
None of this implies that evaluation can proceed without metrics of any kind. Committees need comparability, and citation data, used carefully, can inform judgment. The alternative emerging in policy discussions is not the abandonment of measurement but its recontextualization: article-level metrics that describe how a specific paper has been read, cited, and used; narrative portfolios that ask researchers to explain the significance of their own work; and committee training that treats quantitative indicators as inputs rather than verdicts.
The deeper lesson is about how scientific institutions adopt tools. A metric designed for one purpose—allocating library budgets—was repurposed, without validation, for an entirely different one—judging human beings. The impact factor's career is a case study in metric drift: the process by which a convenient proxy, once embedded in routine, becomes indistinguishable from the thing it was supposed to measure. Undoing that drift will require more than statistical correction. It will require the universities and agencies that depend on the number to accept the harder, costlier task of reading the science itself.
The evidence suggests they are beginning to try. Whether the effort outlasts the convenience of a single, tidy number remains the open question of American research assessment.
