Is impact factor everything?

Although the impact factor is the oldest and, arguably, the most widely used citation metric, it has lost some of its former glory. In recent years, the impact factor has come under considerable criticism for its limitations, and the academic research community is actively looking for alternative metrics that are better indicators of research quality. This has led to the rise of metrics like the h index, Eigenfactor score, and several article-level metrics based on how articles are shared through social media.

Thus, people are increasingly questioning the validity of the impact factor. In fact, a recent study, available freely on arxiv, seems to provide evidence that the era of the impact factor as we know it may be ending altogether. In this study, the authors assessed the association between a journal’s impact factor and the number of citations individual articles in that journal received, for a period of more than 100 years (1902–2009). Their hypothesis that this association should be diminishing, especially since the 1990s, was supported by a sound argument that the impact factor was important at a time when researchers had to read entire issues of print journals to find articles of interest. Librarians would consider the impact factor of journals when determining which journals to stock. However, in the digital age of academic publication, researchers can find articles of their choice by merely performing online keyword searches. Thus, researchers no longer need to read entire journals, and accordingly, the impact factor of a journal itself should not matter as much.

Consistent with their hypothesis, they found that the association between journal impact factor and actual citations per article has been decreasing over the last two decades. This finding was true across subject areas. More interestingly, they assessed how many of the top 5–10% most highly cited papers were in fact published in the top 5–10% journals (based on impact factor) in their respective fields, and found that this proportion has significantly reduced over the past two decades.

So as you can see, the strength of the impact factor is reducing both literally (as the above study indicates), and in the minds of academic researchers. While funding committees may still consider the impact factor of journals in which you have published while evaluating your grant application, I doubt that this factor will serve as the sole basis of their decision. In an earlier post, we explained how the h index is a more reliable measure of researcher quality than the impact factor. To provide the most accurate representation of your research quality and counter any impression that your low-impact-factor publications may have on funding committee reviewers, you should strengthen your application by mentioning a combination of citation metrics, including your h index and other article-level metrics. Clearly, if you have strong objectives and proven expertise for the kind of research you aim to pursue, impact factor is not everything.