New Ways to Measure Research Impact


Content originally published on Learn more about Interfolio’s acquisition of Data180 here.


Academics and university administrators have traditionally measured the success of a faculty member’s published research by tracking the number of times it  was cited in other publications. But some academics are saying that these numbers are misleading and overly simplistic, and are seeking to use technology to create more robust measures of the impact of research articles.

In “Scholars Seek Better Metrics for Assessing Research Productivity,” a 2009 article published in The Chronicle of Higher Education, author David Glenn spoke to numerous academics who are working on big-data systems that could change the way academia measures the impact of scholarly work. While many people agree that better metrics are necessary, Glenn found that it’s difficult to find common ground on exactly what should be measured — and how.

“While it is easy to criticize simplistic measures of citations and ‘impact factors,’ it is not so easy to find agreement about how to improve those metrics,” Glenn writes. “Faculty members sometimes suggest that evaluators should de-emphasize numbers and instead look qualitatively at how research projects affect the public good.”

The article goes on to describe numerous projects underway that could help add dimension to research impact metrics. Johan Bollen, an associate professor of informatics and computing at Indiana University, has created a system called Mesur that aggregates usage information from a variety of academic journals and databases, and then determines which journals and articles have the most influence in the academic world.

Another project, called Eigenfactor, is using similar network analysis tools to determine the reach of different academic journals. And Jorge E. Hirsch, a professor of physics at the University of California at San Diego, has suggested a new metric that he calls the “hbar” for replacing scholarly citation counts. The index is based on a simple mathematical formula that compares the number of articles published by a researcher to the number of times that those articles are cited, and takes into account papers with multiple co-authors.

The complete Chronicle of Higher Education article is available here:

Content originally published on Learn more about Interfolio’s acquisition of Data180 here.