The best way to measure research impact is to use a range of metrics

This is the concluding part of the interview with Melinda Kenneway and Charlier Rapple. Please read the first part of this interview to know more about Kudos and how it helps researchers, institutions, funders, and publishers increase the discoverability of published research. 

Even though millions of articles are published every year, only a few are cited and actually discovered or used by readers. To ensure that their work has maximum impact, researchers must ensure that their papers are easily discoverable by, utilizing their online and offline networks to the fullest. I spoke to Melinda Kenneway and Charlie Rapple, co-founders of Kudos, the web-based service that helps researchers as well as their institutions and funders to maximize the visibility and impact of published articles. In the previous interview segment, Melinda and Charlie explained how Kudos helps researchers, institutions, funders, and publishers increase the discoverability of published research. Here. they talk about the global outreach and future plans of Kudos. They also share their views on topics such as research impact measurement and the role of social media in research promotion.

Melinda Kenneway (Executive Director, Co-Founder at Kudos) is passionate about academic publishing. After a degree in Experimental Psychology, she entered the scholarly communications industry and never looked back since. She has been the Marketing Director of the Journals Division at Oxford University Press and is co-founder of a specialist marketing agency for publishers, institutions, and societies. Charlie Rapple (Sales & Marketing Director, Co-founder at Kudos) loves finding out about new publishing techniques and technologies. She is also an enthusiastic volunteer for the industry association UKSG and an Associate Editor of Learned Publishing. Her previous roles include head of group marketing at Publishing Technology, electronic publishing and account management at CatchWord, and linking and product management at Ingenta. Charlie holds a BA and a postgraduate MDip in Marketing.

How has the global uptake of Kudos been?

Charlie: Kudos was launched in May 2014 and has quickly grown to over 75,000 researcher registrations, supported by over 60 of the world’s leading publishers including Wiley, the American Association for the Advancement of Science (AAAS), and The Royal Society. We’ve also recently launched an institutional service and are working with major universities including Carnegie Mellon in the US, ETH Zürich in Switzerland, the University of Technology Sydney in Australia, and the University of Huddersfield in the UK. We have users in almost every country of the world, including around 5% in China, 5% in India, and 5% in Australia.

Generally, the feedback we’ve had has been overwhelmingly positive with everyone saying that Kudos fills a key gap. And it’s a gap that’s growing. While there are many new emerging platforms through which researchers can network and post their publications (sometimes contravening publisher copyright), and share information about their research activities, Kudos is the only available service that enables them to track which of those sites drive impact for their work.

As with any new service, it can take time to get known and for researchers to see the benefits, but our approach of working closely with publishers and other institutions that support authors is helping drive uptake and engagement.

The impact factor and other citation metrics dominate research evaluation workflows to a great extent. Yet, alternative ways of gauging or evaluating research impact, such as those provided by Kudos, are being developed and adopted. Do you think this is the way forward?

Melinda: Today, there are many metrics that evaluate the impact of researchers’ work. These include traditional metrics such as citations through to emerging metrics (sometimes called “altmetrics”) that report on social shares and media coverage. These metrics are calculated by many separate organizations so for researchers to understand how the impact of their work is being evaluated, they have to know about and seek out each of these different providers of metrics. This makes it difficult to appraise the full impact of a single publication. The best way to fully understand the influence and impact of a research publication is to use a range of metrics instead of relying on a single score. That’s why we combine a range of metrics in our dashboard, meaning that the researcher or institution can decide for themselves which metrics are most important. There are significant differences across disciplines and career stages in terms of what constitutes “good performance.” We’re keen to bring benchmarking into the mix, helping people understand how they compare with their direct peers.

Kudos is not a metrics provider in itself – we are not working to replace the impact factor or any other measure of impact. Our aim is to work with a range of metrics specialists to aggregate their data into one place, the Kudos dashboard, so that are users can easily track and improve those that matter to them.

Charlie, in an interview, you mentioned that in the course of their research, academics build good relationships with experts in their field but often they do not make the most of these networks. Could you share some useful tips that can help researchers understand and use their networks to their fullest potential?

Charlie: Most academics tell us that their careers are based as much on the networks that they build as well as the publications they write. These networks are mostly built offline, through informal introductions, meetings at conferences, and collaborations. Increasingly though, we’re seeing the emergence of new online networking platforms such as ResearchGate and Academia.edu, blogging tools, and other interactive services that enable people to make connections to help benefit their careers. As researchers extend their networks into the online environment, this obviously creates an opportunity for greater sharing of their work.

Some platforms encourage uploading articles to their site, sometimes illegally. It can be difficult for authors to know what they can and can’t share, and with whom. Kudos removes that doubt because we enable authors to explain their work on a page that they can then share legally on any site, and this page then links through to the authoritative version of their article on the publisher’s site. And because all article shares are tracked by Kudos, the researchers can then see the resulting impact. In this way, we help researchers use their growing networks to gain greater exposure for their work with the confidence that they are doing so legitimately.

In terms of how researchers can build up their following in online networks, we’ve recently posted some tips for this in this blog posting. 

Today, more and more people in academic publishing are using social media to talk about their research and build a network. Yet, some researchers are skeptical about using social media platforms to talk about their work. How would you assuage their doubts?

Melinda: Discussions we’ve had with funders imply that they are increasingly seeing outreach as a core part of the scholarly communications process. There is simply too much content now available, on different platforms and in different formats, thus making it difficult to ensure that research finds an audience. As the assessment of impact shifts towards more complex measures of performance, so will the motivation for researchers to take more control over the visibility and impact of their work.

There are cultural challenges here: researchers in some countries are weary of being seen as “promoting” or “marketing” their publications, but that’s not what Kudos aims to encourage. We simply see the opportunity for researchers to use their expertise and interest in their work as key to helping it attract the broadest possible audience. Using social media for sharing links is only one part of the Kudos service. As a starting point, researchers can simply write a plain language summary of their publications and link some related resources to their work - that alone will help attract more readers. One step at a time is generally our advice. Researchers can also find hints, tips, and general advice on how to get started with social media on our site.

What can researchers, publishers, and funders expect from Kudos in the near future?

Charlie: Our vision for Kudos is a world where research has greater impact and researchers benefit from greater recognition. Over the next few years, we envisage a dramatic increase in the number of sites and channels that researchers can use for publishing and promoting their work as well as building a profile online. As this landscape becomes more complex, the need for a tool to guide researchers on how and where their outreach efforts will have most effect will become ever more necessary. Kudos will help meet that need by using aggregated data from individual author activities to help develop “protocols for performance,” tailored to each researcher’s goals and circumstances, and uniting the efforts of publishers, funders and institutions to further boost that effort. We’ll also help researchers, publishers, institutions, and funders to benchmark the “performance” of their publications by providing anonymous, aggregated, cross-publisher, cross-institution, cross-funder datasets to compare their own publications with, in order to understand relative success and determine where and how improvements might be made.

Thanks Charlie and Melinda!

Read the first part of our interview where Charlie and Melinda tell us more about Kudos and how it helps researchers, institutions, funders, and publishers increase the discoverability of published research.