The University of Portsmouth has agreed to sign up to the San Francisco Declaration of Research Assessment, known as DORA. This Declaration aims to help institutions improve fairness and transparency around their use of metrics.
Universities often use bibliometrics such as Impact Factors, journal rankings and academics’ h-indexes. While they can be useful, if taken out of context they can sometimes be misleading and may even unfairly disadvantage individuals.
Take the h-index, for example. This is a ‘score’ assigned to an academic, which combines their number of publications and citations. (To check your h-index, search for yourself in Scopus). It may be tempting to use this to compare academics, but without the context, doing so will give you a false impression. For example –
- The academics may have been working in different subject areas. h-indexes are based on the number of citations that articles receive, and conventions/norms for this differ considerably between disciplines. So you should never compare academics who work in different subject areas.
- The academics may have had different career lengths. It’s almost inevitable that an academic author who has had a longer career will have a higher h-index than an early career researcher or an academic who has had breaks in their career (e.g. due to childcare responsibilities). This may just be because the academic with the longer career has had more time to produce publications and not because they are a ‘better’ researcher.
Journal Impact Factors and other metrics can also be misleading if used out of context.
To tackle this problem, there’s been an international shift towards ensuring such bibliometrics are used responsibly and fairly. Most major funders, including UKRI and Wellcome, have signed up to DORA, along with most UK universities.
DORA is an internationally agreed set of principles that outline how metrics, especially journal metrics, can be used ‘responsibly’.
Its overarching principles are –
The need to eliminate the use of journal-based metrics, such as Journal Impact Factors, in funding, appointment, and promotion considerations;
The need to assess research on its own merits rather than on the basis of the journal in which the research is published; and
The need to capitalise on the opportunities provided by online publication (such as relaxing unnecessary limits on the number of words, figures, and references in articles, and exploring new indicators of significance and impact).
Therefore, metrics can still be used. They are often very useful in understanding someone’s background, research and accomplishments. But they must be used in a way that aligns with DORA, with proper understanding of what the metrics mean and what context is needed to interpret them fairly and responsibly.
Having signed up to DORA, the University will now work towards being compliant with its principles. Over the next year, we will be reviewing areas where DORA may apply, to see whether any changes or updates are needed. This will include areas like recruitment, workload allocation and promotions.
More information will be shared in due course, and key internal stakeholders will be contacted directly. In the meantime, if you have any questions about DORA or the use of metrics, please visit the DORA website, or contact Dr Andy Dixon, Deputy Director, Research and Innovation Services (RIS).