The annual G20 scorecard

Our second figure highlights the citation impact of the research produced by each country/region. The significance of a paper (an article or review) in a research journal is measured by the number of times it is cited in later research. These citation counts grow over time at a rate that varies between research fields, so actual counts are ‘normalized’ for analysis using the global average for field and year of publication. This is called Category Normalized Citation Impact, or CNCI: values greater than 1.0 show a paper is cited more often than world average.

While CNCI provides a useful comparison between different fields, it still lacks some useful context. Research is becoming more collaborative, with internationally collaborative papers tending to accrue more citations than domestic papers. Collaborative CNCI (Collab CNCI) takes this into account by incorporating a collaboration class as part of the normalization process: domestic single institution, domestic multi-institution, international bilateral, international trilateral and international quadrilateral plus (Potter et al. 2020). Again, values greater than 1.0 show a paper is cited more often the world average.

An Impact Profile shows the distribution of CNCI values for a given sample of journal papers and provides a much more informative perspective than a single average. Papers are categorized as either uncited or assigned to various categories chosen to reflect the skewed nature of the citation distribution and provide direct visual comparison with the world average (Adams et al., 2007).

References:
Adams J, Gurney K A and Marshall S. (2007) Profiling citation impact: a new methodology. Scientometrics, 72 (2), 325-344. DOI: 10.1007/s11192-007-1696-x.

Potter, R. W. K., Szomszor, M., and Adams, J. (2020). Interpreting CNCIs on a country-scale: The effect of domestic and international collaboration type. Journal of Informetrics, 14(4), 101075. DOI:10.1016/j.joi.2020.101075.

International collaboration in research has been growing and most of the world’s most highly cited research now has authors from two or more countries/regions. As a result, the specifically domestic part of most nations’ research base has been shrinking and is contributing less to overall national impact (Adams, 2013). The chart on the left in Figure 3 shows the growth of this collaboration.

Some nations, such as Mainland China and Indonesia, however, are following a different trend. Mainland China’s annual output has increased more than three-fold over the last decade, with a selective approach to collaboration that hasn’t kept up with the expansion in domestic research. Indonesia has also been expanding its domestic base after being dominated by international collaboration.

Open access (OA) research publication, where the author or funder pays instead of the reader or a university library paying via journal subscription, is increasing in response to demands from research funders – including governments (see ISI Global Research Report: The Plan S Footprint). The trends and patterns in OA research publication are shown in the chart on the right.

References:
2013. Adams J. The fourth age of research. Nature, 497, 557-560. DOI:10.1038/497557a.

Research Footprints show how a research activity or performance measure varies across disciplines. They show the ‘footprint’ of the country/region on the global research landscape and use eight major discipline groups (see the key in the lower right of the chart) within which there are broadly similar publication and citation patterns.

The left chart shows the distribution of CNCI and Collaborative CNCI (see Figure 2) by discipline, providing a comparison of each nation’s impact against the world average. The chart on the right compares the proportion of each nation’s overall output in each discipline with the G20 average proportions. It also compares the proportion of each discipline’s output that is either international or open access against the corresponding G20 averages. For example, a figure of 2 for All Papers would imply the proportion of output in a particular field is twice the G20 average, and hence a greater focus from that country/region on that field. Comparing the profiles between the two charts helps to identify a country/region’s relative strengths and weaknesses.

Also included are trend charts showing the CNCI and paper counts by discipline over the last decade.

The final figure in this year’s scorecard provides Research Footprints looking at each nation’s contribution to research into the UN Sustainable Development Goals (SDGs). The 17 SDGs were adopted in 2015 and presented in the UN’s 2030 Agenda for Sustainable Development. Sixteen of these SDGs readily translate into areas of research against which nations can be assessed in terms of their relative focus and impact; the 17th goal, Partnerships for the Goals, requires direct government action that can’t be easily measured through academic output.

The left chart looks at the Category Normalized Citation Impact (CNCI) of each nation’s output for each goal, while the right chart shows how the proportion of SDG output for each nation compares with the average proportion for the G20 as a whole. A comparison between the two charts shows where each nation’s priorities lie and how well that research is providing an effective contribution to tackling the world’s most pressing problems.