The (un)sustainability of impact factor. How can we make research more accountable?

Call for papers for: Sustainability Accounting, Management and Policy Journal

Guest Editors:
Michele Andreaus, professor of Accounting, University of Trento, Italy, [email protected]
Carlos Larrinaga, professor of Accounting, University of Burgos, Spain, [email protected]
Ejia Vinnari, professor of Public Financial Management, Tampere University, Finland, [email protected]

Conventional metrics employed in research performance evaluation do not represent the claims of different stakeholders. It is now very well established in the accounting literature that performance measurement and accountability systems are performative. In the research context, this means that performance measurement and accountability systems do not just measure and account for research, but they also transform the research field by creating evaluation rules and promoting specific research areas, while discouraging others. For example, there is evidence of an innovation crisis in health research caused by relying on financial and academic myopic measurement of research output and impact (Zaratin, et al., 2014, Zaratin, et al., 2016). It has been proposed that the paradigm should shift towards co-evaluation, engaging multiple stakeholders to define the measurement and accountability systems that enable the consideration and alignment of a plurality of perspectives about the research process (Pedrini et al., 2018). New jointly agreed metrics are expected to drive common objectives (a common research agenda) and elicit the expected return on investment for each stakeholder involved. This return is not necessarily financial or economic, but first of all social, considering the impact of research on society in the widest sense.

A possible way for making research more accountable and democratic could take into consideration a proper actor engagement (Concannon et al., 2012; Salvetti et al., 2018), considering the different stakeholder perspectives through a negotiation process (see post-normal science approach – Funtowicz and Ravetz 1994). We need to be aware that this approach needs a different definition of the research process and of its evaluation, with specific regard to applied research, but with an eye to basic research as well.
The need for a new approach to research has also been acknowledged by the EU Commission, which created a research action called Responsible Research and Innovation (RRI) .. The RRI is a strategy that anticipates and assesses potential implications and societal expectations with regard to research and innovation, with the aim to foster the design of inclusive and sustainable research and innovation (Mazzucato, 2018). RRI implies that societal actors (researchers, citizens, policy makers, business, third sector organizations, etc.) work together during the whole research and innovation process to align both the research process and its outcomes with the values, needs and expectations of society. The goal of this action is to make the objectivess of the research and researchers closer to the expectations of the society. In other words, the key question is: How can research contribute towards a more just, equitable and sustainable society, considering its impact on human and other forms of life? This approach cuts across all sectors. In very preliminary terms, we can consider healthcare and welfare, climate change and biodiversity loss, human and social rights: which is the real impact of research on these themes?
In broader terms, research assessment exercises can be expected to increase globally as both public and private sector universities compete for funding during and after the covid-19 epidemic. Scarce resources are allocated to research that is assessed to be impactful, and in this context societal impact is likely to become an important criterion.

This special issue aims to bridge debates about research assessment taking place in different scientific fields, as well as in research policymaking, with theories and ideas developing in the accounting and accountability field. In problematizing research impact the special issue will produce new ideas about how more progressive research accountability systems could broaden the scope of research assessment.
The special issue will also make a contribution to the accounting and management fields, where the issue of research assessment has thus far been examined mainly from the perspective of individual researchers (Gendron, 2008, Picard, et al., 2019). This special issue will take a broader stance, seeking to illustrate the limits to innovation, creativity and generation of alternatives that research assessment processes might entail.

Suggested topics
The aim of this Special Issue is to stimulate dialogue and debate on several key issues. But the main question is related to the need to reshape research evaluation approaches to clearly consider the impact of the research on society and to manage the challenges, which meet our societies in the next few years: the key challenges related to welfare, health sector, climate change, biodiversity loss, social and human rights. To be more precise, in the future the term ‘research impact’ could refer to its real impact on society, rather than its impact in terms of citations and inside the academic ivory tower. Of course, excellence in research is and will be a requirement, but how can research contribute towards a better society, for example starting from the SDGs or planetary boundaries?
Based on the above, key themes to be addressed in the special issue include, but are not limited to, the following:

  • How are the goals of research and innovation reflected in the performance measurement and accountability systems used to evaluate research?
  • Which are the implications of performance measurement and accountability systems with regard to the interests of different stakeholders in research? How could those interests be reconciled in different areas of research (health, natural sciences, social sciences, humanities, etc.)? How to identify the relevant stakeholders in each case and how can we take into consideration their goals and their idea of impact of research? The list of relevant stakeholders is often taken for granted, but it is also something that can be problematized and analysed.
  • How to conceptualize the target, methods and metrics of performance measurement and accountability systems used to evaluate research?
  • Which are the consequences of the use of specific performance measurement and accountability systems by funding bodies, research performing organizations and society?
  • How can we identify examples of multi-stakeholder accountability approaches to research?
  • How to characterize the interplay between stakeholder engagement and research accountability? How to engage stakeholders? Can all stakeholders be engaged?

Submitting and schedules

  • Paper submitted to the special issue will undergo a typical SAMPJ triple-blind review process
  • Submission to the journal must be made using ScholarOne Manuscript, the online submission and peer review system
  • The papers should be between 5,000 and 10,000 words (including references)
  • Authors’ guidelines can be found on the journal’s webpage
  • The submission deadline is 30 June 2021

Concannon, T. W., Meissner, P., Grunbaum, J. A., McElwee, N., Guise, J. M., Santa, J., et al. (2012). A new taxonomy for stakeholder engagement in patient-centered outcomes research. Journal of General Internal Medicine, 27(8), 985-991.
Funtowicz, S., & Ravetz, J. R. (1994). The worth of a songbird: ecological economics as a post-normal science. Ecological Economics, 10(3), 197–207.
Gendron, Y. (2008) Constituting the Academic Performer: The Spectre of Superficiality and Stagnation in Academia, European Accounting Review, 17(1), pp. 97-127.
Mazzucato, M., (2018). Mission-oriented research & innovation in the European Union. A problem-solving approach to fuel innovation-led growth. EU Publication.
Pedrini, M., Langella, V., Battaglia, M. A., & Zaratin, P. (2018). Assessing the health research’s social impact: a systematic review. Scientometrics, 114(3), 1227-1250.
Picard, C.-F., Durocher, S. and Gendron, Y. (2019) Desingularization and Dequalification: A Foray Into Ranking Production and Utilization Processes, European Accounting Review, 28(4), pp. 737-65.
Salvetti, M., Lubetzki, C., Kapoor, R., Ristori, G., Costa, E., Battaglia, M. Andreaus, M & Zaratin, P. (2018). Steps towards collective sustainability in biomedical research. Trends in molecular medicine, 24(5), 429-432.
Zaratin, P., Battaglia, M. A. and Abbracchio, M. P. (2014) Nonprofit foundations spur translational research, Trends in Pharmacological Sciences, 35(11), pp. 552-55.
Zaratin, P., Comi, G., Coetzee, T., Ramsey, K., Smith, K., Thompson, A. and Panzara, M. (2016) Progressive MS Alliance Industry Forum: Maximizing Collective Impact To Enable Drug Development, Trends in Pharmacological Sciences, 37(10), pp. 808-10.