Helen Beddow:
Research in the UK has a long tradition of sharing knowledge in innovative ways with businesses and communities. This two-way movement of ideas, experiences, and research outcomes between researchers and those that use it, has led to many innovations for new products, processes and services across many different sectors. Current assessments of university performance centre around research and teaching, and the valuable activities around knowledge exchange can be overlooked.
The Knowledge Exchange Framework, or KEF, aims to recognise and credit knowledge exchange efforts and foster a culture of continuous improvement around knowledge exchange. To find out more about the development and implications of KEF, today I’m joined by Helen Lau, Knowledge Exchange Manager at Coventry University, Hamish McAlpine, Head of Knowledge Exchange, Data and Evidence at Research England, and Chris Hewson, Research Impact Manager for Social Sciences at the University of York.
Helen, Chris, Hamish, welcome to the podcast, it's a pleasure to have you all on here.
Guests:
Hello.
Helen Beddow:
Perhaps a good place to begin, is around the concept of knowledge exchange. Hamish, could you give us a quick overview?
Hamish McAlpine:
What knowledge exchange means, is just the exchange of knowledge from a university or higher education provider, with someone who's not another academic. It's any kind of knowledge-based interaction with the wider world; be that business, public sector, third sector or indeed the general public. That encompasses a very wide range of activities: the very traditional kind of technology transfer, where you're taking an invention, protecting it via patent perhaps, and then exploiting that, creating a spin off company; or licencing that particular piece of intellectual property to an existing company - that's one type knowledge exchange. I think also goes right across into provision of skills training and continuing professional development to new education, through to the provision of specialist facilities and equipment, provision of public space, through to how universities support enterprise and entrepreneurship, through for example, helping graduates to start their own businesses, all the way through to public and community engagement. It’s a really broad conception, which encompasses not just the exploitation of research, but also how teaching plays a role in exchanging knowledge with the wider world.
Chris Hewson:
The key thing there is the word “exchange”. We are looking here at a lot of outputs from universities, so the exchange that we're measuring here is never going to be entirely 50/50. A buzzword at the moment in universities, is “co-production”: funders particularly want to see that research projects are co-produced and the research questions are co-produced because that's the best way of guaranteeing and planning for benefits to derive to those non-academic partners in a coherent way. And when we have seen these things being slightly too supply lead from institutions.
Hamish McAlpine:
I think that's right Chris and it's important to raise because, looking at it from an external perspective, the person trying to find a way into the institution, doesn't really care what your job title is with faculty X or department Y, they just want to be able to interact in the way they want with the institution, or want to find out whether that institution can help my community, can they help my SME overcome this production challenge we have? I think this idea of having a very simple external facing view of what knowledge exchange does, breaks down the internal structures and hopefully makes it a bit simpler for those who wish to interact with their institutions.
Helen Beddow:
And what is the Knowledge Exchange Framework? Could you outline for us what KEF is?
Hamish McAlpine:
KEF is the Knowledge Exchange Framework: it came about in late 2017, when the education minister, Joe Johnson at the time, asked [what was then HEFCE and is now Research England], to develop a new Knowledge Exchange Framework to sit alongside the existing REF (the Research Excellence Framework) and TEF (the Teaching Excellence framework). KEF is quite different from those two, and KEF primarily aims to provide data and tools to allow providers to understand, and hopefully improve, their own performance in knowledge exchange, compared to their peers. A secondary purpose, is being a useful source of information for users and potential users of university generated knowledge, such as businesses and the public, third sectors, and in some cases, the general public. Underpinning both those rationales is the need for greater public visibility and accountability for the increasing amounts of funding going into knowledge exchange at the moment.
Helen Beddow:
What's the importance of KEF? Why does the higher education sector need KEF? What’s its significance now?
Hamish McAlpine:
Knowledge exchange as a concept has been around a long time. It used to be referred to as the “third stream” or “third leg” of the stool alongside research and teaching. More recently, it's become so embedded, it’s such an integral part of many universities’ missions that it's important to recognise. One thing that I hope the KEF will do is bring some parity of esteem to knowledge exchange activities. They’re in no way the poor relation to research or teaching but they're absolutely integral to maximising the benefits of both.
Helen Beddow:
I think what you've outlined around the parity of esteem is really important. How is incentivization working currently and how do you hope to influence this with KEF? What else does KEF aim to do, how will KEF work with the institutional ecosystem?
Hamish McAlpine:
I think the increasing prominence of the impact agenda has helped enormously in recent years, and you see that reflected in the greater weighting given in recent REFs. But I think it's still quite variable; a lot of institutions recognise that knowledge exchange is a very important function for multiple reasons, and some institutions have taken really great strides to recognise this, for example creating a formal career pathway, which knowledge exchange is explicitly recognised in. But then for others, I guess it's harder. I hope one of the things the KEF will do - this parity of esteem thing - will make it easier for institutions to embed the recognition of knowledge exchange activities in their reward and recognition structures.
Helen Beddow:
And what else does KEF aim to do? What do you have to implement within the institutions?
Hamish Mc Alpine:
It’s important to note that KEF is England only at the moment, although we have a very close dialogue with our colleagues in the in the devolved funding agencies in Scotland, Wales and Northern Ireland. But what the KEF is at the moment is an institutional level, largely metrics driven, quite low burden exercise with this idea of a fair comparison as its heart. What we're trying to do is look at what assets and capabilities universities have to do knowledge exchange, and then look at a number of indicators that we think represent outputs of knowledge exchange, and then see how well institutions do compared to their peers. A really important point is this is an institutional level thing - this is not about individual researchers or academics. The other really important point is around the fair comparison, in that we have a very brilliantly diverse HE sector in England, and it would not make any sense to compare everyone to everyone.
Helen Beddow:
It seems it's really about using a wide range of indicators to build a picture of knowledge exchange.
Hamish McAlpine:
Yeah, absolutely. It's not just about knowledge exchange as technology transfer. Technology transfer and commercialization are a really important part of knowledge exchange and actually for many institutions they’re a very small part. We take a very broad view of what knowledge exchange actually is. And I think this is true of the UK’s approach as a whole, we generally take in a very enlightened view. We don't refer to this as knowledge transfer or technology transfer. This is really about exchange and I think that two-way idea of exchange is really important. That encompasses everything, not just about the exploitation of research, but knowledge exchange in our definition is everything from commercialization, tech transfer, exploiting the outputs of research, all the way through to how universities help provide CPD (continuing professional development) and upskilling opportunities in their local area, to how they allow others to use their specialist facilities and equipment, all the way through to community and public engagement and provision of public space. It's a really very broad set of things and what we're trying to do in the KEF, is try and have a basket of measures that represents the breadth of that knowledge exchange.
Helen Beddow:
Can you give us a couple of examples of the metrics for KEF?
Hamish McAlpine:
The metrics are grouped into seven perspectives which cover the breadth of the knowledge exchange. These are derived from a survey that's been running a long time called the Higher Education, Business and Community Interaction, the “HEBCI” survey. So for example in working with business, we're looking at volumes of types of interactions that universities have with business, e.g. volumes of consultancy, volumes of contract research, and within those, we also specifically split out the volume with large businesses compared to small and medium sized enterprises (SMEs). In public and community engagement as another example, a very different one, we actually have developed in association with the National Coordinating Centre for Public Engagement (NCCPE), a structured self-assessment for universities to self-assess their own maturity and how they approach their public and community engagement. The metrics are really tailored to the type of knowledge exchange we're looking at, and we’re trying to within each respective given number of ways that universities can show what they do. We're not trying to suggest that within a particular type of knowledge exchange there is one best way, one metric which will cover all types, we're trying to provide universities the opportunity to demonstrate how what they do is good.
Helen Beddow:
I think this is a good point to get the institutional perspective on KEF – Chris and Helen, from your experience within the institutions, what are the implications of KEF?
Chris Hewson:
One thing that's come out of the process for me from my perspective as a as an impact manager, mainly dealing with REF, mainly dealing with the aspects of KE around the social sciences, is it's been a way of bringing some of these questions to bear about what we are, as an institution. Hamish will give you more detail on this, I don't suppose this will come out very strongly in institutional KEF statements, because I think they're part of an ongoing process, we'll see a development of those, and what they say and how universities, choose to present themselves to the outside world. You've got to remember that impact tends to happen in small corners of institutions. When you then have to talk about this in the round, it kicks off lots of conversations, but it also turns into something else - it's a very iterative process. You start finding out about things that are happening in your own institution that maybe you should have known about. So I think there's actually a wider move across UKRI and we see it in other areas with, for instance, impact acceleration accounts, where there was a realisation that these mechanisms: yes they measure, yes they benchmark, but they also get institutions to think about their own internal processes, and that culture change can be can be nudged, if not forced from the outside. I think it's very interesting, from my perspective, and not being somebody who's directly involved in the KEF, who engaged in those activities relate to the KEF; I dare say Helen will give you a slightly different perspective on this.
Helen Lau:
I think, in terms for institutions across England, the way the KEF is built around seven different perspectives, one of the first things it does for us is it gives us the ability, once the results come out, to see what the specialisms are across different institutions. Having the seven different perspectives you'll be able to see, and institutions will be able to see, specialisms look at benchmarking and areas of expertise, but also start to think about how they will then fold that into their own knowledge exchange strategies, and ways of developing in the future. Hamish mentioned, intellectual property and commercialization or technology transfer; and for some institutions, that's a real strength; and other institutions might focus more on local growth and regeneration, or work with small businesses. It gives a way of looking at the wider piece of knowledge exchange and seeing where specialisms are, as opposed to generalisation. I also think it pulls into the fore, enterprise or knowledge exchange strategies in institutions for a wider conversation. Again, around benchmarking, strategy, and the future, and being able to compare across the sector what institutions are doing. And then strategizing for the future and thinking, this is where we want our institution to go; this is what is important to us in a wider KE landscape, because actually the separation of the perspectives does that quite nicely.
Chris Hewson:
I think Helen makes a very good point there, one thing we need to look at when the results are released, are where different institutions are in their journey, because I think some institutions will be talking about their strategy, what they've been doing, and some institutions will be in a different point in that cycle, they may be literally at the point where they're rewriting a lot of those strategies for various reasons. Maybe it's been a new senior management team or a change of direction for the institution. And of course you can throw COVID in there as well which is also affected how institutions think about particularly regional and local engagements. I think a lot of that will come out in the results. Benchmarking is quite interesting because universities have always had a sense of who their competitors are, but actually having the dashboards to play with, and to see that an institution you thought you were on a par with score a little bit higher, or higher in some areas and lower in some areas than you, will they will help your reflection about whether that's an artefact of the way that data was collected or there was actually something that you can you can do to address that issue.
Helen Lau:
Chris makes an important point, it's a journey that institutions are on, and an important part of that journey, if you like, a subset is the journey that institutions are on around public and community engagement, or civic engagement, and how an institution does that is very specific to institutions themselves, the area they're in, what the actual community is like around them, and the ways in which they engage, or choose to engage and how they're looking at developing that for the future. And I think that's really going to be quite interesting the way it's been pulled into the KEF, and the way different institutions, will look at that, particularly given this year, 2020, with COVID and going forwards into hopefully economic recovery. What institutions do, and how they play a part and the different ways that they play a part. It's going to be a really interesting journey for individual institutions, but as a sector, it will be really nice to see the different ways that can be achieved, and actually celebrate the diversity of the institutions, and the places where they're based as well.
Hamish McAlpine:
That's fantastic and that's music to my ears because neither Chris nor Helen, said league table. I think it's really important to emphasise from my perspective it’s not about a simple league table of who do we think is best. In fact, you talked about diversity, type of knowledge exchange, individual strengths of institutions, the ability to go away and compare and think about yourself, your own institution and where you are on a particular journey in a particular type of knowledge exchange is absolutely what we want to promote, so really great to hear that.
Chris Hewson:
One thing that KEF does is whilst it gives you data to benchmark, it also encourages universities to express their USP - their uniqueness. What is it about their area that makes it different? What is it about the way they set themselves up that makes them different? This is very useful because if you've gone and read a lot of university research strategies which I know you probably all have, they tend to say the same things about where they're positioned on various league tables nationally and internationally. By the KEF coming in and getting people to, in a sense, make public what their KE strategies are, and be encouraged to reflect on them for their own uniqueness without the pressure of them being ranked in a particular way. I think that can only be positive because that information will then be echoed across the institution into other forms of internal and external messaging.
Helen Beddow:
I really like the dashboard approach and how that will enable institutions to really investigate how different areas of knowledge exchange interact and impact on each other, and to see the various mechanisms and dynamics that are allowing them to achieve effective knowledge exchange, and also revealing where the barriers might be. It's a real diagnostic approach. Chris and Helen, how have you found the process of putting together the narrative statements?
Chris Hewson:
I was involved in gathering quite a lot of the information for the public engagement aspect, that was interesting in itself, because the institution at the time at York didn't have a unified strategy, it didn't need one there was perfectly good pockets of activity happening around the institution, but it was a very helpful process to start having conversations across those different bridges about what the commonalities are and what the common themes are. And there's a wider piece of work at the institution around capacity building, linked to various other things around the KEF, that really made us think about what we could say about our strategy how we could present that, but how we could present it again, as a journey with steps towards improvement to get us up that ladder. And I think that's, in many ways, a culmination of the NCCPE’s approach, because I think that most universities that had visits from NCCPE various people have engaged with their agenda we understand it, but I think this really takes it to another level.
Helen Beddow:
It's really interesting to think about what institutions will learn from writing these statements, Helen how have you found leading that process at Coventry.
Helen Lau:
At Coventry I lead on writing the three separate narratives. In the local growth and regeneration one, it was a fantastic opportunity to showcase some of the projects, and interventions that the university group has had across a number of sites in the UK, working with small businesses, and making a difference in those communities. What was really nice is by writing the narrative statements, and then circulating them in the university for approval, it really raised the profile of some of those projects internally, and has given us a great piece of literature which we can start to use and to circulate for our own purposes as well. It was nice to actually pull together success stories, and to have those conversations and to talk about them. With the public and community engagement statement, that was challenging in the fact that it's quite a short document, when you try to put into words, everything that your institution does and embodies around public and community engagement, and I think several institutions will feel the same way. Again, it was really nice to pull out some of the success stories to be circulating that, and to be talking about really positive things, and actually really nice to be talking about really positive success stories, during 2020 actually and be celebrating, saying, “look at these great things we've done that we're now going to tell Research England we've done”.
Chris Hewson:
My sense with the public and community engagement narrative is that it gave you a very good chance to dig out activities that you have no awareness of. It's part of the issue we often have with KE and impact is lots of these activities are hidden, because they're, for want of a better term, almost being done in academic spare time – they’re add ons but they're very valuable. And it could be valuable if the institution knew a little bit more about them.
The advantage we had in York, we are developing a new strategy to be a civic University, a University for the public good, but at the same time, as we're sort of creating these narratives so they’re both feeding into us in different ways. The one thing that we should look at as well are the advantages here around the KEF in bringing to the fore the activities and arts and humanities and social sciences that tend to get almost left off the radar of traditional KE or things that would never ever make it anywhere near the technology transfer office. These are useful ways, because it keeps other parts of the institution interested in the overarching questions of KE strategy and what the KE landscape looks like, how it's changing and how it links to the R&D targets, that otherwise they would be slightly disconnected from.
Helen Lau:
Similarly, actually for Coventry the writing of that public and community engagement narrative really nicely dovetailed into writing of a new strategy for public and community engagement for the institution. Where a lot of focus has gone more on to the communities where we work, and how we engage with them, and what the needs are in those communities. A separate conversation that's feeding into that, about how we can anticipate those needs, are going to change over the next couple of years. And what that's going to mean in terms of health inequalities or unemployment in our campus areas. I presume that's going to be very similar for a lot of institutions that have submitted to the KEF; we've written these narratives at the same time as something huge is going on and actually it's been a good opportunity to pull all of that together and feed that information in. I do think that narrative particularly has been really good for pulling out some of the social sciences, or other types of projects in a range of institutions where actually they're not necessarily the headline projects that people think about when they think about enterprise or knowledge exchange, but it pulls them forwards and it gives them a limelight which is well and truly deserved because there's some excellent knowledge exchange that happens in that public and community engagement space.
Helen Beddow:
Hamish, is this what KEF were hoping for from the narrative statements exercise?
Hamish McAlpine:
Essentially the, the submission of narratives being voluntary this year we've had, by far, the majority of the sector, engage and we've seen a really high uptake of them wishing to submit narratives. And I think this is because the KEF, in my view provides an opportunity for those small but excellent institutions to really show there's some really excellent stuff going on. I've been hugely gratified by the response we've had to the call for narratives.
Just to explain the narrative element: some perspectives, we are taking a purely metrics based approach, but for some of the perspectives, we didn't really consider the existing metrics that we have really to be sufficient to explain the full range of performance and the stuff that institutions are doing. The full narrative element of the KEF is particularly focused on two perspectives: the local growth and regeneration perspective, and the public and community engagement perspective where it's very difficult to capture the huge breadth of activity in a number of metrics. In addition, I think in local growth and regeneration, I think there's been a lot of focus on the role of universities in their local area, not least through the work of the UPP Foundation, the Civic Universities report, and the continued work they're doing in that area. I do think there's perhaps not as such a well developed conceptual framework of what good looks like in that area. So local growth and regeneration, and indeed civic, which I think is even broader than that, encompasses a huge amount of variety of activities. What's been great about reading the narratives in those areas, is we're starting to see patterns of types of activity. In the fullness of time we can do some real analysis on those narratives and start to form a better idea of what good looks like for local growth and regeneration.
Helen Beddow:
So what is good practice around knowledge exchange. What does good look like?
Hamish McAlpine:
Well, I think the metrics that we currently use in the KEF are designed to represent what we think is multiple ways of expressing what good is. It’s worth pointing out that quite a number of the metrics are based on income received by the university for their knowledge exchange activities. And we're not saying that income is the be all end all of knowledge exchange, it’s certainly not, but we use income as a proxy for the kind of impacts generated. So for example, income from contract research or consultancy with business, they'd be very unlikely to pay for something if they didn't think it had any value to them. We've done quite a lot of work to establish the income is a sound proxy for many types of knowledge exchange, and I think it's still important to say. I'm keen to emphasise that this is not about simple league table and Helen and Chris are absolutely right, it's about allowing universities who express the areas in which they have particular strengths in a very diverse sector. I think at the heart of it, there's no getting away from the fact that if you're well below your benchmark as an institution in every single area of knowledge exchange, you might question whether as an institution, you're really taking that seriously enough, and you may want to think about how to improve your performance in the future.
Helen Beddow:
So this is primarily an institution based exercise, but what does KEF mean for individual researchers, what do they need to be aware of?
Hamish McAlpine:
To be absolutely clear and to try and reassure a researcher who might be thinking, “what does the KEF mean for me as an individual?” “What might I have to do, is this going to be another thing that I have to spend time doing?” I don't think that's the case: all of the metrics are derived from existing returns, so there's no new burden of data collection to feed those metrics. The institutional statement and the various narratives associated with this are fairly short, and they're designed to be written by an institution. So a researcher might be called upon to contribute information about a project or something like that, but this is not an exercise in a way the REF is where there's emphasis on what individual researchers and individual academics are contributing in terms of their own outputs and outcomes. I think there's a little downgrade in terms of workload and potentially a reasonable upside for individuals in terms of the recognition they might receive for these valuable activities.
Chris Hewson:
I think the thing that comes out to me, strongest of all is the relational nature of KEF. Granted, there are some transactional elements in there. It's all about relationships: what's the relationship between inside of academia and thinking outside academia, and they can be measured in certain ways, so that the balance of supply and demand is carefully positioned? REF is very much still academics speaking to academics and to a certain extent, marking our own homework, although obviously there are users involved in grading impact case studies. The interesting question for me, and I'm not sure whether this is something that UKRI are going to be looking at, is do we sit there and start looking at REF environment statements? What do they say about knowledge exchange, open research, the porosity of the institution, the individual strategies of submitting units? And try and cross reference that to university REF strategies. Of course the delay has been interesting, because the delay is created a little bit of time to start thinking about some of those things as submissions are finalised for the REF towards spring 2021. I'm not sure whether Hamish has any strong views on that?
Hamish McAlpine:
I think Chris raises an interesting point about the relationship between REF and KEF, and I've covered in a previous blog with Stephen Hill, the Research England director of research, how we see the key differences between REF and KEF. I think it's also really important to note that Research England administer both the KEF and the REF on behalf of England and devolved nations. So in more normal times or when I'm in the office, I'm sitting feet away from the REF team. So there has been interaction throughout this process, and I expect there to still be interaction, particularly when we start reviewing the first iteration of the KEF. While we’re very comfortable about the relationship between a REF impact case studies and what they're trying to achieve, and what the KEF metrics are trying to show, I’m very comfortable there's a very different nature and purpose of those things. I think, a really interesting point, Chris made around the REF environment statements, and it will be super interesting to see how there might be some crossover there how one might reference the other for example, particularly when we're talking about institutional environment, and it'll be interesting to see how that develops and I do think there's opportunity for us to reflect on that and refine that in future iterations of the KEF.
Helen Lau:
So the KEF is, very much like Chris said, individual researchers should look at it from their own institution perspective. I would encourage anybody working in academia to look at the institution you're in, and the way in which that institution has responded, from a metrics perspective into the KEF but also the narrative statements as well, because actually that will tell you far more about what the ethos is and the strategy for knowledge exchange in your institution. Thinking about how that can then be a way of playing your own research or specialisms into knowledge exchange in your institution going forward. So specifically, because the perspectives are much wider, and all encompassing, than necessarily the REF is with the research focus, there's a focus on CPD activity, also a focus on entrepreneurship activity, and also public and community engagement focus, all driven by the strategy of an institution and the way in which the institution is developing its own knowledge exchange, but in quite a broad way and different institutions will have a different focus, across those seven areas. They will have at the moment and they might be looking to maintain or change that into the future. Also, early in 2021, the institutions which sign up to the development year of the knowledge exchange Concordat: The list of that those institutions will be available. This is also looking at the ways in which institutions put forward an action plan to develop their own management and administration of knowledge exchange, including rewarding individuals who are working on knowledge exchange, which could be individual researchers, and how that's done in different institutions. I think if you were individual researcher, looking at knowledge exchange, there's actually about to be, at the end of 2020 and the beginning of 2021, much more information available to you about the institution you're in, and other benchmark institutions, about actually what is happening in the sector to then allow you to think about how that might affect your own research, and what you might actually be able to do, which fits inside this other wider framework.
Helen Beddow:
Helen, you touched on the knowledge exchange Concordat, can you explain a little bit more about that and how it fits in with KEF?
Helen Lau:
The knowledge exchange Concordat has been in development, alongside the knowledge exchange framework. It’s more sector ran by Universities UK, and Guild HE, checking and looking at the development of eight core principles of knowledge exchange and institutions can, at the moment, sign up to a development year for the Concordat and how that will work longer term in practice. The focus is very much though, on individual institutions, looking at their own knowledge exchange, the governance, the policies, management, where the focus is, and developing an action plan for how to improve build and grow from that into the future. So it's very much focused on each individual institution, which then interlinks quite nicely with the way that KEF is separated across perspectives, is that in knowledge exchange this gives us a good opportunity to, instead of compare universities against each other in a really broad way to actually recognise the individual characteristics and strengths of different types of institutions, and to them the Concordat will really allow us to continue to build on that, and to create action plans which pull together the strategy for knowledge exchange in that institution on where the focus will be to continue building strength or create new strength into the future.
Hamish McAlpine:
My very simple take on the relationship between the KEF exercise and the Concordat is two important differences, although I do see them as two pillars of really high performance knowledge exchange. I think the first difference is the Concordat is much more about looking at the internal governance and processes and structures of universities and thinking about how an institution goes about doing something, whereas the KEF is much more about what the institution is doing what are the kind of manifestations, what are the outputs of that internal organisation and that internal effort. I think the other important point of differentiation, is that the KEF looks back, it looks at what has been done over the last couple of years. The concordat is very much a forward looking exercise, it’s to say: Where are we now in 2020? And where do we want to go in the future?
Helen Lau:
I think the award and recognition piece from a KE Concordat is a really good place to start from an individual researcher perspective, particularly because when you look at knowledge exchange you're looking at a broad variety of different things. The focus of knowledge exchange in one institution could potentially be intellectual property and licence income, but in another institution, it could be local growth and regeneration and the number of SMEs to small businesses an institution works with. I think then as a researcher or individual, looking at institutions your own institution or if you were developing in your career to move as well, it's about seeing where those specialisms are and then looking at how institutions are going to develop appropriate reward and recognition schemes and activities that fit within that. Historically, it's been quite easy for individuals or groups in the sector to think along the lines of knowledge exchange which creates large profit margins and actually how is that then rewarding the individual researcher or academic stuff. With knowledge exchange now, we're looking at a much broader range of activity,
Helen Beddow:
Helen it's interesting thinking about return on investment. There are a wide range of activities that institutions or researchers do that do bring in income eventually or support income generation in ways that is hard to quantify. So, how is that wider thinking about return on investment important to support recognise these?
Helen Lau:
I think it's very difficult because you think about return on investment in the wider scope of knowledge exchange, you start to think about public and community engagement and local growth and regeneration. It can very quickly be so associated with the place with which your institution is based, because the prosperity of the place that your institution is based is going to drive a lot of local behaviours. If you're looking at things like health inequalities and local unemployment rates, those are intrinsically going to affect the communities that you're engaged with, and the way in which your institution can affect those, and then, how you then calculate return on investment is quite complicated, because it's in an institution's best interest to support and have a thriving community around it. But actually, a lot of the time, institutions need to fully engage with that to support that activity to happen as much as possible.
Chris Hewson:
This is probably something that Hamish will pick up on but it implies that we need to develop other sorts of softer metrics. The original version had in kind contributions to research partnerships and that that was taken out because it was realised that what the way these things are measured, just wouldn't help any kind of benchmarking whatsoever and from personal experience, I can agree with that. But it doesn't mean we should throw the baby out with the bathwater I think there are probably ways to bring some of the value that that's given from outside to university endeavour that actually brings both parties up to a higher level than it would have been working in isolation. There are lots of ways of working with business that aren't included in those metrics that we can have a think about. Again, COVID has made us think about the needs of our businesses. Our localities have changed radically over the last six to nine months, mainly bad, but actually not all bad, new opportunities have arisen in this uncertain time, and the way the university engages with those businesses can sometimes be missed off all forms of measurement.
Hamish McAlpine:
I do think we have to be slightly careful to remember that not everything an institution does, however, good, and worthy or brilliant is necessarily what I would call knowledge exchange. We have to keep in the back of our minds this focus on knowledge exchange, and by that something exchanged or shared, which is knowledge based on the university being a university, and having this specialist knowledge. It might have a particular role in convening different entities in their local area. And I think it can do that because it's a particular type of institution, they may have a particular type of academic expertise. My first point is we can't just use the KEF as a vehicle to measure absolutely everything good universities do and turn it into a kind of advertising tool. On the question of these hard to measure things, and my kind of generic approach to that has always been to think about a lot of the metrics in the KEF, to be quite frank, are not about the ultimate impact or outcomes are already talked about using income as a proxy for that impact. So my starting point for thinking about how might we capture more types of knowledge based interactions, more types of knowledge exchange in future iterations of the KEF, is to think about, “what's your theory of change?”. “How do you get from your activity to an output to an outcome to this eventual really good impact?”. And if you understand that you can start measuring earlier on down the chain, and those type of things which I call projectory measures, they're not the ultimate outcomes and impacts, are they are giving you a good indication that we can do enough with that stuff you're writing to get some good outcomes and impacts from your activity. And that's the type of thing I'd be interested in. They also tend to be lower burden to capture and they also tend to be much more immediate, so you don't have to wait 10 years to realise the benefits from. But with that low burden requirement also comes the need for metrics to be robust enough to potentially distribute funding on the back of. So we do have to apply a reasonable bar in terms of the robustness and the auditability of the metrics we’re gathering.
The other area where I think we're beginning to think about quite heavily is issues around equality diversity and inclusion and knowledge exchange. So alongside better measures around policy engagement, or knowledge exchange with arts and cultural organisations, the other area where there's frankly embarrassingly little data is around EDI in knowledge exchange. We know, for example, there's quite a gender imbalance in a university based spin out companies, so only about 13% of spinouts have female founders. We also know there are imbalances in the wider funding of start-ups and spinouts, so this isn't particularly about University spinouts, but the wider population of new companies, only about 1% of venture capital funding in the UK goes to all female founding teams. If we look at the ethnicity of founders, looking at black female founders, only about 0.02% of venture capital has gone to black female founders over the last decade in the UK. So important to emphasise that it's not just University spinouts, that's the whole population of start-ups in the UK by suspected to be a similar picture when looking at a university based spinout companies. And that's got us thinking about how whether the KEF would be an appropriate place to start surfacing information on not just gender and ethnicity, but other protected characteristics as a way of beginning to shine a light on those potential imbalances, and therefore how we might go about addressing them. I'm keen to emphasise that I think this is an interesting area to explore one conclusion could be that such issues might be better addressed through other mechanisms such as the KE Concordat, and the action planning institutions do internally, but being, obviously the role I’m in means I'm a great believer if you don't have the data on these things it's very hard to make progress. So I do think we should look very seriously at whether EDI type data and knowledge exchange could be incorporated in future iterations for the KEF, that's certainly something we'll be doing more work on over the coming months.
Helen Beddow:
As you all touched on this is the first iteration of KEF. How do you see it developing in the future?
Chris Hewson:
Ultimately, this is a tremendous resource that can be used to shape and generate new and interesting narratives around what the sector does. By resource I mean Academic Resource, I think there’s a whole lot of secondary analysis, could be done on some of these narratives will be absolutely fascinating. I think one of the great shames of REF in 2014, was we said we'd have all these examples of impact that we could look through and get a sense of what was going on that I don't think it even touched the scides, we had this post REF, rough analysis, obviously all these all the documents are public documents, but the level of analysis wasn't what you'd expect with all that all the effort put into creating those. Maybe it was because there was a distinct lack of coherence. In a sense you have 7000 individual case studies that say their own thing. I think with the KEF narratives I think there's some really interesting pieces of work, particularly around public engagement we can embark on, so it could become greater than the sum of its parts. I don’t know if Hamish can say anything about what for instance NCCP are going to go to do with some of this information?
Hamish McAlpine:
You're right, I think they are, they will be a really interesting resource, I think both the metrics as they're going to be expressed in these interactive very visual dashboards and also the narratives will all be publicly available. I think because they're quite heavily structured, they open up the possibility for more structured comparisons between approaches. So a very basic level of people within institutions, for the first time have a single source if you like single source of structured information about what other institutions are doing in the area of public engagement. I mean, I have to be honest. One thing that's sort of first and foremost in my mind is has everyone scored themselves realistically,? One of my fears when we chose that approach to public and community engagement perspective, was they’d be rampant grade inflation, and people who played high score from what we didn't think look like, state of the art or really excellent activity. Having done some very early analysis on those scores and the range of scores I don't think that's the case. And I think the message has to got across that if everyone scored themselves 25 out of 25 that metric frankly would not survive the review of the first iteration. I do think people are approaching this in kind of honestly, and in the spirit of the KEF it has been a useful tool, not just a means to beat institutions up about their performance. We are going to continue to work with NCCP to see how we can improve the methodology in the future, and potentially again similarly to the local growth and regeneration narratives, look across the kind of ways that people are expressing what good looks like, how they know they're getting good results, how they know that they're meeting the needs of their various publics and communities. And again, seeing whether there's potential for more systematic collection of light in the future.
Helen Lau:
So I think it's different for each institution, the public and the community around that institution is different, by virtue of the place being different. But I think institutions need to look beyond the members of a community that they engage with and researchers need to think beyond just engaging with members of the community, and thinking actually about making co creation into research activities. But from institutions institutional strategy that can actually look at how you work in partnership with other large local institutions to then work and create a difference and share knowledge in that wider community. As we go through the development of the narratives, over time, and as you read through them through points 1-5, you should hopefully get a feel for how an institution is going beyond pulling individuals into research projects, and actually expanding, how the wider knowledge of the institution is shared with the public and with that local community as well. And that's again coming back to what the strategy is in each institution to do that and how to do that, which then is different depending on the place. I think the public and community engagement narratives are just going to be, and designed to be in my opinion, just so specific to that institution, and how the institution, strategizes, and values and feels about that activity.
Helen Beddow:
Yeah, thinking about that, and about public engagement. There's also a question of who you are engaging with. Our communities are very diverse, with many different groups. Are you identifying those at the outset, already invisible? Are there any you're missing? How do you plan to reach these groups, what's the best way to communicate and gauge what they need?
Hamish McAlpine:
Absolutely and the template was really set up and we talked to NCCP a lot about this. It’s set up in a structured way which takes the reader on a journey through that very question. If you don't know who your public are and your communities are and what they need. How do you know if your strategy is any good? How do you know what activities to do? How do you know how to evaluate these activities? So absolutely the template is structured in a way which we think represents good practice in the area, and then really importantly in section five of that template. That's about the organisation articulating how they use those results and how they build on those results how they close the loop to how they use that to continuously improve and refine their, their approach to public engagement.
Helen Beddow:
Thank you so much for talking to us today Helen, Hamish and Chris. And for talking us about KEF. That was a really fascinating conversation.