The monitoring and evaluation process begins when you select your funding partner. That is when you review their needs and ethos and judge whether they match yours. After all, you would not want to be judged by someone operating with different and, perhaps, conflicting criteria.
Some academics will be familiar with this issue, just in their own discipline. How, for example, do different academics evaluate each other's work? An interesting paper on this subject published by Emerald explored this point in relation to American and European approaches to evaluation: "Research at the 'margin' – Challenges for scholars working outside the 'American-European' domain" by Peter G.P. Walters in the Department of Business Studies at Hong Kong Polytechnic University is worth a look.
Make sure you review the funder's explicit processes. Within different funding bodies, monitoring and evaluation processes will vary depending on the size and nature of the project. But, that shouldn't cause any confusion on the researcher's part as to what is important: the common thread is still 'have you delivered what you were funded to deliver?' That is, at the bare minimum, the formal, contractual arrangement.
Some awards in specific schemes are reviewed annually via an annual monitoring form, while some may only have an end-of-project review. Some may be peer-reviewed at their conclusion to assess how the aims and objectives have been met and how specific outputs have been created. If, for example, the award was to fund a book, has the book appeared? This is a process-driven review where the main question is: have you delivered what you were funded to deliver?
European Community projects have strict and frequent monitoring systems, where researchers may be personally evaluated by reviewers twice each year. This is to ensure that money is being spent appropriately, and deliverables that were originally planned for are being achieved and the project is kept on track. If the project is not running according to plan, the group will review why certain milestones are not they being achieved and what needs to happen next.
This is a completely different way of governing the spending of funds from, say, a small research council grant: As one reviewer expressed it –- "It's a very different model, but then again you're talking about significantly large amounts of money".
This is not, however, the strategic relevance of evaluation. Some may think a monitoring and evaluation process is how researchers are held to account for their work, but this is not the only function. Of course, at a programme level the funder must see that public or company money is spent responsibly. At the next level, the monitoring and evaluation process also may pick up examples of both good and bad research practice. If these begin to combine into a common thread, then they can be seen and addressed. But the function is not solely to perform a watchdog role, as one funder explained:
"This is not to force researchers to justify how and why they spent the money. I prefer to think of it in partnership terms. It can give us sufficient and significant information to allow us to explain what we do. It helps us in our representative role and allows us to be well-positioned to speak for the community to the wider society".
Monitoring also helps funder and researcher negotiate changes. The monitoring and evaluation process will reflect the quality of the relationship. Relationships are about reciprocity, about investing time and effort and giving and exchanging things of benefit to each partner. And, like all relationships, a funder-researcher relationship is not fixed and static – it needs to be able to change and grow over time.
After all, we undertake research because we do not know something. That is why as our knowledge grows, the project itself will grow and change. While change may be inevitable, it is how it is managed that is important.
Communication counts for everything in a project when these inevitable changes are unfolding. With communication, a relationship is nurtured and trust developed. Being clear about expected levels and forms of communication and feedback is vital.
Successful researchers will take the initiative to communicate with the funder, even if the funder does not appear to require it. One researcher says that his advice is for the researcher to communicate with them nonetheless, and set and communicate targets.
"It doesn't necessarily mean you'll stick to them because research never goes according to plan, but having no accountability throughout the project life cycle really doesn't do you any favours, to be quite honest".
The other purpose of evaluation is reflexivity. This is where the researcher has an opportunity to learn from the experience. Sometimes, it is hard to remember how the project changed over time from the original concept. The evaluation stage can prompt such reflection.
Have a look at how some funders show their criteria. The Economic and Social Research Council's end-of-award form asks the researcher to summarize the "aims and objectives of the research, noting briefly if these have changed since the original proposal". Think through this carefully in terms of what has been discussed above. By this stage, any changes should have been communicated to the funder to forgo any surprises. The form also asks the researcher to explain, in less than 200 words:
suitable for a lay reader, the findings and most significant achievements of the research. The latter might include: theoretical developments, new findings, new methods, new datasets, impact of the research on academics, policy-makers, practitioners etc.
Ask yourself these questions to help form your response:
Your research is only, as you know, the beginning of a process of learning and teaching. How will you disseminate the results of your research? You do not need a form or a checklist to know whether you have done good research; that is not the only purpose of evaluation. At the very least, any researcher will evaluate their own research by asking "what data do I have? What can I publish? What have I published?".
This may be a required output of your research, but it should certainly also form part of your commitment to both your personal profile and your research community.
Researchers need publication plans, ideally created at the beginning of the research project. In is guidance notes, the ESRC makes the point that researchers must involve the people who will be affected by the research outcomes:
"The Council's new mission places emphasis on ensuring that researchers engage as fully as possible with the users of research outcomes. These may be other academics, government departments, public bodies, businesses, voluntary organizations or other interested parties. Try to consult with and involve people who could make a valuable contribution to the research and who could provide support and interest. Try to do this in the planning of the project and build dissemination activities into the structure of your research plan rather than give them passing reference as an after thought at the end".
A plan also helps because the single most important success criterion in academic publishing is targeting the right publication. Just like unsuccessful research proposals, many unsuccessful journal articles and book proposals are sent to the wrong place. On average, half of all journal papers are rejected before they enter the review process, simply because they have been sent to the wrong journal. Busy people who do not have time to waste need to spend more time on choosing the right publication.
These issues have been fully explored in my previous book How to Get Research Published in Journals (Gower, 1996)
Think beyond your immediate research project to your potential continuing relationship with the same funder, or a new relationship with a different funder. Think about how your relationship with your research community will be nurtured, perhaps by becoming a member of a peer review panel. Most importantly, remember that winning research funding, delivering it, evaluating it and publishing it will help position you not simply as a good researcher who can be trusted to do a good job, but as a colleague and important member of the academic and wider community.
This exclusive series, written by Abby Day, is based on her book, Winning Research Funding, published by Gower/Ashgate, 2003.