Evidence-informed approaches to teaching: what's the evidence?
By Margaret Adolphus
Imagine you are a reasonably successful teacher, who regularly scores well on student evaluation sheets. You recently introduced a new method of teaching on one of your courses which involved students debating online rather than, as is your usual practice, around a table. You put a lot of effort into the online debate, regularly weaving discussions and chasing up lurkers, and the indications are that the approach has been very successful. But, is the experience generalizable? Once you remove your own particular "x-factor", will it succeed elsewhere?
Much of the time with evaluation, you are concerned with a particular course, and with the teacher as much as the approach. But there are times when you need to take a broader view. You may be re-assessing your teaching and learning strategy, perhaps in response to external pressures such as changes to the syllabus, or student body, or a higher teacher-student ratio. In this case, you may want to take a more "scientific", approach to what works, rather than relying on anecdotal evidence.
Not so long ago a university teacher's expertise lay in their research; now they are also expected to be good, and flexible, teachers. The ability to generalize, and repeat in other sessions, is as important for valid teaching as it is for valid research. And research offers a basis for policy: objective evidence that a particular strategy can be effective.
Traditionally, the measure for research has been in publication and citation; while that for teaching has been student evaluations. But is this the most effective form of measurement, if not, then what is, and what constitutes the type of evidence on which policy can be based? Such is the debate going on in most industrialized nations.
There are a number of reasons for this:
- the huge explosion in student numbers;
- the greater concern with the testing of student achievement, and the evidence that it reveals;
- a tendency for students to act more as consumers, and to express vocal dissatisfaction with the education they are receiving; and
- the knowledge explosion powered by the Internet (Centre for Educational Research and Innovation, 2007).
In addition, there is increasing internationalism, as seen in the tendency for universities to link up to offer international courses and campuses, together with greater standardization in the form of initiatives such as the EC's Bologna Process, which seeks to create equivalence in member countries. And finally, there is the quality agenda: institutions having to provide evidence that what they are doing is working.
All this had led to research and teaching (in many countries considered two different, and at times competing, academic activities) coming together. According to Alan Jenkins of Oxford Brookes University, who has researched widely in the area, national policies and funding for research in both the US and in the UK has led to a "structural separation" between research and teaching, to the detriment of the latter (Jenkins, 2004).
This may now be changing: there is a universal concern for sound teaching strategies based on evidence of effectiveness, and teaching methods which encourage students to think like researchers.
The terms "evidence-based" or "evidence-informed" practice are used to describe approaches that critically look at teaching practice in order to provide solid evidence for what works, which can then inform policy.
For example, certain types of learning, such as problem- or enquiry-based, have become popular, but are they effective? Exams are being replaced by continuous assessment, but is this effective and what forms of assessment are best?
If evidence can be accumulated which is not simply relevant to one institution, then good practice can be developed for the sector, which can then become enshrined in policy. However, there is an inherent problem around the nature of acceptable evidence when applied to education.
As a concept, evidence-based practice derives from medical science which is located in the positivist tradition and favours quantitative methods. In the UK, for example, the National Institute for Health and Clinical Excellence laid down guidelines on a range of health-care issues which became policy. There is an obvious benefit for a busy doctor to be able to go to a source which, for example, provides the evidence backed by clinical trials for treatment of unstable angina, and the relative merits of coronary artery bypasses versus other forms of surgical intervention.
Education, on the other hand, is less amenable to a hard scientific approach; it is concerned not just with observable improvement, but with other more subtle and less quantifiable factors such as attitude and motivation. And there is also a considerable overlap with other disciplines, e.g. sociology and psychology. Useful research approaches are just as likely to be qualitative as quantitative: for example, ethnographic methods, focus groups, etc.
Oliver and Connole (2003) provide a critique of evidence-based approaches in the context of e-learning, which they consider to be particularly unsuited to a uniquely quantitative measurement, being rooted in a number of different disciplines and models.
Thus, the authors advocate a "more eclectic research paradigm", in other words, qualitative as well as quantitative methods, for example the use of ethnographic as well as experiment-based techniques.
Having established the need for a broad-based approach to empirical research, the question then becomes whether this is the only form of acceptable evidence. Mike Prosser of the UK's Higher Education Academy claims that while the growing body of pedagogical research is important, evidence can be defined more widely. He lists admissible forms of research as follows (Prosser, 2005):
- Research which makes a significant contribution to the body of knowledge, that is situated in the literature, and which enhances our theoretical and conceptual understanding. Presumably this includes empirical research, as well as conceptualization which draws on empirical research.
- Investigations and evaluations, which explore a local problem or issue, making recommendations, and which are also firmly linked into the literature.
- Literature reviews, where the literature is collected and analysed.
- Scholarship of teaching and learning, which is "evidence-based critical reflection on practice aimed at improving practice".
- Surveys of academic staff and student experiences of teaching and learning.
Although the rest of this article will be concerned with evidence-based practice in higher education, there are other aspects to the growing convergence between teaching and research. As well as an interest in teaching strategies based on evidence of effectiveness, there is also concern with ones that equip the student with the skills of enquiry.
Alan Jenkins has come up with the following definitions of the ways in which teaching can be linked with research (Jenkins, 2005). Thus teaching can be:
- Research-led: the research lies in the subject content, and follows the research interests of the staff. Thus it is the teacher who determines the research agenda.
- Research-oriented: there is as much emphasis on the process by which knowledge is acquired as there is on the knowledge itself, and students are taught enquiry skills and research methods.
- Research-based: here the curriculum is designed around enquiry-based activities undertaken by the students themselves, who thus acquire knowledge rather than have it "handed down" via lectures etc.
- Research-informed: here teaching is consciously drawn from research on the teaching and learning process.
A number of undergraduate programmes in the US, particularly in the top research universities, provide opportunities for students to carry out research projects, which may result in publication. These are often aimed at the brightest students, and those who may in the past have been under-represented, such as women in science or African Americans; they may also take place outside the formal curriculum, for example in a summer internship (Healy and Jenkins, 2009).
For more on teaching research methods, see the insight article, "Teaching research".
One of the problems with education research (at least that in higher education) is that many of the studies are small-scale and locally based, and thus difficult to generalize from. This is not the case with a number of projects which came under the aegis of the UK's Teaching and Learning Research Programme (TLRP) (http://www.tlrp.org/).
Funded by the Economic and Social Research Council, and with a programme spanning the first decade of the twenty-first century, the TLRP took a holistic approach to educational research with a number of projects covering the lifespan, from pre-school to higher education and lifelong learning.
In the area of higher education, a total of 60 higher education institutions of all types (i.e. both old and new, general and specialist), modes of study, and subjects, were investigated.
Showing the influence of the UK's diversity in higher education agenda, there were studies on widening participation and disabled students' experience. Topics were both domain specific (music and teacher education) and general: there was a comparative study of how the learning experience differed at different universities, and another on "congruence" between academics' and students' ways of thinking and practising. Particular approaches, such as problem-based learning, were also covered.
It is fair to say that the research approach was eclectic. Different conceptual and theoretical perspectives, as well as methodologies and designs from a range of disciplines were used, and interdisciplinary approaches were common. Both qualitative and quantitative data were used, and research often spanned several institutions. It thus fully meets the concerns of Oliver and Connole (2003) for an eclectic research paradigm.
From these projects, the commentary derived ten evidence-informed principles about effective pedagogy. These are quoted below (Teaching and Learning Research Programme, 2009, pp 14-19).
- ... demands consistent policy frameworks, with support for learning for diverse students as their main focus. Policies at government, system, institutional and organizational level need to recognize the fundamental importance of learning for individual, team, organizational, institutional, national and system success. Policies should be designed to create effective and equitable learning environments for all students to benefit socially and economically.
- ... depends on the research and learning of all those educators who teach and research to support the learning of others. The need for lecturers, teachers and trainers to learn through doing research to improve their knowledge, expertise and skills for teaching should be recognized and supported.
- ... recognizes the significance of informal learning to developing specific expertise. Learning with friends, families, peer groups and professionals should be recognized as significant, and be valued and used in formal processes in higher education.
- ... fosters both individual and social processes and outcomes. Students should be encouraged to build relationships and communication with others to assist the mutual construction of knowledge and enhance the achievements of individuals and groups. Consulting or collaborating with students as learners about their learning makes this effective.
- ... promotes the active engagement of the student as learner. The main aim of higher learning should be learners' independence and autonomy. This involves engaging students actively in their own learning, and ensuring that they acquire a repertoire of learning strategies and practices, develop positive learning dispositions, and build the confidence to become agents in their own learning.
- ... needs assessment to be congruent with learning. Assessment should be designed for maximum validity in terms of learning outcomes and learning processes, and also should be specific to the type of subject or discipline involved, even if it is interdisciplinary. It should help to advance learning as well as determine whether learning has occurred.
- ... requires learning to be systematically developed. Teachers, trainers, lecturers, researchers and all who support the learning of others should provide intellectual, social and emotional support which helps learners to develop expertise in their learning for it to be effective and secure.
- ... recognizes the importance of prior or concurrent experience and learning. Pedagogy should take account of what the student as learner knows already to plan strategies for the future. This includes building on prior learning but also taking account of the emerging concurrent learning in context, and the personal and cultural experiences of different groups of students as learners.
- ... engages with expertise and valued forms of knowledge in disciplines and subjects. Pedagogy should engage students with the concepts, key skills and processes, modes of discourse, ways of thinking and practising and attitudes and relationships which are most valued in their subject. Students need to understand what constitutes quality, standards and expertise in different settings and subjects.
- ... equips learners for life in its broadest sense. Learning should help individuals develop the intellectual, personal and social resources that will enable them to participate as active citizens, contribute to economic, social or community development, and flourish as individuals in a diverse and changing society. This means adopting a broad conception of worthwhile learning outcomes and taking seriously issues of equity and social justice for all, across social, economic, ethnic and gender differences.
The above principles are fairly general and endorse what are already fashionable approaches such as active learning and peer learning. Interestingly, research also revealed the importance of students learning the craft of their discipline as a whole, and criticized the way courses often focused on isolated module outcomes, at the expense of a more holistic outlook. (For example, understanding of the nature of historical research as opposed to focusing on a particular period or theme.) Problem-based learning, for which previous studies have concentrated on measuring accumulation of knowledge, was also revealed as dissatisfying to students.
The TLRP has produced an excellent commentary on its findings: Effective Learning and Teaching in UK Higher Education (Teaching and Learning Research Programme, 2009).
Gateways and repositories
For people who are interested to follow up particular areas and explore within a discipline, country or particular topic, there are a number of places which offer open access to useful resources. The Teaching and Learning Research Programme, for example, has a repository for all its publications, where you can search by research interest or by title, author or date: see http://www.tlrp.org/dspace/index.jsp.
EvidenceNet, supported by the UK's Higher Education Academy and Higher Education Funding Council, is a more comprehensive resource. It describes itself as "the place to come to find current evidence relating to teaching and learning in higher education", and interprets evidence in the broadest sense as comprising case studies and grey literature as well as large-scale research projects.
EvidenceNet has a good collection of resources, which can be searched or browsed by disciplinary cluster, pedagogic theme, or resource type. Pedagogic themes cover a range of issues from assessment and feedback to teaching and learning practices and strategies, from enhancing learning through technology to lifelong learning, from pedagogic research methods to the research-teaching nexus, from ethics to personal development planning. By far the largest category is equality and diversity (1,613).
As already stated, EvidenceNet is catholic in its approach to research with grey literature as well as published articles. One of the most useful features is the specially commissioned summaries and syntheses of particular topics so that you can get an overview. The content is owned by the community, which can upload and rate resources.
There is also an events section which allows you to search events by region (of the UK), by month and by theme, and one for network (categorized by theme).
The site is fairly comprehensive, but one drawback is a lack of obvious browse facilities: each of the pages has a search box and button, and it's not obvious that if you click the search button, another page appears with all the entries and the categories displayed in a side bar. Once you get used to it, however, it is an invaluable source of information.
Because it is UK funded, EvidenceNet's chief market is those working in UK higher education, the resource collection tends to be biased to the UK. For example, in a search for "inquiry-based learning" in the area of business, management, accountancy and finance, the first 30 items that came up included: two US-based studies, four from Australia, one Irish, one Europe-based, and one comparative study of Britain and South Africa: the rest were UK-based studies. The approach is to use international studies where these are sufficiently important. However, the repository is still of use to those outside the UK providing their interest lies in systems and approaches which are reasonably similar.
Screenshot of EvidenceNet resources website (reproduced by kind permission of the Higher Education Authority)
For those who are particularly interested in online learning, the Sloan Consortium Effective Practices (http://www.sloanconsortium.org/effective) is a useful repository to help people share techniques, strategies and practices. Users are encouraged to submit details, with evidence, of what has worked for them; the practices are then peer reviewed and are eligible for an award.
This repository has a very well organized front end which enables you to browse resources either by institution or by Sloan-C's own quality indicators, which are access, learning effectiveness, faculty satisfaction, student satisfaction, and scale (cost-effectiveness and commitment). However, search is site-wide and you can't confine your search to the repository; it is also difficult if what you are looking for does not fit into Sloan-C's own terms.
The OECD's Directorate for Education also has a repository which gives country-by-country information (not limited to OECD members) for higher education and adult learning: see http://www.oecd.org/infobycountry/0,3380,en_2649_39263238_1_1_1_1_1,00…. The collection of resources is varied, however, and covers education as a whole rather than merely teaching.
Evidence based, or informed, approaches, far from being a straitjacket, can provide a framework within which educational research can develop. Thanks to the work of the TLRP spanning across multiple organizations, we know that active learning works, that learning is social, and that a good learning experience equips one for life. Those who want to find out more about a particular approach have repositories where they can access material, much of it in the public domain. The scaffolding is there on which research into effective higher education can be built.
Centre for Educational Research and Innovation (2007), Evidence in Education: Linking Research and Policy, published by OECD.
Healy, M. and Jenkins, A. (2009), "Developing undergraduate research and inquiry", The Higher Education Academy, York, available at: www.heacademy.ac.uk/assets/York/documents/resources/publications/Develo… [accessed 13 April 2010].
Jenkins, A. (2004), "A guide to the research evidence on teaching-research relations", Higher Education Academy, York [accessed 13 April 2010].
Jenkins, A. (2005), "Strategies for linking teaching and research", Academy Exchange, Issue 2, Autumn 2005, available at: www.heacademy.ac.uk/assets/York/documents/resources/publications/exchan… [accessed 13 April 2010].
Oliver, M. and Connole, G. (2003), "Evidence-based practice and e-learning in higher education: can we and should we?", Research Papers in Education, Vol. 18 No. 4.
Prosser, M. (2005), "Supporting the student learning experience", Academy Exchange, Issue 2, Autumn 2005, available at: www.heacademy.ac.uk/assets/York/documents/resources/publications/exchan… [accessed 13 April 2010].
Teaching and Learning Research Programme (2009), Effective Learning and Teaching in UK Higher Education, available at: www.tlrp.org/pub/documents/UKHEfinal.pdf, accessed 7 April 2010.