Learning outcomes and assessment criteria
Outcomes-based courses and assessment criteria
The outcomes-based approach to teaching and learning is increasingly being used in higher education as the model for best practice in constructing courses and evaluating students' work. Learn more about this approach with this simple, practical guide to building your own outcomes-based programmes.
Developing outcomes-based learning programmes
The outcomes-based approach to course design is intended to make the expectations of the designer/educator more transparent to both the student and any regulatory or accrediting body. Unlike the traditional model of course design in higher education, where the lecturer would decide what to include on a syllabus, based on his or her own judgement of what was important for students to know; or on personal research or other interests; the outcomes-based approach starts with a specification of what the student will be expected to achieve by the end of the unit.
These learning outcomes may be of knowledge acquisition, mastery of skills, or development of attitude or ability. All the different outcomes expected will be specified in publicly shared statements and these will be linked in a clear way to explicit assessment criteria by which they will be measured. The programme is then written; complete with assessments designed to test the criteria, in such a way as to enable students to work towards achieving the stated outcomes.
The outcomes-based approach has been developed in conjunction with credit-based modular frameworks in which each unit carries a specified number of credits, awarded on its successful completion. In order to achieve the desired qualification, the student must amass a given number of credits, usually in stated proportions from different levels.
The Principles of Outcomes-based Course Design
- All learning can be expressed as demonstrable outcomes to be achieved.
- All units are described in terms of their learning outcomes and assessment criteria.
- The type and number of learning outcomes and assessment criteria form the basis for assigning a number of credits and a level to a particular unit.
- For this reason, no unit can be assigned to more than one level.
- Learning outcomes need to be clear and unambiguous.
- Learning outcomes set out the necessary learning, which represents the minimum requirement for a pass grade on the unit.
- Assessment criteria should specify how a satisfactory performance of the learning outcomes will be demonstrated.
- Assessment criteria should be designed to ensure that learning takes place at a level appropriate to the assigned unit level.
- Learning outcomes should contribute to the transparency of the overall qualification gained by enabling students, parents, prospective employers and other educational professionals to understand exactly what has been learned in order to achieve a passing grade.
- This will facilitate student and graduate mobility, internationally, and in a life-long learning context.
Adapted from How to Use Learning Outcomes and Assessment Criteria (David Gosling and Jenny Moon, SEEC, 2002)
A programme can be described by a specification written in terms of programme outcomes similar to the learning outcomes specified for the individual units within it. These larger outcomes are more general and they may not be specifically assessed as part of the course of study but they act as guides in establishing the ethos and direction of the programme. Where the programme aims for validation or accreditation to some larger standard, the programme specification should reflect the requirements of that standard and make reference to available benchmarks.
Writing good learning outcomes
The precision and appropriateness of the learning outcomes are the keys to successful implementation and assessment of outcomes-based programmes of study. Learning outcomes specify what the student is expected to know, understand or be able to accomplish by the end of any given unit of study, therefore they form the basis for all assessment of that learning and for any quality assurance checks, benchmarking or inspection exercise carried out on the unit.
Learning outcomes should not be confused with the aims and objectives of the unit. The two are subtly different. Aims are stated in terms of what is to be taught, and what the intention is behind that teaching; learning outcomes state what the student is expected to learn and have an implication for the standard he or she is expected to attain in order to pass the unit.
For these reasons, it vitally important to ensure that the learning outcomes set for a unit are as well-constructed and clearly written as possible. When drawing up these expected outcomes, they need to be:
The outcome must realistically set out what all students are expected to learn over the time period specified. This needs to be appropriate to the existing knowledge and abilities of students who are eligible to take this unit; and to the specified level the unit occupies within the overall programme (first year undergraduate, second year diploma, taught master's, etc.). All learning outcomes should, in principle, be achievable by all students at that level of the programme.
Learning outcomes do not specify areas of the curriculum but rather, the areas of general learning expected of the students. The outcomes sought are over-arching and do not match the headings or topics of a syllabus, nor should each curriculum area taught have a matching learning outcome.
As far as possible, learning outcomes need to be clear, sharp and unambiguous. Each outcome specified should be capable of only one single interpretation.
Linked to unambiguity is the requirement that learning outcomes be easily understood by all those who will be expected to use them. This group includes students, teaching and inspection staff, potential and current employers, etc. so use of technical or jargon-heavy language should be avoided and the outcomes expressed in the simplest manner possible.
Every learning outcome specified should refer to a significant achievement expected of the student on completion of the unit. This may not represent an exhaustive totality of what the student has actually learned but should include all those features of importance.
In order to determine whether learning outcomes have been achieved, they need to be capable of being assessed by a suitably qualified person, by some reasonable and manageable means, within the time-frame allowed by the programme or institution's regulations. Assessments should be designed so that all the learning outcomes are tested for all students. This often means setting more than one piece of assessed work per unit, and may require the development of different forms of assessment in order to cover all the different types of learning outcome specified.
All the learning outcomes for a unit must be achieved in order for the student to successfully complete that unit. The learning outcomes set out the minimum requirements for passing the unit. Additional 'desirable' outcomes can be specified as part of a grading scheme, allowing students to gain higher marks but these are not the learning outcomes for the unit. It is important to recognize that this approach to developing programmes separates grading of students' work from assessment of whether they have passed or failed to achieve. Learning outcomes are the baseline criteria for passing the unit.
In practical terms, a well written learning outcome will follow the following guidelines:
- It will be expressed in a single, simple sentence
- It will contain a verb that what the student is expected to be able to do at the end of the unit
- It will indicate on what or with what the student will be acting; or, in the case of a skill-based outcome, the way in which that skill is to be performed
- It will indicate what sort of performance is required of the student as evidence that the learning has been achieved
- It will use vocabulary and concepts appropriate to the broader requirements of the programme level at which the unit is pitched.
"By the end of this unit, the student is expected to be able to demonstrate a clear understanding of the different psychological approaches to the study of the individual within the context of management history."
This is a single, straight-forward sentence, using vocabulary appropriate to a second year undergraduate level unit, and including the following:
- "to be able to demonstrate" = verb
- "the different psychological approaches to the study of the individual" = what the student is acting upon
- "a clear understanding" and "within the context of management history" = nature of the required performance
Feedback within an outcomes-based framework
Although informal, verbal feedback is an on-going part of any teaching and learning situation, most educators become aware of feedback when they are assessing students' work. Assessment feedback is usually written, rather than verbal and is therefore accessible to a wider audience (including colleagues and inspectors/evaluators). In addition, many institutions have formal mechanisms for monitoring feedback. Knowing that the words you write about a student's work may themselves be assessed by someone else can be quite daunting but it does serve to focus the mind very clearly when composing assessment feedback.
There are two main purposes of assessment:
- evaluation of student learning against some pre-set, possibly external standard often at or near the end of a course of study (summative assessment)
- discovery of student strengths and weaknesses during the course of study, with a view to guiding and enhancing learning (formative assessment).
Assessment can only serve the latter function if feedback is provided as well as (or instead of) a grade or mark. Most higher education courses contain both elements and single assessment exercises may fulfill both functions.
Formalized systems and approaches may come into effect and there may be a requirement to give written feedback in a particular form or based on pre-determined criteria. The wording and structure of the assessment brief should specify if this is the case and ideally, a stated proportion of the marks awarded will be allocated for following this format.
Accentuate the positive
In order to act as a positive motivator for change, feedback needs to be constructive, comprehensible, honest and useable. Language used in feedback needs to focus on the desired and specified outcomes of the work, and not on the personality or attributes of the recipient.
Useable, constructive feedback should always suggest areas and means for improvement, rather than leave the student with nowhere to go. This applies to those at the top of the achievement spectrum as much as to those at the bottom. It is important to realize that simple exclamations such as: "Excellent! Outstanding!" are as unhelpful to self-development and improvement as: "Irrelevant! Nonsense!". Students at the top of the ability scale can often feel short-changed by cursory feedback that fails to offer them any sort of progression route.
Conversely, those at the bottom end of the scale can easily become disheartened and demotivated if they receive a relentless litany of negative comment and criticism for their work. It is generally recommended that even critical feedback is prefaced by a positive remark and criticism should be phrased in a neutral, unthreatening and depersonalized way. It should explain clearly what was expected of this answer and how the piece of work being assessed could be improved better to meet those expectations.
The "feedback sandwich"
One simple technique for writing feedback is to wrap negative comments up in a "feedback sandwich" with positive, encouraging remarks before and after them. Without deceiving students into believing that their work was better than it actually was, this technique ensures that the assessment is delivered sensitively and with due respect for the effort that has gone into it.
Where formative assessment in the form of detailed written feedback is accompanied (is it generally is) by a grade or mark awarded, it is vital that the feedback remarks match the grade.
As well as relating explicitly to the learning and assessment criteria for the piece of work in question, feedback will often have other specific functions with particular students, at different times in their studies. Sometimes the role of feedback may be more supportive; offering encouragement and spurring the student to greater efforts or increased self-belief and confidence. At other times it may be aimed at refining the student's critical abilities or offering detailed correction of an argument or data.
A final word of caution on the subject of feedback in assessment: it can be never-ending! Limit remarks to the key points of relevance, where the student can gain most benefit and learning. Develop means of streamlining the feedback process to make it a more efficient use of your time as well as a more effective learning tool for your students.
Assessment criteria are statements specifying the standards that must be met and the evidence that will be gathered to demonstrate the achievement of learning outcomes.
The purpose of assessment criteria is to establish clear and unambiguous standards of achievement for each learning outcome. They should describe what the learner is expected to do to show that the learning outcome has been achieved. They should not, however, be confused with the actual assessment tasks. Rather, the assessment criteria specify how the task will be evaluated.
There are three broad types of assessment criteria:
- Threshold standards tell the learner what must be done in order to demonstrate achievement of the learning outcomes of the unit i.e. what is the minimum requirement for passing this unit.
- Grading criteria provide a general description of the standard required for achievement of each pre-established grade, marking band or degree classification i.e. a first class honours award requires 70 per cent or more, an upper second requires 60-69 per cent, etc.
- General criteria provide general outcome descriptors that can be achieved more or less well. Students’ work will be judged to fall at a point within a performance range and marks are allocated accordingly. Typically, criteria of this sort are used to evaluate such things as use of referencing, accuracy of language, use of supporting evidence in drawing conclusions, quality of critical thinking, etc.
Writing assessment criteria
Many of the points to note when writing assessment criteria are the same as those for writing learning outcomes: clarity, unambiguity and brevity are all important and the language should be understandable by both teaching staff and students. The criteria must be measurable in a valid and reliable way and should concern themselves solely with those aspects of performance that are essential for achieving a pass or the specified grade.
The assessment criteria creation process:
- Writing assessment criteria starts with a consideration of the learning outcome being tested.
- Then this needs to be set along side the assessment task.
- Requirements for, or attributes of, successful performance of the task should be listed.
- If necessary, these requirements can be placed into context of expectations at this level of learning.
- The final criteria must focus on what is deemed essential amongst the requirements and these should be formed into clearly worded criteria.
- These criteria need to be checked to ensure that they are reliably measurable and clear in their intention.
- This process can be refined until a satisfactory set of assessment criteria has been created.
Assessment criteria should reflect the overall, published, aims of the programme. If, for example, the course claims to prepare students for entry into a particular profession, then the achievement of the entry requirements for that profession should be specified in the assessment criteria.
The criteria must be informed by the published learning outcomes of the module. They should not, however, merely repeat what has been stated as learning outcomes but must expand on these to make clear how and to what extent the student is expected to use particular skills or knowledge in order to meet these outcomes.
Assessment criteria should reflect the level of the module. Higher level modules will generally require more complex analytical skills and greater depth of knowledge than lower level ones. This must be reflected in the language used to write the criteria, with more descriptive verbs such as ‘define’ or ‘describe’, giving way to increasingly sophisticated analytical and critical ones such as ‘compare’, ‘evaluate’ and ‘critique’.
The criteria must reflect the distinctive epistemological characteristics of the particular subject or discipline being assessed.
Assessment criteria must be comparable to standards set in other institutions offering the same award. Whilst each course will have, and should retain, its distinctive individual features, the meaningfulness of any qualification depends on it representing the same value wherever it has been obtained.
Assessment criteria need to relate to the specific requirements of the assessment task i.e. they should describe the performance required for the task set. Oral presentation criteria will be quite distinct from the criteria set for an essay or portfolio.
Using assessment criteria
Assessment criteria are chiefly of value in so far as they enable students to focus their learning more effectively and make the assessment process more transparent and fair. For this reason, if no other, the expected outcomes and assessment criteria for any module should be discussed with students before they are expected to undertake any assessed work. Such discussions can be facilitated using the following structure:
- decide on the essential criteria
- make the criteria or checklist simple to use
- allow for brief global impressions
- give the criteria to the students before they do the assignment
- if possible, involve them in the design of the criteria and checklist
- encourage students to use the criteria.
The essentials of good criteria are that they:
- match the assessment task and learning outcome
- enable consistency of marking
- can pinpoint areas of disagreement between assessors
- help students to achieve the learning outcomes
- be used to provide useful feedback to students.
Critical to the success of any outcomes-based learning programme is the alignment between assessment methods, assessment tasks, learning opportunities and intended learning outcomes (learning objectives).
This alignment of assessment with other features of a course is the basis of course design. Effective assessment methods and tasks are related to the learning outcomes and the methods of learning. Close links between feedback, criteria and the assessment tasks enable students to achieve the learning outcomes of a course or a programme in a systematic fashion.
The outcomes used as the basis for assessment tasks can be either the programme- or the module-level outcomes. Assessing every outcome of every level in every module can lead to over-assessment of students. Assessing solely for programme outcomes, however, risks not assessing essential knowledge and skills in sufficient detail, although it does give a framework for estimating student progression and achievement. A successful strategy is to ensure that within each module, teaching and learning opportunities are provided which move the students closer to the programme outcomes and that some programme outcomes are assessed in some of the modules so that all are covered at least once over the duration of the course.
Methods of assessment in business and management studies
There is a wide variety of assessment methods available to choose from. Each method has its strengths and weaknesses and some are more suited to the evaluation of certain types of learning outcomes than they are to others. A combination of different assessment methods over a course or programme will allow for the testing of a wider variety of outcome types and help sustain students’ interest and engagement with the course.
The following list of assessment methods is by no means exhaustive and suggestions are listed in order of likely familiarity.
A standard method. There are several types of essays that test different styles of writing and types of thinking. They measure understanding, synthesis and evaluation, if the right questions are posed. They are relatively easy to set and grading based on impressionistic marking is fast. However, marking for feedback can be more time-consuming. Criteria are best kept simple.
Projects, Group Projects and Dissertations
Good all-round ability testing with potential for sampling wide range of practical, analytical and interpretative skills. Allows a wider application of knowledge, understanding and skills to real/simulated situations and provides a measure of project and time management. Group projects can provide a measure of teamwork skills and leadership. Marking for grading can be time-consuming. Marking for feedback can be reduced through peer and self-assessment and presentations. Learning gains can be high particularly if reflective learning is part of the criteria. Variations between markers is possible but use of criteria reduces variability.
These test preparation, understanding, knowledge, capacity to structure, information and oral communication skills. Feedback can come from tutor, self or peers. Marking for grading based on simple criteria is fast and potentially reliable. Measures of ability to respond to questions and manage discussion could be included.
Cases and open problems
These have potential for measuring application of knowledge, analysis, problem-solving and evaluative skills. Short cases are relatively easy to design and mark. Design of more complex cases and their marking schemes can be challenging. Marking for grading and feedback are about the same as for essay marking.
Work based Assessment
A variety of methods is possible, including learning logs, portfolios, projects, structured reports from supervisors or mentors. Supervisors and mentors need training in the use of criteria. Work experiences can be variable so reliability can be low. Validity is dependent upon clear learning outcomes.
Multiple Choice Questions (MCQs)
A standard method. This can sample a wide range of knowledge quickly and has potential for measuring understanding, analysis, problem solving skills and evaluative skills. More demanding MCQs require more time to set. Better ones are based on case studies or research papers. It is easy to mark and analyse results so they are useful for self assessment and screening with potentially high reliability, validity and manageability. Feedback to students is fast. The danger of MCQs is that they often end up testing only trivial knowledge. To save time, look for banks of items on the Web or in US text books. A team of assessors, working to the same learning outcomes, can brainstorm and produce several questions in an afternoon.
A standard method. Problems have the potential for measuring application, analysis and problem solving strategies but complex problems and their marking schemes can be difficult to design. Marking for grading of easy problems is fast. Marking for feedback can be slow. Variation between markers is fairly low when based on model answers or marking schemes. Allow for creative, valid solutions by bright students.
Short answer questions
A standard method with the potential for measuring analysis, application of knowledge, problem-solving and evaluative skills. Easier to design than complex MCQs but still relatively slow. Marking to model answers is relatively fast compared with marking problems but not compared with MCQs. Marking for feedback can be relatively fast.
Learning logs/ diaries
A wide variety of formats exists ranging from an unstructured account of each day to a structured form based on tasks. Some training in reflection is recommended. They are time-consuming for students and require a high level of trust between assessors and students. Measuring reliability is difficult. They may have high validity if the structure matches learning outcomes.
These can come in a wide variety of types, from a collection of assignments to reflection upon critical incidents. The latter are probably the most useful for developmental purposes. There is rich potential for developing reflective learning if students are trained in these techniques but they require a high level of trust between assessors and students. Measuring reliability is difficult. They may be high on validity if the structure matches objectives of training.
Much talked about. Usually software will be used to format multiple choice questions, mark and analyse results. A wider range of graphics and simulations can be used. Optical Mark readers can be used – but some students may still not mark the items clearly. They are time consuming to set but can be marked very fast. Reliability is high but validity (match with outcomes) needs careful attention. Like MCQs, it can be difficult to go beyond and evaluation of the trivial with these.
Single Essay Examination
Three hours on a prepared topic. These are relatively easy to set but attention to criteria is needed. They allow for a wider range of ability tested including the capacity to draw on a wide range of knowledge, to synthesize and identify recurrent themes. Students are able to show depth as well as breadth of knowledge and understanding. Marking for feedback is relatively slow. Marking for grading is relatively fast providing the criteria are simple.
Reflective Practice Assignments
Measure capacity to analyse and evaluate experience in the light of theories and research evidence. These are relatively easy to set. Feedback can potentially come from peers, self and tutors. Marking for feedback can be slow. Marking for grading is about the same for essays. Use of criteria reduces variability.
Test the capacity to present findings and interpretations succinctly and attractively. There is a danger of focusing unduly on presentation methods over content but this can be avoided by the use of simple criteria. Feedback potential exists from tutor, self and peers. Marking for grading is fast. Use of criteria reduces variability.
Modified Essay Questions (MEQs)
A sequence of questions based on a case study. After students have answered one question, further information and a question are given. The procedure continues, usually for about one hour. These are relatively easy to set and they may be used in teaching or assessment for developmental or judgmental purposes. They can be computer- or paper-based and they can encourage reflection and analysis. MEQs have potentially high reliability, validity and manageability.
Test communication, understanding, capacity to think quickly under pressure and knowledge of procedures. There is great potential for immediate feedback. Marking for grading can be fast but some standardization of interview procedures is needed to ensure reliability and validity.
Useful for assessing oral communication skills and for developing ways of giving and receiving feedback on performance. Video-recorded sessions take more time to prepare but are more useful for feedback and assessment. Peer and self assessment can be used. Sensitive oral feedback on performance is advisable. Assessment by simple rating schedule or checklist is potentially reliable if assessors, including students, are trained.
Objective Structured Clinical Examinations (OSCEs)
Initially used in medicine, this technique can be used in business, legal practice, management, psychology, science courses and social work. It is particularly useful for assessing quickly practical and communication skills. OSCEs are fairly hard to design and organise, but easy to score and provide feedback. Group OSCEs can be useful for teaching, feedback and developmental purposes. OSCEs can be used towards the end of a course to provide feedback or to test performance against outcomes. Reliability, validity and manageability are potentially fairly high. Probably less labour intensive than other forms of marking but several assessors are required at one time. Initially, they are time consuming to design.
Adapted from ‘Assessment: A Guide for Lecturers’ by George Brown (Learning & Teaching Support Network, 2001)