Keynote at the ATN Assessment conference 2008

1. Constructive Alignment in Learning, Teaching and Assessment John Biggs Catherine Tang ATN Assessment Conference: “Engaging Students in Assessment” U of SA, 20-21 November, 2008

2. Activity 1 As a teacher, what do you want to achieve in teaching? Share your views with your colleagues.

3. The focus in teaching is not what we teach but what we would like our students to learn and how we can help them achieve that. The first step therefore is to define the intended learning outcomes for our students. Teaching and assessment are then designed and implemented to align to these outcomes.

4. The Intended Outcomes of This Session 1. Explain constructive alignment. 2. Identify intended learning outcomes for one of your teaching units. 3. Design teaching/learning activities to best achieve one of your intended learning outcomes. 4. Design tasks to assess how well the intended learning outcome has been achieved. 5. Reflect on the impact of this session on your teaching and assessment.

6. Implementing Constructive Alignment in Learning, Teaching and Assessment ILO: What the student has to learn Teaching: Engaging the student in the verb in the ILO Assessment: How well the student has achieved the ILO

7. Constructive Alignment Intended Learning Outcomes (ILOs)‏ expressed as verbs students have to enact A The very best understanding that could be reasonably expected: verbs such as hypothesise, apply to “far” domains, generate, create, critically review etc. B Highly satisfactory understanding: verbs such as explain, solve, analyse, evaluate, justify etc. C Quite satisfactory learning, with under- standing at a declarative level: verbs such as describe, elaborate, classify etc. D Understanding at a level that would warrant a Pass: low level verbs, also inadequate but salvageable higher level attempts. Teaching / Learning Activities Designed to elicit desired ILO verbs May be: Large class activities Small class activities Teacher-managed Peer-managed Self-managed Classroom-based Outside classroom as best suits context Assessment Tasks Format such that the target verbs are elicited and deployed in context. Criteria clearly allow judgement as to the quality of the student's performance

8. Designing Constructively Aligned Teaching and Assessment There are four steps in designing such teaching and assessment: 1. describe intended outcomes in the form of standards students are to attain using appropriate learning verbs. 2. create a learning environment likely to bring about the intended outcomes. 3. use assessment tasks enabling you to judge if and how well students’ performances meet the outcomes. 4. develop grading criteria ( rubrics ) for judging the quality of student performance.

9. Intended Learning Outcomes (ILOs) University level What are the attributes of an ideal graduate of the University? Programme level What are the intended learning outcomes for students enrolled in the programme? Subject/Unit level What are the intended learning outcomes for students taking a particular subject/unit at a particular level within the programme?

10. Intended Learning Outcomes (ILOs) . Statements of what students are expected to be able to do as a result of engaging in the learning process (studying a subject/programme). . ILOs should reflect the level of the programme / subject. . Expressed from the students' perspective. . Expressed in the form of action verbs leading to observable and assessable outcomes. . Related to criteria for assessing student performance.

12. Procedures in Designing Subject ILOs 1. Select the topics to be taught. 2. Decide the levels of understanding/performance the students are expected to achieve for the different topics. 3. Consider if all the ILOs are of equal importance. 4. Ensure a clear understanding and agreement of the ILOs within the teaching team and other relevant parties e.g. External Reviewer. 5. Communicate the ILOs to students.

13. The SOLO Taxonomy with sample verbs indicating levels of understanding Competence Prestructural Unistructural Multistructural Relational Extended Abstract one relevant several relevant integrated into generalized to aspect independent aspects a structure new domain Incompetence Fail Incompetent Misses point Identify Name Follow simple procedure Combine Describe Enumerate Perform serial skills List Analyse Apply Argue Compare/ contrast Criticize Explain causes Relate Justify Create Formulate Generate Hypothesize Reflect Theorize . . . .

14. Some vague ILO verbs – to be avoided Appreciate Become aware of Familiarise with Know Learn about Understand These verbs don’t tell the student or the teacher how they would know if the ILO has been met. ILOs need to specify a standard of performance.

15. Activity 2 - Writing Subject ILOs Take a subject that you are teaching. Consider the subject aim and write the subject ILOs by identifying: 1 . The content or topic to be learned. 2. The intended level of understanding/performance to be achieved. Now go across the rows and write out the subject ILOs by stating the content and the intended level of understanding/performance. Subject ILOs: 1. 2. 3. Content / topic Intended level of understanding /performance

16. Programme and Subject ILOs Alignment between the programme and subject ILOs 1. Are the ILOs aligned? 2. Do the subject ILOs appropriately address the programme ILOs? 3. Are there any gaps? Programme ILOs Subject ILOs Subject 1 Subject 2 Subject 3

20. Activity 3 – Designing Teaching and Learning Activities Refer to one of the subject ILOs that you have identified in Activity 2, design TLAs to align with the ILO. Subject ILO: Now double-check if the student learning activities are aligned to the verbs nominated in the subject ILO. Share your ideas with your colleagues. Teaching situation Teaching activities Learning activities (What the teacher does) (What the students do)

21. Assessment Tasks (ATs) . Provide students the opportunity to demonstrate whether or not they have achieved the ILOs and what level their performance is in those ILOs. . Should be appropriately designed or selected to address the ILOs that we want to assess. . Different assessment methods (tasks) address different ILOs There should therefore be several kinds of task. . Provide the evidence allowing teachers to make a judgment about the level of a student’s performance against the ILOs and to award a final grade.

22. Constructive Alignment of ILOs and Assessment Tasks Subject ILO 5 Subject ILO 4 Subject ILO 3 Subject ILO 2 Subject ILO 1 AT 3 AT 2 AT 1 Assessment Tasks Subject ILOs Consider if : all ILOs are being addressed? there is a balanced coverage of the ILOs? the more important ILOs are given appropriate assessment emphasis.

23. Designing Assessment Tasks (ATs) Steps: 1. Select a practicable task that embodies the target ILO verb. (Try using the TLA as an AT first). 2. Make a judgment on how well the ILO has been met by the students' performance in the ATs – developing grading criteria (rubrics).

24. A range of different assessment tasks may be required to address the range of ILOs of a subject. Questions to be asked in selecting assessment tasks: 1 . Are the assessment tasks aligned to their appropriate ILOs? Are the students required to engage in the verbs identified in the ILOs? 2. Are the assessment tasks practicable with respect to available time and resources? 3. Do the assessment tasks reflect the relative importance of the subject ILOs? 4. Is the assessment workload realistic for teachers and students? Selecting Assessment Tasks

25. Common ILOs Possible Assessment Tasks Describe Assignment, essay question exam Explain Assignment, essay question exam, oral presentation Integrate Project, assignment Analyse Case study, assignment Apply Project, case study, experiment Solve problem Case study, project, experiment Design, create Project, experiment, poster Reflect Reflective diary, portfolio, self-assessment Communicate A range of oral, writing or listening tasks addressing the ILOs, e.g. presentation, debate, role play, reporting, assignment, précis, paraphrasing, answering questions etc.

28. Implementing Constructive Alignment in Learning, Teaching and Assessment ILO: What the student has to learn Teaching: Engaging the student in the verb in the ILO Assessment: How well the student has achieved the ILO

29. Using a Venn Diagram 1 2 4 3 psychologist student school Give examples of interactions at: 1. 2. 3. 4.

30. Activity 4 – Designing Assessment Tasks Refer to one of the subject intended learning outcomes you have written in Activity 2, design assessment task(s) to address this ILO. Subject ILO: Now double-check if the student activities are aligned to the verb(s) nominated in the subject ILO. Share your ideas with your colleagues. Assessment Task Student activities to complete the task

31. Assessing by marks or grades?

32. Assessing by Marks For: . Used to it. . Seems to be the logical way to assess in certain subjects. . Logistically easy. Against: . Defines quality in terms of accumulating small quantities. . Measurement error also accumulates, thus invalidating fine discriminations. E.g. there is no valid difference between 74 and 75, yet to the student it can make a BIG difference - an HD or a D! Or worse, the difference between pass or fail. . Sends undesirable messages to students (backwash).

35. Grading Criteria (rubrics) ILO Assessment task

36. Example of Grading Criteria for ILOs Pass Satisfactory Good Excellent D D+ C- C C+ B- B B+ A- A A+ 1.00 1.30 1.70 2.00 2.30 2.70 3.00 3.30 3.70 4.00 4.30 ILOs Explain Able to identify and briefly Able to identify a number Able to identify a full As in “Good” but write about limited points. relevant points with some range of relevant provides views on Very little evidence of details. Uses these points Points with details. possible alternative using these points to to provide a fair reasoning Supported by relevant causes and/or results provide reasoning to or causality. No evidence literature. Points are under changing why they are inter- of a comprehensive organized to provide a conditions. Able to related. overview of reasoning comprehensive and link current or causality. cohesive reasoning or reasoning to causality. situations in real- life professional contexts. Reflect Able to use available Able to use available Able to use available As in “Good”. Able information to self- information to self- information to self- to generalize self- evaluate and identify evaluate and identify evaluate and identify evaluation to beyond limited aspects of own more aspects of own the full range of own existing context. strengths and weaknesses strengths and weaknesses strengths and weak- Suggest ways of in a general sense. No in a general sense. Little nesses. Self-evaluation improving perform- evidence of suggestions application of theory in is based on theory. ance to real-life of ways to improve self-evaluation and limited Increasingly able to professional performance. No evidence suggestions of ways to suggest ways to contest. of theory being used in improve performance. improve performance self-evaluation. in a specific context.

37. Holistic Grading of Assessment Tasks (e.g. a portfolio) Marginal Adequate Good Excellent D D+ C- C C+ B- B B+ A- A A+ 1.00 1.30 1.70 2.00 2.30 2.70 3.00 3.30 3.70 4.00 4.30 The pieces of evidence The evidence is relevant, The evidence presents a As in “B” but with are relevant and accurate and covers good appreciation of higher degree of accurate, but are several aspects of the the general thrust of the originality and isolated, addressing course. Little evidence of course. Good coverage evidence of inter- one aspect of the an overall view of the with relevant and nalization into course. Demonstration course. Demonstrates accurate support. A clear personalized model of understanding in a declarative understanding view of how various of practice. Good minimally acceptable of a reasonable amount of aspects of the course evidence of reflect- way. Poor coverage, no content. Able to discuss integrate to form a ion on own originality, weak content meaningfully. thrust or purpose. performance based justification of portfolio Good coverage but little Good evidence of on theory. items. Inappropriate Application or integration. application of course Generalizes course self-evaluation. Fair justification of items. Content to practice. content to new and Attempted realistic self- Portfolio items well unfamiliar real- evaluation. justified. Realistic life contexts. self-evaluation.

38. Deriving a Final Grade (Quantitatively) . Award individual grades based on the grading criteria. . Convert grades to numerals e.g. using the grade point scale. . Combine (average) the individual grade points to arrive at a final grade point. . Convert the final grade point back to a final grade.

39. Deriving a Final Grade (Holistically) Curriculum and Instruction: A subject in a course for Ed. Psychlsts. Grading will be based on your attaining the following ILOs 1. Apply the principles of good teaching and assessment to chosen contexts. 2. Relate selected aspects of curriculum design and management to the educational system in Hong Kong. 3. Apply the content and experiences in this subject to enhance your effectiveness as an educational psychologist. 4. Show examples of your reflective decision-making as an educational psychologist. Final grades will depend on how well you can demonstrate that you have met all the ILOs: A Awarded if you have clearly met all the ILOs, provide evidence of original and creative thinking, perhaps going beyond established practice. B Awarded when all ILOs have been met very well and effectively. C Awarded when the ILOs have been addressed satisfactorily, or where the evidence is strong in some ILOs, weaker but acceptable in others. F Less than C, work plagiarised, not submitted.

40. Activity 5 Take a few minutes to reflect on the following: 1. One important point that you have gained from this session. 2. One question you still have on designing constructively aligned learning, teaching and assessment. 3. One action you will take in your future teaching and assessment based on the discussion of this session. Please share with us your reflection on the above issues.

41. References Biggs, J. & Tang, C. (3rd Ed) (2007) . Teaching for Quality Learning at University . Maidenhead: Open University Press/McGraw Hill.