Interpreting and Grading Learning

Grading (or marking) is the process of interpreting students' learning products and performance for the purposes of:

• reflecting where students stand in relation to an orderly development of competence
• informing students and teachers of students' current level of learning, and of what they need to do to improve it
• combining a grade with other grades to meet administrative requirements for awarding grade levels for students' overall performance.

Grading is a high-stakes activity. Students use the results to define themselves as learners. Grading is also highly subjective; interpretation relies heavily on the wisdom of practice. Expert assessors are highly skilled in interpreting and grading students' performances and products. They need to:

• possess detailed knowledge of their discipline, of curriculum intentions and of learners and their diverse backgrounds
• have detailed knowledge of assessment options, and understand the limitations of these options
• be very clear as to the purposes of the assessment
• have access to a repertoire of meaningful approaches that have been intentionally developed for the interpretation of students' learning performances
• be aware of contextual influences on their practice, the limitations of their own interpretations and judgements and the ethical and practical implications of the way they conduct grading.

When planning an assessment-as-learning paradigm, you should consider the questions "When should I use grading?" and "How should I interpret learning?" Ought students' learning products and performances to be graded at all?

Once you have decided to use grading, it's also important that you address the question "How should students be graded?" Assessment-as-learning challenges traditional assessment methods such as objective examinations. It recommends alternative assessment practices such as authentic assessment, standards-based assessment and performance-based assessment. Grading according to principles of assessment-as-learning entails taking account of:

• inclusive assessment
• graduate capabilities
• constructivism
• instructional strategies
• formative and summative evaluation
• rubrics
• peer and self-review and assessment.

Benefits

Standards-based assessment has a number of benefits:

• Students are graded on the quality of their work against standards or criteria. Bias cannot arise from comparison with other students' work.
• Students know the criteria against which they will be judged. The primary purpose of criteria-based assessment and grading is to make students aware, before they complete assessable work, of the quality of the work expected.
• This awareness enables them to shape their work appropriately to the standards expected, and assures them that assessors will not be influenced in judging their work by how other students perform, or by their own previous performances.
• When standards are established and communicated before and during the course – to students, assessors and anyone reviewing the grade distributions – they will function properly for both formative and summative assessment.

Challenges

Grading according to the principles of assessment-as-learning while limiting personal bias presents a number of challenges:

• developing transparent, authentic and fair standards
• ensuring alternative means of assessment for diverse student populations
• developing ways to explicitly communicate the standards to students and staff
• being a proficient user of standards-based assessment and grading.

Strategies

It's important to keep the established standards in mind when grading, rather than ranking students against each other. Ranking-focused grading activity produces the greatest anxiety and provokes the most controversy and opposition. Yet it advantages learning the least, sometimes not at all. Boud and associates (2011) argue that "while marks and grades may provide a crude tracking measure of how well students are doing, they do not help students move beyond their present standard of performance."

While the course or class teacher is usually the one who interprets and assesses students' learning performances, other people can also contribute to the process of assessing:

• external examiners
• expert professionals and community representatives
• computers (automated assessment)
• other teaching colleagues
• students assessing themselves
• students assessing their peers.

Students develop self-critical and independent learning when they are involved in interpreting and judging their own learning performance. With proper training and support, their interpretations and judgements about their own and their peers' learning performances are marginally more consistent and reliable than those of multiple sessional tutors. Being involved in the process of assessment is also a useful learning activity in its own right. The nature and purpose of the assessment task will determine who will contribute to the assessment, and how they will determine the final grade. A grade is a single indicator of the standard of a student's work, but multiple interpretations of a student's work can contribute to deciding it.

In the overall management of assessment processes, allow scope for moderating final grades to ensure that they accurately represent each student's demonstrated capabilities and performance.

Points of reference and grading criteria

When you interpret and grade, you are comparing what you observe with one or more criteria, and points of reference, based theoretically on the purpose or intentions of a particular assessment task. Points of reference can be of three types:

• pre-established criteria: "Does the student performance or learning product demonstrate or address the criteria for which the task was established?"
• pre-determined behavioural norms: "How does the student performance or learning product compare with established norms for this particular level of students?"
• ideographic: "How does the performance or product compare with this student’s earlier performances or products?"

In practice, experienced academics' points of reference are not always clear-cut and rational. They can include:

• other students' learning products
• recall of classroom events and conditions
• broad pedagogical objectives and the specific intended learning objectives
• knowledge of content
• recall of previous assessment events
• an incrementally developed construct based on the assessor's perceptions of form, process and content cues in their students' work
• performance standards.

While it is virtually impossible to grade objectively and impartially all of the time, being aware of the possibility of bias can help minimise its effect.  For some recommendations on how to reduce the likelihood and magnitude of bias, see "Strategies to enhance grading reliability", below.

Representativeness, accuracy and consistency

Assessors should focus not on the grading validity and consistency of a single test, but on the achievement overall of:

• representativeness, where assessors question the meaningfulness of the information the student has generated and the extent to which it reveals the student's cognitive activities
• accuracy, where they map a student's typical performance against clearly outlined criteria
• consistency, where they use consistent, established criteria, but in tasks that best suit individual students, acknowledging that not all students demonstrate their learning in the same manner.

Using assessment rubrics

Use rubrics to articulate and communicate performance expectations and standards. They add value in a number of ways:

• guiding the unit design
• communicating expectations to students
• giving students an idea of where they sit in a framework of orderly development towards increased expertise in a learning domain
• being used as a peer and self-evaluation tool
• aiding consistency, accuracy and representativeness in interpreting, grading and reporting learning outcomes using multiple markers.

To begin creating a rubric, identify the generic capabilities being assessed and the differential levels of attainment for each, as shown in Table 1.

Table 1: Example levels of attainment in a rubric

DEVELOPING	FUNCTIONAL	PROFICIENT	ADVANCED
Not yet at the basic level of expectations Some minimal desired features may be present but not enough to pass May be enough to ask for further work and resubmission	Meets basic requirements at pass level Can be carried out in part without support but some guidance may still be needed There may be a high degree of reliance on authority Little translation or integration of concepts into students' own language or existing knowledge schema	Exhibits independence, translation, integration and application (relational knowledge) Competently analyses and applies conceptual knowledge to novel contexts May correspond to a credit grade	Performance beyond core expectations Highly independent, creative, critically reflective, generative and transformative Uses evidence to formulate defensible personal viewpoints and hypotheses and generate new ideas May correspond to a distinction or high distinction

Within each cell in a rubric grid, statements serve as descriptors of the level of attainment for each criterion. In the example row in Table 2 (from Murray-Harvey, Silins & Orrell, 2003), descriptors relate to each level of attainment for the broad attribute "discipline knowledge and understanding".

Table 2: Example levels of attainment for one criterion

ATTRIBUTES	DEVELOPING	FUNCTIONAL	PROFICIENT	ADVANCED
Discipline knowledge and understanding	Limited understanding of required concepts and knowledge Inaccurate reproduction of text and lectures Does not translate concepts into own words	Encyclopaedic discipline knowledge; accurately reproduces required facts and definitions Shows adequate breadth, but limited depth of understanding of basic concepts	Exhibits breadth and depth of understanding of concepts in the knowledge domain Uses terminology accurately in new contexts and has transformed the ideas to express them appropriately in own words Aware of limits of own understanding	Exhibits accurate and elaborated breadth and depth of understanding of concepts in the knowledge domain Shows understanding of how facts are generated Appreciates the limited and temporary nature of conceptual knowledge in the discipline or field

Strategies to enhance grading reliability

Even where assessors are trained to recognise the subjectivity in their grading processes and to ignore influences that might interfere with their making good judgements, the following factors have been shown to profoundly affect the grades assigned to students' learning products and performances:

• the visual appeal of the assignment's presentation
• the quality and legibility of the student's handwriting and drawings
• the correctness of grammar and spelling
• the quality of the introductory paragraph alone
• the quality of the other papers being assessed (especially the five preceding papers)
• the teacher's own knowledge and expectations of particular students based on classroom events
• the teacher's own "assessment personality"; for example, the tough grader or the encourager of students
• the teacher's own beliefs about grading and education
• the teacher's experience in grading; for example, less-experienced assessors tend to focus on transmission of content, whereas more-experienced assessors tend to focus on learning and transformation.

You can significantly improve the reliability of your grading if you plan how you will reduce the effect of some of the above factors. Some strategies are:

• establishing and maintaining standards by using model answers to benchmark standards at different grades
• annotating model answers to identify performances of different levels on specific criteria
• avoiding sorting assessment products into predicted grade categories prior to marking and assigning grades
• marking papers that have been anonymised
• multiple marking of the same paper by either the same assessor or by two different assessors
• assigning markers to mark the same question in assignments or tests composed of multiple sections
• involving neutral external examiners and assessors
• using computer-aided marking; for example, with machine-readable multiple-choice quiz sheets, or online automated marking.

Managing grading

To effectively manage grading, at the very least you should develop and implement:

• the use of an online submission and marking process
• clear statements to students about their responsibility to keep a copy of all work submitted until grading is concluded for the unit
• a system for allocating time, immediately after grading, to review the grade distribution and any impressions of how students managed the task. Do this individually and as a team or department.
• a program-based or department-based moderation process that is collegial, educative and developmental rather than punitive, and focuses on successes and effective practices as well as providing support for improvement where practices have not been so effective
• a routine recording of reflections after review of the grade distribution. For example, respond to these questions:
  • What can we learn to improve the assessment process for next time?
  • What factors have influenced any unexpected results?
  • What other information should we provide to the head of department, head of Faculty or examinations committee, so that they can understand the grade outputs for the unit?