Assessing Qualitative Differences in Student Work

Overview

Providing fair, transparent, and objective assessment of students' qualitative work is challenging. Students deserve to be assessed in the same way, using the same criteria, as the rest of their cohort. However, as students progress in the sophistication of their understanding and skills, their work can shift from largely quantitative to primarily qualitative. It is up to the assessor to ensure that they guard against subjectivity and intuitive evaluation. One way to assess qualitative work fairly and transparently (that is, so that the student can see that they are being assessed in the same way as their cohort, and can understand how the assessor arrived at their mark) is the Structure of the Observed Learning Outcome (SOLO) Taxonomy (Biggs & Collis, 1982).

The SOLO Taxonomy

The SOLO Taxonomy (that is, a classification system for student work) is a proven framework designed to:

• categorise student responses to open-ended questions and
• focus on qualitative differences between students’ responses.

The taxonomy focuses on the degree to which students' responses express explicit relationships between the components that make up their answers. 

At the first level, called the "pre-structural" level (or "missing the point"), a student might give a response that does not actually address the question (perhaps by merely restating it, or by declining to answer altogether).

At the second level, called the "uni-structural" (or "single point") level, a student might give a response that addresses only one aspect or component of the question; the response might also be vague and lack depth, or have only limited relevance.

At the third level, called the "multi-structural" (or "multiple unrelated points") level, a student's response might focus on several relevant aspects, but treat them independently, without identifying relationships among them. Students at this level  may understand isolated aspects of the concept they're examining, but don't know how to look for how these aspects relate to each other.

At the fourth level, called the "relational" (or "logically related points") level, the student has shown how the different components of their response are connected:  how each affects and is affected by the others. They may also have identified overall patterns and examined the question from different perspectives.

At the fifth level, called the "extended abstract" (or "unanticipated extension") level, the student goes beyond what the original question asked for. They could, for example, offer comparisons to similar situations outside the immediate scope of the question, provide more-complex analysis of causality and consequences, or generalise their findings to a new area.

The first three levels of the taxonomy are primarily quantitative; the fourth and fifth are primarily qualitative.