Overview
Providing fair, transparent, and objective assessment of students' qualitative work is challenging. Students deserve to be assessed in the same way, using the same criteria, as the rest of their cohort. However, as students progress in the sophistication of their understanding and skills, their work can shift from largely quantitative to primarily qualitative. It is up to the assessor to ensure that they guard against subjectivity and intuitive evaluation. One way to assess qualitative work fairly and transparently (that is, so that the student can see that they are being assessed in the same way as their cohort, and can understand how the assessor arrived at their mark) is the Structure of the Observed Learning Outcome (SOLO) Taxonomy (Biggs & Collis, 1982).
The SOLO Taxonomy
The SOLO Taxonomy (that is, a classification system for student work) is a proven framework designed to:
- categorise student responses to open-ended questions and
- focus on qualitative differences between students’ responses.
The taxonomy focuses on the degree to which students' responses express explicit relationships between the components that make up their answers.
At the first level, called the "pre-structural" level (or "missing the point"), a student might give a response that does not actually address the question (perhaps by merely restating it, or by declining to answer altogether).
At the second level, called the "uni-structural" (or "single point") level, a student might give a response that addresses only one aspect or component of the question; the response might also be vague and lack depth, or have only limited relevance.
At the third level, called the "multi-structural" (or "multiple unrelated points") level, a student's response might focus on several relevant aspects, but treat them independently, without identifying relationships among them. Students at this level may understand isolated aspects of the concept they're examining, but don't know how to look for how these aspects relate to each other.
At the fourth level, called the "relational" (or "logically related points") level, the student has shown how the different components of their response are connected: how each affects and is affected by the others. They may also have identified overall patterns and examined the question from different perspectives.
At the fifth level, called the "extended abstract" (or "unanticipated extension") level, the student goes beyond what the original question asked for. They could, for example, offer comparisons to similar situations outside the immediate scope of the question, provide more-complex analysis of causality and consequences, or generalise their findings to a new area.
The first three levels of the taxonomy are primarily quantitative; the fourth and fifth are primarily qualitative.
Question: What causes traffic problems in Sydney? 1. First-level response:
This answer avoids the question. Repeats the question and fails to make a genuine attempt to tackle the question. This response to the task misses the point.
2. Second-level response: We have a traffic problem in Sydney because there are too many cars. This answer is based on only one relevant aspect of the task, and its conclusion is limited and dogmatic. This response to the task is limited to a correct single point.
3.Third-level response: There are too many cars in Sydney and there are not enough roads and the Government does not control the traffic well. There is always a traffic jam. This is still not a very satisfactory answer to the task, as it is limited to multiple unrelated points.
4. Fourth-level response:
This answer offers connected ideas and logically related points.
5. Fifth-level response: This response includes all the connected ideas and logically related points given in the fourth-level response, but adds more:
This is an excellent answer not only giving a logically related response to the task, but going beyond, offering an unanticipated extension. This is could be considered a distinction or high-distinction response. (This example used is adapted from a similar task first developed in CLEAR, Chinese University of Hong Kong.) |
Resources
- The who, what, when, where, and why of SOLO Taxonomy. https://www.nzcer.org.nz/nzcerpress/set/articles/who-what-when-where-an…
- A teacher's guide to SOLO Taxonomy. https://www.structural-learning.com/post/what-is-solo-taxonomy
Biggs, J. B., & Collis, K. F. (1982). Evaluating the quality of learning: The SOLO taxonomy (Structure of the Observed Learning Outcome). Academic Press.
Svensäter, G., & Rohlin, M. (2023). Assessment model blending formative and summative assessments using the SOLO taxonomy. European Journal of Dental Education, 27(1), 149-157.