Assessing Qualitative Differences in Student Work

How do you assess qualitative differences in student work?

Here is a well tried and proven framework that was designed to:

  • categorise student responses to open-ended questions, and
  • focus on qualitative differences between students’ responses.

The framework called the SOLO Taxonomy.

SOLO stands for the Structure of Observed Learning Outcomes

SOLO Taxonomy has 5 levels in assessing differences in student work. The first three are quantitative phases and the second two are qualitative phases.

They are:

 

Sample task

Here is a sample task that assesses each answer to the task using the SOLO five levels

What causes the traffic problem in Sydney?

1. First response to this task:

  • The traffic is always bad.
  • The traffic problem is caused by cars.

This answer avoids the question. Repeats the question and fails to make a genuine attempt to tackle the question.

This response to the task - Misses the Point

2. Second response to the task:

We have a traffic problem in Sydney because there are too many cars.

This answer is based on only one relevant aspect of the task, its conclusion is limited and dogmatic.

This response to the task – is limited to a correct Single Point

3.Third response to this task:

There are too many cars in Sydney and there are not enough roads and the Government does not control the traffic well. There is always a traffic jam.

This is still not a very satisfactory answer to the task - we have Multiple Unrelated Points

4. Fourth response to this task is logical and related and has connected ideas:

  • too many cars
  • limited road space
  • poor traffic management
  • poor city planning
  • overused roads requiring regular maintenance which further reduces usable road space.
  • poor traffic management/ poor city planning leading to ineffective road use.

This is a good answer offering connected ideas and logically related points.

A response of this nature addresses the question set and deserves a good solid mark.

5. Fifth response to this task is also logically related.

But this response does much more. It has an unanticipated extension to the question asked.

The student has really thought about the question.

Her response includes all the connected ideas and logically related points given in level 4, but adds more, including:

  • Discuss the causes of the traffic problem and their interrelations.
  • Point out that like many other social problems, the key issue is proper management of limited resources.
  • Suggests research should be conducted to identify the key problem area.
  • Compares the situation in Sydney with that of Melbourne and Singapore.

This is an excellent answer offering not only a logically related response to the task given but goes beyond, offering an unanticipated extension to the question.

This is more akin to a distinction or higher distinction level response

SOLO in summary

The SOLO has 5 levels, designed to be used to assess differences in student work.

SOLO is useful in structuring feedback to students.

References and Further Readings

Traffic problem example used is adapted from a similar task first developed in CLEAR, Chinese University of Hong Kong.

  • Biggs, J.B. & Tang, C. (2011). Teaching for Quality Learning at University. (4th ed.). Maidenhead: McGraw Hill Education & Open University Press.
  • Biggs, J. (1995). Assessing for learning: Some dimensions underlying new approaches to educational assessment. The Alberta Journal of Educational Research, 41(1), 1-17.
  • Biggs, J.B. & Collis, K.F. (1982). Evaluating the Quality of Learning. The SOLO Taxonomy. New York: Academic Press.