Assessing Authentically

Ideally authentic assessment should provide long-term student engagement with learning and can occur at any stage of the teaching program. It should utilise a variety of resources and perspectives over a sustained period of time, as well as peer collaborations to promote engaging and open conversation.

Authentic assessment should be based in an environment that the student could work in, learn in or utilize post task and include the development of a scenario. This type of learning and teaching is preferable toward the end of a degree when students are comfortable collaborating and working on ill-defined problems, and have skills in reflection.

Authentic assessment can benefit students in many ways:

• Students generally accept authentic assessment as a valid approach, and favour it as a method that motivates them to engage in deeper and more productive learning.
• Because it involves addressing "ill-structured", unpredictable challenges, it helps students rehearse for the complex ambiguities of working and professional life, and to visualise themselves as real professionals.
• It requires students to construct unique responses rather than selecting from pre-existing options. In this way it challenges students to undertake complex higher order reasoning, and to think independently and creatively.
• Students can reflect on and assess their own work and effort. They can see, meaningfully, in situ, how effectively they apply conceptual learning.
• Students can integrate their learning in a holistic way, bringing together work samples collected over time, perhaps in the preparation of a portfolio.
• External stakeholders, such as industry groups and professional bodies, favour universities offering work-relevant experiences. Authentic assessment enhances graduate employability by developing students' "work-readiness" capabilities.
• It can creatively disrupt the traditional power balance in assessment by allowing external markers to give feedback and/or grade students' work, and students to actively engage in self and peer assessment.
• Authentic assessment addresses Boud and Falchikov's (2005) calls for "sustainable assessment". It helps equip learners with relevant workplace skills and competencies and prepares them for lifelong learning.

The very characteristics that make authentic assessment so meaningful for students also present significant challenges.

• Unpredictability increases the potential for things to go wrong and jeopardise a student's chance to demonstrate their achievements and capabilities.
• Authentic assessment in actual workplaces can place a significant burden on associated staff. Arranging each student's unique setting in advance, negotiating their individual tasks and interpreting and grading their work can be very time-consuming.
• Developing appropriate simulations is resource intensive, although it can yield long-term returns.
• Assessment tasks, if their scope is not carefully articulated at the outset, can expand to create an unreasonable workload for students. The resulting problems or delays in completing tasks may place students and staff in an awkward position with host organisations or supervisors.

Authentic assessment tasks may be problematic for particular students. For example:

• Access to off-campus activities may be difficult for students with mobility issues. Block placements may place prohibitive additional burdens on students with carer responsibilities.
• Students may feel anxious about whether they will fit in, whether they will be able to communicate effectively and so on, particularly when high-stakes assessment will depend on their capacity to interact within the authentic setting.
• Make sure you tell students what accommodations and alternative arrangements are available in such circumstances. Also, be aware of students' diverse levels of experience in the types of authentic settings established for learning and assessment activities, so that students can be well prepared and supported at all stages.

In planning for authentic assessment, ensure that the tasks and activities possess the distinctive characteristics of authenticity, as outlined in Figure 1 (adapted from this Resource from University of Wollongong 2005).

Figure 1: Characteristics of authentic task design

Designing authentic assessment requires considerable work prior to the commencement of a course. However, a great deal of this effort can be expected to have ongoing value for future classes, in the form of:

• relationships developed with host organisations
• assessment resources developed for students about the processes, and
• simulation resources developed to replicate authentic settings.

Figure 2: Dimensions of authentic assessment (after Mueller, 2010)

Designing authentic assessment involves the following steps:

Determine the broad characteristics of authenticity

• How authentic can the assessment be, in terms of various dimensions of authenticity?
• Some significant characteristics of authenticity, in terms of how it contrasts with more traditional forms of assessment in academic settings such as exams, are set out in Figure 2 above.

Align assessment with the intended learning outcomes

• Ideally, associate learning outcomes that reflect real-world contexts with authentic assessment tasks.
• You may need to revise the assessment plan and/or intended learning outcomes to some extent, to ensure effective alignment.

Design the real-world conditions

This involves describing problems or scenarios, finding placements in authentic settings, or designing a learning environment, taking account of the following:

• Placements in real-world contexts can be high-risk for both the student and those with whom they will interact. Ensure that everyone exercises appropriate duty of care in the preparation and ongoing management of students in these contexts, and recognises the needs of all parties. The more natural the context, the greater the potential risks involved.
• Where placement in a real setting is not possible or desirable, technologies can be exploited to design scenario-based virtual learning environments in which conditions, characters, circumstances and parameters simulate a real-life context for learning (Herrington et al., 2003).

Ensure that students have the knowledge and skills needed

Ensure that students have the knowledge and skills they need to carry out the tasks, that they are well prepared and equipped for their engagement in the setting, and that they understand the assessment requirements.

For example:

• Are they aware of what a report looks like, in contrast to more academic essays?
• Do they know how to make written or verbal presentations to the public?
• Are they aware of the legal implications of designing a website?
• Do they appreciate the ethical issues involved in reporting their observations?

Highlight the importance of students understanding their roles when engaging in real-world activities

• Students are ambassadors of the university; they should present themselves appropriately within the wider community.
• Students should be well prepared, and behave responsibly and professionally.

Design the assessment task(s)

According to Wiggins (1993), the tasks for assessment should, as much as possible:

• be essential tasks that need to be done in the setting, and not needlessly intrusive
• be enabling, in that they guide students toward more sophisticated use of skills or knowledge
• entail the integration of skills and knowledge contextualised to the authentic setting in all its complexity, rather than being disaggregated to correspond to individual learning outcomes
• depend on the student's own research or use of knowledge
• emphasise higher order reasoning rather than simple recall or description
• be representative rather than comprehensive, giving students the opportunity to probe deeply rather than to gain broad but shallow experience
• be interesting and worthwhile, engaging students' interest and motivation
• revolve around complex, ambiguous or "wicked" problems (problems whose solutions create further problems).

Additionally, authentic assessment tasks ideally will address the needs of an authentic audience (Rule, 2006).

Manage the assessment load

In authentic assessment situations, students may be over-zealous, producing very large portfolios or very long reports. It is important to set limits on the size of the submission, if for no other reason than to manage students' and staff workloads.

For example, if students are to keep journals, have them submit brief periodic reflective statements based on their journals, rather than submitting the journals themselves.

Plan for improved reliability in grading authentic assessment tasks

The more complex the assessment, the more judgment is required from markers, and the greater the need to incorporate reliability measures into the grading process.

• Establish clear assessment criteria
• Include process indicators in assessment criteria, as well as product indicators
• Develop an assessment rubric outlining standards at different grade levels
• Incorporate multiple sources of evidence of student achievement
• When grading, involve others such as host supervisors and marking teams, and students themselves as self- or peer assessors
  Develop dialogue between assessors and learners
• Provide opportunities for students to present additional evidence (Hager & Butler, 1996).

Examples of authentic assessment

Students are increasingly being involved in authentic learning and assessment tasks that are typically, but not exclusively, set in such work-integrated contexts as:

• professional education programs
• authentic clinical, production or research contexts
• application and problem solving with preclinical concepts and laboratory work
• professional internships throughout a range of disciplines
• cross-disciplinary integration.

The following examples show the range of activities possible with this form of assessment.

Problem-based learning (PBL)

For an extended period, a team of students evaluate what they know and what they need to learn in order to gain the necessary capacities to respond to a real-world problem or task.

Problem-based learning requires students to work with one another to identify and define problems and to formulate and test hypotheses, searching for and applying theoretical knowledge and skills to new and ill-defined contexts.

Objective Structured Clinical Examinations (OSCE)

OSCEs use actors in simulated real-world health problems. Students must observe, diagnose and treat these simulated patients in a limited period. This activity is more individualistic than PBL.

OSCEs are labour-intensive and expensive, but are replicable so that all students are subjected to the same challenges, which is not possible in real clinical contexts.

Scenarios

Scenarios can require students to:

• notice what is important
• explain it using theoretical concepts of the course
• plan and theoretically justify an intervention.

Or they can require students to:

• notice critical factors in a given situation
• investigate the implications and prepare
• present a report for a prescribed audience for a prescribed purpose.

Portfolios

Portfolios require that students

• understand and internalise the learning outcomes of a unit of study, and then
• plan their own set of activities that will generate validated evidence of their performance capability and skill mastery.

The most important feature of the portfolio is the contents-and-commentary page, where the student directs the assessor to particular evidence in relation to specific learning outcomes, to explain and justify the learning achievements.

Designing a solution

Designing a solution to a real community or workplace problem and presenting the solution to its intended audience can be a very engaging activity for students. It can be combined with learning to conduct small-scale research and surveys of contemporary issues that are published in a report for a specific audience.

Analysing "wicked problems" entails considering authentic, complex problems, any solutions to which will create other problems.

Writing for publication

Writing a journal article or short story for publication can be extended to requiring students to form editorial panels, review the work produced and undertake full responsibility for producing a publication, for example, an edited collection of papers.

Staging an exhibition, performance or conference

Events organised by a class group can involve community members, industry experts or professional bodies. They can represent the culmination of a course or program, and expert assessors and judges can be recruited to provide critiques.

Constructing a website

Design and development of a website or other public education resource involves educating a community group about a contemporary issue that students have identified, studied and researched. Not only do students investigate the often conflicting explanations and viewpoints, they also consider the needs and motivations of the community in relation to the issue.

Placements

Workplace or community placements require students to draft, negotiate and establish their own learning outcomes that are congruent with the learning objectives of the unit, and that reflect their context. Students then gather validated evidence that they have achieved the learning outcomes.

Forensic problem-solving

Forensic problems can be set across a range of disciplines. In the sciences, they may be based on scenarios that require students to gather, record and analyse materials for their normal bench work laboratory education. Student groups may have to report or receive findings in relation to another group of students. This requires multiple levels of systematic organisation, teamwork and communication.

Forensic problem-solving can be combined with dramatisations such as court cases, where results and conclusions are argued and defended. In this way, the routine laboratory skills that are learned and developed derive greater meaning from the "big picture" setting and real-life interest established by the scenario.

The following video series includes:

• discussion with Dr Kerry Thomas (from COFA) and her students on how she prepares future art teachers through the assessment of performance tasks.
• mini documentary on the use of authentic assessment by the Engineering Faculty's Dr Sami Kara.
• discussion on various aspects of authentic assessment with Dr Patsie Polly from the School of Medical Sciences and Gwyn Jones, Learning Advisor from the UNSW Learning Centre.
• interview with Chris Walker from the School of Social Sciences and International Studies on his use of role play to bring policy theory to life in an extended case study course.

Videos - Further Aspects of Assessing Authentically

The Connections Seminar series and the annual Learning and Teaching Forum provide platforms for UNSW staff to explore different aspects of learning and teaching, share ideas and get feedback on practice and research.

Recordings and presentations can be found on the respective Moodle course sites (self-enrolment key provided)

• 2019 Learning and Teaching Forum on 26 November 2019: Designing a post-graduate program for online delivery, authentic experience and active engagement presented by Dr Thuy Vu, Mr Steven Parker and Dr Toni Ferrara, Teaching and Learning Unit, Faculty of Science (self-enrolment key: lntforum)
• 2019 Learning and Teaching Forum on 26 November 2019: Collaborate Cambodia presented by Ms Eva Lloyd, Education Focussed Academic, Faculty of Built Environment(self-enrolment key: lntforum)

Mueller, J. (2010). Authentic assessment toolbox

University of Wisconsin, Stout (2007). Authentic assessment: Online assessment resources for teachers

Boud, D. (2000). Sustainable assessment: rethinking assessment for the learning society. Studies in Continuing Education 22(2), 151–167.

Boud, D. and Falchikov, N. (2005). Redesigning assessment for learning beyond higher education. In A. Brew and C. Asmar (eds), Research and Development in Higher Education 28, HERDSA, Sydney, 34–41.

Hager, P and Butler, J. (1996). Two Models of Educational Assessment. Assessment and Evaluation in Higher Education 21(4), 367–378.

Herrington, J., Oliver, R. and Reeves, T. C. (2003). Patterns of engagement in authentic online learning environments. Australian Journal of Educational Technology 19(1), 59–71.

Herrington, J., Reeves, T. and Oliver, R. (2010). A guide to authentic e-learning. London and New York: Routledge.

Isaacs, G. (1996). Assessment programs: Characteristics and constraints. TEDI, University of Queensland.

Journal of Authentic Learning, State University of New York at Oswego.

Lombardi, M. & Oblinger, D. (2008). Making the Grade: The Role of Assessment in Authentic Learning, ELI Paper 1, EDUCAUSE Learning Initiative.

Rule, A. C. (2006). Editorial: The Components of Authentic Learning. Journal of Authentic Learning 3(1), 1–10.  

The contributions of staff who engaged with the preparation of this topic are gratefully acknowledged.

This topic was inspired by a resource developed for the Macquarie University Assessment Toolkit.