Selecting Assessment Technologies
Integrate digital devices into your assessment plan only if adding them improves the quality of the student learning experience. Improving assessment management should be only a secondary consideration.
Bringing technology to assessment processes can:
- allow students to submit assignments online
- enhance students' assessment-as-learning experiences
- give students more ways to learn and to demonstrate their learning
- help staff reconsider learning and teaching approaches
- help staff assess in innovative ways
- help staff give timely and more comprehensive feedback, and
- make it easier to manage large volumes of marking and administration.
Often if you improve assessment management using technologies, you also benefit students educationally. For example, setting online multiple-choice quizzes with automated marking and feedback can reduce staff marking loads and give students immediate feedback on their learning performance.
Plan carefully, and manage assessment-by-technology to ensure that it enhances learning. Check also that it doesn't disadvantage students, especially students with a disability. Be strategic about the integration, so that staff workloads remain sustainable and the university continues to comply with statutory and legislative frameworks.
Dr Carol Russell discusses some strategies, benefits and challenges for assessing with technologies, and Tam Nguyen (Faculty of the Built Environment) talks about using online portfolio tools.
When to use
The main reason to use computers for assessment should be to enhance student learning.
Technology integration is a whole-of-curriculum matter. It can support all aspects of assessment: you can present content and tasks, assess students individually and in groups, provide feedback, and share and manage information about assessment.
You can use it for diagnostic assessment, presenting low-stakes tasks that show students and teachers what students are ready for in terms of learning activities.
Formative assessment activities (ungraded assessments that help students manage their learning) can also be presented online.
Graded, summative assessments (contributing to the final course grade) lend themselves to online presentation. Some tests can be partly or completely computer-marked; others allow students to review each other's work.
You can also create integrative assessment activities on digital devices, so that students can review their work and assess how well they've learned, reflect on how closely they've met discipline standards and teacher expectations, and analyse their own work in relation to sample good-quality responses to online tasks.
Digital assessment can help learners engage more productively and flexibly in learning, and can help staff manage assessment more efficiently and effectively. Here are some benefits that teachers have reported:
- Students can choose when and where they access content to learn or engage in assessment (e.g. at home, outside normal university operating hours). This flexibility helps them regulate their own learning.
- Online tests themselves can be more flexible, with new question formats possible (hot spots, Likert scales, embedded-answer questions and drag-and-drop exercises).
- You can quickly analyse student or group performance on specific questions, and determine an assessment's validity.
- Using tools such as discussion forums and wikis, you can record group processes and support your students as they progress through assessment tasks.
- Students can re-take online quizzes as often as they like, and gain formative feedback on basic conceptual competence.
- With simulations, virtual worlds, online role plays and online access to subject experts and resources, students can experience authentic assessment at no great cost, and appreciate the relevance of their learning. And you can track each student's contribution to a complex task like an online collaborative role play or scenario.
- Intelligent tutoring systems feed individualised feedback to students about their responses to problem-solving tasks. They can also show students how their performance compares with that of experts. This is useful for both formative and summative assessment.
- Students can incorporate multimedia and interactivity when presenting their learning for assessment.
- You can rapidly disseminate learning support and feedback to all students in online environments.
- You can keep electronic records, for efficiency in compiling later marks and for quality assurance purposes. For example, you can record students' learning processes, their automated assignment receipts, the marks they were awarded and the feedback you gave them.
- Operating in technology-rich assessment environments, students can develop and be assessed on their generic technology literacy.
- You can use automated text-comparison services such as Turnitin's OriginalityCheck tool to detect plagiarism as well as to help students develop their paraphrasing and citation skills.
- You can have students use blogs, wikis and ePortfolios to conduct self-assessment and peer review activities.
When deciding whether to assess electronically, and which technologies to choose, plan carefully. Think about:
- Access and equity: An advantage for one student can be a disadvantage for another. Provide choices to counter this disadvantage. The UNSW: Accessibility Support website provides guidelines, and this page of accessibility tips is also useful.
- Staff workload: For example, developing online quizzes with comprehensive feedback is time consuming, but may be worthwhile for large classes that must master many basic concepts. In addition, it increases the speed and consistency of feedback.
- Online tests and quizzes, while they can provide excellent formative feedback for students, can be problematic if used for high-stakes summative purposes, especially if IT system outages and malfunctions occur during timed assessments.
- Student stress: Ensure that students know where to find help (including how to download specific software, if necessary) when they are using and troubleshooting assessment technologies.
- Consider copyright issues when you are deciding which technologies to use in learning and teaching.
The eLearning for Staff page lists the UNSW-supported technologies you can use for learning and assessment, depending on what type of learning outcome you seek. The site also suggests related learning and assessment activities.
Here are just a few examples of how you can incorporate learning technologies into assessment designs to engage students.
Prepare your students for an online assessment
- Make sure they have had practice using the relevant online tool.
- Trial the task with at least two people to check that the instructions are clear and comprehensive.
- Explain why you are using online assessment.
- Ensure that the timeframe is realistic for students to complete the task.
- Evaluate whether the weighting of the assessment warrants the time required to successfully complete the online task.
- Provide support material such as guidelines, templates and marking criteria to help students manage the task successfully.
- Provide instructions on what to do if the technology fails.
Clearly communicate assessment requirements
- Using a system such as Echo360, or a voice tool in a Moodle course, record a general lecture about assessment in which you provide background information about tasks and answer students' frequently asked questions.
- Create an assessment activity gateway within your Moodle course where you integrate task requirements, assessment rubrics, related readings (copyright material must be held in the Library e-reserve, but you can link to it), access to other resources and support, and tools to facilitate group work.
- Provide online worked examples of an electronic assessment rubric to illustrate performances at different levels of attainment.
Create a place for students
- Use computer-based interactive assessment activities in campus laboratories and clinics to engage students actively in learning.
- Set up online discussion forums and monitor and assess students' contributions to the threads.
- Provide students with authentic learning challenges using online simulations, games and virtual worlds.
- Use web conferencing tools to involve international experts, leading scholars and stakeholders in discussions.
- In between face-to-face meetings, use online groups to brainstorm assessment tasks.
- Set up wikis to facilitate multiple contributions to the preparation of an assessment product, and enable assessment of individuals' contributions.
- Save lecture notes, study guides and learning materials in an accessible format, so that students with a disability can rapidly convert files.
- Build in choices for students both in assessment design and in the technologies you employ to support assessment. This will make it easier to accommodate special needs as they arise.
- Provide text alternatives for visual content, to aid students with visual disabilities, slow Internet connections or other technology-limiting factors.
- Read this page of accessibility tips.
Encourage demonstration of higher order thinking and reasoning
- Use integrated blogs and wikis in your online course to facilitate students' reflective contributions as they prepare for assessment tasks.
- Set up groups in virtual worlds to enable students to contribute creatively to authentic assessment tasks.
- Design assessment tasks to require students to produce digital artefacts—for example, videos or websites.
- Have students present their work for assessment using ePortfolios such as Mahara. This can increase the portfolios' accessibility, so that you can more easily involve peers and external experts in giving feedback.
Create learning communities around assessments
- Set up the groups yours for group work projects in the online course, to help ensure that group memberships reflect the diversity of the class.
- Enable your students to communicate about assessment tasks with students studying the same material in another university, or even in another country. This can foster a more globalised learning perspective.
- Invite off-shore students to join a synchronous virtual classroom with local students, where the teacher provides generic feedback on assessment and then students break out into small discussion groups.
Give students more responsibility for learning
- Use a Moodle course to provide a central space where students learn and are assessed.
- Provide a bank of computer-based, randomly generated test items for students to practise on, prior to a summative test.
- Require students to put their text-based assignment through Turnitin's OriginalityCheck tool before they submit it, and to include a report on the extent of text-matching with their assignment submission.
Provide safe practice in simulated high-risk tasks
You can provide virtual:
- accident and emergency wards
- expensive equipment, for manipulation practice
- clients, for practice in counselling prior to working with real clients
- world community planning exercises
- high-risk laboratories, where students can practise safety procedures and teachers can assess their competence.
See the simulations page for more information.
Provide guidance and feedback to students
- Electronic response systems, such as clickers, can provide immediate feedback on students' learning in lectures.
- Use web conferencing software such as Blackboard Collaborate (either within Moodle or independently of an LMS) to set up a virtual office for one-to-one consultations. This can be more time-efficient than face-to-face meetings.
- Use email for private discussion with students.
- Open up online public discussion of learning challenges that many students share, so that everyone can benefit from your feedback.
- Record audio of your assessment feedback as you mark students' work. This lends immediacy and invites dialogue with students about assessment.
- Prepare FAQs about assessment tasks and put them online. This can save you a lot of time in the long run.
Manage and streamline assessment processes
Require students to submit assignments using Moodle. These Learning Management Systems allow you to set up cut-off dates, automated receipts and late submission rules.
You can mark, and mark up with feedback, digital copies of student assignments using tools such as GradeMark. This makes it easier for course convenors to manage staff marking teams at a distance, monitoring the quality of feedback given by sessional tutors and modelling best practice for new tutors.
Set up a folder to share access with the marking team and make it easier to coordinate the marking and grading process.
LMS tools and assessment examples
Here are some examples of online assessment activities that can be provided in UNSW Moodle:
Prior to a face-to-face class, have students complete a short computer-marked quiz so that it is clear what aspects of the topic need to be clarified.
Evaluate students' contribution to discussion to encourage participation in collaborative work
If providing course content online, include some s-tests so that students can evaluate their own progress.
With a written assignment, have the students exchange and comment on each others' drafts before final submission.
Workshop (peer review)
Support students in acquiring academic writing skills by having them use text similarity detection software while they are writing.
Use online assessment to manage grading of written work, and giving feedback
Grademark, Moodle marking guide, assignment rubric
Use audio tools to provide audio feedback on assignments—research shows that this is very time efficient and appreciated by students.
In this collection of videos, Dr Carol Russell discusses some strategies, benefits and challenges for assessing with technologies.
Teaching Gateway pages about designing online assessment and learning activities
- Group assessment tasks—planning—planning, choosing LMS tools for group work
- Creating lesson plans in Moodle
- Online learning activities—planning, integrating, using LMS tools
- Assessing online participation—integrating activities and assessment, assessing participation, which tools to use
Useful UNSW sites
- UNSW Accessibility Support website, UNSW Guidelines on Website Accessibility
- UNSW copyright guidelines
- UNSW TELT Gateway, Matrix for selecting technologies
- Centre for the Study of Higher Education:
- Australian Flexible Learning Framework, Flexible Learning Toolboxes
- Edith Cowan University, Technology Supported Learning Database
- Macquarie University's Assessment Toolkit contains this page, which includes several podcasts of staff talking about their assessment practices
Almond, P., Winter, P., Cameto, R., Russell, M., Sato, E., Clarke-Midura, J., Torres, C., Haertel, G., Dolan, R., Beddow, P. and Lazarus, S. (2010). Technology-Enabled and Universally Designed Assessment: Considering Access in Measuring the Achievement of Students with Disabilities—A Foundation for Research. Journal of Technology, Learning, and Assessment 10(5).
Herrington, J., Mantei, J., Herrington, A., Olney, I. and Ferry, B. (2008). New technologies, new pedagogies: Mobile technologies and new ways of teaching and learning. In Hello! Where are you in the landscape of educational technology? Proceedings of the ASCILITE Conference Melbourne, 419–427.
Joint Information Systems Committee (JISC) (2007). Effective Practice with e-Assessment: An overview of technologies, policies and practice in further and higher education.
Nicol, D.J. and Milligan, C. (2006). Rethinking technology-supported assessment in terms of the seven principles of good feedback practice. In C. Bryan and K. Clegg (eds), Innovative Assessment in Higher Education, London: Taylor & Francis Group Ltd.
Nicol, D (2008), Technology-supported assessment: A review of research. Unpublished manuscript.
New Media Consortium (NMC). (2008). The Horizon Report. Washington DC: New Media Consortium and Educause Learning Initiative.
University of Strathclyde (2008). Creating accessible assessments and examinations for disabled students.