Digital assessment exemplar 15

Open book exams allow students to have notes and other resource materials during exams as they test students’ ability to find and apply information and knowledge to answer questions. Is it possible to deliver open book digital exams and at the same time prevent students from (1) getting answers via social media or (2) leaking or sharing questions using email accounts?

Dr Nirmani Wijenayake used the Safe Exam Browser (SEB) for delivering two open book digital assessments (worth 25% each) in Fundamentals of Biochemistry (BIOC2181), in July and August 2019, catering for 240 students. In each exam the students were divided into two cohorts (morning and afternoon) with approximately 120 students per session. The Moodle quizzes, one for each cohort, were set up with password protection where students had to turn up to the exam to obtain the passwords. First password is used to open the Safe Exam Browser and the other, to start the quiz. In addition, a quit password was in place to prevent students from closing the Safe Exam Browser accidentally.

As this was an open book exam, with SEB, students were able to attempt the Moodle quiz while accessing their notes through LabArchives, an electronic lab notebook, that was used to store their lecture notes. The beauty of this system is that you can set up the SEB to only give access to specific sites you want the student to interact with during the exam. It uses a black and white listing system that gives you the full control of which sites can be used by students. Other resources in lab’s computers were also locked down temporarily (access to disk drives, un-authorised URLs, Operating System’s task manager, closing browsers, etc). While invigilation was used for both exams, it was considerably simpler with SEB as invigilators can be certain that access to online resources are limited. Without SEB, invigilator must be more vigilant in case students are sending out questions or asking questions via social media.

Digital assessment using Moodle and SEB is fully reliant on Moodle being functional with no network outages during the exam. Therefore, it is essential that you liaise with UNSW IT prior to running exams to ensure a smooth process with minimal disruptions. Once you get over these hurdles, it is an easy and flexible system that can be used in large courses.

Important steps for the deployment:

• Moodle Quizzes must be ready, assigned to the right groups and accessible by students.
• A .seb file must be created for each quiz; this file allocates resources and the quiz that will be available during an exam. Each student must download this file and run it before starting the exam.
• On the day of the exam, each student is provided with a handout with step by step instructions on how to access the exam.
• Contact UNSW IT in advance to setup Safe Exam Browser in each PC in the lab.
• Let UNSW IT and ETS Moodle teams know that you are running fully online exams (course, dates, times, student numbers). The aim will be to avoid UNSW IT undertaking computer or system updates just before or during exam time. Request ETS to provision suitable Moodle capacity. Also request ETS to advise you of any planned maintenance Moodle that may impact the exam session.

Pros

• Allows you to conduct open book exams that gives students access to specific preapproved sites.
• Prevents cheating via social media and unauthorized resources.
• Running the exam in a computer lab prevents contract cheating.
• Answers are saved every 30 seconds. In an event of temporary network outage, entered answers will not be lost (a notice will be displayed on the screen). Students can continue working on the current page and their responses will re-save to the server upon reconnection.
• Multiple choice questions can be marked instantly.
• Less invigilation needed.
• Don’t have to deal with printing and sorting large amount of paper.
• Exam answers can be stored on Moodle indefinitely.
• Have the potential to use personal computers to do the exam in the future.

Cons

• Exams could fail due to power failure, network outages or hardware failures. This means you need to have a backup plan in case this happens.
• Initially it can be stressful for students and academics who have never done an online exam.
• Some students are not very proficient in using PCs which were used in the exam.
• SEB is currently not set up to be used on personal computers.

• Make sure you assign class time to do a formative trial or mock exam using Safe Exam Browser so students can get used to the system before doing a high-stake exam. This will decrease the stress for students as they are more familiar with the system prior to the real exam.
• Start early so you have enough time for setting up and testing the .seb file.
• Provide enough time for UNSW IT to setup the lab you booked; Safe Exam Browser on every computer.
• Inform UNSW IT about the exam, so that computer updates do not take place during exam.
• Do not provide any links inside a quiz that would allow students to navigate away from the exam.
• It is generally a good idea to allow extra time before and after the expected exam duration so you have some contingency to setup and to fix any issues that may occur. e.g. schedule a 3-hour class block to run a 2-hour exam. Or a 1-hour class time to undertake a 40-minute test.

To get started, please get in touch with Dr Mathew Hiller and Dr Nirmani Wijenayake.

The following video was produced for BIOC2181 students to access a quiz via SEB.

Please get in touch with Dr Mathew Hiller and Dr Nirmani Wijenayake for more demos.