The PVC Education Office is calling for expressions of interest (EOI) to participate in TELT pilots to evaluate the use of selected tools in digital assessments. The PVCE has completed an initial technical evaluation of these tools to ensure their compatibility with the University's existing IT and educational technology, including Moodle. The tools include:
- Safe Exam Browser; this is a client based (free) software that integrates with Moodle for locking down computer devices to restrict access to other resources on the device or online during digital assessments.
- Questionmark OnDemand and Examsoft; these are specific assessment platforms that provide greater assessment functionality than Moodle, including a wider range of assessment types, security features and reporting.
The primary purpose of these pilots is to test the utility of these tools at UNSW and to obtain feedback on the staff and student experience. These initial pilots will be conducted under controlled conditions to avoid any adverse impact on student assessment. The use of the tools and associated support will be restricted to these pilots in 2019.
- The pilots are restricted to solely formative assessments i.e. the assessment cannot contribute any mark, irrespective of weighting, to the course result.
- The types of computer devices may include:
- Desktop devices in existing computer laboratories; clearly this will restrict participation to those with access to computer laboratories and appropriate local IT support.
- Students’ own devices; while this will allow greater flexibility and broader participation, it will also entail the additional local responsibility of supporting students’ devices. The PVCE Office is unable to provide backup devices or desktop support.
- The types of assessment items:
- Selected response items e.g. MCQs, T/F Qs, graphical interactions.
- Constructed responses involving short text answers or simple numerical responses.
- Constructed responses involving long text answers (essays).
- Subject to local support availability, the preferred number of students per setting is less than 100.
Staff participating in the pilots will be responsible for:
- Writing the assessment items in the preferred platform (Moodle, Questionmark OnDemand or Examsoft); basic support will be provided to staff in the use of these platforms.
- Implementing the preferred tool to deliver the assessment; the School’s professional and technical staff will be required to implement the assessment; basic support will be provided to staff in the use of these tools.
- Recruiting students to participate in the pilot; student participation may be part of the usual course activities or may be a separate activity. If the pilot is conducted as a course activity, students who do not wish to participate or who are unable to participate must be offered the same formative assessment under usual conditions.
- Providing technical support to students using their own devices prior to and during the assessment; local IT staff will be needed for this purpose. The PVCE Office will provide generic resources to inform students.
- Locating the physical space for the pilot; dedicated physical spaces for the pilot will not be available; the pilots will need to be conducted in available teaching spaces.
- Providing invigilation, if required.
- Encouraging students to complete evaluation forms at the completion of the pilot.
- Completing staff evaluation forms after the pilot.
- The pilots are being conducted as an evaluation activity, not research. Staff who wish to obtain additional information for research purposes should obtain Ethics approval separately.
- Provide support to staff on the use of tools and their associated integrations with TELT.
- Provide information to staff for supporting students and provide an escalation point on difficult problems.
- Manage application capacity and service availability with the vendors.
- Conduct evaluation, including providing evaluation forms, and report back to participants.
Safe Exam Browser (SEB) has been selected as the preferred add-on application for securing devices using Moodle digital assessment tools. SEB can lock down the device so that the student cannot access unauthorised resources during the assessment. SEB is integrated with Moodle. The application must be downloaded to the student’s device prior to the assessment.
Further information about SEB is available here https://safeexambrowser.org/about_overview_en.html
Additional conditions
- The assessment must be delivered using Moodle assessment tools e.g. Quiz, to allow SEB to be configured.
- Students should be informed about the purpose and nature of SEB; information will be provided by the PVCE Office for this purpose.
- SEB can be used with Windows, Mac and iOS devices.
- SEB can be configured to allow access to specified resources; staff wishing to use this function should seek advice from the PVCE Office and their local IT support staff.
Alternative assessment platforms to complement the assessment tools in Moodle are being evaluated. Questionmark has been used by Medicine for several years and provides greater features in terms of available item types, quality control processes, secured online and offline delivery, and detailed reports. Examsoft has similar features although it is only delivered in an offline mode and requires students to download the application. Examsoft has been used previously at UNSW by Law.
Further information on both platforms is available:
Additional conditions
- The whole assessment process from writing items to delivery and marking must be conducted within the assessment platform. Student access to Questionmark OnDemand during the assessment will be through Moodle. Student access to Examsoft during the assessment will be through their Examplify client software.
- Students should be informed about the purpose and requirements for using the assessment platform; information will be provided by the PVCE Office for this purpose.
- Both platforms can be used with Windows, Mac and iOS devices.
For further information on the Digital Assessment project please contact the Academic Lead: Digital Assessment (PVC-E office) Dr Mathew Hillier m.hillier@unsw.edu.au