The PVC Education Office is calling for expressions of interest (EOI) to participate in TELT pilots to evaluate the use of selected tools in digital assessments. We welcome those new to digital assessment and those already conducting digital assessments (we are keen to collaborate!). The tools include:
- Safe Exam Browser for Moodle: this software integrates with Moodle (specifically Quiz) to lock down computer devices to restrict access to other resources on the device or online during digital assessments. It works with all Moodle quiz question types including STACK used in STEM disciplines. The initial phase of work will be to look at SEB in on-campus labs before moving on to possible BYOD use.
- WIRIS for Moodle: a plugin for Moodle that allows input of Mathematical and chemical formulae by button bar or by using handwriting recognition via pen or touch device. This can be used in conjunction with Moodle quiz or other Moodle tools where the Moodle content editor is available.
- QuestionMark OnDemand: a separate assessment platform that provides additional question types to that of Moodle and a question bank, additional analysis and reporting features.
- Examplify from ExamSoft: a separate assessment platform designed to work with BYOD that focuses in basic selected response and long form essay style responses. It provides a question bank, question categorisation, additional analysis and reporting features.
The key purpose of these pilots is to test the utility of these tools at UNSW and to obtain feedback on the staff and student experience. These initial pilots will be conducted under controlled conditions to avoid any adverse impact on student assessment.
To get involved:
- Please read the detailed information below, then,
- Complete the expression of interest form
We will discuss with you the best way to proceed according to your context and experience with the selected digital assessment tool/approach.
- In general we plan to run pilots as either:
- Proof of concept done as a mock assessments or formative assessment i.e. the assessment cannot contribute any mark, irrespective of weighting, to the course result. We recommend that those new to digital assessment try a mock assessment first before moving on to undertaking a real assessment.
- Low stakes pilot assessments (real assessments) i.e. the assessment will contribute to student's mark in the course. We recommend this for those already conducting digital assessment or those who have already run a proof-of-concept.
- The types of computer devices may include:
- Desktop devices in existing computer laboratories; this will restrict participation to those with access to computer laboratories and appropriate local IT support.
- Students' own devices; while this will allow greater flexibility and broader participation, it will also entail the additional local responsibility of supporting students' devices. The PVCE Office is unable to provide backup devices or desktop support.
- The types of assessment items (dependant on the tool chosen):
- Selected response items e.g. MCQs, T/F Qs, graphical interactions.
- Constructed responses involving short text answers or simple numerical responses.
- Constructed responses involving long text answers (essays).
- Subject to local support availability, the preferred number of students per setting is less than 100.
Staff participating in the pilots will be responsible for:
- Writing the assessment items in the preferred platform (Moodle, Questionmark OnDemand or Examsoft); basic support will be provided to staff in the use of these platforms.
- Implementing the preferred tool to deliver the assessment; the School’s professional and technical staff will be required to implement the assessment; basic support will be provided to staff in the use of these tools.
- Recruiting students to participate in the pilot; student participation may be part of the usual course activities or may be a separate activity. If the pilot is conducted as a course activity, students who do not wish to participate or who are unable to participate must be offered the same assessment under usual conditions (e.g. an option for pen-on-paper).
- Providing technical support to students using their own devices prior to and during the assessment; local IT staff will be needed for this purpose. The PVCE Office will provide generic resources to inform students.
- Locating the physical space for the pilot; dedicated physical spaces for the pilot will not be available; the pilots will need to be conducted in available teaching spaces.
- Providing invigilation, if required.
- Encouraging students to complete evaluation forms.
- Completing staff evaluation forms after the pilot.
- The status of ethics protocol approval will be discussed as part of our preparatory discussions. As of April 2019 an ethics protocol is not in place but the PVCE team aim to do so in due course. At this point in time - Pilots would be conducted an 'evaluation activity' only, not research. Staff who wish to obtain additional information for research purposes should obtain Ethics approval separately until PVCE has obtained ethics approval.
- Provide support to staff on the use of tools and their associated integrations with TELT.
- Provide information to staff for supporting students and provide an escalation point on difficult problems.
- Manage application capacity and service availability with the vendors.
- Conduct evaluation, including providing evaluation forms, and report back to participants.
a) Safe Exam Browser (SEB) has been selected as the preferred add-on application for securing devices using Moodle digital assessment tools. SEB can lock down the device so that the student cannot access unauthorised resources during the assessment. SEB is integrated with Moodle. The SEB application must be installed on the computer being used for the assessment (i.e. in a lab or on the student's device prior to the assessment). Additional restrictions may need to be established - such as to limit Internet access in on-campus labs.
Further information about SEB is available here https://safeexambrowser.org/about_overview_en.html
b) WIRIS is a plugin for Moodle that adds the ability to enter formulae for Math and Chemistry using either a nested button bar or via handwriting recognition using touch or digital pen input. This is possible anywhere the Moodle content editor is visible - e.g. when large text input boxes such as for content or essay length response fields. This can be used in tests where the student may return responses containing formulae in a Moodle quiz.
Further information about WIRIS is available http://www.wiris.com/en/solutions/moodle
A demo Moodle site for WIRIS is at http://www.wiris.com/moodle/demo/login/index.php
It is possible that SEB and WIRIS could be used in combination within Moodle.
Additional conditions
- The assessment must be delivered using Moodle assessment tools. If SEB is being used then this must be using Quiz. This is to allow SEB to be configured.
- Students should be informed about the purpose and nature of SEB; information will be provided by the PVCE Office for this purpose.
- SEB can be used with Windows, Mac and iOS devices.
- SEB can be configured to allow access to specified resources; staff wishing to use this function should seek advice from the PVCE Office and their local IT support staff.
Alternative assessment platforms to complement the assessment tools in Moodle are being evaluated. Questionmark has been used by Medicine for several years and provides greater features in terms of available item types, quality control processes, secured online and offline delivery, and detailed reports. Examsoft has similar features although it is only delivered in an offline mode and requires students to download the application. Examsoft has been used previously at UNSW by Law.
Further information on both platforms is available:
Additional conditions
- The whole assessment process from writing items to delivery and marking must be conducted within the assessment platform. Student access to Questionmark OnDemand during the assessment will be through Moodle. Student access to Examsoft during the assessment will be through their Examplify client software.
- Students should be informed about the purpose and requirements for using the assessment platform; information will be provided by the PVCE Office for this purpose.
- Both platforms can be used with Windows, Mac and iOS devices.
For further information on the Digital Assessment project please contact the Academic Lead: Digital Assessment (PVC-E office) Dr Mathew Hillier m.hillier@unsw.edu.au