Digital Assessment Project Pilots

We will discuss with you the best way to proceed according to your context and experience with the selected digital assessment tool/approach.

• In general we plan to run pilots as either:
  • Proof of concept done as a mock assessments or formative assessment i.e. the assessment cannot contribute any mark, irrespective of weighting, to the course result. We recommend that those new to digital assessment try a mock assessment first before moving on to undertaking a real assessment.
  • Low stakes pilot assessments (real assessments) i.e. the assessment will contribute to student's mark in the course. We recommend this for those already conducting digital assessment or those who have already run a proof-of-concept.
• The types of computer devices may include:
  • Desktop devices in existing computer laboratories; this will restrict participation to those with access to computer laboratories and appropriate local IT support.
  • Students' own devices; while this will allow greater flexibility and broader participation, it will also entail the additional local responsibility of supporting students' devices. The PVCE Office is unable to provide backup devices or desktop support.
• The types of assessment items (dependant on the tool chosen):
  • Selected response items e.g. MCQs, T/F Qs, graphical interactions.
  • Constructed responses involving short text answers or simple numerical responses.
  • Constructed responses involving long text answers (essays).
  • Digitally inked responses.
• Subject to local support availability, the preferred number of students per setting is less than 100.

Staff participating in the pilots will be responsible for:

• Writing the assessment items in the selected platform; basic support will be provided to staff in the use of these platforms.
• Implementing the preferred tool to deliver the assessment; the School’s professional and technical staff will be required to implement the assessment; basic support will be provided to staff in the use of these tools.
• Recruiting students to participate in the pilot; student participation may be part of the usual course activities or may be a separate activity.  If the pilot is conducted as a course activity, students who do not wish to participate or who are unable to participate must be offered the same assessment under usual conditions (e.g. an option for pen-on-paper).
• Providing technical support to students using their own devices prior to and during the assessment; local IT staff will be needed for this purpose. The PVCE Office will provide generic resources to inform students.
• Locating the physical space for the pilot; dedicated physical spaces for the pilot will not be available; the pilots will need to be conducted in available teaching spaces.
• Providing invigilation, if required.
• Encouraging students to complete evaluation forms and surveys.
• Completing staff evaluation forms after the pilot.
• The status of ethics protocol approval will be discussed as part of our preparatory discussions. As of August 2019 an ethics protocol is in place with a defined set of data collection tools. Individual academics may want to be added to the protocol and/or suggest changes to the data collection tools. With that in mind the time line for changes to ethics protocols needs to be kept in mind.

• Provide support to staff on the use of tools and their associated integrations with TELT.
• Provide information to staff for supporting students and provide an escalation point on difficult problems.
• Manage application capacity and service availability with the vendors.
• Conduct evaluation, including providing evaluation forms, and report back to participants.

a) Safe Exam Browser (SEB) has been selected as the preferred add-on application for securing devices using Moodle digital assessment tools. SEB can lock down the device so that the student cannot access unauthorised resources during the assessment. SEB is integrated with Moodle. The SEB application must be installed on the computer being used for the assessment (i.e. in a lab or on the student's device prior to the assessment). Additional restrictions may need to be established - such as to limit Internet access in on-campus labs.

Further information about SEB is available here https://safeexambrowser.org/about_overview_en.html

b) Digital inking using Poodll has been implemented and is now available for use in UNSW Moodle - however this is still in pilot phase and as such limited support is available at this time. UNSW has only implemented the 'drawing tools from the Poodll suite. With that in mind information about Poodll anywhere (only the simple whiteboard tool) https://moodle.org/plugins/tinymce_poodll and the Poodll recording question type (only the 'picture' function) https://moodle.org/plugins/qtype_poodllrecording Note: Digital Inking can be combined with SEB for use in summative assessment.

Additional conditions

• The pilot assessment must be delivered using Moodle assessment tools. If SEB is being used then this must be using Quiz. This is to allow SEB to be configured.
• Students should be informed about the purpose and nature of SEB; information will be provided by the PVCE Office for this purpose.
• SEB can be used with Windows, Mac and iOS devices. However full computers or laptops are preferred.
• SEB can be configured to allow access to specified resources; staff wishing to use this function should seek advice from the PVCE Office and their local IT support staff.
• Use of Digital Inking tools require the use of digital pen input devices. Examples include commodity USB graphic pen tablet (approx $60) or touchscreen laptop with pen input device. Schools and faculties may be called upon to fund the purchase of these devices for use by students.

For further information on the Digital Assessment Project please contact Karsten Sommer k.sommer@unsw.edu.au