Description
UNSW REVIEW is an online criteria-based marking and feedback system developed originally at University of Technology Sydney. It is used at UNSW Business School and several other faculties at UNSW in over 150 courses per term for online marking, feedback, marks management and assurance of learning (Program) reports.
REVIEW is popular with teaching and learning oriented academics, as it efficiently supports marking and enables the provision of targeted feedback to students. One long-term user is Gigi Foster (School of Economics, UNSW Business). Gigi has used REVIEW since 2011 in her ECON1401 course. She values REVIEW amongst other things for its ease of use (including online storage and accessibility of marker feedback), visual clarity, capacity to elicit and store student self-assessments, and structure centring on the learning criteria for each assessment item.
Each course is configured to include the marking rubric for each assessment, with each assessment criterion on the rubric displayed in Review and linked to Program learning goals. To mark assessments, Gigi and her team of markers grade students’ performance and enter feedback on each criterion through Review’s online interface. She can monitor the marking of her tutors within the course in real time while marking is still in progress via a ‘staff-averages’ report that tracks the quantity of feedback and the criterion-level averages and histograms of the marks awarded by each marker. This report helps to identify marker inconsistency as an input to the moderation process before the publication of marks, improving the quality and consistency of marking and feedback.
For several years, Gigi has embedded self-assessment on each criterion into her assessments on REVIEW. Typically, Gigi gets over 90% of students to complete self-assessments and write a short reflection justifying the marks they awarded themselves and, for later tasks, describing how feedback from markers on prior assessments was incorporated and responded to in the current task. Thousands of students have now undertaken this activity in ECON1401. Student feedback has been generally positive, and this activity introduces students to a foundational skill highly prized in the Business School graduates: the development of independent evaluative judgement and reflection skills.
As well as through self-assessment, REVIEW encourages student engagement with assessment through peer feedback practices. Staff can develop whole-of-class or group-based surveys delivered to students through a web-form. These allow individuals to rate other team members, or members of other teams to rate (and optionally give feedback to) other teams. This feature has proved very popular due to its intuitive and visual interface and is now used in dozens of UNSW Business School courses every term (including ECON1401).
Gigi has contributed to the development of marking data visualization screens in REVIEW. The marking visualization screen contains a series of intuitive, high-value graphs that provide ‘actionable intelligence’ on a prime concern of those awarding or moderating marks, specifically, mark distributions. The visualization of the numeric data allows practitioners to quickly see what’s happening in their marking, saving time and enabling prompt response if needed.
Deployment
While currently used in only four faculties, all courses in the university and the student enrolment lists for all courses are present in REVIEW via a daily automated feed, similar to the Moodle feed. Access to the platform is provided through request to the local Faculty REVIEW / TELT administrative team. Once set up, staff enter the platform and set up the assessments, which can be cloned from previous courses, a process taking one or two minutes. Once assessments are configured, staff provide / unhide a link to the REVIEW platform for students to access via the course Moodle page. The REVIEW link is a standard inclusion on the UNSW Business Moodle template. Students can simply use Z-pass credentials to log in to REVIEW.
As the course begins and assessments are submitted by students via a separate platform (e.g., Turnitin, Moodle Assignment Tool, etc.), Gigi and her course tutors mark individual students’ work (via the in-built Tutorial groups filter in REVIEW, if desired) or group work (where a mapping of students to groups is uploaded through a CSV import) in REVIEW. Students are marked (individually or as a group) against the criteria created on REVIEW on the standard university 0 – 100 scale and provided with criterion-specific typewritten feedback. Markers can also save further time by accessing the universe of comments made by the whole marking team from the current or previous term’s marking, which are stored in the platform. As marking proceeds, REVIEW automatically calculates and displays the emerging overall assessment mark and grade for each student by using pre-defined weightings of the criteria.
Once marks and feedback have been entered and moderation has occurred, email notifications can be sent to students stating feedback has been released. At this point students can log into REVIEW access to their grading dashboard (like a home page). Feedback and grades can be released separately if desired – Gigi might choose to do this to encourage students to read their feedback first to ensure they have absorbed this information, rather than solely focusing on their grades. Students can export a PDF of their assessment grades and feedback which many have used to supplement their work portfolios. Crucially, students retain access to all their feedback from all courses on REVIEW right throughout their degree. As access to the REVIEW courses is not term-limited, students can develop and access an ‘e-Portfolio’ of their assessment experience and feedback, which they can export as evidence of their task and course-based performance throughout their degree.
Finally, marks can be ported over to other platforms (e.g. ASTRA / ECCLES / Moodle) by staff or TELT administrators.
Pros and Cons
Pros
- Encourages criterion- and rubric-based marking and linked feedback practices
- Is visual and user-friendly for staff and students, providing online access for students to task feedback from the moment that feedback is approved for release
- Enables student engagement in assessment through self- and peer-assessment, developing student’s evaluative judgement and reflection
- Used continuously for ten years; stable. reliable, Award-winning software
- Provides a centralised location for assessment performance data throughout a student’s degree program
- Supports evidence-based moderation and standardisation in marking
- Polished interface that is accessed using Z-id credentials
- Automatic grade calculations
- Built-in graphical visualisation of marking allows for interpretation and analysis of assessment;
Cons
- Teachers can only map one program goal to each assessment criterion, whereas often a criterion might map to several program learning goals, and vice versa
- REVIEW is still a separate platform – it is not closely integrated with Moodle or other assessment gathering platforms
- Assessment tasks must be submitted through a different platform (e.g., Turnitin, in-class presentations, etc.)
- Grading export and importing to central platforms (e.g., ECCLES) is still a manual process – the marks transfer process at the end of a term takes a few minutes per task / course.
- Not designed for close-editing and marking-up of student work
Getting started
REVIEW is currently supported directly by Faculty administrative staff in the UNSW Business School, School of Art and Design, Built Environment and UNSW Global. Staff in other faculties should contact either the UNSW Business Digital Learning Team or the PVC(E) TELT team to discuss access.
Login to REVIEW via https://unsw.review-edu.com/unsw. If you can’t see your course code, request access from the relevant support team in your Faculty.
Staff and students log in to REVIEW using standard UNSW Z-id and ZPass. To activate a REVIEW course or add staff, please quote the course code and semester and the staff first and last name and Z-id to your Faculty contact. Once the Lecturer in Charge has been added, that person can then find and add other staff markers without further assistance from administrative staff.
Students are automatically added to REVIEW courses via an overnight university enrolment feed. The recommended way of giving students access is to REVIEW is to add a url link on Moodle. You may also wish to send out an announcement via Moodle with the link in it as well. Example instructions follow:
REVIEW Link on Moodle
Title: Get Task Feedback on REVIEW
URL: https://unsw.review-edu.com/unsw
Open the “Showcase” section for more guides.
Best practice tips
When creating criteria, think about the association of the specific criteria to the longer -term skills and attributes of your degree. Link each criterion to its closets ‘match’ from the Program Goal options that REVIEW presents as part of the task – criteria set up.
Don’t be afraid to enable self-assessment in REVIEW and embed it in your expectation of student engagement with your assessments. Tell students to self-assess prior to submitting their work. This step can act as both a ‘check-list’ to self-check that they’ve completed all required task elements, and a self-identification step regarding the quality of the work they’re submitting. If students identify a weak performance area on a particular criterion, they have the opportunity to improve their work before submission. Everyone wins!
Showcase
Coming soon.