Digital assessment exemplar 18

How would you ensure a team of tutors to mark consistently and to the same standard across a large cohort 450 – 1500 students? Lynn Gribble is using GradeMark to meet these requirements.

GradeMark is a tool within the Turnitin suite of products (also known as Turnitin Studio). Commencing with giving in text feedback you can create and save your most frequently used comments and then drag and drop them to highlighted sections of the text, you can type comments over a piece of text strike through text or create individual comments as you might if hand marking, however you can save any comment as a ‘quick mark’ storing it in a bank of commonly used comments reducing your time to provide feedback.

The rubric feature means you can set criteria and standards to a rubric. By assigning each criterion it reduces marker bias and ensues that sections are weighted in terms of importance for each component of the assessment. It also auto adds each assigned mark for each criterion.

There are two ways to provide overall feedback about the assignment via a general text comment box which in our course we have created some standards for each grade to ensure this is given to each student plus we ask each tutor to add one or two sentences specific to each student. Alternatively, a voice message can be left for each student if the course is mainly online or if you feel this better supports your learning outcomes.

The strategy was to ensure consistent grading but more importantly to show students that grading was closer to a science than an arbitrary mark given. The rubrics are released to students at least two weeks prior to due date for submission. This way, the students are focused on what we will be marking, and it means. When students ask questions about the assessment the team use the rubric as a means to discuss what is required demonstrating how they will earn marks.

This is crucial as often students can go off track ‘failing to answer what has been asked or required’ the rubric shows exactly what is expected and to certain levels from fail through to high distinction. There are only grades awarded reducing the temptation to say it is a high credit when the marks awarded for the section might only be 5 or 10%. In the way the rubric ‘forces’ the decision to look at what level the students achieved or completed the component thus reducing bias or mark forcing.

Since 2011, Lynn Gribble has been managing large teams of tutors marking. All papers are submitted through Turnitin so it also reduces the risk of lost assessments and ensures they are time stamped for when they are submitted, meaning that both the student and tutor can clearly see when it was submitted. As we have large cohorts, with multiple tutors or new tutors grade mark means I can easily oversee what tutors are doing, co mark with them, blind mark when needed and also quality check comments, overall marks and grades and how they are assignment. From an equity perspective it has also meant we can ensure students that no one tutor was harder or easier as a marker. If someone gets sick or has an emergency, it is easy to redistribute the marking among other tutors. It is also available as an app, which works via thin server, meaning you can mark anywhere any time via iPad or a tablet, even on a plane! All your work is stored in the app and uploads once reconnected to the internet.

The outcomes have been excellent with less than one remark requested per year on over 7,000 papers marked annually. Tutor’s confidence and self-efficacy has increased, as has Lynn's ability to confidently tell students they papers have been assessed to a standard.

1. Once setting you assignment, design your marking rubric determining components and weightings. Inform students that all assignments will be marked online within the Turnitin system. Set submission date and time
2. Set post date to one year later to reduce any risk of assessments being released back to students before you are ready.
3. Publish your rubric to Moodle or via Turnitin or both. Inform students to work to the grading and set up Turnitin link including building and attaching the rubric. Share rubric with tutors and discuss the standards. If you have created standard in text comments share the QMS file which all tutors can important to their own Turnitin account
4. Ensure all tutors have Moodle access, if they plan to work remotely, they will need to download the app, generate an access code from within Turnitin via Moodle to be able to download all assignments. If you have a large cohort separate your class by tutorial groups for ease for each tutor.
5. On submission day as Lecturer in Charge you can sort by groups, surname, time submitted or percentage matched, based on the LIC needs you can check all of this prior to tutors commencing marking.
6. Ask tutors to notify you when they have marked between 3 – 5 papers and you can check for consistent application of grades, and comments easily anywhere any time.
7. You can also check progress of all markers and averages. Once you know everything has been completed and you are happy with all results you can set the post date to a day and time that suits your course I.e. not during class or late Friday afternoon.

Pros

• Quick easy anytime anywhere access
• Consistency and quality of application of marks
• You can see if students have read their feedback
• Students appreciate the knowledge of how their papers have been assessed
• Once you have created a rubric you can easily reuse it or repurpose it as needed.
• No need to rewrite the same comments over and over and ability to link comments to rubric criterion
• Students can access all feedback from any smart phone, or device linked to the internet from anywhere.
• Allocate students to their tutor or to other markers as needed
• You can have blind or non-identified markers.
• Digital learning teams can assist you
• You can mark offline

Cons

• If tutors use the same comments to a whole class, the class will easily know.
• From time to time Turnitin goes offline and thus if you are not using the app you could lose access for up to an hour but it is very infrequent.

If you are already using Turnitin just add a rubric. At UNSW we have an institutional license.

• Consider what you are assessing by criterion and design a rubric when you design your assessment.
• Not sure if this is for you? Next time you are marking think about how many times you write the same comment and set up a bank of commonly used comments when you are setting up your rubric to share with your tutors and markers.

Reach out to Lynn Gribble ([email protected]) or the Business Digital team for assistance. Once you get started you will save time, and student angst as everyone can transparently and consistently see what is being assessed