Evaluation of peer assessment

Nagisa Fukui

I have been coordinating the first-year Japanese course for over 10 years in the School of Humanities & Languages. The course is a gateway course for the Japanese major program, and it has between 350-500 student each year. The Japanese program at UNSW aims to foster students who can connect with each other and with other Japanese speakers by using Japanese. Behind this aim is the sociocultural understanding of languages and learning. Our program is envisaged as a network of Communities of Practice (Wenger 1998) where our students engage in Boundary Crossing (Ishiyama 2018). CoP is a type of community where its members regularly and interactively engage in participatory practices to achieve shred goals. In the UNSW Japanese Cop, students the Japanese language in networked communities, within and beyond the classroom and the course.

Kar Ming Choong

I teach a third-year, undergraduate accounting course in the School of Accounting, Business School. This course has enrolments around 700 students each year and has adopted the flipped classroom mode of delivery since 2011. A combination of the flipped delivery and learning outcomes means that student teams are critical features of this course. I implemented peer assessment practices in team environment to provide feedback to students and address teamwork challenges such as freeloading and fairness.

Nagisa Fukui

I use two different types of peer evaluations within the course. The first one is a task conducted every week. At the end of each tutorial, our students perform a self-created role-play for a particular situation. Then two pairs become a group and evaluate the other pair’s performance by using an evaluation sheet that is created by me. This practice not only encourages our students to use Japanese with each other, but it also creates bonding between students. In the class, they pair-up with different peers every week as a group. As a result, they get to know each other very well and a positive learning environment is created. The other peer evaluation tool is used as a part of the formal assessment. In the course, we have an Interaction Test which is worth 25%. Our students perform their role-play in front of the class, and the students evaluate them and select the top 3 performance pair. Those placed as the top 3 pairs receive 2 marks as bonus marks.

Kar Ming Choong

I am currently using the peer evaluation software that was developed specifically for the Business School. Initially, I used the software to gauge students’ contributions in team projects at the end of the semester (summative). Later, I incorporated continuous peer evaluation by having it 4-5 times in the semester (summative). Then, I realised that students would benefit from receiving anonymous feedback from their team members in the first round of evaluation (formative) before proceeding with the summative rounds. I also used the peer evaluation software as a way to punish students (i.e., lower contributions will lead to lower team project marks) and to reward students (i.e., out of 5% standalone mark). I included extra questions in the evaluation survey and discussed with some students to obtain feedback about this assessment.

Nagisa Fukui

I organised a pilot study on the second evaluation system in 2014 by conducting follow up interviews two years after the evaluation tool was applied. I also organised an evaluation survey. My students appreciated the fact that their voices counted formally as a part of their assessments. Secondary to this is that this method made our students watch and oversee student performances and Japanese usages. As a result, the students reported that they gained valuable learning from their peers. Additionally, the students disclosed that it boosted their confidence and motivation. They realised that they actually understood Japanese quite well. They were able to acknowledge that their class mates’ usage of Japanese was of high calibre and they recounted that this inspired them. The students reported that the test atmosphere was more fun and engaging and I believe that this created a nicely shaped conclusion to their achievement.

Kar Ming Choong

Results show:

1. the means for the overall question are remarkably consistent across the rounds of evaluations (i.e., from week 4 to week 12);
2. the means for the overall question are skewed to the right of the distribution that is, on the generous side of the evaluation scale (i.e., 4.8/5%).;
3. although anonymous in nature, students were still reluctant to evaluate another team member harshly as they felt it was not truly anonymous and can therefore expect retaliation in the next round of evaluation;
4. in the formative round, students prefer to provide written, qualitative feedback instead of quantitative feedback as they felt the latter was too “judgemental” in the early phase and could ruin team harmony;
5. students prefer more items/criteria to evaluate their team members;
6. the peer evaluations encouraged students to improve their teamwork performance to enhance team effectiveness (scale SD [1]-SA [7], mean 4.81; mid-point = 4);
7. the peer evaluations encouraged other team members to improve their teamwork performance to enhance team effectiveness (scale SD [1]-SA [7], mean 4.76; mid-point = 4); and
8. students agree that the requirement to continuously evaluate their team members increase their preference for teamwork (scale SD [1]-SA [7], mean 5.79; mid-point = 4) .

Nagisa Fukui

When I conducted this pilot study, I had not designed the first type of evaluation (task one) yet. One of the comments that I received from students when I was conducting this study was that they were not used to evaluating peers and felt uncomfortable. I responded to the students reporting on this through introducing my classes to practice peer evaluation every week. The impact on students of the regular use of this task was that the students became more comfortable, happier and more able after the Interaction test.

Kar Ming Choong

Using the feedback above and a reference below, I am now adopting peer assessment in teamwork with the following features:

1. punish students who did not contribute sufficiently in team projects, rather than rewarding students (out of 5%);
2. maintain the formative nature of peer evaluation in the first round of evaluation but solicit written feedback only and exclude peer ratings in the first round;
3. no feedback will be released to students from subsequent rounds of evaluations (where ratings are requested) until the end of the semester to obtain a more honest assessment without the fear of retaliation from team members;
4. emphasise the two main findings from research at Google Inc to students (see reference). To create a successful team, the team must have 2 key characteristics: 1. Equal voices: When everyone in the team feels like they can speak up without fear or embarrassment. 2. Social sensitivity: When team members show they are sensitive to how one another feels.

Nagisa Fukui

http://japaninstitute.anu.edu.au/sites/default/files/u6/02_Nagisa_Fukui-JSAA2013.pdf

Kar Ming Choong

https://www.nytimes.com/2016/02/28/magazine/what-google-learned-from-its-quest-to-build-the-perfect-team.html?ref=oembed