Comparisons of some different ways to evaluate teaching

I am the first year director in the school of Physics. Most of the courses I am responsible for are large first year courses which are blended or entirely online. My largest course is Physics 1A, a blended course with up to 1600 students which is predominantly taken by first year engineering students. I have also designed an entirely online course, Everyday Physics, which has around 1000 enrolments each year. In 2018 I introduced an online graduate certificate in physics for science teachers.

The main tool I use are concept inventory tests. Concept inventory tests use multiple choice questions to test the students understanding of core concepts. The tests are given to students before they start a topic and then again at the end of the topic. The results of the pre and post tests are compared to find out how much the students have learnt. Normalized gains can be calculated for a cohort, the normalized gain the average post test score minus the average pre test score (how much they have learnt) divided by 100% minus the average pretest score (the maximum possible amount a student could learn). I give these tests to the students online and have a bonus mark to motivate the students to attempt them.

I have compared the learning gains of students in the online course I designed to a face-to-face course which delivers similar material. I found that the learning gains were slightly higher in the online course. I also looked at gender differences, in the online course female students had higher learning gains than male students. This was not the case in the face-to-face course.

In the large Physics 1A course we have not found very significant differences in the learning gains of male and female students. This is a good result as it is quite common in physics courses for learning gains of female students to be lower.

The positive results from the online courses that I introduced gave me the confidence to develop an online degree. Having found that students can learn as much (or even more) in a well-designed online course I developed an entirely online degree; the Graduate Certificate in Physics for Science Teachers. If students were learning less in an online setting, I would have had reservations about introducing an online degree. Further impact has been demonstrated by through my numerous conference presentations. I have given a number of conference presentation about my work, including a Plenary talk at the Australian Institute of Physics meeting. As a result, a number of institutions are looking at introducing courses similar to Everyday Physics. I have published two papers on this subject but my main focus is on applying what I have learnt and found to improve student learning in my courses.

Concept inventory tests can be downloaded from Physport: https://www.physport.org/

A seminal paper using results from concept inventory tests to show the importance of interactive engagement:

Hake, R. R. (1998). Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses. American journal of Physics, 66(1), 64-74.

My education papers:

Malik, U., Angstmann, E. J., & Wilson, K. (2019). Learning and Conceptual Change in Thermal Physics Concepts: An Examination by Gender. International Journal of Innovation in Science and Mathematics Education (formerly CAL-laborate International), 27(1).

Ng, W., & Angstmann, E. (2017). Promoting Physics Literacy through Enquiry-Based Learning Online. Journal of Education in Science, Environment and Health, 3(2), 183-195.