Digital assessment exemplar 16

Concept and knowledge mapping have been demonstrated to enhance meaningful learning and critical thinking in a variety of disciplines through visualisation of interrelated concepts. However, providing manual feedback on students’ maps is not feasible for large classes without using technology. Accordingly, a user-friendly, valid and reliable, automated online tool for assessment and feedback of students’ maps might have significant benefits for learning. No such tool providing automated scoring and feedback on students’ maps is currently commercially available.

Knowledge Maps educational software was developed at UNSW utilising a Learning and Teaching Innovation Grant. Knowledge Maps is a web-based system integrated with Moodle that can be used to create, edit and share maps, as well as providing automate feedback on students’ inputs. Knowledge Maps enables the development and implementation of online concept / knowledge maps as learning activities and/or formative and summative assessment activities with automated scoring and feedback. This tool has potential benefits for learning in a variety of disciplines.

A novel approach to assessment is the use of online concept and/or knowledge maps. Concept and knowledge maps are graphical representations of knowledge and have been used in all levels of education to promote meaningful learning, critical thinking, and problem-solving skills. concepts (enclosed in boxes) and are joined by lines or arrows and labeled with linking phrases. Characteristic features of concept and knowledge maps are a hierarchical structure (with broader concepts at the top) and cross-links (showing how ideas from different domains relate to each other).

Gary Velan and colleagues trialed Knowledge Maps for novice medical students enrolled in their first course of a six-year undergraduate medicine program, in the second formative assessment for the 2017 course. This assessment included a modified essay question on the pathogenetic mechanisms underlying the clinical features of acute inflammation (a major focus of the course).

Following submission of their answers to the modified essay question (and prior to receiving feedback on their response), students were provided with a link to an online knowledge-mapping activity on acute inflammation covering similar concepts to the modified essay question. No training was provided to students other than the brief instructions at the top of the activity, as seen in the main figure above. Participation was voluntary and students were free to continue with their formative assessment without accessing or completing the map. 103 students attempted both the mapping activity and the modified essay question.

Gary Velan and colleagues found that overall, there was a significant correlation between map scores and modified essay question scores. Responses to the questionnaire regarding use of knowledge maps for assessment were positive.

Many students perceived that the mapping tool improved and motivated learning, helped them identify important concepts, was simple to use, and they would recommend to others. The students also commented that the maps were easy to use, and that they helped link everything together.

Using Knowledge Maps for assessment minimises cueing effects, because answering by the process of elimination is unlikely to be effective when faced with lists that contain 20 or more options, typical for a knowledge map, which is akin to extended matching MCQs. Teachers have the advantage of being able to determine different options for each testable node and set up functioning distractors based on known misconceptions. The platform also enables automated grading that alleviates the marking burden and could be beneficial in formative settings by increasing the opportunity for students to receive feedback on their learning. Additionally, the same map can be used to assess different aspects of a topic by testing different nodes, as well as varying the degree of scaffolding by altering the number of nodes tested.

1. Please refer to the Knowledge Maps User Guide to add a Knowledge Maps activity in UNSW Moodle. This guide shows how to create, edit, share Knowledge Maps, as well as the difference between Activity Mode and Assessment Mode.
2. Provide a brief instruction for students, in the map, explaining how to navigate and use the knowledge map.

Pros

• In Activity Mode, Knowledge Maps can be used as a tool for learning about relationships between concepts in any discipline, while providing automated feedback on students’ submissions.
• Studies suggest that Knowledge Maps in Assessment Mode provide a similar indication of students’ understanding of a topic as a modified essay question.
• High re-usability. The same map can be used to assess different aspects of a topic by testing different nodes, as well as varying the degree of scaffolding provided by altering the number of nodes tested.
• Time saving with instant, consistent computer grading and feedback.
• Web-based, accessible on all devices.
• Teachers can import Expert Maps created with CMap Tools.

Cons

• Current lack of web-presence (web-based docs and tutorials).
• Current lack of institutional support via the TELT platform.

To get started, please refer to the Knowledge Maps User Guide or get in touch with Gary Velan.

Please see below links to 3 screenshot videos showing staff how to create a knowledge map, create a learning activity using the map, then creating a link to the learning activity in Moodle:

Creating an Expert Map

Creating a Mapping Activity

Linking to Moodle

• Create an Expert Map first, then create a Scaffold Map by removing nodes and/or linking phrases from the Expert Map.
• It is recommended that dropdown lists be used to enable cueing for novice learners in a discipline, whereas free text may be appropriate for more advanced learners.
• Provide a short and concise guide on how to control and navigate the map.

Please contact Gary Velan for demos.