Clinical Workplace Assessment App (CWAapp) for assessing clinical skills

The Clinical Workplace-based Assessment application (CWAapp) has been extensively used in Phase 1 for communication skills assessment tasks and has now been successfully employed for miniCEX tasks in the rest of the Medicine program. The App is available for students to download on their phone or tablet (iOS and Android) and allows them to improve the efficiency of in-training assessments whilst providing a flexible system for them to record their clinical experiences. 

The app provides mobile access to in-training assessment forms for clinical supervisors to record students’ performances during clinical placements. The data recorded in the app is then synced to eMed (the Medicine LMS) for collation of results and to be included in the student’s portfolio. 

It also allows students to record non-identifiable data related to patient encounters, effectively acting as a clinical logbook. The information will be stored in UNSW Medicine eMed in the student’s portfolio for later reflection. 

Finally, the app is configured so that changes to in-training assessment requirements in the future can be readily addressed, and new or variant forms adapted and uploaded without onerous administration. 

It starts with having the students download the app to their phone or tablet device via either the Google Play store for android devices, or the Apple app store for iOS devices. Students can search “UNSW CWA” in these stores to find the app quickly. The app also syncs with the e-Med LMS system for synchronization of grades and student data. 

Once downloaded, students then log into the app using their Z-pass credentials. Within the app they will see the series of electronic forms and patient logs elevant to their stage of training and potentially covering a range of different clinical assessments and/or logbooks. When students are about to undertake a clinical assessment, for example a miniCEX in a real ward or other clinical workplace (e.g. GP surgery or community child health clinic etc), they will set up the form ready for use by entering details of the assessor and patient (non-identifiable) or skill to be assessed. The student then hands their device to the assessor (any clinician or healthcare worker qualified to perform the relevant assessment) who will use the form on the app to record the results of the student’s performance, including written feedback and grades.  

The assessor then signs off the on the assessment form within the app before handing the device back to the student. The app will also record metadata to assist in verifying authenticity of the assessment result including time, location and the frequency of submissions. Also, an email will be dispatched to the assessor to verify that they were indeed the assessor of this student for this specific assessment. 

Finally, students are also able to access an area in the app which allows them to enter their own self-reflection on each assessment they’ve undertaken. 

Pros

• Large admin and logistic benefits of not having to deal with any paper-based forms, overall saving a considerable amount of time for both students and teachers; 
• App is always available for students to access, facilitating self-paced and self-initiated assessment schedules and assisting them to complete the reflective cycle; 
• It provides a reliable and secure assessment recording environment that integrates with faculty records; 
• Provides for more efficient teacher feedback and intervention for lower performing students; 
• Analytics on student completions is available which can be used for accreditation and strategic requirements, as well as research purposes;
• Does not require consistent network connection – works offline in wifi dead zones in hospitals! 

Cons

• Some students and assessors, in this relatively early period of use, have submitted feedback on the app itself that suggests it isn’t the easiest to navigate, and may be more difficult than paper forms they’re already used to; 
• Both students and assessors need some degree of introductory training on how to use the app; 
• Students may still face technical issues due to the nature of using their own device and not a centrally allocated device. Medicine is able to loan devices to students in some, but not all locations to circumvent this type of problem;
• Some students are reluctant to use their own device for privacy reasons, though notifications can be disabled when the app is in use;
• Some assessors do not like using the students’ ‘foreign’ devices and find it more difficult to type on smaller devices – loaned devices and voice-to-text can assist in these circumstances. 

The CWAapp has been custom built for UNSW Medicine but a similar solution could be applied to other domains. Many tools currently already exist for the creation of electronic forms on separate platforms, to even simple editable PDFs. Depending on user needs, it may be possible to obtain similar functionality from Qualtrics, Microsoft Forms and others. They are also an increasing number of highly sophisticated, enterprise level system solutions – but at a cost! 

• Introduce the app (or similar solution) to the students and assessors early in the program to ensure they can use it effectively. Start with a pilot and be proactive in involving workplace assessors and offer training if at all feasible. 

• Do not roll out across a program all at once. Start in Year 1, and familiarise new students with the technology, and enhance or increase the number of possible assessments or logbooks as that cohort progresses through your program. 

• Look for champions in both the assessor and student group. Positive word-of-mouth is the best way to promulgate such technology. Respond to negative feedback as quickly and as effectively as possible. Be aware that everyone is learning new tricks and that can lead to pushback – be respectful of this and be prepared to learn better ways to do things from your users. 

The CWAapp can be downloaded here for iOS devices, and here for Android devices.

Test student credentials can be made available. This is best organised through meeting with Dr Silas Taylor ([email protected]) who is responsible for the design, development and implementation of this app in Medicine. 

You can download a presentation on the CWAapp here.

There is a short video from the perspective of an assessor using the app here.