Assessing with Role Play and Simulation
Role play and simulations are forms of experiential learning (Russell & Shepherd, 2010). Learners take on different roles, assuming a profile of a character or personality, and interact and participate in diverse and complex learning settings.
The terms "role play" and "simulation" are sometimes used inconsistently or interchangeably. However, "simulations" often involve a familiar or realistic situation in which a participant’s role may not be as prominent or distinctive as it would be in a role play. Frequently simulations incorporate role play, leading to the term "role-playing simulation". The difference is generally one of degree rather than kind.
Role plays and simulations function as learning tools for teams and groups or individuals as they "play" online or face to face. They alter the power ratios in teaching and learning relationships between students and educators, as students learn through their explorations and the viewpoints of the character or personality they are articulating in the environment. This student-centered space can enable learner-oriented assessment, where the design of the task is created for active student learning. Students are actively involved in both self and peer assessment and obtain sustainable formative feedback.
When to use
Good-quality learning design provides opportunities for situated and authentic learning. As Figure 1 indicates, high quality learning is situated in a real-life context and simulates the learning activities of the graduate's employment area.
Figure 1: eLearning opportunities to address principles of high-quality learning design (Wills, Leigh & Ip, 2011).
|Boud & Prosser: Summary of Principles|
|Engage learners||Consider learners' prior knowledge and desires and build on their expectations.|
|Acknowledge learning context||Consider how the implementation of the learning design is positioned within the broader program of study for the learner.|
|Challenge learners||Seek the active participation of learners, encouraging them to be self-critical.|
|Provide practice||Encourage learners to articulate and demonstrate to themselves and their peers what they are learning.|
|Siemens & Tittenberger: Summary of Principles|
|Social||Learning is a social process and knowledge is an emergent property of interactions between networks of learners.|
|Situated||Learning occurs within particular situations or contexts, raising the importance of educational activities mirroring actual situations of use.|
|Reflective||Learners require time to assimilate new information.|
|Multi-faceted||Learning incorporates a range of theory, engagement, "tinkering" or bricolage, and active construction.|
Authentic learning and task design provides students with:
- opportunities to reflect on the way knowledge will be gained in real life
- activities that are authentic in nature
- modelling of expert performances and processes
- the opportunity to learn about new perspectives and roles in life
- opportunities to reflect on learning.
- opportunities to see how tacit knowledge can be made explicit
- scaffolding and coaching at critical times in the learning and assessment process
- assessment that is aligned with learning objectives within the task.
Role plays and simulations significantly contribute to students' learning and assessment when they allow students to view multiple perspectives on their responses in a safe but challenging environment.
Widespread evidence suggests that educators and students experience satisfaction with assessment-as-learning through role play, games and simulation (Russell & Shepherd, 2010). Simulated learning environments (SLEs) provide a safe, supportive environment where students can develop their clinical skills, competency and agency.
SLEs are also flexible and controllable, and educators find they can design suitable and varied education events within them.
The blended learning environment can provide face-to-face students with a virtual classroom where students and educators can deliver content and interact in a simulated learning environment. This benefits both staff and students; it has, however, meant a shift in practice for the educator as the room changes to accommodate the required infrastructure and seating arrangements.
The benefits listed below are attributed to either role plays or simulations; however, some benefits can be attributable to both forms to some extent.
Benefits of assessing by role play
- Role play is an excellent means of evaluating decision-making and interpersonal communication skills.
- Role play is particularly useful to students who will operate in a tense professional environment (e.g. diplomacy, acute or sensitive medical care settings, psychology and counselling) or requires complex decision-making.
- Scenarios can be scaffolded, gradually increasing in complexity to ensure that students reach a sufficient level of competence.
- Role plays help you evaluate students' ability to work under pressure and with others, including providing opportunities for inter-professional learning.
- With online SLEs, students can role play anonymously.
Benefits of assessing by simulation
- Simulation is a form of authentic assessment. When exposed to active, experiential, reflective and contextual learning approaches such as simulated environments, students can see the direct relevance of their educational experience to their future practice.
- Educators can assess a student's preparedness for the practical placement component of their degree.
- Technology-based forms of simulation can enable instant feedback to students.
- Simulations are effective means of evaluating students' competencies, such as their professionalism, as well as their content knowledge.
In the medical and allied health literature, SLEs are consistently found to have 3 significant benefits. They:
- promote an increase in self-efficacy in clinical decision making,
- improve clinical communication skills, and
- promote greater awareness in students of the role they play in a collaborative care setting.
As with the benefits, the issues are listed here as characteristic of either role plays or simulations; however, some apply to both forms.
Challenges of assessing by role play
- Role plays are resource intensive and both costs and available time will constrain them. Constraints can be reduced by developing a bank of role play scenarios and sharing role play resources, as has been done with Project enROLE. Many universities utilise shared training and evaluation centres in the fields of medicine and allied health.You can reduce the setup cost of role plays by utilising the university platform. You can also use many learning designs that are distributed online at no cost.
- A new platform for learning, requiring students to learn new skills just to participate in the learning, can distract them from the conceptual learning the role play was intended to promote.
- Institutional Learning Management Systems that require students to tick a box to ensure that each of their posts is anonymous can compromise student anonymity.
Challenges of assessing by simulation
- For students who struggle with public speaking or group participation, simulation assessments can create so much anxiety that it affects their performance or participation.
- It is impossible to genuinely recreate authenticity in a simulated environment. The most you can do is use different aspects of simulation to cater to the assessment needs of the students.
- Students need to be guided throughout a simulation, and learning must be scaffolded.
- It is advisable to flag the timing of simulations with your colleagues, as preparing for a simulation can prevent students from completing other learning and assessment work.
- Setup costs can be significant.
- Staff and students' accessibility can present challenges during simulation setup.
- Facilitators sometimes need to invest significant time learning the tools required to develop a simulation, to track and structure activity and to monitor and communicate with students during simulations. This investment can only benefit their teaching, but expect time-pressed teachers to be resistant at first.
When assessing role plays and simulations, we recommend that you:
- align the task with the learning outcomes and structure it accordingly
- provide clear and explicit information as to what is expected of students
- ensure that the task is authentic and real-world based
- scaffold the learning experience, breaking tasks down to manageable size
- use both formative feedback and summative assessment.
Once debriefing sessions have been held, evaluate the learning design. Gredler (1996) suggests using a 3-step evaluative procedure to redesign a role play or simulation:
- Document the design validity of the innovation.
- Verify the cognitive strategy and the social interactions using formative feedback and redesign them where needed.
- Conduct follow-up evaluation and research on specific processes and effects of the learning and assessment.
Online role play is described by Project EnROLE as having the following characteristics (Wills, Leign & Ip, 2009):
- It is designed to increase understanding of real-life human interactions and dynamics.
- Participants assume someone else's role or place themselves in someone else's situation.
- Participants undertake authentic tasks in an authentic context.
- The task involves substantial in-role interaction with other roles for collaboration, negotiation and debate.
- Interaction between roles takes place substantially online.
- Learning outcomes are assessable and generate opportunities for student reflection.
These characteristics can also aid in setting up the face-to-face learning environment of a role play or simulation.
As Figure 2 demonstrates, adaptive learning in role plays includes modelling and input from students that can alter the learning outcomes. These disciplines can utilise this type of active and adaptive learning and can film it for evaluation (including peer- and self-evaluation). Actors can be used to perform the role of a patient or client, so that students' communication and clinical decision-making skills can be explored. Actors are usually trained in the details of a case, in the array of issues and behaviours a patient or client is likely to present to the health professional, and to replicate the performance from student to student to ensure standardisation of assessment . Studies have shown the level of standardisation achieved is usually very high.
In the following table, Siemens and Tittenberger (2009) outline the ways in which you can use role play with other experiential learning. They enumerate the opportunities that role-based eLearning provides for high-quality learning design and the tools that can be integrated into the learning experiences. The elements of design for authentic eLearning specified by Lombardi (2007) include basing a learning task on real-life problem solving, within a meaningful context. Online role plays embody Lombardi's suggestion that a learning task provide long-term student engagement with learning, involve a variety of resources and perspectives over a sustained period of time, and entail collaborations to promote engaging, open conversation.
Figure 2: Role play in comparison with other experiential learning activities (Wills, Leigh & Ip, 2011, modified from Siemens & Tittenberger, 2009).
|Type of learning activity||What is it?||Media forms||Technique||Technologies||Tools|
|Assimilative||Processing narrative media—managing and structuring information||Lectures, DVDs or reading texts||Concept mapping, brainstorming, buzzwords, crosswords, defining, mind maps, web search||Word processing software, presentation software, text, image, audio, video||CMAP, Hot Potato, Google, MS Office products, social bookmarking, blogs, wikis, pageflakes, Google reader|
|Adaptive||An environment that changes according to learner input||Simulations, games||Modelling||Virtual worlds, models, simulations, games||Second Life, MMORPG|
|Communicative||Discussing||Asynchronous or synchronous discussions, chats, text messages||Reasoning, arguing, coaching, debate, discussion, negotiation, performance||Electronic whiteboards, email, discussion boards, chat, instant messaging, VOIP, videoconference, web conferencing, blogs, wikis||Online bulletin board, Skype, IM, Facebook, social bookmarking, blogs, wikis|
|Productive||Learners producing something||Creating, producing, writing, drawing, composing, synthesising, remixing, mashups||Artefact, book report, thesis, essay, exercise, journalling, literature review, multiple choice questions, puzzles, voting portfolio, product, test||Creative applications (image editing, CAD, design software) computer-aided assessment tools, electronic learning environments||Indesign, Photoshop, YouTube, Google Video, Office software, Sketch|
|Experiential||Interactive activities that focus on problem solving||Practising, applying, mimicking, experiencing, exploring, investigating, performing||Case study, experiment, laboratory, field trip, game, role playing, scavenger hunt||Virtual lab, 3D immersive environment||Google Earth, MMORPG, Second Life|
Online role play/simulation
Important steps in the process:
- Design the problem.
- Design the rules and roles.
- Set up the scenario.
- Assign student roles.
- Consider the practical limitations of playing out the simulation, and make adjustments to the task design as necessary.
- Develop moderation and other necessary skills.
- Assess the technological requirements and develop them as necessary.
- Assist in developing students' skills in asynchronous posting online.
- Develop students' understanding of the "story" that accumulates as they post.
- Develop their skills in reflective practice; use a blog, journal or wiki space during the role play or simulation, and afterwards to assess students' participation and to evaluate the role play or simulation as a learning experience.
You can use role play and simulation within a game, or use a game within a role play or simulation. Games are engaging, can be highly authentic and can incorporate a competitive element, up to and including advancement to the next stage or problem, or winning a prize at the conclusion of the game. Game feedback is generally immediate, reinforcing the student’s application of subject matter knowledge.
The success of using games in learning relies on the application of strict rules. In the academic setting they should meet 2 requirements (Gredler, 1996):
- Random factors should not contribute to winning.
- Winning should depend on the application of knowledge of the subject matter.
Games have 4 general purposes in learning and assessment, says Gredler:
- for practice and refinement of skills
- to aid in identifying gaps or weaknesses in knowledge
- for review or evaluation
- to learn new ways of investigating concepts and principles in the learning of problem-solving skills.
Crookall and Saunders (1989) view academic games as a simulation—a representation of an authentic real-world system that can itself take on some aspects of reality for participants or users. Games are useful tools as feedback responses for students; a key characteristic of game learning is that one cannot progress to the next stage of a game without gaining the knowledge to accomplish the requisite task. Groups as diverse as the American military and the National Association of Home Builders in the United States invest in games that represent and instruct their particular content and views (Squire, 2006). "Serious games" such as the US Army's America's Army are designed to impart their content by immersion of the players in game-playing activities.
Simulations have the "potential to develop students' mental modes of complex situations as well as their problem solving strategies" (Gredler, 1996). You can use experiential simulations in a number of strategic ways for groups or individual students, and assess them using various techniques. Some examples of simulations are:
- data management—often team-oriented and containing variables that are manipulated
- diagnostic and crisis management—cause and effect contingencies are drawn from real cases; experts aid in the working through of the task.
- social-process—These simulations require the learners to personally interact with the situations, and can have unexpected outcomes.
Experiential learning that focuses on the interactive activities of problem solving fits with Kolb’s experiential learning cycle (Kolb (1984). Kolb differentiates learners according to which feature of the experiential learning cycle they prefer: concrete experience, active experimentation, reflective observation or abstract conceptualisation. Kolb (1984) developed this concept of the learning process to "ensure that teaching and tutoring activities give full value to each stage of the process. This may mean that for the tutor or mentor, a major task is to 'chase' the learner round the cycle, asking questions which encourage reflection, conceptualization, and ways of testing the ideas" (Atherton, 2010).
- In the course outline, give students adequate warning of the workload requirement for the simulation. This gives them a chance to opt out if their current load is already heavy.
- Arm students with the required content.
- Have them work on a position paper and an objective sheet from the initial stages.
- Conduct surveys before and after the simulation, and implement a debrief questionnaire.
- Dedicate the final class to debriefing the students about the process and evaluating the learning within the simulated environment.
Virtual reality and other online learning tools
Virtual reality and other online tools can play an important role in both online teaching and simulated environments. They provide what Russell and Shepherd (2010) referred to as optimal elements of learning design, a complex social learning space and reflective practice.
Educators commonly combine these simulated environments for assessment. For example, combining a part-task trainer within a role play scenario, or a DVD can create a high-fidelity simulation environment.
Low-fidelity SLEs such as case studies and role-plays are being overlooked "despite an established base of research to support their effectiveness". Published research does indicate that higher fidelity is better; however, the effectiveness of any simulation technology depends on how it is used (Beaubien & Baker, 2004, 55).
High- and medium-fidelity mannequins
Mannequins are commonly used by medical and allied health disciplines to assess clinical competency, such as self-confidence, clinical judgment, interpersonal communication and inter-professional teamwork. More common to the medical and allied health disciplines are part-task trainers, which utilise anatomical models or computer software models that replicate a specific physical intervention; for instance, a spinal simulator to assess physiotherapy students' ability to perform passive oscillatory movements.
- Online simulations: Harvard Business for Educators
- Project enROLE: encouraging role based online learning environments
- Resources from Project enROLE
- Health Workforce Australia National Simulated Learning Project. Report for Physiotherapy. This report looks at assessing with high- and medium-fidelity mannequins.
Atherton, J. S. (2010) Learning and Teaching: Experiential Learning, retrieved 15 February 2012.
Beaubien, J.and Baker, D. (2004). The use of simulation for training teamwork skills in health care: how low can you go? Quality & Safety in Health Care 13(s1): i51–i56.
Chin, J., Dukes, R. and Gamson, W. (2009). Assessment in Simulation and Gaming: A Review of the Last 40 Years. Simulation and Gaming 40(4), 553-568.
Crookall, D. and Saunders, D. (1989). Towards an integration of communication and simulation. In D. Crookall and D. Saunders (eds), Communication and simulation: From two fields to one theme. Clevedon, UK: Multilingual Matters.
Decker, S., Sportsman, S., Puetz, L. and Billings, L. (2008). The evolution of simulation and its contribution to competency. Journal of Continuing Education in Nursing 39(2), 74–80.
Freitas, Sara I. (2006). Using Games and Simulations for Supporting Learning. Learning, Media and Technology 31(4), 343–358.
Gaba, D. (2004). The future of simulation in health care. Quality and Safety in Health Care 13(s1), i2–i10.
Gredler, M.E. (1996). Educational games and simulations: A technology in search of a research paradigm. In D.H. Jonassen (ed.), Handbook of research for educational communications and technology (pp. 521–539). New York: Macmillan.
Hofstede, G.J., de Caluwe, L. and Peters, V. (2010). Why simulation games work—in search of the active substance: A synthesis. Simulation and Gaming 41(6), 824–843.
Klassen, K. J. and Willoughby, K. A. (2003). In-Class Simulation Games: Assessing Student Learning. Journal of Information Technology Education 2.
Kolb, D.A. (1984). Experiential Learning. Englewood Cliffs, NJ: Prentice-Hall.
Kolb, A. and Kolb, D. (2005). Learning Styles and Learning Spaces: Enhancing Experiential Learning in Higher Education. Academy of Management Learning and Education 4(2), 193–212.
Lombardi, M. (2007). Authentic Learning for the 21st Century: An Overview. ELI Paper. EDUCAUSE.
Reckien, D. (2010). Urban Sprawl: Using a Game to Sensitize Stakeholders to the Interdependencies Among Actors' Preferences. Simulation and Gaming 41(2), 260–277.
Russell, C. (2009). A systemic framework for managing e-learning adoption in campus universities: individual strategies in an institutional context. ALT-J Research in Learning Technology 17(1), 3–19.
Russell, C. and Shepherd, J. (2010). Online role-play environments for higher education. British Journal of Educational Technology 41(6), 992–1002.
Shute, V. J., Ventura, M., Bauer, M. I. and Zapata-Rivera, D. (2009). Melding the power of serious games and embedded assessment to monitor and foster learning: Flow and grow. In U. Ritterfeld, M. Cody and P. Vorderer (eds), Serious games: Mechanisms and effects (pp. 295–321). Mahwah, NJ: Routledge, Taylor & Francis.
Siemens, G., & Tittenberger, P. (2009). Handbook of emerging technologies for learning. Winnipeg: University of Manitoba.
Squire, K. D. (2006). From content to context: Video games as designed experiences. Educational Researcher 35(8), 19–29.
Vincent, A. and Shepherd, J. (1998). Experiences in Teaching Middle East Politics via Internet-based Role-Play Simulation. Journal of Interactive Media in Education 1998(3).
Walker, C. (2009). Teaching Policy Theory and its Application to Practice Using Long Structured Case Studies: An Approach that Deeply Engages Undergraduate Students. International Journal of Teaching and Learning in Higher Education 20(2), 214–225.
Wills, S., Leigh, E. and Ip, A. (2011). The power of role-based e-learning: Designing and moderating online role play. New York: Routledge.
Zagal, José P. (2010). Time in Video Games: A Survey and Analysis. Simulation and Gaming 41(6), 844–868.
The contributions of staff who engaged with the preparation of this topic are gratefully acknowledged.