Building on the new wave of AI LLM capability, our AI Conversation Sim project sets out to allow academics to craft specialised scenarios aimed at helping their students learn specific skills related to conversations in their discipline areas. The students can then practice these simulated conversations through VR or Web interfaces that present a 3D environment and virtual AI agents that talk with the player in real-time. Leveraging an authoring tool, teachers can create scenarios covering use-cases such as journalism interviewing practice, board room negotiations and doctors taking patient histories.
Note that this project is being done in coordination with the UNSW IT AI guardrail processes to ensure that development and exploration in the use of AI is performed in safe, ethical and secure ways.
Project Details
The project is in active development and the initial Proof of Concept experiments in several courses have produced exciting results. The simulation uses a game engine visualisation which communicates with specialised AI models like ChatGPT to deliver a fluid conversational experience for the students in a carefully constructed experience relating to their coursework or discipline. There is exciting potential in being able to build experiences with briefing information, objective checking, feedback, timers, emotion, 3D characters, virtual reality, multiple AIs, transcripts, security, achievements, props, gestures and more.
WIP
The work in progress AI Conversation Sim has been explored in several classes to refine the development.
The Journalism class saw students practicing their interviewing skills in VR with an AI Senator and and AI Celebrity with fascinating results. The students reported increased confidence in these sorts of interviews following the experience. One student even commented that she wasn't so comfortable doing celebrity interviews anymore and having to ask those personal questions. This really shows how engaged with the exercise the students were and the level of impact the simulation had.
In the AI Literacy course students debated with the virtual agent and it was fascinating seeing them learn new arguments from the AI which they then used when they switched sides on the topics.
TECHNOLOGY & DEVELOPMENT
The team of staff and work integrated learning students are using Unity as the core of the platform with web services to manage project editing and the API calls to a range of AI endpoints from ChatGPT to Hume.AI.
Use Cases
The aim of the project is to allow academics to create and hone their scenarios within the AI Conversation Sim via the web authoring toolset. Along with the students being able to access the experiences by both VR and Web this will allow the project to work at the scale desired for each course.
Current Use-Cases under Development
- ADA Language Learning – Practice conversations
- ADA Journalism – Interviewing skills
- ADA Education – Parent teacher interviews
- Business – Employees with a disability
- Business – Negotiation approaches
- Business – Board room presentation Q&A
- Business - AI literacy & debate
- UNSW College – English confidence
- PVCE – AI literacy module
- Medicine – Optometry clinical communication
- Medicine – Emergency Dr-Patient interactions
- Medicine – Pharmacy patient history taking
- Medicine – Psychiatry mental health interactions
- Science – Science communication
- Engineering – Collaborative communication
- Engineering - Photovoltaics client meeting
- Law – Class participation practice
Get Involved
There are internal Teams Channels and working groups covering the development and use-cases. To join the team, get a demo or have your use-case part of the development just get in touch.
- Email: [email protected]