Introduction
ChatGPT has not been endorsed or approved by UNSW IT Cyber Security. In the situation where ChatGPT is used, it is crucial not to enter your own or others' academic work or research, as well as student work, into ChatGPT. Submitting personal or sensitive information to ChatGPT means it could be used for training the large language models (LLMs), risking potential release to third parties. UNSW has adopted the specific generative AI tools identified below to support teaching at UNSW.
What AI tools has UNSW adopted for staff use?
Turnitin has developed a tool to detect writing within an assignment submission that has been AI generated. The tool is featured in the Similarity report and will appear as an additional box that gives a percentage of the paper identified as AI-generated. Clicking on the box will open a report that highlights text in the submission potentially written by AI.
July 2024 Turnitin AI Report Updates
On July 16th, Turnitin are releasing updates to their AI writing detection tool, with the launch of AI paraphrasing detection. In addition to Turnitin’s existing tool identifying text that is generated by AI (rather than a human), we understand that Turnitin’s updated functionality will also identify passages that may contain AI-generated text that has been modified to disguise its origin using text spinners – tools that rephrase or rewrite text to convey the same meaning with different wording (e.g., QuillBot).
The AI paraphrasing detection is part of a package of updates that are being automatically rolled out. Turnitin say that the AI paraphrasing detection will not identify originally human-written text that was passed through a text spinner. At UNSW, we believe this is because the detector is identifying AI ‘fingerprints’ left from the original generation.
Passages where ‘AI -paraphrased’ text has been identified will be highlighted in a different colour to the existing AI generation detection. This will assist in allowing you to distinguish between the two detection methods.
UNSW has not yet had an opportunity to test the new feature. As with the existing AI detection tool, please use professional judgement when interpreting any reports. We will provide a further update if any issues come to light.
Additional updates included in this July 2024 release:
- AI score presentation change – To avoid potential incidence of false positives, where a submission has less than 20% AI writing detection, no score or highlights will be shown on the submission (i.e. 20% is now the lowest score that will be reported). This will apply to all submissions made after July 16th.
- Increased supported word count – The maximum word count limit for AI writing detection will be increased from 15,000 to 30,000 words.
Should I use Turnitin’s AI writing detection tool?
There is no expectation to use the tool, but staff will likely want access to it and should apply professional judgment to its reports.
Like the Similarity test, the Turnitin AI detection tool will not definitively prove misconduct but alerts markers to investigate further. Other detection tools are being announced but should not be used until they comply with UNSW data privacy obligations.
There is no fool-proof way to detect AI-generated text. The issue is complex because authors can add quirks during editing that do not resemble AI, such as deliberate spelling mistakes or asking GenAI to imitate a specific author's style.
Who can see the Turnitin AI detection report?
The report is only visible to teachers and not students.
How accurate is the Turnitin AI detection report?
A high similarity percentage in Turnitin's report is not definitive proof of student misconduct. It should be taken as an indication of possible unauthorised AI use in assessments.
Care must also be taken in the reliability and accuracy of the tool since Turnitin has indicated that:
- The tool only picks up text generated from GPT-3 and GPT-3.5 but not the recently released ChatGPT-4 (Turnitin plan to expand their detection capabilities to other models in the future).
- The occurrences of potential false positives where the tool can incorrectly identify fully human written text as AI generated (see next section on false positives).
When could a false positive occur?
Turnitin’s AI writing detection model only looks for prose sentences contained in long-form writing, such as an essay or extended response. Hence it can return false positives for:
- Short documents with only a few hundred words, where the prediction will be mostly “all or nothing.”
- Lists without a lot of structural variation.
- Text that literally repeats itself.
- Text that has been paraphrased without developing new ideas.
After further testing, Turnitin has determined that there is a higher incident where the tool can incorrectly identify fully human written text as AI generated when there is a score of less than 20% detection of AI writing in a submission. From July 16th, to avoid further potential instances of false-positives, where a submission has less than 20% AI writing detection, no score or highlights will be shown on the submission (i.e. 20% is now the lowest score that will be reported).
Is there any AI-generated content the Turnitin AI detection tool cannot detect?
The tool cannot detect AI-generated text in the form of non-prose, such as poetry, scripts or code.
Nor does it detect short-form/unconventional writing such as bullet points, tables, or annotated bibliographies.
Investigating for AI content
Colleagues are reminded, if Turnitin identifies potential AI writing or AI paraphrasing in a student’s submission, this is merely a flag that may indicate the need for an academic investigation. Markers should rely on their own professional judgement, not just the AI detection tool. All investigations of academic misconduct must follow the UNSW Plagiarism policy and procedures. Some helpful links;
- Check the UNSW Key principles of AI usage in assessment.
- Have a look at these Conversation Starters and the Investigating Student use of Generative AI guide if you think your student has misused Gen-AI.
- Connect with your School Student Integrity Adviser (SSIA) or the UNSW Conduct & Integrity Office for additional support. Contact the team via Conduct & Integrity Office website or Conduct & Integrity SharePoint.
Guide
Find out how the Turnitin AI detection tool works
FAQs for AI Writing Detection Capabilities
Got questions? Access answers to any additional questions you might have
Microsoft Copilot (previously known as Microsoft Bing Chat) is a generative AI tool which can create images, sounds and text, including essays, summaries, songs, stories and poems. Copilot is now available for use at UNSW. Staff and students can log into Microsoft Copilot using a UNSW [email protected] account at copilot.microsoft.com.
The AI Ecosystem has made Microsoft Copilot available for general use at UNSW because Copilot provides users with commercial data protection. When UNSW staff and students access Copilot using a UNSW zID, their data is encrypted and not seen by humans. The tool does not store or use the information entered once a session ends. This ensures that a user's work and private information are not used to train AI tools.
As an educational institution, UNSW embraces the use of innovative technologies, but also emphasises using these technologies responsibly. When using Copilot and other forms of AI, please remember to review IT’s guiding principles for Ethical and Responsible Use of Artificial Intelligence at UNSW.
Staff and students may have concerns about what AI tools such as Copilot mean for privacy and academic integrity. It is important to stress that the extent to which students can use Copilot is entirely dependent on what is specified in their course. Approved assessment instructions templates to help define acceptable uses of AI in coursework and assessments are available to use and tailor for specific courses. UNSW Current Students has also provided official guidelines on Referencing and acknowledging the use of artificial intelligence tools for students.
More in-depth information, including a rationale for Copilot's availability at UNSW, can also be found at UNSW IT Microsoft Copilot.
Adobe Firefly is a generative AI system that creates images and improves creative workflows using text prompts. It can also add or remove objects from an image, transform text and more. The AI is trained on Adobe Stock images, openly licensed and public domain content, making it safe for commercial use. For additional information, please access this guide to Adobe Firefly.
UNSW staff and students now have access to Scite.ai, an AI tool for discovering and evaluating articles. Scite.ai predominantly focuses on the sciences, but also includes some humanities journals.
Scite.ai allows users to see how a publication has been cited by providing the context and describing whether the article provides supporting or contrasting evidence, or whether it only mentions the publication.
UNSW has an organisation-wide license for students, teaching staff and research staff to use Scite.ai. Register or log in to Scite.ai here: Scite.ai UNSW Registration and Login
Generative AI has the potential for increased efficiencies for academics in research as it does for students. Any use of GenAI should abide by the ethical requirements of the academic field and the required disclosures of any particular publication.