Staff from across all Faculties at UNSW, as well as from teaching and learning support units, engage in research and scholarly investigations into teaching and assessment approaches. Scholarly outputs from their research can be accessed through UNSWorks, the online institutional repository of UNSW research output.
Throughout the Assessment Toolkit, you will find references to scholarship by UNSW staff in assessment. Also here you will find a collection of refereed papers by UNSW staff titled "Marking Time—Leading and Managing the Development of Assessment in Higher Education". Abstracts of the accepted papers appear below. This collection is co-edited by Kathryn Coleman and Adele Flood and is to be published by NEW LEARNING, Common Ground Publishing. The Learner publishes cutting edge ideas on teaching and learning - ideas which point to the emerging shape of a new world of learning.
Abstracts:
MLearning: the potential benefits and limitations in higher education
Dr William Ashraf
Manager, Evaluation Services,
Learning and Teaching Unit (LTU)
University of New South Wales
Sydney, Australia
By 2015, 80 percent of people will access the internet from a mobile device (The Horizon Report, 2011) and it is likely that internet enabled devices with outnumber desk bounded wired computers. The challenge for universities and educators is to keep in step with the students and their insatiable appetite for media rich quality digital content on demand and on the move.
Mobile devices such as smartphones and hand held table devices are becoming ubiquitous and affordable for many especially when bundled with 3G and WiFi access to the internet.
It is perhaps surprising that few universities have grasped the full potential of using these mobile devices for learning, teaching and assessment. This may be due in part to the rapid pace of change in technology and appliances on offer and the fact that our students have been far more ready to accept / embrace the advantages of such devices for communication, online social presence and entertainment.
Potential applications for mobile technologies include broadcasting teaching sessions thereby providing synchronous or asynchronous location-independent learning opportunities, utilising smartphone apps for learning and assessment and finally exploring the use of online games to encourage fuller student participation in learning. The main challenges for faculty when using these technologies are likely to be that they are working in unfamiliar spheres / cultural contexts, the institutional robustness of the campus / teaching space WiFi or 3G signal or load capacity and ensuring equality in terms of their students' access to the necessary resources to support these ventures.
This paper aims to summarise the current use of mobile technologies in the teaching and learning space by exploring the potential benefits and limitations in higher education.
Constructing a Creative Curriculum
Dr Adele Flood
Manager Curriculum Research, Evaluation and Development
Senior Lecturer School of Education
Learning and Teaching Unit (LTU)
University of New South Wales
Sydney, Australia
This paper will discuss how, by changing our focus, we offer opportunities for students to become more autonomous and engaged in their own learning experiences.
In section 1, I will outline recent developments in the Australian tertiary sector with specific reference to the current development of National Threshold Learning Outcomes. I will then outline the current Assessment Project being implemented across UNSW to effect change in Teaching and Learning practice.
In section 2, I will discuss the need to change our perspective towards teaching and learning to incorporate a student-action centred learning approach. I will present a model that supports this approach and will conclude by providing an exemplar of a Visual Arts curriculum that involves Graduate Capabilities and Learning Performance Indicators within the parlance of a creative approach to teaching and learning. This will be considered alongside assessment processes that are "fit for purpose" to provide an ongoing learning dialogue for the student.
Scaffolding student learning by managing development of academic literacies through an oral presentation assessment
Gwyn Jones
The Learning Centre
University of New South Wales,
Kensington, NSW, 2052 Australia
Dr Patsie Polly
Department of Pathology,
School of Medical Sciences,
Faculty of Medicine,
University of New South Wales,
Kensington, NSW, 2052 Australia
Science students often struggle with assessment based on oral presentation of their research findings.
PATH3205 Molecular Basis of Disease is a third year course in Pathology offered to Science, Advanced Science and Medical Science students (n=77). The "Research Group Presentation" assignment has been designed to engage students with their 'Community of Practice' by embedding academic literacies (Wenger, 2008). This assessment facilitates a collaborative learning environment and enables oral communication skills through reporting research by inviting the students to investigate a disease topic and present the molecular basis of that disease.
"The role of the researcher" is the threshold concept that is central to this assignment (Myer & Land, 2006). This requires the students to research their topic, engage in "research thinking and evaluation" of literature, negotiate their findings as a group and graphically communicate their understanding by oral presentation using Power Point.
The process of student learning was supported by embedding two distinctive academic literacy tools within the course. The first being "Science Communication" (SC) 1 and 2, consisting of two consecutive workshops which focused on presentation and collaborative learning skills and the second being the "Research Group Consult" (RGC), a 1hour consultation on presentation structure, format and dialogue specifically targeted to each Research Group. This study reports the process of engagement and the 'researchers' reaction to it. It is supported by formative practical feedback as well as peer feedback.
Study outcomes demonstrated that students in PATH3205 adopted the "role of the researcher" with enthusiasm and engaged with their "community of practice" to meet the requirements of the assessment criteria and communicate their research effectively. The embedding of academic literacy skills had a transformative impact on student learning with improved performance, research thinking, evaluation, teamwork and oral communication. This model could be applied to other courses which seek to develop a similar threshold concept as well as the graduate attributes of oral communication and independent learning.
Production Planning Game: Putting theory into practice
Associate Professor Sami Kara
Director of Research and founder of the
Life Cycle Engineering Research Group at UNSW,
School of Mechanical and Manufacturing Engineering,
University of New South Wales,
Kensington, NSW, 2052 Australia
Kathryn Coleman
Educational Designer
Learning and Teaching Unit (LTU)
University of New South Wales
Sydney, Australia
Through authentic planning and assessment based on a simple production environment, authentic learning is constructed for students through tasks that include problem solving based on real world contexts, supported by scaffolded and modelled support that allow for the development of knowledge construction. As Nunes and MacPherson (2006) confer, "learning activities must be authentic... embedded in realistic and relevant contexts" and "learners must be provided with the opportunity to explore multiple perspectives on an issue" (p.4) through learning experiences that are driven by sound educational choices and pedagogy. This learning should be based on the environment that the student will work in, learn in or utilise post task. The students must also have ownership of both the learning process and the product while having ownership of the learning process throughout.
As Herrington, Reeves and Oliver (2010) confer, the learning space must cater for the needs of the students where possible through cued and modelled support that provides a critical and challenging learning environment. Real world learning supports these knowledge experiences that provide unique learning opportunities. Utilising both personalisation and scaffolding in the face-to-face classroom, students in the School of Mechanical and Manufacturing Engineering learn production planning and control through a Role Play simulation. The three stages of this physical game simulation involve pre-game analysis, role playing and post-game analysis.
Enhancing student learning in written literacy through assessment and community of practice in a second year Pathology course
Dr Patsie Polly
Department of Pathology,
School of Medical Sciences,
Faculty of Medicine,
University of New South Wales,
Kensington, NSW, 2052 Australia
Gwyn Jones
The Learning Centre
University of New South Wales,
Kensington, NSW, 2052 Australia
An integrated contextualised approach to assessment design enhances learning success. Assessments in Science courses are often focused on the description of content and to not directly engage students in critical or evaluative writing opportunities.
PATH2201/2202 Processes in Disease, a second year core course in Pathology (n=220), was the ideal starting point for the development of academic literacies in medical research writing as these are foundational Pathology courses offered to a mixed cohort of second-year Science students within the School of Medical Sciences (SoMS), UNSW.
The "Media Assignment" has been designed to engage Science students into the research experience within their "community of practice" (Wenger, 2008). The assignment places the learners in 'the role of the researcher' by engaging them in their own research inquiry based on a current issue in disease presented in the media.
The aim of this assessment is to develop graduate attributes specifically focused on written communication skills for medical scientific research as well as to enhance self-directed learning and critical thinking skills. To facilitate students writing like researchers, a multifaceted "decoding" tool for research skills in Pathology was developed based on the findings of Middendorf and Pace (2004). This approach incorporated three fundamental stages: 1) Collaboration; 2) Decoding the discipline of Pathology and 3) Design of assessment.
A key element to this process has been the training and engagement of tutors, who were former students in PATH2201/PATH2202 and are now post-graduate or post-doctoral medical research scientists with SoMS. These tutors have made an impact in that they contribute professional expertise to enhance student learning and engagement in this assessment, underpinning the community of practice in medical research.
Findings from this study indicated that innovative assessment design coupled with distinctive learning and teaching approaches in managing both tutor and student support led to increased student confidence about writing like a researcher with improved and sustained quantitative outcomes. This integrated model could be applied to other courses which seek to embed and contextualise graduate attributes for written communication.
Assessment for learning: Postgraduate research and writing skills through scaffolded online peer review
Dr Arianne Rourke
Senior Lecturer
School of Art History and Art Education,
College of Fine Arts
University of New South Wales
Sydney, Australia
Kathryn Coleman
Educational Designer
Learning and Teaching Unit (LTU)
University of New South Wales
Sydney, Australia
An important aspect of the elearning landscape is to provide a well-planned OCL (online collaborative learning) environment that supports the formation of an evolving learning community of practice (Levinsen, 2006; Bullen, 1998).
Powers & Guan (2000) suggested that the future of online learning spaces needed to include an emphasis on improving the outcome of OCL, with prominence being placed on the design of these elearning environments as well as the establishment of clear roles for both the instructor and the student within the virtual learning platform.
Weinberger, Ertl, Fischer & Mandl (2005) recommended that emphasis should also have been placed on the importance of support and effective coordination of the collaborative learning process within the elearning environment through the provision of "scaffolding" of thoughts and ideas.
There have been many studies on the usefulness of peer review in higher education particularly for facilitating OCL (Carlson & Berry, 2003; Raadt, Toleman, & Watson, 2005). Including Brunk-Chavez and Miller (2006) that has suggested that collaborative learning could empower the student and "encourages exploratory talk" (p.5) between students as they explore their learning and conduct their research.
Through the utilisation of Blackboard 9 learning management system, visual learning objects were developed to assist predominately visual learners to assess both their writing and their peers while working through the stages of writing a 10,000-word research paper in one University Semester time-frame.
This paper will discuss how online self and peer review was utilized as an authentic assessment practice to teach Postgraduate students how to write a research paper on Art Administration.
Use of a grid format answer in extended legal problems
Associate Professor Alex Steel
Associate Dean Education
Faculty of Law
University of New South Wales, Australia
Dr Dominic Fitzsimmons
Academic Fellow - Learning and Teaching
Faculty of Law
University of New South Wales, Australia
This paper discusses the use of a grid format in answering extended legal problems as an alternative to the traditional essay format answer. The paper discusses the advantages of such an approach, including:
- Students tend to write discursive essay style answers to legal problem questions. Less able students often gloss over finer points doing this.
- The grid forces students to at times pedantically refer to each element, match a fact and make a simple legal argument
- This shows them how the turning point in a legal argument might well be a technical issue they had overlooked.
- Emphasises the utility of the pigeonhole/mailbox way of thinking in approaching a complex problem
- It creates an efficiency in marking in that the student's reasoning is clearly exposed and reading the far right column is often sufficient to allow an accurate grade—the white space around the cells means it is easy to scan over the other columns quickly to make sure the issues were correctly identified.
The paper goes on to place this approach to writing and grading in a broader theoretical framework of learning, and assesses its place in the broader range of skills and capabilities well rounded law graduate require.
Authentic Assessment, Experiential Learning, and Cinematic Engagement in Architectural Education
Dean Utian
Lecturer
Faculty of Built Environment
University of New South Wales,
Kensington, NSW, 2052 Australia
The study of cinema in schools of architecture provides for the critical engagement of one art form with another. In addition to looking at film for a new perspective and understanding of architecture, the cinematographic techniques applied to the making of short films provides for experiential learning where students develop skills in creative and critical thinking, as well as problem solving. The study of the literature and theory of both film and architecture adds a further layer of critical analysis.
This paper reviews the impact of the cinematic engagement with architecture through the experience of a new 2011 summer semester course called Cinematic Space, within the Architecture Program, Faculty of the Built Environment, University of New South Wales. The major assessment for the course was a short film (no longer than 7 minutes) that explored the theme of space, place and time. Successes in the course were the rich and rewarding experiences for students, a high satisfaction in their short films, and deep evaluation of architectural concepts. The challenges included meeting the right balance of theory in cinema and architecture to technical learning.
Using an Integrated Structured Viva Voce to Assess Application of Foundation Concepts in Professional Practice
Associate Professor Gary Velan
School of Medical Sciences,
Faculty of Medicine
University of New South Wales,
Kensington, NSW, 2052 Australia
Professor Philip Jones
Associate Dean Education
Faculty of Medicine
University of New South Wales,
Kensington, NSW, 2052 Australia
Professor Rakesh Kumar
School of Medical Sciences,
Faculty of Medicine
University of New South Wales,
Kensington, NSW, 2052 Australia
This paper presents how using an Integrated structured viva voce assessed understanding of biomedical sciences at multiple stations in authentic clinical contexts. Each station consisted of multiple structured questions from different disciplines, which were coarsely graded to improve reliability. Pairs of examiners from appropriately matched clinical and biomedical sciences backgrounds were utilised at each station.
As the first cohort of students in the new integrated Medicine program at UNSW reached their senior years, we developed new assessments that aligned with the radical re-design of biomedical sciences teaching. Whereas this had previously been front-loaded in the first three years of a six-year program, a key tenet of the new program was that students should continuously build on their understanding of the scientific basis of Medicine over all six years.
To ensure that knowledge of biomedical sciences was learned and applied in a context relevant to medical practice, we designed an integrated oral assessment that aligned with the new curriculum for Phase 3 (Year 5) Medicine.
In the development process, we had to address the challenges presented by geographical dispersion of both students and teachers, because in the senior years the Medicine program is delivered not only at four metropolitan Clinical Schools but also at 4 campuses of the Rural Clinical School. These issues have been overcome by utilising standardised, structured viva stations. Clear instructions are provided to examiners regarding the standard of expected responses for each question at each station, which permit local and travelling examiners to run examinations at multiple campuses. Examiner training DVDs assist with calibration of grading, thereby improving reliability.
Student evaluations indicated that the examination was an effective stimulus for them to build on their learning in biomedical sciences from earlier years of the program. Senior clinicians remarked on the value of examining with biomedical scientists, thus bringing diverse members of the Faculty together. Our use of structured, coarsely graded vivas has been recognised as a relevant and valid approach by the Royal College of Pathologists of Australasia. Such vivas are also suitable for adaptation to other professional programs where it is desirable to assess foundation concepts in a real-world context
Patchwork texts and the role of feedback in learning
Ann Wilson
Project Coordinator - Teaching Development
Learning and Teaching Unit (LTU)
University of New South Wales
Sydney, Australia
While feedback is a key element in formative assessment, and an activity that takes up much teacher time and energy, we do not know how our feedback is understood by the learner, or what meaning they make of it. Often we do not provide opportunities for learners to demonstrate if they have understood our feedback, and thereby an important opportunity is lost in the learning cycle. In fact, this renders the cycle incomplete because the final stage of it must be that the learner has opportunity, capacity and inclination, to respond to our feedback. Patchwork texts offer this opportunity.
In response to a programme review at the AGSM we introduced the assessment method of Patchwork texts. Winter (2003) explains:
"The essence of a patchwork is that it consists of a variety of small sections, each of which is complete in itself, and that the overall unity of these component sections, although planned in advance, is finalised retrospectively, when they are 'stitched together'.'' (Winter 2003)
"... a form of assessment for learning since the series of patches requires students to accumulate knowledge, adding new knowledge or perspective in each patch, as they head towards an integrated and comprehensive understanding of a topic." (Trevelyan and Wilson, 2011)
Feedback is intrinsic to the efficacy of patchwork text, students are required to submit their first "patch"—for which they get feedback and then resubmit increasingly refined "patches" until the assessment task is complete. This paper will explore the structure of, and student response to, patchwork text assessment.
A collaborative approach to successful assessment review
Professor Alan Crosky
Faculty of Science
School of Materials Science and Engineering
University of New South Wales
Sydney NSW 2052
Belinda Allen
Educational Designer
Learning and Teaching Unit (LTU)
University of New South Wales
Sydney, Australia
Reviewing and mapping course assessments at a program level allows the streamlining and scheduling of assessments to improve student experience and outcomes, and to assess more efficiently. But it can be a daunting and time-consuming exercise, requiring program directors to recruit all course convenors to co-operate in the process.
The School of Materials Science has recently undertaken a program level review and mapping of the courses and assessments in their undergraduate program. This was driven by the need to improve the efficiency and effectiveness of assessment in the program, in particular reducing student and teacher workload and avoiding scheduling clashes for students.
Professor Alan Crosky led this review, and rather than impose top-down implementation, his strategy of asking course convenors in a less formal way to collaborate in the exercise was successful in engaging them to have input into the whole of program structure, with a resulting improvement in assessment efficiency. This approach has had benefits beyond the original intentions, including fostering a sense of ownership over program development by individual course convenors, and the development of collegiality and community within the discipline.
This process will be further developed to map program-level learning outcomes and graduate capabilities in a holistic way.
What's wrong with feedback?
Dr William Ashraf, Sabina Hussain and Sunah Kyan
Dr William Ashraf
Manager, Evaluation Services,
Learning and Teaching Unit (LTU)
University of New South Wales
Sydney, Australia
The higher education sector, both nationally and internationally, has a undergone massive transformation over a few short years with the move from an elite to a universal educationally platform. Students are now a very heterogeneous group with varied abilities, aspirations and "pulls" on their time. It could be argued that today's Gen-Y students require even more support and guidance compared to previous generations to keep pace with the rapid change, particularly with technically based degree programmes, as well as having compete in an ever more aggressive job market. Traditionally, student feedback has been viewed as the vehicle to help students navigate through the complex learning pathway.
Therefore, it perhaps surprising that universities score relatively poorly in their key function of helping students to improve and accelerate their learning. Indeed our students have consistently supplied us with feedback in this regard and we seem reluctant to address, at least at an institutional level, or take the necessary effective strategic and operational steps to improve the situation.
We all need feedback in our daily and professional lives in order for us to improve our performance and excel. But what exactly is feedback? As educators we (and the organisations we work in) seem devote an enormous amount of time and resource into providing comment, advice, collection and distribution of written work for marking/grading. However, results from questionnaires aimed at capturing student voice indicate that this is the least appreciated aspect of our university system.
This paper aims to summarise and evaluate, using statistical analysis were appropriate, the current national and international data on perceptions of the helpfulness of student feedback in the higher education sector, suggest innovate strategies for change and where technology can be harnessed to promote good practice within a learning organisation.
Publication
Accepted papers will be edited as part of a published collection entitled "Marking Time—Leading and Managing the Development of Assessment in Higher Education".
This edited collection will be co-edited by Kathryn Coleman and Adele Flood and is to be published by NEW LEARNING, Common Ground Publishing. The Learner publishes cutting edge ideas on teaching and learning—ideas which point to the emerging shape of a new world of learning http://thelearner.com