In pursuit of pukka assessment: bringing together Generation Z and Generative AI

Generative AI promises to transform the world of work. Graduates with skills in ethical and proactive use of such tools are in great demand. Digital Technologies and the Future of Work is an online module for undergraduate students from across the university. It was developed in 2018 as an Education Incubator Project, linking students in Exeter and Penryn without the need to travel between campuses.  

The module provides students with opportunities to study new digital developments while putting the skills they acquire directly into practice. The largely asynchronous, fully online format accommodates students facing health, work, or other commitments, enabling flexible, inclusive participation.  

As the module Educators, we took a sanguine rather than adversarial approach to student use of Generative AI in learning situations, by demonstrating that its use need not focus simply on capacity for plagiarism. 

For their assessed work, students were asked to develop a portfolio of evidence analysing their experiential learning journey through the module. Within this, our focus on Generative AI was a deliberate ploy to encourage reflection on the range of technical competencies, interpersonal abilities, and personal attributes required for graduate success.  

What exactly was the task? 

Students were asked to pose a rather vague question in ChatGPT related to the module curriculum: “In what ways are businesses making use of tools such as ChatGPT?”  

They then had to “improve” the initial response by thinking critically about successive, more specific, iterations of the question, cross reference the responses received with other sources to check for accuracy and add relevant contemporary examples. They were assessed on their reflection of the value obtained from working through this process, not on the content of the final answer.     

When planning the tasks, we put a lot of thought into what guidance should be provided to reassure students about the value of working in a very new and controversial area that is changing all the time: 

• We were concerned that media “hype” about generative AI often painted a very negative picture – by focusing on concerns over cheating, or on doomsday scenarios for many of the careers students may have aspired to follow. As one student noted: “I never thought I’d see the day when I was asked to use ChatGPT in one of my essays”. 
• We wanted to make sure that the playing field was level in terms of access to the tools, and did not advantage those able to pay for premium versions. To address this, we acknowledged that while new and “better” tools were emerging on a regular basis, we would mandate use of the free ChatGPT  3 for this task.  
• We also had a “back up” assessment prepared which required a more “traditional” approach, just in case anyone voiced particular objections to using such tools. Given that all the students had chosen to study this optional module specifically focused on digital technologies and the future of work, we certainly hoped the backup would not be necessary (it wasn’t!).  
• Finally, we sought advice from a number of colleagues in terms of the most appropriate wording of the task and the extent of the support students might need in preparing their work.  

In week 8 of the module, the students were invited to a webinar demonstration of how to use ChatGPT effectively. It was led by Pierre de Brosses, a GenerativeAI Specialist and one of our Digital Learning Developers. As a recent graduate himself, he was able to empathise with the students who had never completed this type of task before, and who had varying degrees of prior experience in using ChatGPT:

The idea was to enable them to take their first steps into GenAI, and understand the transferable skills of persistence, iteration and feedback that will be useful across their whole skillset. Students seemed to resonate with the idea of using GenAI to extend their abilities and go further than they ever could on their own. (Pierre de Brosses, Digital Learning Developer) 

Following the webinar, they had 3 weeks in which to finalise their portfolios and submit them for assessment. Many students highlighted this session as the best part of the module, with one claiming that he was unable to stop thinking about it afterwards.    

So how did the assessment go?! 

The degree of effort and enthusiasm students applied to this task exceeded anything we have previously worked on. The relevance of the task to their current university priorities and preparation for the workplace was very obvious. The module grades awarded were higher than in previous years, and some students improved what might have been an underwhelming grade by producing significantly stronger work in the Generative AI aspects.  

As an example of the type of feedback we received, Felicity said:  

Interacting with ChatGPT for this task encouraged me to think critically and strategically. The process of asking questions to ChatGPT and exploring different angles of the same question has increased my critical thinking and communication skills. The action of changing a few words to enable a more informative and evaluative response has enabled me to create better styled questions for any queries during my learning at University and for any future career plans. (Felicity Thomas, BSc Business Strategy and Marketing)   

What are the next steps? 

We are currently sharing our results via event presentations and posts like this to raise awareness of the potential of generative AI to improve the learning experience and employability of our students. Similar tasks which build on our term 1 experience are currently in progress within a large term 2 module (360 students). 

In addition, we hope our work may encourage a long overdue change in assessment priorities and help inspire a culture of resilience and critical thinking in our learning environments.  To conclude with a challenge:  

There is no part of our graduates’ future careers that will not be impacted by AI over the coming decades. We need to find more examples like this one which normalise and integrate AI into the Higher Education experience in ways that will equip them for this new era. (Raphael Dennett, Deputy Director, University of Exeter Centre for Entrepreneurship) 

For more information please contact: