Varvara Papazoglou is a Teaching Associate in the School of Computer Science, University of Sheffield. In this post Varvara shares the work she led to investigate student perceptions of AI that found growing confidence, excitement and acceptance. The work was presented at the 18th annual International Conference of Education, Research and Innovation (ICERI 2025).
The study started during the academic year 2023-24 and was
intended to track how the perceptions, expectations, and technical
understanding of our undergraduate students shifted as they gained direct
experience with GenAI technologies. With a team that included Alex Lucas and Prof. Rob Gaizauskas, we redesigned the first
two lab sessions for a first-year module called Machines and Intelligence so
that they included a practical introduction to Generative Artificial
Intelligence (GenAI). Our study spanned two academic years, and
tracked how the perceptions, expectations, and technical understanding of
students shifted as they gained experience with GenAI technologies . To measure
this, we invited volunteer participants from both cohorts to complete two
online surveys, one before and one after engaging with the GenAI-focused
curriculum of the module.
These questionnaires were administered three to four weeks apart, using unique IDs to match participant responses while preserving their anonymity. Structurally, they consisted of 13 core questions common to both iterations. Questions identified the specific GenAI tools students utilised and their typical purposes for doing so with, and 5-point Likert scales were used to evaluate viewpoints on GenAI's performance, the identification of AI-generated content, and general feelings of excitement or concern. The perceived impact of AI on future careers, potential job obsolescence, and the ethics of GenAI use in university and workplace settings were explored. To ensure a comprehensive analysis, most of these questions included non-mandatory open-ended sections for qualitative insights, concluding with a final opportunity for participants to offer additional comments.
Testing Turing
The first lab focused on a practical adaptation of the Turing Test.
●
As interrogators, whether they
successfully identified the machine and what specific linguistic cues or
mistakes led to that conclusion.
●
As the interrogated group, whether
they successfully deceived the interrogators into making the wrong
identification, and whether this success was attributed to the AI’s
capabilities or their own careful prompt engineering or other factors.
●
Whether the Turing Test remains a
valid benchmark for machine intelligence, based on their experience with the
given task.
The second lab shifted towards the ethical and practical implications of AI in Higher Education. After reviewing online University resources regarding guidelines and policy on the use of GenAI, students analysed and critically evaluated 10 short descriptions of use cases on GenAI in Higher Education, split into two groups of five, involving use by educators and five use by students, ranging from AI-generated lecture content to code authoring with GenAI.
Groups debated the advantages and disadvantages of these scenarios, helping them develop an informed, critical perspective on how to use these tools responsibly throughout their degree.
Confidence, Excitement and Acceptance
To understand how these labs changed perspectives, we
conducted pre- and post-task surveys, analysing 33 matched responses from the
2023-24 and 2024-25 cohorts. The results revealed the following in student
literacy:
●
Initially, students were fairly
confident that GenAI could handle complex tasks. After the labs, this
confidence actually dropped (from an average of 3.8 to 3.5 on a 5-point scale),
suggesting that students gained a first-hand understanding of the inherent
"hallucinations" and limitations.
●
Student confidence in their
ability to identify AI-generated content (based on cues such as unnatural or
"over-polished" text) rose slightly, from an average of 3.2 to 3.5.
Overall, as indicated by their responses to our survey questions, most participants
had some experience and a good understanding of cues that can reveal GenAI
output before our teaching interventions. However, among participants with no
prior experience, greater confidence in recognising AI-generated material was
observed.
●
Interestingly, despite becoming
more aware of the risks, students' willingness to use GenAI in their studies
increased (from 3.7 to 4.3). In addition, in their comments, students
emphasised that, because GenAI tools and technologies are here to stay, the
priority must be acceptance and education, so people are aware of how to use
them effectively to maximise utility while minimising risk.
●
While the media often portrays AI
as a threat to jobs, our students largely viewed it as a transformation. Many
saw the obsolescence of certain tasks not as a negative, but as an opportunity
for new roles and increased productivity. Interestingly, students expressed
similar feelings of excitement and acceptance for GenAI use in both Higher
Education and the workplace.
Conclusion
Our findings suggest that early, structured engagement with GenAI, tied to subject-specific material, can foster more balanced and informed perspectives among students. Students with more awareness of risks and limitations seem to be more optimistic of the impact and potential of GenAI. However, we are mindful of the modest scope of our current research. Given the relatively small sample size and the specific context of our introductory module, these results serve as preliminary insights rather than definitive conclusions. We have identified areas for improvement, particularly in deepening students’ awareness of ethical and societal risks while reducing the likelihood that students feel overly confident about their ability to identify and manage AI-generated content without necessarily being aware of the full range of risks.
Varvara Papazoglou is a Teaching Associate in
the School of
Computer Science, University of Sheffield. Her research interests include Artificial
Intelligence, Music Information Retrieval, Computational Linguistics and
Natural Language Processing, as well as Educational Research in Computer
Science.
Citation: V. Papazoglou, A. Lucas, R. Gaizauskas (2025) SHIFTING ATTITUDES TOWARDS GENERATIVE AI: A STUDY WITH FIRST-YEAR COMPUTER SCIENCE STUDENTS, ICERI2025 Proceedings, pp. 4438-4448. https://doi.org/10.21125/iceri.2025.1290