Skip to main content Accessibility Statement
Log in
Home Quality and standards August's Quality Quick Win
Quality and standards

August's Quality Quick Win


Author



Sophie Leslie 
Education Quality, Policy and Governance Manager

Quality Quick Wins are practical, topical, bite-sized insights that quality teams can use immediately in their day to day work.

Faster thematic analysis of student survey comments


Using GenAI to identify key themes in free-text student survey responses, ensuring safeguards for data protection, ethics and consent.

The challenge we are trying to address 

Free-text comments from student surveys, such as the NSS, module evaluations and postgraduate surveys, are among the richest sources of insight into student experience, but coding them for analysis manually is highly time-consuming at scale. Quality teams are increasingly resource poor, so valuable feedback risks being missed, skimmed, sampled, or reviewed too late to inform enhancement action. 


What to do 

Use GenAI to run a first-pass thematic analysis of free-text survey comments, identifying recurring themes, sentiment and emerging challenges to focus follow-up action. Data protection, ethics and consent must be built in to surveys from the outset to enable this. 

Quality professionals define the scope and safeguards; GenAI does the coding and synthesis. 


What this looks like in practice
  • Before analysis begins, the quality team must confirm comments were collected under consent enabling AI-assisted analysis. 
  • Safeguarded comments are loaded into an approved, institution-controlled GenAI tool, configured to identify recurring themes, sentiment and specific challenges. 
  • The GenAI tool returns a structured report grouping comments under themes, for example assessment, timetabling or learning resources, with anonymised quotes and frequency/sentiment indicators. 
  • The quality team checks no comment could identify an individual, validates the groupings against a manual sample, and triages findings into suggested key actions. 

Data protection, ethics and consent considerations 

This use case involves personal, potentially sensitive, student data, so safeguards are integral to the process, not an optional extra:
  • Confirm the original consent or privacy notice covers AI-assisted analysis; anonymise data or update consent where it does not. 
  • Anonymise or pseudonymise comments before sharing with any GenAI tool, especially for small cohorts where comments could identify an individual. 
  • Use an approved, institution-controlled environment for any personal or special category data, and complete a Data Protection Impact Assessment where required. 
  • Be transparent with students about AI use, ideally at the point comments are collected, and retain human oversight: treat GenAI themes as a first draft, not a final judgement. 
  • Human bias: reviewers can unconsciously look for the issues they expect (e.g. assessment, timetabling) and skim past others students raise as important – agree the coding frame in advance and use more than one reviewer for the validation sample.
  • AI bias: summarisation can lose or distort context, or add interpretation that isn’t in the original text. Require verbatim quotes rather than paraphrases, and check outputs against the source comments. 

Why this works

 

Free-text survey comments are among the richest but most time-consuming sources of student feedback to analyse manually. GenAI, used with the right safeguards: 

  • Processes large volumes of comments consistently and quickly, surfacing themes a manual sample might miss. 
  • Identifies patterns across cohorts and years that would otherwise take significant staff time to spot. 
  • Builds institutional assurance and trust, through documented safeguards and manual validation of AI-identified themes against unverified over-reliance.

Example impact

 

Swansea University’s quality team piloting GenAI-assisted analysis of internal survey comments (for an appropriately prepared survey) enabled near-immediate initial analysis of all comments, and the identification of key themes, along with the potential for suggested actions to address key student feedback, saving hours of effort. 


Suggested prompt to use with GenAI 

Analyse the attached anonymised student comments. Identify the recurring themes only (e.g. assessment, timetabling, learning resources).

 

For each theme give: theme name, number of comments, overall sentiment, and 2-3 verbatim quotes copied exactly as written – do not paraphrase, shorten or summarise any comment, and add no interpretation beyond the themes and quotes requested.

 

List anything that doesn’t fit a theme as ‘Uncategorised’.

 

Present as a table: Theme | Comments | Sentiment | Quotes.  



Key takeaways for implementing tomorrow

  • Check what GenAI tools are already approved by IT and data protection teams before doing anything else; never paste comments into a public chatbot. 

  • Ensure that any data collected has appropriate consent embedded before data collection commences. 

  • Pick one small, low-risk dataset to start with, such as one module's evaluation comments, rather than an institution-wide dataset. 

  • Anonymise that dataset yourself first, then sense-check the AI's themes against your own quick read of a sample, so you know how much to trust the output. 

  • Note the safeguards you applied as you go, then share draft themes with one programme team as a conversation starter before scaling up. 

  • Write strict prompts: fix the themes/format required and state explicitly that comments must not be summarised, paraphrased or interpreted – only quoted and grouped by theme.