
King’s guidance on generative AI for teaching, assessment and feedback
Supporting the adoption and integration of generative AI
Generative AI is poised to create transformative changes, impacting not only the educational sector but also the wider professional world. In the face of these advancements, a purely defensive approach, centred on detecting and reducing AI usage, does not sufficiently equip our students for an AI-driven future.
The PAIR (Problem, AI, Interaction, Reflection) framework, developed by King's College London Professor Oguz A. Acar, aims to proactively integrate AI into our curricula. See more for further details of the framework.
The framework is designed to be (i) simple, providing a straightforward structure to harness the potential of generative AI; (ii) customisable, allowing adaptation to align with specific learning objectives and student characteristics; and (iii) compatible, building on established pedagogical approaches such as problem/inquiry-based learning and active learning, making it suitable for different disciplines.
The three key tenets of the framework are:
Viewing AI as a tool to augment, not replace, human insight, judgment and creativity.
Focusing on the development of transferable skills (i.e. problem formulation, exploration, experimentation, critical thinking, and reflection) over mastering specific tools.
Promoting effective and responsible use of generative AI tools.
This guidance document outlines the fundamentals of the PAIR model and discusses its implementation across various disciplines and courses.
The PAIR framework is designed with simplicity, customisability, and compatability in mind, allowing it to be seamlessly adapted across various disciplines. It emphasises the cultivation of skills that are transferrable and not tied to specific algorithms or AI tools.
The PAIR process is encapsulated in four steps (see Figure below):
Students define the problem or challenge they aim to address.
Students identify and choose the most suitable generative AI tools for their needs. This involves exploring, comparing, and evaluating the features of various generative AI options.
Students deploy the selected generative AI tools to tackle their problem, experimenting with diverse inputs and critically evaluating the outputs.
Students assess and report their experiences with the generative AI tools.
Acar, Oguz A. (2023). “Are Your Students Ready for AI? A 4-Step Framework to Prepare Learners for a ChatGPT World”, Harvard Business Publishing Education. Accessed at: https://hbsp.harvard.edu/inspiring-minds/are-your-students-ready-for-ai?
The PAIR offers flexibility in application across diverse coursework designs. Below are some ways how PAIR can be tailored to suit different academic objectives and contexts.
The PAIR framework can be seamlessly incorporated into existing coursework or serve as a foundation for a new (formative) assignment.
For the former application, module leaders could simply invite students to tackle the whole task or challenge highlighted in the assignment, or parts of it, using the PAIR framework. For the latter, the module leaders could identify new problems that could potentially benefit from harnessing generative AI.
It is worth noting that "problem" is broadly interpreted as the task or challenge integral to the assignment. This encompasses, but is not limited to, diverse tasks such as analysis or comparison (for instance, of legislation, data, code, case studies, theories, movements, or ecosystems) and creation or design tasks (like developing algorithms, artworks, models, business plans, research protocols, simulations, or chemical compounds).
In addition, the framework is suitable for both team-based and individual assignments.
Please see Section 4 for a specific example of how to implement this, with sample material that could be directly used or modified for your own module.
Each component within the framework can be adjusted based on learning goals, available resources, university and assessment-specific marking criteria, and the unique characteristics of the student body to ensure the framework complements the learning environment and supports achieving learning objectives.
This is the foundation of PAIR. Approaches to this can range:
This can also be tailored per the module’s needs. Module leaders might, for example, suggest a list of freely accessible foundational models, such as Bard, Bing AI, ChatGPT 3.5, or Claude 2.1, in alignment with the learning objectives. In addition, or alternatively, students might be encouraged to explore and select AI tools based on their research. If the second option is preferred, it is important to ensure all students can access the recommended AI tools and recommend that they solely rely on freely available tools.
While there are many AI tools available, with dozens emerging weekly, the majority of current freely-available foundational models are sufficient for most purposes. Table 1 presents the key characteristics, strengths, and weaknesses of the primary freely available tools.
This stage encourages students to actively engage with their chosen AI tools. Emphasis can be placed on experimenting with different techniques to understand the tools' capabilities, or on the critical evaluation of results.
Depending on learning objectives, module leaders may guide students to delve deeper into understanding advanced prompting strategies, feedback mechanisms, and the nuances of AI responses. Alternatively, the focus might shift to scrutinising the quality, potential biases, and underlying logic behind AI-generated outputs.
This phase encourages students to introspectively analyse their experiences with the AI tools. Module leaders can provide avenues for students to contemplate the broader implications of AI outputs.
Topics of reflection might encompass AI's societal effects, its influence on specific disciplines, professions, or tasks as well as their identity. Aligning these reflective prompts with course objectives encourages students to better contextualise their learnings and the potential ramifications of AI in the wider world.
In adapting the PAIR framework to coursework, it is important to tailor the complexity of the assignment (e.g. whether an inquiry is open vs. closed) based on student characteristics such as their prior knowledge and cognitive abilities. It is also important for module leaders to have basic familiarisation with the main AI tools presented in Table 1.
Chat GPT 3.5 |
The free version of Chat GPT. This is the most popular generative AI tool and one of the best ones. It is fast and performs well in various tasks including writing and coding. |
Bing AI |
A very capable free tool that requires a Microsoft account (which could be created for coursework purposes) and Microsoft Edge browser. Its main advantage is internet access. It offers three modes: creative, balanced, and precise. In "creative mode", it grants free access to the model behind the paid version of ChatGPT (GPT4). It can also be used to generate images building on DALLE text-to-image model of Open AI. |
Claude 2.1 |
A powerful model performing well in writing and coding tasks (surpassing even ChatGPT4 model in some coding tests). A distinct advantage is its ability to handle large documents (PDF, Word), and it possesses a larger memory than other foundational models. With a later training cut-off (compared to ChatGPT 3.5), it offers more recent information. This model is viewed by some as more “ethical” as it uses a model called Constitutional AI (which draws principles from the UN Universal Declaration of Human Rights, among other sources). |
Bard |
Similar to Bing AI in that it has access to the Internet. It requires a Google account to sign in. While this is currently less capable than other options, Google is expected to release a new, highly capable model soon. |
Note: These models are regularly updated; as a result, while the details provided here are accurate as of the time of writing, they may change in the future.
This section provides an illustration of the incorporation of the PAIR Framework into traditional coursework, such as an essay or report that involves researching and writing about a topic (see also another sample assignment for a marketing module).
Note that the content in italics represents materials that could be shared with students directly or after minor modifications in your coursework guide and/or module outline.
Firstly, module leaders will want to clarify for students that they are encouraged to use generative AI tools for this assignment. The following is a suggested text that could be included in the coursework guidance:
In this assignment, you are not only allowed but also encouraged to use AI tools. The underlying motivation is preparing you for a future that will be increasingly AI-driven by equipping you with key skills to harness these tools effectively and responsibly.
To guide you in integrating AI tools in your assignment, we will be using the PAIR (Problem, AI, Interaction, Reflection) framework.
Next, you will want to expand on the different components of the PAIR framework.
Regarding problem definition, module leaders need to decide whether they want an open, closed, or semi-open inquiry. Using the broad topic of “The Impact of AI in the Next Decade” as an illustration, the following are examples that vary in their levels of 'openness':
Regardless of the type of inquiry chosen, it is vital to stress to students the importance of having a clear understanding of their “problem”— in this case the objectives and requirements of the essay:
Before engaging with AI, make sure that you have clear understanding of your task, and its requirements and boundaries.
Emphasise to students the importance of understanding their needs before exploring relevant AI tools for their coursework. You can offer them the AI tools listed in Table 1 as a starting point, as these tools could assist with various tasks involved in writing their essay (such as brainstorming, generating an outline, conducting research, etc.). It is also beneficial to highlight the strengths and limitations of each tool, as detailed in Table 1, to help students gauge their potential utility.
In addition, consider encouraging students to explore tools beyond those listed, and ask them to compile an overview of the tools they have explored. Students should provide a brief justification for their choices; this overview can be attached as an appendix to their essay. Guidance along the following lines might be used:
Before selecting an AI tool, determine what you need. Ask yourself questions like:
Next, explore, evaluate and select the most suitable AI tools for your task, taking into account their features, functions, potential benefits, and limitations. As part of your exploration, record your findings in a table, documenting the tools you’ve explored, chosen or discarded, accompanied by brief justifications for your choices. Include this table as an appendix to your report.
This step involves students actively engaging with their chosen AI tools. It is useful to highlight the importance of hands-on experimentation and critical assessment of AI-generated outputs as well as reiterating the significance of problem formulation.
Encourage students to fully immerse themselves in the functionalities of their selected AI tools. This goes beyond merely querying the tool; they should also understand its features, capabilities, and potential limitations.
Tasks might include generating content ideas, crafting an essay outline, seeking clarifications on specific topics, receiving feedback on arguments, checking for grammatical accuracy and flow, or even creating visuals (for instance, using Bing).
Equally important is to remind students to critically evaluate the output— ensuring the information is relevant, accurate, and free from bias. The following offers general guidance worth sharing with students:
Engage proactively with your selected AI tools. Interact with the AI, provide feedback, experiment with various prompts, and critically evaluate the responses. Keep a record of the prompts that yielded the desired results, as well as those that fell short. Should you observe any consistent biases or inaccuracies in an AI tool's outputs, record these discrepancies.
In an appendix to your report, ensure to:
In addition, students would greatly benefit from understanding certain strategies before interacting with AI tools. Here are some key pointers to make the most out of generative AI tools:
The key to effective prompting is a strong understanding of the problem that you want to address. See a more comprehensive explanation of problem formulation (Oguz A. Acar). But, in essence, you should clarify what exactly you want to achieve. Do you want to generate ideas, craft an outline for your essay, search the relevant sources, or get counterpoints on your arguments? The clearer you are about your questions; the better the AI's response will be.
Break down your task into smaller and simpler components when possible. For example, various phases of essay writing - such as idea generation, outlining, drafting, and refining - can be considered distinct components to tackle individually.
Similarly, treat the different sections of your essay - the introduction, body, and conclusion - as separate elements. If your essay deals with the impact of AI in education, consider narrowing your scope to delve into specific aspects like its benefits, challenges, real-world applications, and future potential. Craft tailored AI prompts to address each of these individual components effectively.
When seeking information or content for a specific section of your essay, be explicit. Provide the AI tool with some context and background about the project or the target audience of your output. For example, if your essay focuses on AI in healthcare, you can prompt:
To get some creative and out-of-the-box outputs, you can play around with constraints (e.g, impose arbitrary constraints, modifying or removing existing boundaries) to explore novel perspectives. Some examples:
Don’t rely on a single prompt to get everything you want. Engage with the AI in a back-and-forth manner, refining your questions based on its responses. If you want AI to change the tone of writing or incorporate a specific argument, tell it. If you find the output too complex ask it to simplify (e.g., “Your explanation on the how generative AI models work was too complex. Can you simplify it for a 10-year old”).
If you are unhappy with the solution set, try reframing the question: could the same problem be asked from a different perspective? Sometimes, it is better to start over, as past conversations often influence outcomes.
The more you use and interact with the AI, the better you will get at eliciting the responses you want. Use it frequently to familiarise yourself with its strengths and limitations.
AI tools themselves are getting better at crafting prompts. So, it can help you too. Some examples:
You should always critically evaluate the output, corroborate it with other sources, and scrutinise for biases or inaccuracies. Before incorporating anything to your output:
Following these recommendations should allow you to better harness the value of generative AI tools.
In addition, module leaders might provide further guidance by sharing other resources and explaining main prompting techniques. See, for example, below:
Here is also a useful practical guide by Open AI for which include some additional strategies and tactics specifically for ChatGPT, but they are mostly appropriate for other chatbots too.
There are also different prompting techniques one can use. The most common ones include: zero-shot, few-shot and chain-of-thought prompting.
A key part of the PAIR framework is the reflection part, and it is useful to encourage students to also include a reflection report. The following guidance would be helpful:
Compose a reflective report in which you ponder over the lessons learned from your essay writing process with the AI tools. In this brief report (500-750 words), answer the following questions:
It is also important to provide information to students about how to use AI and how to report their usage. The following items could be used for this purpose:
This guidance was developed by Oguz A. Acar, who extends thanks to numerous colleagues across King’s College London for their feedback on the framework and earlier versions of the document, particularly to Karl Nightingale, Caitlin Bentley, Martin Compton, Ricardo Twumasi, Eleanor Dommett, and Vincent Giampietro.
For questions and feedback, please reach out to Professor Oguz A. Acar.
Supporting the adoption and integration of generative AI
Overview of key terminology and contextual information for generative AI
Find out more about learning and teaching at King's