XClose

Teaching & Learning

Home
Menu

How does Generative AI work?

How GenAI works and the sources of data that it is trained on.

MediaCentral Widget Placeholderhttps://mediacentral.ucl.ac.uk/Player/EiJbEc8E

 

In this video: How does GenAI work?

Artificial Intelligence is a topic in computer science that has been researched in universities since the 1950s. Various approaches to AI have been developed, many of which are now built into other software and used routinely.  

Developments since 2010 in big data and machine learning have allowed the creation of foundation models that are trained on broad data sets and have many use cases. These attracted public attention with the release of Stable Diffusion and ChatGPT in 2022.  

Common GenAI tools such as large language models (LLMs) and text-to-image tools are pre-trained using large quantities of text or images. They are given prompts by a user and then use statistical methods to generate a plausible output. There are often guardrails added to remove any offensive content.  

A technique called ‘Retrieval Augmented Generation' can be used to supplement prompts with documents you provide, giving the model accurate sources about your context that it can cite.  

As well as the prompt, there are often other parameters you can change. These can alter the length of the response, its tendency to repeat text, and its randomness or creativity.  

There is no reliable readily available way to verify whether a piece of text was written by a human or by a language model. We advise against using systems that claim to do this, as they can generate false positives which cannot be verified. 

 

Things to know 

The Wall Street journal reports that OpenAI have developed a reliable method for watermarking its outputs so that ChatGPT use can be reliably detected. They have decided to hold off on its release.   

Things to try 

Real or fake text? Can you guess whether text is computer generated or written by a human?  

Further information 

Financial Times explainer on how large language models generate text. Generative AI – a primer.