How Large Language Models Generate Text

Part of the Prompt Engineering course

You now know what a prompt and a completion are, and that the same prompt can come back differently worded each time. This lesson shows you what happens on the other side of that prompt: how the model actually turns your words into a reply. Understanding the mechanism now will make the five principles of prompting, coming up next, feel less like rules to memorize and more like consequences of how the machine works.

Before a model sees your prompt, the text is chopped into small chunks called tokens, which are subword pieces such as a short word, part of a longer word, or a single punctuation mark. The process of cutting text into these chunks is called tokenization. ...