The Sticky Notes

Imagine you are sitting at a kitchen table. You have a corkboard in front of you. On that board, you have a small stack of bright yellow sticky notes. Each note holds one thought. You write down what you just said. You write down the question you asked. Every single sentence gets its own little square of paper.

This is how the AI remembers your conversation. It does not store the whole chat in a giant library. It keeps the recent parts on this board. This board is called the KV cache. The letters stand for Key and Value. Think of the Key as the label on the note. It tells the AI what the note is about. Think of the Value as the actual words written on the paper. The label helps the AI find the right note. The words help the AI understand what to say next.

Now, look at the board again. When you start a new chat, the board is clean and empty. There are no notes yet. The AI has nothing to search through. It can answer your first question quickly. It looks at the few notes you just added. It finds them easily. It reads them fast. It gives you a reply. Then you ask another question. You add a new sticky note to the corkboard. The AI reads this new note. It also looks back at the old notes. It needs to know what you said before. This is how it keeps the conversation flowing. It does not forget the beginning. It remembers the middle. It remembers the end. But here is the catch. Every single sticky note stays on the board. The AI does not throw them away. It keeps them all. You might ask a question that refers to something you said ten minutes ago. It might ask you to summarize the whole chat. It needs the whole board. It needs every note. This is where the trouble starts. The board gets crowded. The AI has to search through more notes. It has to look at every single one. It checks each note. It checks the middle notes. It does this for every new word it writes. Imagine you are looking for a specific recipe in a library. If the library has only three books, you find the recipe quickly. You scan the spines. You pull the book. You open it. You read. It takes seconds. Now imagine the library has three thousand books. You have to walk down every aisle. You have to check every shelf. You have to look at every spine. It takes minutes. It takes longer. The AI does this with your sticky notes. It scans the whole board. It scans every note. It scans the notes one by one. It looks for the right connection. This is what we call the Key-Value cache. The Key is the note itself. The Value is what the note means to the AI. The AI stores both. It stores the word. It stores the meaning. It keeps them together. This pair is the sticky note. The AI does not read the whole board every time. It reads the notes it has already made. This is why it feels fast at first. You ask a short question. The board has only a few notes. The AI finds the right note quickly. It finds the right answer quickly.

If you remember only one thing

The KV cache is the AI's short-term memory, like sticky notes on a corkboard. Every new sentence adds a new sticky note, so the board gets crowded. The more notes, the longer it takes to find the right one, so answers slow down.