The Illustrated Guide to Local AI

Deck

Local AI

Cards about running AI models on your own computer.

Local AI cover
Model
Model Legendary
Concept · Base Set
LegendaryConceptBase Set
Power 9 · Danger 5 · Speed 4 · Difficulty 6

Description

It is a large file that holds all the knowledge a computer has learned. You load it to make the machine think, but if you use the wrong one, it will give you wrong answers.

Analogy

It is like a massive library book that contains every fact the librarian has ever read.

Example

model.load("my_brain.bin")

Flavour

The frozen mind of the dragon, waiting for the key to wake it.

Effect

It is a large file that holds all the knowledge a computer has learned. (POWER 9, DANGER 5, SPEED 4, DIFFICULTY 6)

Learn this in chapter 2 →
Parameters
Parameters Epic
Concept · Base Set
EpicConceptBase Set
Power 10 · Danger 9 · Speed 1 · Difficulty 8

Description

These are the tiny internal settings that decide how a model thinks. The number tells you how complex the brain is, and changing them can make the machine brilliant or completely broken.

Analogy

It is like tuning a piano with millions of strings, where one wrong twist ruins the whole song.

Example

model.parameters()

Flavour

The silver runes carved into the dragon’s heart, each one holding a whisper of memory.

Effect

These are the tiny internal settings that decide how a model thinks. (POWER 10, DANGER 9, SPEED 1, DIFFICULTY 8)

Learn this in chapter 2 →
Weights
Weights Epic
Concept · Base Set
EpicConceptBase Set
Power 10 · Danger 9 · Speed 1 · Difficulty 8

Description

These are the tiny numbers that hold all the knowledge inside an AI. They are the memories the machine learned, stored as values that change how it thinks.

Analogy

It is like the specific arrangement of beads on a necklace that determines the pattern.

Example

model.save_weights("model.h5")

Flavour

The silver runes carved into the stone, holding the memory of the dragon’s breath.

Effect

These are the tiny numbers that hold all the knowledge inside an AI. (POWER 10, DANGER 9, SPEED 1, DIFFICULTY 8)

Learn this in chapter 2 →
Quantization
Quantization Rare
Concept · Base Set
RareConceptBase Set
Power 4 · Danger 1 · Speed 8 · Difficulty 2

Description

It shrinks a large model by rounding its numbers to simpler values. This makes the program run faster and use less memory, like saving a photo at lower quality.

Analogy

It is like drawing a detailed map with only the main roads, skipping the tiny side streets.

Example

quantize(model, bits=8)

Flavour

The silver rune that trims the heavy stone to fit in a pocket.

Effect

It shrinks a large model by rounding its numbers to simpler values. (POWER 4, DANGER 1, SPEED 8, DIFFICULTY 2)

Learn this in chapter 3 →
Context Window
Context Window Rare
Concept · Base Set
RareConceptBase Set
Power 4 · Danger 1 · Speed 5 · Difficulty 2

Description

It is the amount of conversation the AI can remember at one time. If the chat gets too long, the oldest parts fall off the edge and are forgotten.

Analogy

It is like a small desk where you can only stack a few papers before the rest fall to the floor.

Example

max_tokens=4096

Flavour

The silver desk that holds only what fits within its glowing borders.

Effect

It is the amount of conversation the AI can remember at one time. (POWER 4, DANGER 1, SPEED 5, DIFFICULTY 2)

Learn this in chapter 4 →
KV Cache
KV Cache Epic
Concept · Base Set
EpicConceptBase Set
Power 9 · Danger 1 · Speed 10 · Difficulty 8

Description

It remembers every word you have said so the computer does not have to read the whole story again. It saves time by keeping a sticky note for each sentence.

Analogy

It is like a librarian who keeps a bookmark in every book so you never have to start from page one.

Example (longer chats need a bigger KV cache (more RAM))

cache = {}

Flavour

The dragon sleeps on its hoard of whispers, guarding the past so the future can speak.

Effect

It remembers every word you have said so the computer does not have to read the whole story again. (POWER 9, DANGER 1, SPEED 10, DIFFICULTY 8)

Learn this in chapter 5 →
VRAM
VRAM Rare
Concept · Base Set
RareConceptBase Set
Power 4 · Danger 1 · Speed 8 · Difficulty 2

Description

It is the fast memory built into the graphics card. It holds the images and data the computer is working on right now, so the processor can grab them instantly without waiting.

Analogy

It is like the chef's immediate workbench, where ingredients are kept within arm's reach for quick cooking.

Example

nvidia-smi

Flavour

The silver runes that hold the vision close to the heart of the machine.

Effect

It is the fast memory built into the graphics card. (POWER 4, DANGER 1, SPEED 8, DIFFICULTY 2)

Learn this in chapter 6 →
Inference
Inference Rare
Concept · Base Set
RareConceptBase Set
Power 4 · Danger 1 · Speed 5 · Difficulty 2

Description

It is the moment the computer actually answers your question. You ask, it thinks, and it gives you the result. This is the only time it speaks to you.

Analogy

It is like asking a librarian for a book and getting the answer right away.

Example

model.generate("Hello")

Flavour

The silver tongue that speaks only when spoken to.

Effect

It is the moment the computer actually answers your question. (POWER 4, DANGER 1, SPEED 5, DIFFICULTY 2)

Learn this in chapter 7 →
Token
Token Uncommon
Concept · Base Set
UncommonConceptBase Set
Power 3 · Danger 1 · Speed 5 · Difficulty 2

Description

It is a small piece of text, like a few letters, that the computer reads one by one. The model counts these pieces to understand your words and charges you for them.

Analogy

It is like counting the number of apples in a basket to decide the price.

Example

"Hello" is 1 token, "world" is 1 token.

Flavour

The silver scales that weigh every whisper in the cold hall.

Effect

It is a small piece of text, like a few letters, that the computer reads one by one. (POWER 3, DANGER 1, SPEED 5, DIFFICULTY 2)

Learn this in chapter 7 →
Mixture of Experts
Mixture of Experts Epic
Concept · Base Set
EpicConceptBase Set
Power 9 · Danger 4 · Speed 7 · Difficulty 8

Description

It wakes up only the right specialists for each question instead of asking everyone. This saves energy and makes answers faster because only the experts who know the topic are active.

Analogy

It is like a hospital where only the heart doctor and the lung doctor are called for a chest pain, not the whole staff.

Example

model.generate(input, top_k=2)

Flavour

The council of silver masks, where only the chosen voices speak in the cold hall.

Effect

It wakes up only the right specialists for each question instead of asking everyone. (POWER 9, DANGER 4, SPEED 7, DIFFICULTY 8)

Learn this in chapter 8 →
Agent
Agent Legendary
Concept · Base Set
LegendaryConceptBase Set
Power 9 · Danger 8 · Speed 7 · Difficulty 6

Description

It is an AI that does not just answer questions, but actually performs tasks for you. It can search the web, write files, or send emails on your behalf.

Analogy

It is like a loyal squire who fetches your sword and armor while you sit on the throne.

Example

agent.run("find the best price for this item")

Flavour

The silver knight that acts before the king speaks.

Effect

It is an AI that does not just answer questions, but actually performs tasks for you. (POWER 9, DANGER 8, SPEED 7, DIFFICULTY 6)

Learn this in chapter 10 →
Fine-tuning
Fine-tuning Uncommon
Concept · Base Set
UncommonConceptBase Set
Power 4 · Danger 2 · Speed 3 · Difficulty 3

Description

It takes a smart computer model and gives it extra practice on one specific task. You show it many examples of your favorite style or topic, and it learns to copy that pattern perfectly.

Analogy

It is like a master chef who already knows how to cook, but spends a week practicing only one specific soup recipe until it is perfect.

Example

python train.py --epochs 10

Flavour

The silver rune that whispers a single song into the mind of the machine.

Effect

It takes a smart computer model and gives it extra practice on one specific task. (POWER 4, DANGER 2, SPEED 3, DIFFICULTY 3)

Learn this in chapter 11 →
Embedding
Embedding Uncommon
Concept · Base Set
UncommonConceptBase Set
Power 4 · Danger 1 · Speed 5 · Difficulty 3

Description

It turns words into numbers so the computer can measure how similar things are. It helps the machine understand that "dog" and "cat" are closer than "dog" and "car".

Analogy

It is like sorting apples by size and color so you can find the ones that look alike.

Example

embedding = model.encode("hello")

Flavour

The silver runes that bind every word to its true shape.

Effect

It turns words into numbers so the computer can measure how similar things are. (POWER 4, DANGER 1, SPEED 5, DIFFICULTY 3)

Learn this in chapter 5 →
Temperature
Temperature Uncommon
Concept · Base Set
UncommonConceptBase Set
Power 3 · Danger 2 · Speed 5 · Difficulty 2

Description

It controls how wild the AI's answers are. Low numbers make it safe and predictable, while high numbers make it creative but risky.

Analogy

It is like turning up the heat on a stove, where low heat keeps things calm and high heat makes them boil over.

Example

temperature = 0.7

Flavour

The dial that decides if the dragon whispers or roars.

Effect

It controls how wild the AI's answers are. (POWER 3, DANGER 2, SPEED 5, DIFFICULTY 2)

Learn this in chapter 7 →
Prompt
Prompt Uncommon
Concept · Base Set
UncommonConceptBase Set
Power 4 · Danger 1 · Speed 5 · Difficulty 2

Description

It is the instruction you give to the machine. You write what you want, and the model tries to follow your words. A clear prompt gets a clear answer.

Analogy

It is like handing a chef a specific recipe ticket at the counter.

Example

Write a short poem about the moon.

Flavour

The silver scroll that commands the spirit within the glass.

Effect

It is the instruction you give to the machine. (POWER 4, DANGER 1, SPEED 5, DIFFICULTY 2)

Learn this in chapter 4 →
LLM
LLM Epic
Concept · Base Set
EpicConceptBase Set
Power 9 · Danger 5 · Speed 4 · Difficulty 6

Description

It reads your words and writes new ones back to you. It guesses the next word based on patterns it learned from millions of books. It can help you write, but it can also make things up.

Analogy

It is like a very fast parrot that has memorized every book in the library.

Example

prompt = "Write a short poem about rain."

Flavour

The silver dragon that speaks in echoes of forgotten voices.

Effect

It reads your words and writes new ones back to you. (POWER 9, DANGER 5, SPEED 4, DIFFICULTY 6)

Learn this in chapter 2 →

← All decks