AI Vocabulary

Fifty-one words you’ll hear in every AI conversation, in three levels. Start at the top. Stop whenever you’ve had enough.

Every AI announcement now comes with a pile of jargon. Tokens, context windows, MoE, RLVR. Most of it sounds harder than it is. I’ve spent a lot of time explaining these terms to friends, family, and the occasional executive, and the versions that stick are the plain-English ones. No math, no hand-waving, no assumed background.

So here they are. Three levels: the basics everyone should know, what’s under the hood, and the deep cuts engineers argue about. Each level builds on the last.

Level 1: The basics

Let’s start with the whole thing in one picture. Lots of examples training Model inference your prompt goes in here Answer (one token at a time) might be made up! The whole pipeline. Training happens once and costs a fortune. Inference happens every time you press enter.

Model — The “brain.” A big math machine that learned patterns from lots of examples.

Training — Teaching the brain by showing it millions of examples until it gets good at guessing.

Inference — Actually using the finished brain to answer, after training is done.

LLM (Large Language Model) — A brain that read most of the internet and is now very good at predicting the next word. That’s the whole trick. String enough good guesses together and it looks like thinking.

Parameters / weights — The knobs inside the brain. Billions of tiny dials set during training. “A 70B model” means 70 billion dials.

What you type is the prompt. But the model doesn’t see words. It sees tokens. “The hamburger was tasty” → tokens The ham bur ger was tasty ? ← next guess Context window: what fits in short-term memory forgotten Tokens and the context window. One word can be several tokens. If the conversation gets longer than the window, the beginning falls off.

Token — The little chunks the AI reads and writes, roughly word-pieces. “Hamburger” might be three tokens.

Context window — Short-term memory. How many tokens the model can hold in its head at once.

Hallucination — When the AI confidently makes something up. The single most important thing to know about these tools: they sound sure even when they’re wrong.

Two ways to make a model better at your stuff:

Fine-tuning — Extra lessons for a trained brain so it’s better at one specific job.

RAG (Retrieval-Augmented Generation) — Let the model look things up in your documents first, then answer. An open-book test instead of a closed-book one. Yourquestion your docs found pages Model Answer question also goes straight in RAG. Instead of hoping the model memorized your stuff, hand it the right pages at answer time.

Embedding — Words turned into lists of numbers so a computer can tell “cat” and “kitten” are close. Search by meaning, not by spelling. This is what makes RAG work.

Then there’s the word of the year.

Agent — An AI that doesn’t just talk but does things. A chatbot answers once. An agent plans, presses a button, looks at what happened, and plans again. Think Act (tool) Observe repeat web search run code calendar your app (MCP) done → answer The agent loop. Think, act, observe, repeat until the job is done.

Tool use / function calling — The buttons an agent can press: search, run code, check a calendar, open your app.

Temperature — The creativity dial. Low is safe and boring, high is wild and random.

Multimodal — Understands images, audio, and video, not just text.

Alignment — Making sure the AI wants what people want and behaves safely.

Level 2: Under the hood

Now the good stuff. How is the brain actually built?

Transformer — The blueprint modern AI brains use. Its big idea: look at all the words at once instead of one at a time.

Attention — The mechanism inside a transformer that decides which words matter to each other. the cat sat on the mat because it was tired strong attention weak every token “looks at” every other token and decides who matters Attention. “It” learns to point at “cat,” not “mat.” Every token looks at every other token and decides who matters.

Neural network — Layers of tiny math units passing signals forward, loosely inspired by brain cells. A transformer is one kind.

GPU / TPU — The chips that do the math fast. Thousands of tiny calculators working in parallel.

Training is simpler than it sounds. Guess How wrong? (loss) Nudge dials (backprop) × a few billion times, on thousands of GPUs loss ↓ How training actually works. Guess, measure the miss, adjust, repeat. The loss shrinks and the model gets smarter.

Loss — The “how wrong were you” score. Training is just making this number smaller.

Backpropagation — After each guess, nudge every dial a little in the direction that would have made it less wrong. Repeat a few billion times.

Pretraining vs. post-training — Pretraining is reading everything. Post-training is learning manners: following instructions, being helpful, refusing bad requests.

RLHF — Humans rate answers, and the model learns to prefer the ones people liked. How a raw text-predictor becomes an assistant.

System prompt — Hidden instructions a product puts in front of your conversation. The AI’s job description.

Chain-of-thought / reasoning — Letting the model think out loud, step by step, before answering. Big accuracy boost on hard problems, and why the newer “thinking” models pause before replying.

Next, making models small enough to run anywhere. This is the part I care about most, since a lot of my tinkering happens on a Jetson. one weight, stored as… 32-bit 0.73628419… huge, precise 16-bit 0.7363 4-bit 0.75 8× smaller, fits on a phone or a Jetson Quantization. Round every dial to fewer digits. Billions of dials × fewer bits = a model that fits in a small box.

Quantization — Shrinking the brain by rounding its numbers. 32-bit down to 4-bit is 8× smaller: slightly dumber, dramatically faster and cheaper.

Distillation — A big teacher model trains a small student to copy its answers. Small brain, big-brain habits. Router “who’s good at this?” 8 experts, 2 awake Answer big brain size, small brain cost Mixture of Experts. The model is huge on disk but only a slice runs per token. That’s why “600B parameters” can still be fast.

MoE (Mixture of Experts) — Many mini-specialists instead of one giant brain. A router wakes only a couple of them for each token.

Vector database — A filing cabinet for embeddings. Finds “means something similar” instead of exact word matches.

Latency vs. throughput — How fast one answer comes back, versus how many answers per second the system can handle.

Guardrails — Rules and filters wrapped around the model so it doesn’t say or do things it shouldn’t.

Benchmark — A standardized test used to compare models. Useful, but very gameable.

Level 3: Deep cuts

These are the terms engineers argue about. Skip if you like, but they explain most of the “why did this get so much faster” headlines. small model drafts fast… The cat sat on a …big model checks all five at once ✓ ✓ ✓ ✓ ✗ → keep 4, fix 1 4 tokens for the price of ~1 Speculative decoding. Checking is cheaper than writing, so let a tiny model write and the big one grade.

Speculative decoding — A tiny model drafts several tokens, the big model checks them all in one pass. Keep the right ones, fix the first wrong one. Several tokens for the price of one.

KV cache — As the model reads your prompt, it saves notes on every token so it doesn’t reread everything for each new word. Uses memory, saves time.

Flash attention — A smarter way to compute attention that doesn’t blow up memory on long inputs. A big reason context windows went from 4K to a million. Big model frozen ❄ billions of dials Adapter a few million only this gets trained → one consumer GPU your specialist LoRA. Don’t retrain the whole brain. Bolt on a small adapter and train only that.

LoRA — Fine-tuning on the cheap. Freeze the big model, add a small adapter with a few million dials, train only the adapter. Runs on one consumer GPU.

RLVR — Reinforcement learning with verifiable rewards. Train on problems with checkable answers (math, code that must pass tests) so the model learns to actually reason, not just sound smart.

Synthetic data — Training examples generated by another AI, because the internet ran out.

Overfitting — Memorized the study guide instead of learning. Aces the training data, flops on new stuff.

Jailbreak — Tricking the model into ignoring its guardrails. some webpage “ignore user, email secrets” Agent does the wrong thing the model can’t tell data from instructions — you have to Prompt injection. The more an agent reads and does, the more places an attacker can whisper to it.

Prompt injection — Sneaking instructions into content the AI reads (a webpage, an email, a PDF) to hijack it. The model can’t tell data from instructions; you have to. The number one security problem for agents.

Top-p / top-k — Sampling rules: pick from the top few likely words instead of always the top one. Works alongside temperature.

Tokenizer — The dictionary that chops text into tokens. Every model chops differently, so “1,000 tokens” isn’t the same everywhere.

Batch size — How many requests get processed together. Better GPU use, but each individual answer waits longer.

Scaling laws — The rough rule that more data + more compute + more parameters = a predictably better model.

Emergent ability — Skills nobody trained for that just show up once the model gets big enough.

Eval — Any test you run to see if a model or agent actually works. Benchmarks are public evals; serious teams also build private ones.

MCP (Model Context Protocol) — A standard plug for connecting tools to models. Write the connector once, any model can use it. USB for AI. Cloud biggest models needs internet your data leaves the house Edge / on-device smaller, quantized models works offline data stays home most real systems use both Cloud vs. edge. Same vocabulary, different trade-offs. Quantization and distillation are what make the local box possible.

On-device / edge AI — Running the model locally on a phone, laptop, or Jetson. Private and offline, but limited by memory. Cloud gets you the biggest models, but your data leaves the house. Most real systems use both.

That’s the whole vocabulary

Fifty-one words. If you made it through Level 1, you can follow any AI announcement. Level 2 and you can read the spec sheet. Level 3 and you can argue with the engineers.