PythonIntermediate~4h

LLM Fundamentals

The arithmetic behind the model

A model is a thing you send bytes to. This is the arithmetic of those bytes: what a line costs in tokens, what fits inside the window, what silently falls out when it does not, and what a temperature does to the distribution a word is drawn from. You leave holding a budget simulator you wrote.

Start this projectSee the modules
Modules
5
Drills
10
Build steps
2
Time
~4h

Walk out able to

Count the tokens in a request, say which message a window will drop and when, and predict what a sampling setting does to a stated distribution.

What you work out

01

Tokens, counted

What a line costs, and what a rewrite saves.

02

The window as a budget

What fits, what drops, what the tenth turn costs.

03

Temperature and filters

Exactly what each dial does to a distribution.

04

What it can see, and why

Which facts arrive, and where a wrong answer came from.

The pitch

What you practise, and what you leave with

A context-budget simulator in Python: it counts tokens, trims a conversation to fit a window, and reports what ten turns really cost.

You will practise

  • Counting tokens and costing a rewrite
  • What a window holds and what it drops
  • Temperature, top-k and top-p, exactly
  • Reading a failure back to its mechanism

Afterwards you can

  • Count the tokens before you send them
  • Say what the window will drop, and when
  • Predict what a temperature does to a gap
  • Name the mechanism behind a wrong answer

Modules

5 modules, 25 items

Lessons explain one idea. Drills repeat it until it sticks. Build steps make something that exists afterwards.

  1. 1

    Count what you send

    ~48 min

    Produce a token count for any line, and say which of two spellings is cheaper.

    • LessonWhy count what you send4 min
    • LessonSee where a token boundary falls7 min
    • LessonSay where the cost actually lands6 min
    • DrillCount a line of tokens9 min
    • DrillSay what a rewrite saves8 min
    • Build stepWrite the counter14 min
  2. 2

    Fit it in the window

    ~51 min

    Say what a window holds, what falls out of it, and what the tenth turn costs.

    • LessonBudget the window in four terms9 min
    • LessonSay when it starts dropping6 min
    • DrillSay what falls out of the window10 min
    • DrillCost the tenth turn10 min
    • Build stepMake it drop the oldest turns16 min
  3. 3

    Turn the dials

    ~44 min

    Say exactly what a temperature, a top-k and a top-p do to a stated distribution.

    • LessonDivide the scores by a temperature5 min
    • LessonDelete candidates with a filter5 min
    • LessonTake the greedy path, and lose6 min
    • DrillScale a gap by a temperature9 min
    • DrillCut the tail off a distribution9 min
    • DrillTake the greedy path, and the best one10 min
  4. 4

    Write what it can see

    ~32 min

    Decide which facts reach a given turn, and name the change that supplies a missing one.

    • LessonDecide what it can actually see8 min
    • LessonName the repair that fits6 min
    • DrillName the fact that actually arrived9 min
    • DrillName the one change that helps9 min
  5. 5

    Read the failure back

    ~43 min

    Trace a wrong answer to the mechanism that made it, and ship the budget simulator.

    • LessonTrace a wrong answer to its cause7 min
    • LessonRead the settings, not the tone6 min
    • DrillName the mechanism behind it10 min
    • LessonShip the budget report20 min

More in AI & machine learning

Keep going

All AI & machine learning projects

RAG & Embeddings

Retrieval is arithmetic. Do the arithmetic.

Intermediate~4h5 modules

PyTorch: Training

Write a loop that learns, and fix one that does not.

Advanced~4h4 modules

PyTorch: Tensors & Autograd

Say the gradient before backward() does.

Intermediate~4h4 modules

Tokens are countable. A window is arithmetic.

The first item is free. ~4h of focused work, at your own pace.

Start LLM FundamentalsAll projects

Intermediate~4h5 modules