PyTorch: Tensors & Autograd

Read the graph the engine built

Every tracked operation leaves a node behind, and backward() walks those nodes in reverse. This is practice at predicting both halves — what got recorded, and the exact number that comes back — from shapes and dtypes through to a gradient worked by hand, finishing with an engine of your own.

Start this project See the modules

Modules: 4
Drills: 9
Build steps: 7
Time: ~4h

Walk out able to

Say what shape a tensor expression has, why a gradient is None, and what number backward() will put in any tensor of a small graph.

What you work through

What a tensor carries

Shape, dtype, and whether anything is watching.

Shapes that line up

Rearranging calls, batched @, the silent broadcast.

What the engine records

grad_fn, leaves, and the five reasons for None.

Walk the graph backwards

Gradients by hand, then an engine that agrees.

What you get

Skills that stick

Every project mixes short lessons, focused drills and real build steps, so you finish with working code and the judgement behind it.

You will practise

Dtypes, shapes and batched @
Which lines built graph, and why None
What the gradient buffer holds
Gradients by hand, broadcast ones too

Afterwards you can

Say the shape before running the line
Diagnose a None gradient from the code
Compute a small graph’s gradients by hand
Write a working reverse-mode engine

Modules

Step by step

Each module builds on the last. Read a short lesson, drill it until it's second nature, then put it to work.

1
What a tensor carries
~43 min
Name the three labels on any tensor, and predict the dtype a construction lands on.
- LessonWhy an array library keeps a diary5 min
- LessonRead the three labels a tensor carries9 min
- LessonPredict the dtype a line lands on5 min
- DrillSay the dtype before it bites10 min
- Build stepMake a tensor and read it back7 min
- Build stepTotal two columns, then divide7 min
2
Shapes that line up
~1h
Say what shape any rearranging call or any matrix multiply produces — or that it raises.
- LessonGive a tensor the axis it needs10 min
- DrillLine two shapes up, or make them12 min
- LessonTwo spellings for the same rearrange6 min
- DrillSay the shape after the call12 min
- LessonWhat the axes in front of a matmul do6 min
- DrillSay the shape a matrix multiply produces12 min
- Build stepFind the broadcast that did not raise7 min
- Build stepGive it the axis it was missing7 min
3
What the engine records
~1h
Say which lines built graph, where a gradient is kept, and what the buffer holds.
- LessonWatch the engine write it down8 min
- LessonFind out where a gradient is kept8 min
- LessonSwitch the recording off on purpose6 min
- DrillSpot the line that built no graph10 min
- DrillSay why the gradient is None12 min
- DrillSay what the gradient buffer holds10 min
- Build stepBreak the second backward pass12 min
4
Walk the graph backwards
~1h
Compute a small graph’s gradients by hand, then build the engine that does it.
- LessonWalk a graph backwards by hand11 min
- DrillCompute it before the engine does14 min
- LessonA gradient wears its own shape6 min
- DrillSum a gradient back to its shape12 min
- LessonWhy backward() wants one number5 min
- Build stepBuild the engine that does it for you18 min
- Build stepMake it agree with torch, digit by digit14 min