Skip to content
Technology

Model distillation

A technique for training one model on the answers of another: it turns a large model into a smaller one with little loss of quality and lets catching-up labs narrow the gap with the leader. In episode 189 Max Grigoriev uses it to explain why Chinese models stay months behind.

18
mentions in the archive
1
central to an episode
2
episodes with a mention
0
fragments with a timecode
Nov 2026
first mention
Nov 2026
peak · 2 in a month
115
links to other entries

Trends

Model distillation in the author's trends

all 20 thoughts →

A link is an episode of the thought where Model distillation is central or discussed in depth, or an author's quote that names it. The bar counts the shared episodes; blue bars are the months they meet on the June 2024 — November 2026 scale.

6 of the author's 20 thoughts are linked to Model distillation · “Details” shows the shared episodes and a quote

Not linked · 14

thoughts whose episodes don't feature Model distillation centrally or in depth