Model distillation

A technique for training one model on the answers of another: it turns a large model into a smaller one with little loss of quality and lets catching-up labs narrow the gap with the leader. In episode 189 Max Grigoriev uses it to explain why Chinese models stay months behind.