In their own words.
Our plain version above is ours. This is the official or expert wording we based it on.
“The weights and biases that a model learns during training.”
Related words.
Fine-tuning
Extra training given to an already-trained model to make it better at a particular job, style or subject.
Distillation
Training a smaller, cheaper model to copy the behavior of a bigger one, so it runs faster at some loss of ability.
Training data
The text, images, code and other material a model learns from during training.
Training run
One long job, often lasting weeks to months on thousands of chips, in which a model learns from its training data. Its size is usually measured in FLOP of compute.
Compute
Computing power: the chips, and the amount of calculation they do, used to train and run AI. It is often counted in FLOP.
FLOP (floating-point operation)
One basic arithmetic step on decimal numbers. The total FLOP used to train a model (e.g. 10^25) is a standard way to say how big a training run was.