In their own words.
Our plain version above is ours. This is the official or expert wording we based it on.
“large-scale machine-learning models that exceed the capabilities currently present in the most advanced existing models”
Related words.
Fine-tuning
Extra training given to an already-trained model to make it better at a particular job, style or subject.
Distillation
Training a smaller, cheaper model to copy the behavior of a bigger one, so it runs faster at some loss of ability.
Parameters (weights)
The numbers inside a model that are adjusted during training and hold what it has learned. Big models have hundreds of billions or more.
Training data
The text, images, code and other material a model learns from during training.
Training run
One long job, often lasting weeks to months on thousands of chips, in which a model learns from its training data. Its size is usually measured in FLOP of compute.
Compute
Computing power: the chips, and the amount of calculation they do, used to train and run AI. It is often counted in FLOP.