Transformer architecture
Türkçe: Transformer mimarisi
The neural-network architecture introduced in 2017 that computes relationships between all words in a text in parallel using an attention mechanism.
// EXPLANATION
The attention mechanism learns how much weight each token should give to the others. Because it suits parallel computation it scales efficiently on GPUs; nearly all of today's LLMs are based on it.
// ALSO KNOWN AS
transformer architecturetransformer model
// RELATED TERMS
LLM (large language model)A neural network trained on very large amounts of text that generates text by probabilistically predicting the next piece (token) of a sequence.
MoE (mixture of experts)An architecture in which the model consists of many "expert" sub-networks and only a few of them run for each token.
GPU (graphics processing unit)A processor with thousands of small cores that performs the same operation on lots of data in parallel, used for graphics and AI computation.