Bandwidth
Türkçe: Bant genişliği
The maximum amount of data a bus or memory can transfer per second (GB/s).
// EXPLANATION
In AI inference the limiting factor is often not compute but memory bandwidth, because most of the model weights are read from memory for every token. That is why HBM and wide memory buses matter on graphics cards.
// ALSO KNOWN AS
bandwidthmemory bandwidth
// RELATED TERMS
VRAMThe high-bandwidth memory on the graphics card itself; an AI model's weights have to fit into it.
PCIeThe lane-based serial bus standard that connects graphics cards, SSDs and other components to the motherboard.
LatencyThe time between starting an operation and receiving its result; it measures how quickly data responds.
LLM (large language model)A neural network trained on very large amounts of text that generates text by probabilistically predicting the next piece (token) of a sequence.
