4 GPUs, 2x speed: why PyTorch DDP stalled on a PCIe-only 4x RTX A6000, and two flags that took it to 3.6x
All-reduce at 4.3 GB/s through the host bridge halved our scaling. bf16 gradient compression plus gradient accumulation got most of it back.
Latest articles
How long until a rented GPU runs CUDA? Launch to first matmul on six GPU types, timed
Same API, same image, same script: 186 seconds on one GPU type, 707 on another. Where the time goes, and why the newest PyTorch broke on a V100.
vLLM on three GPUs: Qwen2.5-7B tokens per second and cost per 1M tokens on RTX A6000, L40S and H100
Same model, same benchmark, three rented GPUs: $0.12 to $0.22 per 1M output tokens when the batch is full, $4.86 to $9.47 when it is not.
Embed your docs on a rented GPU or call an embeddings API? We measured both with bge-m3
527 chunks, the same model both ways, real bills. The API won for our docs; the GPU wins past about 630k tokens.