2026 GPU Selection Guide — From L40S to B300
"Is H100 enough, or should we go with B200?" — this is the most common question teams face when choosing a GPU. As models grow larger and workloads diversify, the GPU landscape has expanded far beyond what it was just one generation ago.
In this guide, we compare the specs of 8 major GPUs available in the cloud today and break down which GPU fits each workload. The criteria are straightforward: VRAM, compute performance, memory bandwidth, and GPU interconnect.
Core Spec Comparison
Here's a side-by-side comparison of all 8 GPUs, focused on the four factors that matter most for workload selection: VRAM, memory bandwidth, compute (FP8), and interconnect.
| GPU | Architecture | VRAM | Memory BW | FP8 (dense) | Interconnect | TDP |
|---|---|---|---|---|---|---|
| L40S | Ada Lovelace | 48 GB GDDR6 | 864 GB/s | 733 TFLOPS | PCIe 4.0 | 350W |
| RTX Pro 6000 | Blackwell | 96 GB GDDR7 | 1.6 TB/s | —* | PCIe 5.0 | 600W |
| A100 SXM | Ampere | 80 GB HBM2e | 2.0 TB/s | 312 (FP16)* | NVLink 3.0 (600 GB/s) | 400W |
| H100 SXM | Hopper | 80 GB HBM3 | 3.35 TB/s | 1,979 TFLOPS | NVLink 4.0 (900 GB/s) | 700W |
| H200 SXM | Hopper | 141 GB HBM3e | 4.8 TB/s | 1,979 TFLOPS | NVLink 4.0 (900 GB/s) | 700W |
| B200 SXM | Blackwell | 180 GB HBM3e | 8.0 TB/s | 4,500 TFLOPS | NVLink 5.0 (1,800 GB/s) | 1,000W |
| GB200 NVL72 | Blackwell | 13.4 TB HBM3e (rack-scale) | 8.0 TB/s / GPU | 4,500 / GPU | NVLink full-mesh (72 GPUs) | —** |
| B300 | Blackwell Ultra | 288 GB HBM3e | 8.0 TB/s | ~7,000 TFLOPS | NVLink 5.0 (1,800 GB/s) | 1,400W |
* RTX Pro 6000 is based on the Blackwell architecture with 5th-gen Tensor Cores. The Server Edition delivers 120 TFLOPS FP32 and approximately 4,000 TFLOPS FP4 (dense). A100 does not support FP8 — the 312 TFLOPS figure is FP16.
** GB200 NVL72 is a rack-scale system with 36 Grace CPUs and 72 Blackwell GPUs. At rack scale, it provides 13.4 TB of HBM3e memory. A single GB200 Grace Blackwell Superchip provides 372 GB of HBM3e memory.
How to Read These Specs
VRAM: Can Your Model Fit?
VRAM is the first thing to check. Model parameters and activation memory must fit in GPU memory for both training and inference. As a rough guide, a 7B model needs about 14 GB in FP16 for inference, and a 70B model needs about 140 GB. Training requires 3–4× more memory than inference due to optimizer states.
Memory Bandwidth: How Fast Tokens Come Out
In inference, token generation speed is largely determined by memory bandwidth. H100 (3.35 TB/s) and H200 (4.8 TB/s) have the same compute, but H200 delivers higher inference throughput — the difference is bandwidth.
Compute (FP8 TFLOPS): Training Speed
For training, compute directly determines how fast your model learns. FP8 is becoming the standard for modern model training, and the Blackwell generation supports FP4 as well. Note that A100 doesn't support FP8 — compare it using FP16.
Interconnect: The Multi-GPU Bottleneck
When a single GPU isn't enough, how fast GPUs can communicate with each other becomes critical. NVLink provides direct GPU-to-GPU connections, while PCIe routes through the motherboard — much slower. If you need multi-GPU training, SXM form factor with NVLink is the way to go.
| Interconnect | Bandwidth | GPUs |
|---|---|---|
| PCIe 4.0 / 5.0 | 64–128 GB/s | L40S, RTX Pro 6000 |
| NVLink 3.0 | 600 GB/s | A100 SXM |
| NVLink 4.0 | 900 GB/s | H100, H200 |
| NVLink 5.0 | 1,800 GB/s | B200, B300 |
| NVLink 5.0 (full-mesh) | All 72 GPUs connected | GB200 NVL72 |
GPU Recommendations by Workload
Now let's match GPUs to real workloads. There's no single "right answer," but there are sensible starting points based on model size and task type.
| Workload | Recommended GPU | Why |
|---|---|---|
| LLM Inference (7B–13B) | L40S, RTX Pro 6000 | 48–96 GB VRAM is sufficient. Great cost-efficiency, and quantization (INT8/INT4) lets you serve even larger models |
| LLM Inference (70B+) | H200, RTX Pro 6000 | 141 GB / 96 GB VRAM handles large models. H200's HBM3e bandwidth (4.8 TB/s) makes token generation fast |
| Fine-tuning (LoRA/QLoRA) | A100, H100 | 80 GB VRAM + NVLink enables LoRA on models up to 70B. The most proven, cost-effective combination |
| Full Fine-tuning (70B+) | H200, B200 | 141–180 GB VRAM + high bandwidth. Optimizer states fit in GPU memory for better training efficiency |
| Pre-training (mid-scale, ~30B) | H100, B200 | High FP8 compute + NVLink + InfiniBand multi-node. H100 has a mature ecosystem; B200 offers 2.3× more compute |
| Pre-training (large-scale, 100B+) | GB200 NVL72, B300 | GB200 connects 72 GPUs in a single NVLink domain and provides 13.4 TB of HBM3e at rack scale. B300 maximizes single-GPU VRAM (288 GB) and compute |
| Image / Vision Models | H100, B200 | Diffusion model training needs high compute + bandwidth. B200's 8 TB/s bandwidth handles large batches well |
| Data Preprocessing / Embeddings | L40S, A100 | Compute-light batch workloads. Solid performance at the lowest hourly cost |
| Cost-First Experimentation | L40S, A100 | Lowest cost per hour. Ideal for rapid prototyping and iterative experiments |
Availability on VESSL Cloud
All of the above GPUs are available through VESSL Cloud. Some can be provisioned instantly from the console, while others are available through our sales team.
| GPU | Availability | Notes |
|---|---|---|
| L40S | ✅ Instant | Provision from the console immediately |
| RTX Pro 6000 | 🔜 Coming Soon | Reach out if you're interested |
| A100 SXM | ✅ Instant | Provision from the console immediately |
| H100 SXM | ✅ Instant | Provision from the console immediately |
| H200 SXM | 💬 Contact Sales | Available through our sales team |
| B200 SXM | 💬 Contact Sales | Available through our sales team |
| GB200 NVL72 | 💬 Contact Sales | Custom configuration after workload consultation |
| B300 | 💬 Contact Sales | Custom configuration after workload consultation |
GPUs on VESSL Cloud are all SXM-based (except L40S and RTX Pro 6000). As a Persistent GPU Cloud, your workspace environment — packages, data, and configurations — is preserved even when you pause your GPU.
FAQ
I'm using H100 — should I switch to B200?
It depends on your workload. If you're running into training time or VRAM limitations on H100, B200 is a clear upgrade — 2.3× compute, 2.3× VRAM, 2.4× bandwidth. But if H100 is working fine for you, there's no rush to switch. H100 has the most mature ecosystem and lower hourly cost than B200.
GB200 vs. B300 — which should I choose?
They serve different purposes. GB200 NVL72 connects 72 GPUs in a single NVLink domain, so it's ideal for very large workloads within a single rack. Larger-scale expansion happens through an InfiniBand or Ethernet cluster. B300 maximizes single-GPU VRAM (288 GB) and compute. Choose B300 if you want to minimize GPU count while maximizing per-GPU efficiency. Choose GB200 if you need a 72-GPU NVLink domain inside a rack and plan to scale further through an InfiniBand or Ethernet cluster.
Can L40S and RTX Pro 6000 be used for training?
Yes — for small-scale fine-tuning and experimentation. However, since they use PCIe (no NVLink), GPU-to-GPU communication is slow for multi-GPU training. They're best suited for single-GPU LoRA fine-tuning or inference experiments. RTX Pro 6000's 96 GB VRAM lets you handle fairly large models on a single GPU.
How do I estimate how much VRAM my model needs?
Here are rough guidelines:
- FP16 Inference: Parameters × 2 bytes. 7B model ≈ 14 GB, 70B model ≈ 140 GB
- INT8 Inference: Parameters × 1 byte. 70B model ≈ 70 GB
- Training (FP16 + Adam): Parameters × ~18 bytes. 7B model ≈ 126 GB
- LoRA Fine-tuning: Base model memory + ~10–20% extra
Actual usage varies with activation memory, batch size, and sequence length. If you're not sure, just reach out — we'll recommend a configuration based on your workload.
How do I get started?
L40S, A100, and H100 can be provisioned instantly from VESSL Cloud. H200, B200, GB200, and B300 are available through our sales team. You don't need exact requirements — just tell us about your current situation and we'll suggest realistic options.
References
VESSL AI