Cloud GPU Services Compared: Renting Compute for Training and Inference
AWS, Google Cloud, Azure and Lambda compared on the same unit: dollars per GPU-hour on demand, plus the traps — full-node minimums, egress and quotas.
By Galactic News Editorial · Published 2026-11-07T15:04:07+00:00 · Updated 2026-11-07T15:04:07+00:00
Cloud GPU Services Compared: Renting Compute for Training and Inference
Renting a GPU in the cloud beats buying one for most teams. The hard part is comparing prices. Vendors quote per GPU-hour, per node, and per month, and sometimes only for eight-GPU blocks. This comparison puts AWS, Google Cloud, Azure, and Lambda side by side on the same unit: dollars per GPU-hour, on demand. The prices below are list prices from October 2026 roundups that cite the vendors' own price pages. Check the live page before you budget, because GPU prices move often.
Cloud GPU Services Compared at a Glance
| Provider | Flagship GPU shapes | H100 on demand, per GPU-hour | Billing | Best for |
|---|---|---|---|---|
| AWS | P5 (H100), P5en (H200), P6 (B200 and B300), G6e (L40S) | $6.88 | On-demand, Spot, Savings Plans, and Capacity Blocks; billed per second | Teams already on AWS; the cheapest hyperscaler H100 |
| Google Cloud | A3 (H100), A3 Ultra (H200), A4 (B200), A2 (A100), G2 (L4) | About $10.98 | On-demand billed per second with a one-minute minimum; Spot discounts up to 91% | Teams on GCP, GKE, or Vertex AI |
| Azure | ND H100 v5, ND H200 v5 | About $12.29 | Pay-as-you-go, with reservations for longer terms | Microsoft shops and enterprise buyers |
| Lambda | H100 SXM, H100 PCIe, A100, B200, GH200 | $3.99 in an 8-GPU node; $3.29 for a single H100 PCIe card | Per-minute billing; no egress fees | Straightforward GPU rental without a hyperscaler stack |
The gap is wide. Lambda's H100 runs about $3.99 per GPU-hour in an eight-GPU node, against $6.88 on AWS, about $10.98 on Google Cloud, and about $12.29 on Azure. The hyperscalers charge more, and they bundle more: identity, storage, networking, and managed services. Lambda sells raw GPUs with less around them. The right choice depends on what else you need, not just the hourly rate.
Training vs Inference: Different Machines
Training and inference need different hardware. Training a large model needs many GPUs in one node, fast links between them, and fast links between nodes. AWS P5 and P6, Google Cloud A3 and A4, and Azure ND-series machines are built for that. They use NVLink inside the node and high-speed fabric between nodes. Google Cloud's A3 Mega, for example, pairs eight H100 cards with up to 1,800 Gbps of node networking.
Inference is lighter. A single GPU often suffices, and cheaper cards work well for smaller models. Google Cloud's G2 machine, with the NVIDIA L4, runs about $0.70 per GPU-hour on demand. It fits models that fit in 24 GB of memory after quantization. Lambda's A100 40 GB card costs $1.99 per GPU-hour, and its A100 80 GB card costs $2.79. For fine-tuning on a budget, an A100 is the pragmatic middle ground: far cheaper per hour than Hopper or Blackwell cards, with broad availability.
Match the card to the model size first. A 7-billion-parameter model does not need an H100. A 70-billion-parameter training run does not belong on a single L4. Overspending on the card is the most common waste in cloud GPU bills.
The Price Traps in Cloud GPU Services
Five traps catch buyers. Each one is documented on the vendors' own pages.
- Full-node minimums. The headline per-GPU rate often assumes an eight-GPU node. Google Cloud's a3-highgpu-8g is the only A3 H100 shape sold on demand, at $87.83 an hour for the node. Smaller A3 shapes need Spot or Flex-start VMs. AWS P5 and P6 eight-GPU nodes work the same way. Lambda's H100 SXM is also an eight-GPU node at $3.99 per GPU-hour, though its H100 PCIe card is available as a single GPU at $3.29 an hour.
- Reservation-only shapes. Google Cloud's newest hardware is reservation-first. A4 machines with B200 cards cannot be launched as standard on-demand VMs. They need a reservation, a Spot VM, or a Flex-start VM. Plan ahead or pick an older shape.
- Egress fees. Moving data out of a cloud costs money. AWS gives 100 GB a month free, then charges $0.09 per GB. Lambda charges no egress fees at all, including for data on attached storage. If your workload moves large datasets out, egress can rival the GPU bill.
- Quotas. AWS P5 instances need a service quota increase before they will launch at all. The default quota is zero. Request the increase days before you need the machines, not the morning of the run.
- Spot and preemptible pricing. Spot VMs on Google Cloud run at discounts of up to 91% off on-demand rates, but they can be preempted with 30 seconds of notice. AWS Spot and Azure low-priority VMs work the same way. Use them for fault-tolerant batch work, never for a live inference endpoint.
On-Demand, Spot, or Reserved: Pick a Purchase Model
On-demand is the default. You pay the hourly rate, start when you want, and stop when you are done. It is the right choice for short jobs, tests, and anything with an unknown finish time. It is also the most expensive way to run a long job.
Spot and preemptible VMs cut the price sharply. Google Cloud's Spot discounts reach up to 91% off on-demand rates for most machine types and GPUs. AWS Spot and Azure low-priority capacity work the same way. The trade is interruption. A Spot VM can be taken back with short notice, so use Spot only for work that can restart, such as batch training with checkpoints.
Reservations and capacity blocks suit steady workloads. AWS sells Savings Plans and EC2 Capacity Blocks for machine learning, which reserve hardware for a defined window. Lambda sells 1-Click Clusters with a two-week minimum commitment for larger training runs. If you know you will run the same job every week for months, ask the vendor for a committed-use quote before you pay on-demand rates. A dated snapshot from September 2026 put three-year committed H100 pricing near $2.97 per GPU-hour on AWS, against $6.88 on demand. Committed rates change, so get a fresh quote.
How to Choose in Four Questions
- Where does your data live? If it sits in S3, BigQuery, or Azure storage, the matching cloud usually wins on transfer cost and setup time, even at a higher GPU rate.
- Do you need one GPU or eight? Single-GPU work points to Lambda's PCIe cards, Google Cloud's A2 or G2 shapes, or AWS G6 instances. Multi-node training points to P5, P6, A3, or A4.
- Can the job be interrupted? If yes, Spot pricing cuts the bill by half or more. If no, pay the on-demand rate and sleep well.
- What is the total bill? Add storage, egress, and managed-service fees to the GPU hours. A cheap GPU-hour with expensive egress is not cheap.
Bottom Line
For raw GPU-hours, Lambda is the cheapest of the four, at $3.99 per GPU-hour for H100 in an eight-GPU node, with no egress fees. For teams already on a hyperscaler, AWS has the cheapest H100 among the big three at $6.88 per GPU-hour, and its P5 family is the simplest path to large-scale training. Google Cloud suits teams on GKE and Vertex AI, but its newest B200 shapes need reservations. Azure suits Microsoft shops willing to pay the premium. Whatever you pick, price the whole job, not the headline rate, and check the live pricing page on the day you buy.