How much does an idle NVIDIA H100 cost in the cloud?
The lowest per-GPU rate for the NVIDIA H100 is on p5.48xlarge on AWS (us-east-2), at $6.88/GPU-hr on-demand. That whole box left running around the clock costs about $40,179/mo. Scheduling it off outside business hours saves up to $28,221/mo (est.).
Cheapest per GPU
$6.88/GPU-hr
p5.48xlarge · us-east-2
Whole instance 24/7
$40,179/mo
730-hour month
Saved with a schedule (up to, est.)
$28,221/mo
up to 70% off the 24/7 bill (est.)
Instances carrying the NVIDIA H100
| Instance | Provider | GPUs | vCPU / Memory | $/hr | $/GPU-hr | Spot $/GPU-hr |
|---|---|---|---|---|---|---|
| p5.48xlarge | AWS | 8× GPU | 192 vCPU · 2048 GB | $55.040/hr us-east-2 | $6.88 | $1.79 |
| a3-highgpu-8g | Google Cloud | 8× GPU | 208 vCPU · 1872 GB | $87.832/hr us-central1 | $10.98 | $6.59 |
About the NVIDIA H100
The H100 is the default answer to "we need to train something real": mature software support, multi-cloud availability, and enough per-GPU memory (80 GB) for most practical training and fine-tuning work. Both AWS and Google Cloud sell it in eight-GPU nodes designed to be rented for a run — which is exactly how the economics make sense.
The failure mode is the gap between runs. An eight-H100 node kept "warm" while results are reviewed, datasets rebuilt, or the next experiment argued about, bills at full rate for every idle hour — and those in-between weeks routinely cost more than the training time itself. Comparing per-GPU-hour rates across clouds is worth doing once; controlling idle hours pays every week.
Common questions
AWS or GCP for H100 training?
Rates differ by region and change; check the live table above rather than folklore. The tie-breakers are usually non-price: where your data already lives, which quota you can actually get, and how your orchestration stack handles each cloud's node shapes. For bursty work, how fast you can stop and restart matters more than a small rate delta.
How much of a typical H100 bill is idle time?
For teams that train in bursts, the idle share is often the larger half — nodes provisioned for a deadline outlive it, and eight GPUs bill whether one job or none is running. An automatic stop at lease expiry converts that from a recurring surprise into a non-event, with savings that can reach up to the full off-hours share of the bill (est.).
Other GPU tiers
Stop paying for idle cloud GPU servers
Idlefy runs a free audit of your fleet, then flips dev servers to stopped-by-default: lease one when you need it, and
when the lease ends it shuts itself off — stop, not terminate, persistent disks intact. Timers are set by your team, never
guessed, and Idlefy only manages instances tagged idlefy=enabled — on AWS that boundary is enforced by the IAM policy itself.
Prices are on-demand list price for Linux (AWS) / standard (GCP) instances and exclude storage, network, data transfer, and any negotiated or volume discounts. Actual bills vary by usage and account-level pricing. Spot figures are indicative — AWS values are derived from AWS's published typical savings vs on-demand for the region; GCP values are the listed Spot price. Spot capacity can be reclaimed by the provider. Stopping preserves EBS and persistent-disk volumes; data on local instance-store or local-SSD drives does not survive a stop. Prices as of 2026-08-31.