How much does an idle NVIDIA L4 cost in the cloud?
The lowest per-GPU rate for the NVIDIA L4 is on g2-standard-4 on Google Cloud (us-central1), at $0.71/GPU-hr on-demand. That whole box left running around the clock costs about $516/mo. Scheduling it off outside business hours saves up to $362/mo (est.).
Cheapest per GPU
$0.71/GPU-hr
g2-standard-4 · us-central1
Whole instance 24/7
$516/mo
730-hour month
Saved with a schedule (up to, est.)
$362/mo
up to 70% off the 24/7 bill (est.)
Instances carrying the NVIDIA L4
| Instance | Provider | GPUs | vCPU / Memory | $/hr | $/GPU-hr | Spot $/GPU-hr |
|---|---|---|---|---|---|---|
| g2-standard-4 | Google Cloud | 1× GPU | 4 vCPU · 16 GB | $0.707/hr us-central1 | $0.71 | $0.42 |
| g6.xlarge | AWS | 1× GPU | 4 vCPU · 16 GB | $0.805/hr us-east-2 | $0.80 | $0.27 |
| g2-standard-8 | Google Cloud | 1× GPU | 8 vCPU · 32 GB | $0.854/hr us-central1 | $0.85 | $0.51 |
| g6.2xlarge | AWS | 1× GPU | 8 vCPU · 32 GB | $0.978/hr us-east-2 | $0.98 | $0.28 |
| g2-standard-12 | Google Cloud | 1× GPU | 12 vCPU · 48 GB | $1.000/hr us-central1 | $1.00 | $0.60 |
| g6.12xlarge | AWS | 4× GPU | 48 vCPU · 192 GB | $4.602/hr us-east-2 | $1.15 | $0.35 |
| g6.4xlarge | AWS | 1× GPU | 16 vCPU · 64 GB | $1.323/hr us-east-2 | $1.32 | $0.50 |
| g6.8xlarge | AWS | 1× GPU | 32 vCPU · 128 GB | $2.014/hr us-east-2 | $2.01 | $0.71 |
| g6.16xlarge | AWS | 1× GPU | 64 vCPU · 256 GB | $3.397/hr us-east-2 | $3.40 | $0.92 |
About the NVIDIA L4
The L4 is the current default for cost-efficient inference on both clouds — AWS wraps it in the g6 family, Google Cloud builds the g2 machine series around it. 24 GB of VRAM and strong efficiency per watt make it the natural landing spot for production model serving, media pipelines, and lightweight fine-tuning.
Because it is the "small, sensible" GPU, L4 machines multiply: one per service, one per environment, one per experiment. Each looks harmless on the bill. Fleet-wide, the always-on ones dominate — an inference box that serves traffic around the clock earns its keep, but its staging twin and the two experiment boxes beside it usually do not. The savings from scheduling those off scales with fleet size, not box size.
Common questions
AWS g6 or GCP g2 for L4 serving?
The silicon is the same; the decision usually follows your existing platform — data gravity, orchestration, and committed-use economics. Check the live per-region rates above for the current gap, and remember that a modest rate advantage disappears quickly if one cloud's box runs 24/7 while the other's stops off-hours.
Do staging and dev copies of inference services need their own GPUs?
They usually get them, because a GPU code path is hard to exercise without a GPU — and then they sit unused between deploys. The fix is rarely deletion: give each non-production box an owner and an expiry, so it exists when someone claims it and stops itself when the claim runs out. The same fleet then bills for a working week instead of a calendar week.
Other GPU tiers
Stop paying for idle cloud GPU servers
Idlefy runs a free audit of your fleet, then flips dev servers to stopped-by-default: lease one when you need it, and
when the lease ends it shuts itself off — stop, not terminate, persistent disks intact. Timers are set by your team, never
guessed, and Idlefy only manages instances tagged idlefy=enabled — on AWS that boundary is enforced by the IAM policy itself.
Prices are on-demand list price for Linux (AWS) / standard (GCP) instances and exclude storage, network, data transfer, and any negotiated or volume discounts. Actual bills vary by usage and account-level pricing. Spot figures are indicative — AWS values are derived from AWS's published typical savings vs on-demand for the region; GCP values are the listed Spot price. Spot capacity can be reclaimed by the provider. Stopping preserves EBS and persistent-disk volumes; data on local instance-store or local-SSD drives does not survive a stop. Prices as of 2026-08-31.