H100 Cloud Rental Cost 2026: Provider Price Guide

H100 Cloud Rental Cost 2026: Provider Price Guide

Photo by Jakub Zerdzicki on Pexels

Quick Answer: In 2026, H100 cloud rental ranges from $1.20/hr (io.net spot) to $4.73/hr (AWS on-demand). The best value for most AI workloads is Lambda Labs at $2.49/hr (reliable, good support, consistent performance) or RunPod at $2.29/hr (good availability, community features). For the absolute cheapest H100 access, io.net spot at $1.20-1.80/hr is 75% cheaper than AWS but with 10-20% failure rates. For production serving with SLA requirements, Lambda Labs or AWS are the only viable choices. New H200 and B200 GPUs are available at 10-20% premium over H100 pricing.

H100 Pricing Comparison: All Providers

Single H100 Pricing ($/hr)

8x H100 Instance Pricing ($/hr)

8x H100 is where the price gap widens dramatically: AWS charges $151/hr for 8x H100 while Lambda Labs charges $19.92/hr (87% less). AWS includes NVLink, better networking, and enterprise support — but the 7.5x price premium is hard to justify for most workloads.

On-Demand vs Spot vs Reserved Pricing

Savings by Commitment Type

When to Use Each

Spot Preemption Rates

H100 vs H200 vs B100 Pricing

New GPU Pricing (2026)

Is the H200/B100 Worth the Premium?

For most teams, H100 is the sweet spot. H200 gives 24% more throughput for 20% more cost — marginally better value. B100 is faster but at 40% premium over H100. Upgrade only if you're bottlenecked on throughput.

Photo by Andrey Matveev on Pexels

Cost per Million Tokens: The Real Metric

70B Model Serving Cost

8B Model Serving Cost

Cheapest per-token inference: io.net spot H100 ($0.0007/1M tokens) for 70B. For small models, consumer GPUs on DePIN networks are dramatically cheaper per token than H100s.

Training Cost: H100 vs Consumer GPU Rental

Fine-Tuning Cost Comparison (Llama 3.1 70B, QLoRA, 1 epoch, 1000 samples)

Full Training Cost (70B, LoRA, 1 epoch, 10K samples)

Hidden Costs: Storage, Transfer, and Engineering Time

Storage Costs

Data Transfer (Egress)

Real Monthly Cost Example

Workload: Batch inference, 70B, 50M tokens/day, H100

Provider Rankings by Use Case

Best for Production Inference (SLA Required)

1. Lambda Labs  ★★★★★  $2.49/hr, 99.5% uptime, good support
2. RunPod       ★★★★☆  $2.29/hr, 98% uptime, community support
3. AWS          ★★★☆☆  $4.73/hr, 99.9% uptime, enterprise pricing

Best for Training

1. RunPod       ★★★★★  $2.29/hr (on-demand), $1.85/hr (spot)
2. Lambda Labs  ★★★★★  $2.49/hr, consistent performance
3. Vast.ai      ★★★★☆  $1.89/hr, but variable quality

Best for Batch Inference (Cheapest)

1. io.net spot  ★★★★★  $1.20/hr — can't beat the price
2. Vast.ai bid  ★★★★☆  $1.50/hr — more reliable than io.net
3. Akash        ★★★★☆  $2.20-3.00/hr — most decentralized

Best for Experimentation

1. Vast.ai      ★★★★★  $1.50-1.89/hr, widest selection
2. RunPod       ★★★★★  $1.85-2.29/hr, best UX
3. io.net       ★★★★☆  $1.20-2.59/hr, cheapest spot

Related Reads

• RunPod vs Vast.ai vs Lambda Labs: Best GPU Cloud for LLMs
• Akash Network H100 Pricing: How Cheap vs AWS?
• Self-Host Cost Per Million Tokens: Calculator & Provider Data

Optimizing H100 Workloads for Cost Efficiency

Beyond choosing the right provider, small adjustments to workload configuration can yield outsized cost savings. For inference, batch size is the most critical lever: increasing batch size from 1 to 32 on a 70B model boosts throughput by 8x (from 55 to 450 tokens/sec) while only doubling GPU memory usage. This reduces cost per million tokens by 75%—transforming a $0.0029/1M token AWS deployment into a $0.0007/1M token operation on io.net. However, larger batches introduce latency; for real-time applications, benchmark your model’s latency vs. throughput tradeoff at batch sizes of 1, 2, 4, 8, 16, and 32 to find the optimal balance.

For training, mixed precision (FP8) and gradient checkpointing can cut costs by 30-50% without sacrificing model quality. FP8 reduces memory usage by 50% compared to FP16, enabling larger batch sizes or fitting larger models on a single H100. Gradient checkpointing trades compute for memory, reducing VRAM usage by 30-40% at the cost of a 20-30% increase in training time. When combined, these techniques can reduce the cost of a 70B fine-tuning run from $80 to $40 on RunPod. Providers like Lambda Labs and RunPod offer pre-configured templates with these optimizations enabled, while AWS requires manual setup via custom AMIs or Docker containers.

Network Topology and Multi-GPU Scaling

The performance gap between single H100s and 8x H100 instances isn’t linear due to networking bottlenecks. AWS’s p5.48xlarge instances include NVLink and 3.2Tbps networking, enabling near-linear scaling for distributed training (e.g., 7.8x speedup for 8x GPUs). In contrast, providers like Lambda Labs and RunPod use PCIe 4.0 or 5.0 for multi-GPU communication, which caps scaling efficiency at 6-7x for 8x GPUs. For workloads like full fine-tuning or large-scale inference, this difference can add 20-30% to runtime costs. Benchmark your workload’s scaling efficiency by comparing single-GPU performance to 2x, 4x, and 8x configurations—if scaling efficiency drops below 80%, consider splitting workloads across multiple single-GPU instances instead of using a single 8x instance.

For inference, multi-GPU setups are rarely cost-effective unless you’re serving at scale. A single H100 can handle ~500 tokens/sec for a 70B model at batch=32, which translates to ~130M tokens/day—enough for most production workloads. If you need higher throughput, deploy multiple single-GPU instances behind a load balancer (e.g., Nginx or Traefik) rather than using an 8x H100 instance. This approach improves fault tolerance and reduces costs by 30-50% compared to a single 8x instance, as you can scale horizontally with spot instances.

Provider-Specific Quirks and Workarounds

Each H100 provider has unique limitations that can impact cost and performance if not accounted for:

• AWS: EBS boot volumes are slow (100-200 MB/s); use instance storage for datasets or cache. Egress costs ($0.05-0.09/GB) can exceed compute costs for large datasets—compress data or use AWS’s free tier (100GB/month) for transfers.
• Lambda Labs: No spot instances, but their 1-month reserved pricing ($1,793/mo) is effectively a 2% discount. Their NVMe storage is included up to 200GB, making them ideal for storage-heavy workloads like video processing.
• RunPod: Spot instances have a 5-minute preemption warning, but their API doesn’t expose this—use their webhook feature to trigger checkpointing. Their $0.01/GB egress is the cheapest among providers, but transfers are throttled to 1Gbps.
• Vast.ai: Provider quality varies wildly; filter for "H100 PCIe 4.0" or "NVLink" in the search bar to avoid underperforming instances. Their bid system can save 20-30% over on-demand, but lowball bids may get preempted quickly.
• io.net: Spot instances are the cheapest ($1.20/hr) but have no preemption warning—implement a heartbeat system to detect failures. Their storage is ephemeral, so use external storage (e.g., S3 or Backblaze B2) for datasets.

For production workloads, test each provider’s networking performance with tools like ib_write_bw (for InfiniBand) or iperf3 (for TCP). AWS and Azure offer 100Gbps+ networking, while alternative providers typically cap at 25-50Gbps. If your workload involves frequent data transfers (e.g., distributed training), this can add 10-20% to runtime costs. For storage-bound workloads, benchmark disk I/O with fio—Lambda Labs’ NVMe storage delivers 3-5GB/s, while AWS’s gp3 EBS tops out at 1GB/s.

Key Takeaways

• For most AI workloads, Lambda Labs ($2.49/hr) or RunPod ($2.29/hr) offer the best balance of cost, reliability, and support—avoid AWS ($4.73/hr) unless you need enterprise SLAs or tight cloud integration.
• Spot instances (e.g., io.net at $1.20/hr) can cut costs by 75% but carry 30-50% preemption risk; use them only for fault-tolerant workloads like batch inference or hyperparameter sweeps with checkpointing every 10-15 minutes.
• H100 remains the cost-performance sweet spot in 2026: H200 offers 24% more throughput for a 20% price premium, while B100’s 40% premium is only justified for throughput-bottlenecked workloads like high-volume 70B inference or full fine-tuning.
• For 70B inference, io.net spot H100s deliver the lowest cost per million tokens ($0.0007), but consumer GPUs (e.g., RTX 4090 on Vast.ai at $0.0008/1M tokens) are dramatically cheaper for smaller models like 8B.
• Hidden costs add up: AWS charges $0.08/GB-month for storage and $0.05-0.09/GB egress, while providers like RunPod ($0.01/GB egress) or Lambda Labs (500GB free egress) can reduce total monthly costs by 20-40%.
• For full fine-tuning (LoRA, 10K samples), expect $50-80 on RunPod/Vast.ai vs $600 on AWS; spot instances can further reduce training costs by 15-30% if checkpointing is implemented.