NVIDIA H100 Tensor Core GPU (80GB)

Home
Accelerators
Nvidia
NVIDIA H100

Now Available

Introduction

An Order-of-Magnitude Leap for Accelerated Computing

The NVIDIA H100 GPU delivers exceptional performance, scalability, and security for every workload. Built on the NVIDIA Hopper™ architecture, the H100 features fourth-generation Tensor Cores and a dedicated Transformer Engine with FP8 precision, accelerating large language models by up to 30X over the prior generation. From enterprise inference to exascale HPC, the H100 is the proven foundation for data center AI.

Built on the NVIDIA Hopper Architecture

The Hopper architecture delivers a massive leap in accelerated computing performance. With fourth-generation NVLink at 900 GB/s, a dedicated Transformer Engine, and support for PCIe Gen5, the H100 powers everything from trillion-parameter models to scientific simulations.

Learn About the Hopper Architecture

Workloads

Securely Accelerate Workloads From Enterprise to Exascale

4Xfaster

AI Training on GPT-3

30Xfaster

AI Inference (Megatron 530B)

7Xfaster

HPC Applications

Up to 4X Higher AI Training on GPT-3

Projected performance subject to change. GPT-3 175B training A100 cluster: HDR IB network, H100 cluster: NDR IB network | Mixture of Experts (MoE) Training Transformer Switch-XXL variant with 395B parameters on 1T token dataset, A100 cluster: HDR IB network, H100 cluster: NDR IB network with NVLink Switch System where indicated.

Transformational AI Training

The H100 features fourth-generation Tensor Cores and a Transformer Engine with FP8 precision, delivering up to 4X faster training over the prior generation for GPT-3 (175B) models. Combined with fourth-generation NVLink at 900 GB/s and NDR InfiniBand networking, the H100 scales efficiently from small enterprise clusters to massive multi-node GPU deployments.

View Training Performance Benchmarks

Up to 30X Higher AI Inference Performance on the Largest Models

Megatron chatbot inference (530 billion parameters)

Projected performance subject to change. Inference on Megatron 530B parameter model based chatbot for input sequence length=128, output sequence length=20 | A100 cluster: HDR IB network | H100 cluster: NVLink Switch System, NDR IB

Real-Time Deep Learning Inference

The H100 extends NVIDIA's inference leadership with up to 30X higher performance on the largest models. Fourth-generation Tensor Cores accelerate all precisions — FP64, TF32, FP32, FP16, INT8, and FP8 — reducing memory usage while maintaining accuracy for large language models deployed at scale.

Explore NVIDIA's AI Inference Platform

Up to 7X Higher Performance for HPC Applications

Exascale High-Performance Computing

The H100 triples the double-precision Tensor Core FLOPS over the prior generation, delivering 60 teraflops of FP64 computing. New DPX instructions provide up to 7X higher performance on dynamic programming algorithms like Smith-Waterman for genomic sequencing and protein structure prediction.

Learn More About High-Performance Computing

Enterprise-Ready: MIG & Confidential Computing

Multi-Instance GPU (MIG) technology lets IT managers partition a single H100 into up to seven isolated GPU instances, maximizing utilization across workloads. The H100 is also the first accelerator with built-in Confidential Computing, providing hardware-based trusted execution environments to protect data and AI models in use.

Learn More About MIG

H100 NVL

Supercharge Large Language Model Inference With H100 NVL

The PCIe-based NVIDIA H100 NVL with NVLink bridge is optimized for LLMs up to 70 billion parameters (Llama 2 70B). With 188GB of combined HBM3 memory across an NVL pair, it delivers up to 5X performance over NVIDIA A100 systems while maintaining low latency in power-constrained, air-cooled data center environments. The H100 NVL is ideal for enterprise rack designs that require flexible GPU configurations without liquid cooling infrastructure.

Server Compatibility

Compatible Dell PowerEdge Servers

Dell PowerEdge XE9680

Supports 8× H100 SXM5 GPUs via HGX board
Purpose-built for large-scale Al training

View Server

Dell PowerEdge R760xa

Supports H100 NVL PCle GPUs
Versatile 2U platform for Al and HPC

View Server

Dell PowerEdge XE7740

Supports H100 NVL with up to 8 GPUs
17th Gen, optimized for dense Al workloads

View Server

Dell PowerEdge R7725

H100 NVL compatible, AMD EPYC powered
17th Gen 2U rack server

View Server

Dell PowerEdge R770

H100 NVL compatible, Intel Xeon powered
17th Gen 2U enterprise server

View Server

Dell PowerEdge R7625

H100 NVL compatible, AMD EPYC powered
16th Gen 2U rack server

View Server

View Full GPU Compatibility Matrix

Specifications

NVIDIA H100 GPU

	H100 SXM	H100 NVL
FP64	34 teraFLOPS	30 teraFLOPS
FP64 Tensor Core	67 teraFLOPS	60 teraFLOPS
FP32	67 teraFLOPS	60 teraFLOPS
TF32 Tensor Core^*	989 teraFLOPS	835 teraFLOPS
BFLOAT16 Tensor Core^*	1,979 teraFLOPS	1,671 teraFLOPS
FP16 Tensor Core^*	1,979 teraFLOPS	1,671 teraFLOPS
FP8 Tensor Core^*	3,958 teraFLOPS	3,341 teraFLOPS
INT8 Tensor Core^*	3,958 TOPS	3,341 TOPS
GPU Memory	80GB HBM3	94GB HBM3
GPU Memory Bandwidth	3.35 TB/s	3.9 TB/s
Decoders	7 NVDEC 7 JPEG	7 NVDEC 7 JPEG
Max Thermal Design Power (TDP)	Up to 700W (configurable)	350–400W (configurable)
Multi-Instance GPUs	Up to 7 MIGs @ 10GB each	Up to 7 MIGs @ 12GB each
Form Factor	SXM	PCIe Dual-slot air-cooled
Interconnect	NVIDIA NVLink™: 900GB/s PCIe Gen5: 128GB/s	NVIDIA NVLink: 600GB/s PCIe Gen5: 128GB/s
Server Options	NVIDIA HGX H100 partner and NVIDIA-Certified Systems™ with 4 or 8 GPUs NVIDIA DGX H100 with 8 GPUs	Partner and NVIDIA-Certified Systems with 1–8 GPUs

^* With sparsity.

View Datasheet View Hopper Architecture Whitepaper

Interested in This GPU?

Get pricing, availability, and compatibility details from the ServerMonkey team.

Request a Quote

NVIDIA H100 GPU

An Order-of-Magnitude Leap for Accelerated Computing

Built on the NVIDIA Hopper Architecture

Securely Accelerate Workloads From Enterprise to Exascale

Up to 4X Higher AI Training on GPT-3

Transformational AI Training

Up to 30X Higher AI Inference Performance on the Largest Models

Real-Time Deep Learning Inference

Up to 7X Higher Performance for HPC Applications

Exascale High-Performance Computing

Enterprise-Ready: MIG & Confidential Computing

Supercharge Large Language Model Inference With H100 NVL

Compatible Dell PowerEdge Servers

Dell PowerEdge XE9680

Dell PowerEdge R760xa

Dell PowerEdge XE7740

Dell PowerEdge R7725

Dell PowerEdge R770

Dell PowerEdge R7625

NVIDIA H100 GPU

Interested in This GPU?

What are you looking for?

Get to Know Us

Resources

What We Do

Customer Service

Follow Us