High Bandwidth Memory (HBM)Architecture OverviewDevelopersJEDEC, SK Hynix, AMD, Samsung, MicronReleased2013 (HBM1 standard)Market segmentGPUs, High-Performance Computing (HPC), Data Centers, AI AcceleratorsPhysical StructureDie configuration4, 8, 12, or 16 DRAM core dies stacked over 1 base logic dieVertical stackingThrough-silicon vias (TSVs)Packaging interfaceSilicon interposer (2.5D packaging) or 3D direct stackingBus width1024 bits per stack (up to 2048 bits in HBM4)Performance & SignalingGenerationsHBM1, HBM2, HBM2E, HBM3, HBM3E, HBM4Max bandwidthUp to 1.2–1.5+ TB/s per stack (HBM3E / HBM4)Signaling typeNRZ / PAM4 (generation dependent)Manufacturing & ProcessDRAM nodes1b nm / 1c nm class (latest iterations)Base logic nodeAdvanced nodes (e.g., TSMC 5nm / 12nm FinFET)Key foundriesTSMC, Samsung Foundry, Intel Foundry
High Bandwidth Memory (HBM) is a computer memory interface for 3D-stacked synchronous dynamic random-access memory (SDRAM), initially developed by Samsung, AMD and SK Hynix. It is often used in conjunction with performance-oriented graphics accelerators, network devices, FPGAs and ASICs; some CPUs use HBM as on-package cache or RAM, such as the NEC SX-Aurora TSUBASA and Fujitsu A64FX.ISSCC 2014 Trends page 118 "High-Bandwidth DRAM" The first HBM memory chip was produced by SK Hynix in 2013, and the first devices shipped with HBM were the AMD Fiji GPUs in 2015.
HBM was adopted by JEDEC as an industry standard in October 2013.High Bandwidth Memory (HBM) DRAM (JESD235), JEDEC, October 2013 The second generation, HBM2, was accepted by JEDEC in January 2016. JEDEC officially announced the HBM3 standard on January 27, 2022, and the HBM4 standard in April 2025.
In 2025, the world's largest manufacturers of HBM include SK Hynix, Samsung Electronics, and Micron Technology.
TSMC produces the base die for HBM and is planned to be the foundry for several HBM companies in 2026.
HBM has had an unprecedented demand increase, and in general DRAM (DDR4, DDR, and flash memory/NAND) price has in early 2026 "experienced compounded increases, some exceeding 200%, since early 2025 ... [because of] unprecedented demand coming from the AI sector ... HBM is crowding out commodity DRAM capacity. Micron noted a 3-to-1 conversion ratio between HBM and DDR5 wafer capacity, meaning every HBM ramp directly compresses general-purpose memory supply."
Technology Type Specification establishment date max datarate speedper pin Stack per Stack max capacity max data rate HBM 1 Oct 2013 1.0 Gb/s 8×128 bit 4 dies × 1 GB = 4 GB 128 GB/s HBM 2 Jan 2016 2.4 Gb/s 8 dies × 1 GB = 8 GB 307 GB/s HBM 2E Aug 2019 3.6 Gb/s 12 dies × 2 GB = 24 GB 461 GB/s HBM 3 Jan 2022 6.4 Gb/s 16×64 bit 819 GB/s HBM 3E May 2023 9.8 Gb/s 16 dies × 3 GB = 48 GB 1229 GB/s HBM 4 April 2025 8 Gb/s 32×64 bit 16 dies × 4 GB = 64 GB 2048 GB/s HBM achieves higher bandwidth than DDR4 or GDDR5 while using less power, and in a substantially smaller form factor.HBM: Memory Solution for Bandwidth-Hungry Processors , Joonyoung Kim and Younsu Kim, SK Hynix // Hot Chips 26, August 2014 This is achieved by stacking up to 32 DRAM dies and an optional base die which can include buffer circuitry and test logic. The stack is often connected to the memory controller on a GPU or CPU through a substrate, such as a silicon interposer. Alternatively, the memory die could be stacked directly on the CPU or GPU chip. Within the stack the dies are vertically interconnected by through-silicon vias (TSVs) and microbumps. The HBM technology is similar in principle to, but incompatible with, the Hybrid Memory Cube (HMC) interface developed by Micron Technology.Where Are DRAM Interfaces Headed? // EETimes, 4/18/2014 "The Hybrid Memory Cube (HMC) and a competing technology called High-Bandwidth Memory (HBM) are aimed at computing and networking applications. These approaches stack multiple DRAM chips atop a logic chip."
The HBM memory bus is very wide in comparison to other DRAM memories such as DDR4 or GDDR5. A HBM1 stack of four DRAM dies (4Hi) has two 128bit channels per die for a total of 8 channels and a width of 1024 bits in total. A graphics card/GPU with four 4Hi HBM stacks would therefore have a memory bus with a width of 4096 bits. In comparison, the bus width of GDDR memories is 32 bits, with 16 channels for a graphics card with a 512bit memory interface.Highlights of the HighBandwidth Memory (HBM) Standard . Mike O'Connor, Sr. Research Scientist, NVidia // The Memory Forum – June 14, 2014 HBM1 supported up to 4 GB per package.
The larger number of connections to the memory, relative to DDR4 or GDDR5, required a new method of connecting the HBM memory to the GPU (or other processor). AMD and Nvidia have both used purpose-built semiconductor devices, called interposers, to connect the memory and GPU dies. This interposer has the added advantage of requiring the memory and processor to be physically close, decreasing memory paths. However, as semiconductor device fabrication is significantly more expensive than printed circuit board manufacture, this adds cost to the final product.
Interface
The HBM DRAM is tightly coupled to the host processor die with a distributed interface. The interface is divided into independent channels. The channels are completely independent of one another and are not necessarily synchronous to each other. HBM DRAM uses a wide-interface architecture to achieve high-speed, low-power operation. HBM1 DRAM used a 500 MHz differential clock CK_t / CK_c (where the suffix "_t" denotes the "true", or "positive", component of the differential pair, and "_c" stands for the "complementary" one). Commands are registered at the rising edges of CK_t and CK_c. Each channel interface maintained a 128bit data bus operating at this double data rate (DDR). HBM1 supported transfer rates of 1 GT/s per pin (transferring 1 bit), yielding an overall package bandwidth of 128 GB/s.
HBM2 The second generation of High Bandwidth Memory, HBM2, also specified up to eight dies per stack and doubled pin transfer rates up to 2 GT/s. Retaining 1024bit wide access, HBM2 was able to reach 256 GB/s memory bandwidth per package. The HBM2 spec allowed up to 8 GB per package. HBM2 was predicted to be especially useful for performance-sensitive consumer applications such as virtual reality.
On January 19, 2016, Samsung announced early mass production of HBM2, at up to 8 GB per stack. SK Hynix also announced availability of 4 GB stacks in August 2016.
HBM2E In late 2018, JEDEC announced an update to the HBM2 specification, providing for increased bandwidth and capacities. Up to 307 GB/s per stack (2.5 Tbit/s effective data rate) was then supported in the official specification, though products operating at this speed had already been available. Additionally, the update added support for 12Hi stacks (12 dies) making capacities of up to 24 GB per stack possible.
On March 20, 2019, Samsung announced their Flashbolt HBM2E, featuring eight dies per stack, a transfer rate of 3.2 GT/s, providing a total of 16 GB and 410 GB/s per stack.
August 12, 2019, SK Hynix announced their HBM2E, featuring eight dies per stack, a transfer rate of 3.6 GT/s, providing a total of 16 GB and 460 GB/s per stack. On July 2, 2020, SK Hynix announced that mass production has begun.
In October 2019, Samsung announced their 12-layered HBM2E.
HBM3 In late 2020, Micron unveiled that the HBM2E standard would be updated and alongside that they unveiled the then next standard known as HBMnext (later renamed to HBM3). This was to be a big generational leap from HBM2 and the replacement to HBM2E. This new VRAM would have come to the market in the Q4 of 2022. This would likely have introduce a new architecture as the naming suggests.
While the architecture might have been overhauled, leaks pointed to performance similar to the updated HBM2E standard. This RAM was likely to be used mostly in data center GPUs.
In mid 2021, SK Hynix unveiled some specifications of the HBM3 standard, with 5.2 Gbit/s I/O speeds and bandwidth of 665 GB/s per package, as well as up to 16-high 2.5D and 3D solutions.
On 20 October 2021, before the JEDEC standard for HBM3 was finalised, SK Hynix was the first memory vendor to announce that it had finished development of HBM3 memory devices. According to SK Hynix, the memory would have run as fast as 6.4 Gbit/s/pin, double the data rate of JEDEC-standard HBM2E, which formally topped out at 3.2 Gbit/s/pin, or 78% faster than SK Hynix's own 3.6 Gbit/s/pin HBM2E. The devices supported a data transfer rate of 6.4 Gbit/s and therefore a single HBM3 stack might have provided a bandwidth of up to 819 GB/s. The basic bus widths for HBM3 remained unchanged, with a single stack of memory being 1024-bits wide. SK Hynix would offer this memory in two capacities: 16 GB and 24 GB, aligning with 8-Hi and 12-Hi stacks respectively. The stacks consisted of 8 or 12 16 Gb DRAMs that are each 30 μm thick and interconnected using Through Silicon Vias (TSVs).
According to Ryan Smith of AnandTech, the SK Hynix first generation HBM3 memory had the same density as their latest-generation HBM2E memory, meaning that device vendors looking to increase their total memory capacities for their next-generation parts would need to use memory with 12 dies/layers, up from the 8 layer stacks they typically used until then. According to Anton Shilov of Tom's Hardware, high-performance compute GPUs or FPGAs typically used four or six HBM stacks, so with SK Hynix's HBM3 24 GB stacks they would accordingly get 3.2 TB/s or 4.9 TB/s of memory bandwidth. He also noted that SK Hynix's HBM3 chips were square, not rectangular like HBM2 and HBM2E chips. According to Chris Mellor of The Register, with JEDEC not yet having developed its HBM3 standard, might have meant that SK Hynix would have needed to retrofit its design to a future and faster one.
JEDEC officially announced the HBM3 standard on January 27, 2022. The number of memory channels was doubled from 8 channels of 128 bits with HBM2e to 16 channels of 64 bits with HBM3. Therefore, the total number of data pins of the interface was still 1024.
In June 2022, SK Hynix announced they started mass production of industry's first HBM3 memory to be used with Nvidia's H100 GPU expected to ship in Q3 2022. The memory would provide H100 with "up to 819 GB/s" of memory bandwidth.
In August 2022, Nvidia announced that its "Hopper" H100 GPU would ship with five active HBM3 sites (out of six on board) offering 80 GB of RAM and 3 TB/s of memory bandwidth (16 GB and 600 GB/s per site).
HBM3E On 30 May 2023, SK Hynix unveiled its HBM3E memory with 8 Gbit/s/pin data processing speed (25% faster than HBM3), which was to enter production in the first half of 2024. At 8 GT/s with 1024-bit bus, its bandwidth per stack is increased from 819.2 GB/s as in HBM3 to 1 TB/s.
On 26 July 2023, Micron announced its HBM3E memory with 9.6 Gbit/s/pin data processing speed (50% faster than HBM3). Micron HBM3E memory was a high-performance HBM that used 1β DRAM process technology and advanced packaging to achieve the highest performance, capacity and power efficiency in the industry. It could store 24 GB per 8-high cube and allowed data transfer at 1.2 TB/s. There would be a 12-high cube with 36 GB capacity in 2024.
In August 2023, Nvidia announced a new version of their GH200 Grace Hopper superchip that used 141 GB (144 GiB physical) of HBM3e over a 6144-bit bus providing 50% higher memory bandwidth and 75% higher memory capacity over the HBM3 version.
In May 2023, Samsung announced HBM3P with up to 7.2 Gbit/s which would be in production in 2024.
On October 20, 2023, Samsung announced their HBM3E "Shinebolt" with up to 9.8 Gbit/s memory.
On February 26, 2024, Micron announced the mass production of Micron's HBM3E memory.
On March 18, 2024, Nvidia announced the Blackwell series of GPUs using HBM3E memory