The document provides an in-depth overview of High Bandwidth Memory (HBM), including its features, operating modes, and standards such as HBM2 and HBM2E. HBM is utilized in various applications like GPUs, AI training, and high-performance computing, and it offers superior bandwidth and efficiency compared to traditional memory types. Key topics discussed include channel addressing, command operations, and the specifics of read and write procedures.

1. HBM(High Bandwidth
Memory)
-Harinath reddy

2. Contents
Introduction
HBM2 and HBM2E
HBM Features
Global and each Channel signals
HBM Operating Modes
HBM Channel Addressing
Mode Register set
Write and Read Operation
ROW Commands
COLUMN Commands
References

3. Introduction
• HBM stands for high bandwidth memory and is a type of memory interface
used in 3D-stacked DRAM (dynamic random access memory) in GPUs, as
well as the server, machine-learning DSP , high-performance computing
and networking and client space.
• HBM uses less power and posts higher bandwidth than on DDR4 or
GDDR5 memory with smaller chips.
• HBM technology works by vertically stacking memory chips on top of one
another. The memory chips are connected through through-silicon vias
(TSVs) and microbumps.
• High bandwidths are essential for the complex AI/ML algorithms needed to
rapidly execute massive calculations and safely implement real-time
decisions on the road.

5. HBM2 and HBM2E
• HBM memory stack consists of five chips four storage dies above a single logic
die that controls them and speed upto 128 GBps.
• HBM2 debuted in 2016, and in December 2018 the JEDEC updated the HBM2
standard.
• The HBM2 standard called for up to 8 dies in a stack (as with HBM) with an
overall bandwidth of 256 GBps.
• The HBM2E standard allows up to 12 dies per stack for a max capacity of 24GB.
The standard also pegs memory bandwidth at 307 GBps, delivered across a
1,024-bit memory interface separated by 8 unique channels on each stack.
• Outstanding bandwidth, capacity and latency in a powerefficient, compact footprint
make HBM2E memory a superior choice for AI training hardware.
• According to an Ars Technica report, HBM3 is expected to support up to 64GB
capacities and speeds up to 512 GBps.

6. HBM Features
• 2n prefetch architecture with 256 bits per memory read and write access
• BL = 2 and 4 · 128 DQ width + Optional ECC pin support/channel
• Legacy Mode and Pseudo Channel Mode Operation; (64 DQ width for Pseudo
Channel Mode)
• Differential clock inputs (CK_t/CK_c)
• DDR commands entered on each positive CK_t, CK_c edge. Row Activate
commands require two cycles. All other commands are one cycle command.
• Semi-independent Row & Column Command Interfaces allowing
Activates/Precharges to be issued in parallel with Read/Writes.
• Data referenced to strobes RDQS_t/RDQS_c and WDQS_t/WDQS_c. 1 strobe pair
per DWORD.
• Semi-independent Row & Column Command Interfaces allowing
Activates/Precharges to be issued in parallel with Read/Writes.

7. • Data referenced to strobes RDQS_t/RDQS_c and WDQS_t/WDQS_c. 1 strobe pair per
DWORD. ·
• Up to 8 channels/stack
• 8 or 16 banks per channel; varies by device density/channel
• Bank Grouping supported · 2K or 4K Bytes per page; varies by device density/channel
• DBIac support configurable via MRS
• Data mask for masking WRITE data per byte
• Self Refresh Modes · I/O voltage 1.2 V
• DRAM core voltage 1.2 V, independent of I/O voltage
• Channel density of 1 Gb to 32 Gb
• Unterminated data/address/cmd/clk interfaces
• Temperature sensor with 3-bit encoded range output

8. • Each channel provides access to an independent set of DRAM banks.
• Channels are independently clocked, and need not be synchronous.
• Each channel consists of an independent command and data
interface. RESET, IEEE1500 test port and power supply signals are
common to all channels.
• no channel may access the memory storage for a different channel.
• Each channel interface provides an independent interface to a
number of banks of DRAM of a defined Page size.

9. Single Channel Signal Count & global signal count

10. HBM operating Modes
• HBM DRAM defines two mode of operation depending on channel
density. 1)Legacy Mode and 2)Pseudo Channel Mode
• Legacy mode provides 256 bit prefetch per memory Read and Write
access. Address bit BA4 is a “Don’t Care” in this mode.
• Pseudo Channel mode divides a channel into two individual sub-channels
of 64 bit I/O each, providing 128 bit prefetch per memory Read and
Write access for each Pseudo channel.
• Both Pseudo channels operate semi independent.
• Both Pseudo channels also share the channel’s mode registers. All I/O
signals of DWORD0 and DWORD1 are associated with Pseudo channel 0,
and all I/O signals of DWORD2 and DWORD3 with Pseudo channel 1.

11. Pseudo Channel Mode Operation

12. HBM Channel Addressing

13. Mode Registers
• The bank group feature is configurable via MRS(Mode Registers set).
• All mode registers are programmed via the Mode Register Set (MRS)
command and retain the stored information until they are
reprogrammed, chip reset, or until the device loses power.
• Mode registers must be loaded when all banks are idle and no bursts are
in progress;

14. Mode Registers

15. Operation
• Clocking overview:
• The HBM device captures data on row bus and column bus using differential
CK_t/CK_c.
• The HBM device has uni-directional differential Write strobes (WDQS_t/WDQS_c)
and Read strobes (RDQS_t/RDQS_c) per 32 DQ bits (DWORD).
• HBM Write Data Mask (DM) and Data Bus Inversion (DBIac) Function:
• HBM device supports Data Mask (DM) function for Write operation and Data Bus
Inversion (DBIac) function for Write and Read operation.
• DBI pin is a bi-directional DDR pin and is sampled along with the DQ signals for
Read and Write operation.
• DM pin is bi-directional DDR pin and is sampled along with DQ signals for Read or
Write operation; however DM is input only and is only used for Write operation.

16. Write & Read Operation
Write Operation:
• HBM device inverts Write data received on the DQ inputs in case DBI is
sampled HIGH, or leaves the Write data non-inverted in case DBI is
sampled LOW. Note that DM input is not affected by the DBIac function.
Read Operation:
• HBM device counts the number of DQ signals that are transitioning from
previous state. Note that DM output is not affected by the DBIac
function.
• The HBM device inverts Read data and sets DBI HIGH when the number
of transitioning data bits within a byte is greater than 4, or when the
number of transitioning data bits within a byte equals 4 and DBI was
High; otherwise the HBM device does not invert the Read data and sets
DBI LOW.

17. Cont…

19. ROW Commands
• Row No Operation Command (RNOP):
• Bank and Row ACTIVATE Command (ACT):
• Precharge Command (PRE/PREALL)
• AUTO PRECHARGE
• REFRESH Command (REF)
• SINGLE BANK REFRESH Command (REFSB)

20. • Row No Operation Command (RNOP):
• It is used to instruct the HBM device to perform a NOP as row command; this
prevents unwanted row commands from being registered during idle or wait
states.
• Bank and Row ACTIVATE Command (ACT):
• Before a READ or WRITE command can be issued to a bank, a row in that bank
must be opened. This is accomplished via the ACTIVATE command, which selects
both the bank and the row to be activated. Once a row is open, a READ or WRITE
command could be issued to that row.
• Precharge Command (PRE/PREALL):
• The PRECHARGE command is used to deactivate the open row in a particular
bank (PRE) or the open rows in all banks (PREALL). The bank(s) will be in idle state
and available for a subsequent row access a specified time tRP after the
PRECHARGE command is issued.

21. • AUTO PRECHARGE:
• Auto Precharge is a feature which performs the same individual-bank precharge
function described as Precharge, but without requiring an explicit PRECHARGE
command.
• REFRESH Command (REF):
• The REFRESH command is used during normal operation of the HBM device. The
command is received on the row command inputs R[5:0] and requires a CNOP
command on the column command inputs C[7:0].
• Parity is evaluated with the REFRESH command when the parity calculation is
enabled in the Mode Register.
• SINGLE BANK REFRESH Command (REFSB):
• The SINGLE BANK REFRESH command provides an alternative solution for the
refresh of the HBM device. The command initiates a refresh cycle on a single
bank while accesses to other banks including writes and reads are not affected.

22. COLUMN Commands
• Column No Operation Command (CNOP)
• Read Command (RD, RDA)
• Write Command (WR, WRA)
• Mode Register Set (MRS)
• Power-Mode Commands (Power-Down (PDE, PDX))
• Self-Refresh (SRE, SRX)

23. • Column No Operation Command (CNOP):
• It is used to instruct the HBM device to perform a NOP as column command; this
prevents unwanted column commands from being registered during idle or wait states.
• Read Command (RD, RDA):
• A read burst is initiated with a READ command . The bank and column addresses are
provided with the READ command and auto precharge is either enabled or disabled for
that access.
• Parity is evaluated with the READ command when the parity calculation is enabled in the
Mode Register.
• Write Command (WR, WRA):
• A Write burst is initiated with a WRITE command. The bank and column addresses are
provided with the WRITE command and auto precharge is either enabled or disabled for
that access.
• Parity is evaluated with the WRITE command when the parity calculation is enabled in
MR0

24. • Mode Register Set (MRS):
• The MODE REGISTER SET (MRS) command is used to load the Mode Registers of
the HBM device. The command is received on the column command inputs C[7:0]
and requires a RNOP command on the row command inputs R[5:0].
• Power-Mode Commands:
• Power-Down is entered when CKE is registered LOW along with RNOP and CNOP
commands.
• CKE must not go LOW when read or write operations are in progress.
• CKE can go LOW while any other operations such as row activation, precharge,
auto precharge, or refresh are in progress, but the power-down specification will
not apply until such operations are complete.
• Self-Refresh (SRE, SRX):
• Self-refresh can be used to retain data in the HBM device, even if the rest of the
system is powered down. When in the self-refresh mode.

25. References
• https://www.cs.utah.edu/thememoryforum/mike.pdf
• https://www.design-reuse.com/articles/41186/design-considerations-for-high-
bandwidth-memory-controller.html
• https://www.rambus.com/interface-ip/ddrn-phys/hbm/
• https://go.rambus.com/packaging-solutions-for-ai-and-hpc
• https://go.rambus.com/hbm2e-gddr6-memory-solutions-for-ai
• https://www.rambus.com/rambus-advances-hbm2e-performance-to-4-0-
gbps-for-ai-ml-training-applications/
• https://www.rambus.com/blogs/hbm2e/
• https://www.tomshardware.com/reviews/glossary-hbm-hbm2-high-
bandwidth-memory-definition,5889.html
• https://www.tomshardware.com/reviews/glossary-hbm-hbm2-high-
bandwidth-memory-definition,5889.html