Servers demand memory that is continually higher performance and lower power while the manufacturing base quickly transitions to sub-20nm process nodes. DRAM refresh-related single bits continue to dominate memory “failure” paretos. However, given appropriate system design these highly unpredictable yet correctable events can be adequately addressed, even with leading edge memory. This discussion will focus on the performance and tradeoffs of implementing ECC and the reliability concerns of refresh-related single bits prevalent in the ramp of new process nodes.

1. Addressing Process Scaling
Challenges in Server Memory
Eric Caward, Business Development Manager
Linley Spring Processor Conference: April 11, 2018
©2018 Micron Technology, Inc. All rights reserved. Information, products, and/or specifications are subject
to change without notice. All information is provided on an “AS IS” basis without warranties of any kind.
Statements regarding products, including regarding their features, availability, functionality, or
compatibility, are provided for informational purposes only and do not modify the warranty, if any,
applicable to any product. Drawings may not be to scale. Micron, the Micron logo, and all other Micron
trademarks are the property of Micron Technology, Inc. All other trademarks are the property of their
respective owners.

2. 0
2
4
6
8
CY16 CY17 CY18 CY19 CY20 CY21 CY22
BGB
Explosive Cloud Growth
DRAM TAM
Market
DRAM Cloud
22% 32%
2016-2022 CAGR

3. Delivering Data Center Access to More Bits
Test/Debug
(low volume)
Design Qualification
(ramping volume)
Full Production
(high volume)
MatureBaseline
TraditionalSupply
0%-100%

4. Reliable Bits w/ECC
Delivering Data Center Access to More Bits
Test/Debug
(low volume)
Design Qualification
(ramping volume)
Full Production
(high volume)
MatureBaseline
TraditionalSupply
0%-100%
Additional Supply

5. “Good” bits vs. “Bad” bits

6. Scope of Good Bits vs. Bad Bits“Good” bits vs. “Bad” bits

7. Undetected Detected
SoftHard
Uncorrectable Correctable
lexicon
the vocabulary of a
particular language, field,
social class, person, etc.
noun [lek-si-kon, -kuh n]
Understanding Error Types

8. 2009 2019
50nm 1Ynm
100% 10%
DRAM Cell Scaling
Physics is the Root Cause of Refresh Related Single Bits

9. 2009 2019
50nm 1Ynm
100% 10%
DRAM Cell Scaling
Physics is the Root Cause of Refresh Related Single Bits

10. 2009 2019
50nm 1Ynm
100% 10%
0
16
32
48
64
2009 2011 2013 2015 2017 2019
Variable Retention Time
Effective Internal Refresh
(Ext tREF=64ms)
DRAM Cell Scaling
Physics is the Root Cause of Refresh Related Single Bits

11. 10’s of Millions of Modules Shipped
 < 0.05% were confirmed fails
 ~ 0.005% were estimated uncorrectable
− Includes post production damage, the most
predominant of failure mechanisms
Predominate Module Field Failures are Correctable
Pass Correctable Uncorrectable

12. Robust Error Correction in Today’s Data Center
 100% Correctable
ECC: Single Bit
Beat Example Data (72 bits)
0 00 FF00FF01 FF00FF00
1 55 AA55AA55 AA55AA55
2 CC 33CC33CC 33CC33CC
3 99 66996699 66996699
4 88 77887788 77887788
5 44 BB44BB44 BB44BB44
6 DD 22DD22DD 22DD22DD
7 11 EE11EE11 EE11EE11

13. Robust Error Correction in Today’s Data Center
 100% Correctable  x8 Components
 Standard Config
 Lockstep Config
ECC: Single Bit
 x4 Components
 Able to correct errors in
different halves of burst
ECC: Multi Bits
Beat Example Data (72 bits)
0 00 FF00FF01 FF00FF00
1 55 AA55AA55 AA55AA55
2 CC 33CC33CC 33CC33CC
3 99 66996699 66996699
4 88 77887788 77887788
5 44 BB44BB44 BB44BB44
6 DD 22DD22DD 22DD22DD
7 11 EE11EE11 EE11EE11
Beat Example Data (72 bits)
0 00 FF00FE00 FF00FF00
1 55 AA55AB55 AA55AA55
2 CC 33CC32CC 33CC33CC
3 99 66996799 66996699
4 88 77887788 778877F8
5 44 BB44BB44 BB44BBF4
6 DD 22DD22DD 22DD22FD
7 11 EE11EE11 EE11EEF1
Beat Example Data (72 bits)
0 00 FF00FF00 FF00FF00
1 55 AA05AA55 AA55AA55
2 CC 330C33CC 33CC33CC
3 99 66996699 66996699
4 88 77887788 77887788
5 44 BB5BBB44 BB44BB44
6 DD 22DD22DD 22DD22DD
7 11 EE11EE11 EE11EE11

14. Correctable Errors Do Not Foretell Uncorrectable Errors
SETUP

15. Correctable Errors Do Not Foretell Uncorrectable Errors
SETUP

16. Correctable Errors Do Not Foretell Uncorrectable Errors
SETUP

17. Correctable Errors Do Not Foretell Uncorrectable Errors
SETUP

18. Correctable Errors Do Not Foretell Uncorrectable Errors
SETUP

19. Correctable Errors Do Not Foretell Uncorrectable Errors
565,113,525
0
200,000,000
400,000,000
600,000,000
0
1,000,000
2,000,000
3,000,000
1
11
21
31
41
51
61
71
81
91
101
111
121
131
141
151
161
171
181
191
201
211
221
231
241
TotalNumberofErrors
NumberofErrorsPerDay
Days tested
Number of Correctable Errors
System Correctable fails Cumulative Fails Log. (System Correctable fails)
1 241

20. Correctable Errors Do Not Foretell Uncorrectable Errors
565,113,525
0
200,000,000
400,000,000
600,000,000
0
1,000,000
2,000,000
3,000,000
1
11
21
31
41
51
61
71
81
91
101
111
121
131
141
151
161
171
181
191
201
211
221
231
241
TotalNumberofErrors
NumberofErrorsPerDay
Days tested
Number of Correctable Errors
System Correctable fails Cumulative Fails Log. (System Correctable fails)
1 241
Note: Application tests only finds 1,170 (or 0.00021% of errors found)

21. 565,113,525
0
200,000,000
400,000,000
600,000,000
0
1,000,000
2,000,000
3,000,000
1
11
21
31
41
51
61
71
81
91
101
111
121
131
141
151
161
171
181
191
201
211
221
231
241
TotalNumberofErrors
NumberofErrorsPerDay
Days tested
Number of Correctable Errors
System Correctable fails Cumulative Fails Log. (System Correctable fails)
Correctable Errors Do Not Foretell Uncorrectable Errors
1 241
0 Uncorrectable Events Detected
Note: Application tests only finds 1,170 (or 0.00021% of errors found)

22. NoAppreciable Performance Penalty
SETUP

23. NoAppreciable Performance Penalty
SETUP

24. NoAppreciable Performance Penalty
SETUP

25. Context Switch Performance Cost
NoAppreciable Performance Penalty
Results
0.00000%
0.00025%
0.00050%
0.00075%
0.00100%
2 4 6 8 10 12 14
PerformanceCost@1/s
Number of Processes
lmbench lat_ctx
(size=64k ovr=2.68)
Context Switch
0.00028% @ 1/s

26. Context Switch Performance Cost Single Bit Performance Cost
NoAppreciable Performance Penalty
Results
0
100,000
200,000
300,000
400,000
0%
10%
20%
30%
40%
50%
60%
Errors
Bandwidth%Normalized
Monitor Data
Total data bus B/W % iMC:N0_H1_C1[4]-Errs
0.00000%
0.00025%
0.00050%
0.00075%
0.00100%
2 4 6 8 10 12 14
PerformanceCost@1/s
Number of Processes
lmbench lat_ctx
(size=64k ovr=2.68)
Context Switch
0.00028% @ 1/s

27. Context Switch Performance Cost Single Bit Performance Cost
Context Switch
0.00028% @ 1/s
Error Correction
0.0000319% @ 1/s
NoAppreciable Performance Penalty
Results
0
100,000
200,000
300,000
400,000
0%
10%
20%
30%
40%
50%
60%
Errors
Bandwidth%Normalized
Monitor Data
Total data bus B/W % iMC:N0_H1_C1[4]-Errs
0.00000%
0.00025%
0.00050%
0.00075%
0.00100%
2 4 6 8 10 12 14
PerformanceCost@1/s
Number of Processes
lmbench lat_ctx
(size=64k ovr=2.68)
>>

28. Achieve Form, Fit, Function,
Performance, and Reliability
Leverage System Level ECC to Enable More Bits Today
www.micron.com/scale