Jorge Tonfat defended his Ph.D. thesis on a frame-level redundancy scrubbing technique for mitigating soft errors in SRAM-based FPGAs. The thesis outlined the increasing soft error rate issues in newer FPGA technologies, described existing mitigation techniques like triple modular redundancy and memory scrubbing, and proposed a new frame-level redundancy scrubbing approach. Experimental validation through radiation testing and fault injection showed the effectiveness of the frame-level redundancy scrubbing technique.

1. Frame-Level Redundancy Scrubbing
Technique for SRAM-based FPGAs
Ph.D. Thesis Defense
By Jorge Tonfat
Advisor: Ricardo Reis
Co-advisor: Fernanda L. Kastensmidt

2. Outline
• Motivation
• SRAM-based FPGA overview
• Radiation Effects in SRAM-based FPGAs
• SEU Mitigation Techniques in SRAM-based FPGAs
• Frame-level Redundancy Scrubbing Technique
– Performance Analysis
• Validation of FLR-Scrubbing
– Radiation Test Results
– Fault Injection Results
• Conclusions
Jorge Tonfat - Ph.D Thesis Defense 2

3. Outline
• Motivation
• SRAM-based FPGA overview
• Radiation Effects in SRAM-based FPGAs
• SEU Mitigation Techniques in SRAM-based FPGAs
• Frame-level Redundancy Scrubbing Technique
– Performance Analysis
• Validation of FLR-Scrubbing
– Radiation Test Results
– Fault Injection Results
• Conclusions
Jorge Tonfat - Ph.D Thesis Defense 3

4. SRAM-based FPGAs for critical applications
Transportation
Systems
Medical Devices
Aerospace Systems
Jorge Tonfat - Ph.D Thesis Defense 4
Motivation

5. SRAM-based FPGAs for critical applications
Transportation
Systems
Medical Devices
Aerospace Systems
Main
characteristic:
A system error is
unacceptable
Main
characteristic:
A system error is
unacceptable
Jorge Tonfat - Ph.D Thesis Defense 5
Motivation

6. FPGAs are attractive to critical systems:
• Reconfigurable on the field: the original design
can be updated/improved during lifetime
• High-density logic integration to implement
complex circuits
• Low design costs and low development time
compared to ASICs
SRAM-based FPGAs for critical applications
Jorge Tonfat - Ph.D Thesis Defense 6
Motivation

7. But SRAM-based FPGAs are susceptible to soft errors or
SEUs (Single Event Upsets).
SRAM-based FPGAs for critical applications
7
Motivation
Source: http://www.cotsjournalonline.com/articles/view/102279

8. Radiation Environment
Protons,
Electrons
Neutrons
Solar flares
Galactic Cosmic
Rays (GCR)
Van Allen belts
Protons, heavy ions
Trapped Particles
Jorge Tonfat - Ph.D Thesis Defense 8
Motivation

9. Outline
• Motivation
• SRAM-based FPGA overview
• Radiation Effects in SRAM-based FPGAs
• SEU Mitigation Techniques in SRAM-based FPGAs
• Frame-level Redundancy Scrubbing Technique
– Performance Analysis
• Validation of FLR-Scrubbing
– Radiation Test Results
– Fault Injection Results
• Conclusions
Jorge Tonfat - Ph.D Thesis Defense 9

10. SRAM-based FPGA
Jorge Tonfat - Ph.D Thesis Defense 10
SRAM-based FPGA Overview

11. SRAM-based FPGA
Configuration Layer
Main
source of
reliability
threats
ICAP
Application Layer
DSP
DSP BRAM
BRAM
LOGIC
LOGIC
Jorge Tonfat - Ph.D Thesis Defense 11
SRAM-based FPGA Overview

12. FPGA Config. Mem. (Virtex 5 LX50T)
1 Frame has the
height of 1 row and
the width of 1 bit.
1 Frame has the
height of 1 row and
the width of 1 bit.
1 Frame – minimum fragment for read or write
1 Frame = 1312 bits = 41 words of 32
bits
Row 0
Row 1
Row 2
Row 0
Row 1
Row 2
Jorge Tonfat - Ph.D Thesis Defense 12
SRAM-based FPGA Overview

13. FPGA Config. Mem. (Virtex 5 LX50T)
1 Frame – minimum fragment for read or write
1 Frame = 1312 bits = 41 words of 32
bits
TOP
BOTTOM
Jorge Tonfat - Ph.D Thesis Defense 13
SRAM-based FPGA Overview

14. FPGA Config. Mem. (Virtex 5 LX50T)
1 Frame – minimum fragment for read or write
1 Frame = 1312 bits = 41 words of 32
bits
Row 0
Row 1
Row 2
Row 0
Row 1
Row 2
Jorge Tonfat - Ph.D Thesis Defense 14
SRAM-based FPGA Overview

15. FPGA Config. Mem. (Virtex 5 LX50T)
1 Frame – minimum fragment for read or write
1 Frame = 1312 bits = 41 words of 32
bits
CLB
COLUMN
BRAM
COLUMN
Each column is
composed of a group of
frames.
Each column is
composed of a group of
frames.
Jorge Tonfat - Ph.D Thesis Defense 15
SRAM-based FPGA Overview

16. FPGA Config. Mem. (Virtex 5 LX50T)
Number of frames depends
on the type of column (CLB,
BRAM, DSP, etc):
Number of frames depends
on the type of column (CLB,
BRAM, DSP, etc):
Type Width (frames)
IOB 54
CLB 36
BRAM 30
DSP 28
CLK 4
GTP 32
1 Frame – minimum fragment for read or write
1 Frame = 1312 bits = 41 words of 32
bits
CLB
COLUMN
BRAM
COLUMN Frame36
…Frame1
Frame2
Jorge Tonfat - Ph.D Thesis Defense 16
SRAM-based FPGA Overview

17. Outline
• Motivation
• SRAM-based FPGA overview
• Radiation Effects in SRAM-based FPGAs
• SEU Mitigation Techniques in SRAM-based FPGAs
• Frame-level Redundancy Scrubbing Technique
– Performance Analysis
• Validation of FLR-Scrubbing
– Radiation Test Results
– Fault Injection Results
• Conclusions
Jorge Tonfat - Ph.D Thesis Defense 17

18. SRAM-based FPGAs under Radiation
Configuration memory bits
ICAP
Application Layer
DSP
DSP BRAM
BRAM
LOGIC
LOGIC
Bit-flips (SEUs) have persistent effect
Correct upsets by reconfiguration
Configuration Layer
composed of millions of SRAM cells
Jorge Tonfat - Ph.D Thesis Defense 18
Radiation Effects in SRAM-based FPGAs

19. SRAM-based FPGAs under Radiation
Configuration memory bits
ICAP
DSP
DSP BRAM
BRAM
LOGIC
LOGIC
Bit-flips (SEUs) have persistent effect
Correct upsets by reconfiguration
SEU and SET have transient effect
Mask errors by
Triple Modular Redundancy (TMR)
Configuration Layer
composed of millions of SRAM cells
Application Layer
Jorge Tonfat - Ph.D Thesis Defense 19
Radiation Effects in SRAM-based FPGAs

20. Trends On Soft Error Rate (SER) in SRAM-
based FPGAs
More density of
Bitstream
Increase SER!
Jorge Tonfat - Ph.D Thesis Defense 20
Radiation Effects in SRAM-based FPGAs
SER = Soft Error Rate

21. Trends On Soft Error Rate (SER) in SRAM-based FPGAs
At nominal VDD
and temperature
At nominal VDD
and temperature
Jorge Tonfat - Ph.D Thesis Defense 21
Radiation Effects in SRAM-based FPGAs
Data from Xilinx Reliability Report second half 2014
Real Time Soft Error Rate

22. Trends On Soft Error Rate (SER) in SRAM-
based FPGAs
22
Radiation Effects in SRAM-based FPGAs

23. F. L. Kastensmidt, J. Tonfat, T. Both, P. Rech, G. Wirth, R. Reis, F. Bruguier, P. Benoit, L. Torres, and C. Frost, “Voltage scaling
and aging effects on soft error rate in SRAM-based FPGAs,” Microelectronics Reliability, 2014.
Trends On Soft Error Rate (SER) in SRAM-based FPGAs
Low VDD + Low Power Techniques, Aging effects
23
Radiation Effects in SRAM-based FPGAs

24. IBE, E. et al. “Impact of Scaling on Neutron-Induced Soft Error in SRAMs From a 250 nm to a 22 nm Design Rule”. IEEE
Transactions on Electron Devices, 2010.
Trends On soft error rate (SER) in SRAM-
based FPGAs
Frame A
Frame B
Inter-Frame
Multiple bit-flips
(MCU)
Intra-Frame
Multiple bit-flips
(MBU)
MBU events are
increasing
24
Virtex -5 65 nm
Radiation Effects in SRAM-based FPGAs

25. M. Wirthlin, D. Lee, G. Swift, and H. Quinn, “A Method and Case Study on Identifying Physically Adjacent Multiple-Cell
Upsets Using 28-nm, Interleaved and SECDED-Protected Arrays,” IEEE Transactions on Nuclear Science, 2014.
Trends On soft error rate (SER) in SRAM-based
FPGAs
25
Kintex-7 FPGA (28 nm)
Radiation Effects in SRAM-based FPGAs

26. M. Wirthlin, H. Takai, and A. Harding, “Soft error rate estimations of the Kintex-7 FPGA within the ATLAS Liquid Argon (LAr)
Calorimeter,” Journal of Instrumentation, 2014. 26
Trends On soft error rate (SER) in SRAM-based
FPGAs
Radiation levels at CERN are higher than at space
Kintex 7 325T (Series-7 FPGA) :
Configuration bits = 67.9 Mbits
BRAM bits = 16.4 Mbits
SEU rate for configuration memory (Kintex-7 325T): 1 SEU each 150 secs!
Fast
accumulation
of SEUs
Radiation Effects in SRAM-based FPGAs

27. Problem Definition
Increasing
SEUs
More density
Increasing
MBUs
Closer and smaller
transistors
Increasing
Density
(Smaller transistors =>
More transistors per
area, low power,
faster devices)
scrubbing
More transistors to
correct at same speed
Higher scrubbing rate
Accumulation of
faults and MBUs
Higher probability of
system error
Reduction of mean time
to failure (MTTF)
Technology
trend
Threats Problem
Voltage
Scaling
and Aging
Jorge Tonfat - Ph.D Thesis Defense 27
Radiation Effects in SRAM-based FPGAs

28. Problem Definition
Increasing
SEUs
More density
Increasing
MBUs
Closer and smaller
transistors
Increasing
Density
(Smaller transistors =>
More transistors per
area, low power,
faster devices)
scrubbing
More transistors to
correct at same speed
Higher scrubbing rate
Accumulation of
faults and MBUs
Higher probability of
system error
Reduction of mean time
to failure (MTTF)
Technology
trend
Threats Problem
Voltage
Scaling
and Aging
Jorge Tonfat - Ph.D Thesis Defense 28
Radiation Effects in SRAM-based FPGAs

29. Outline
• Motivation
• SRAM-based FPGA overview
• Radiation Effects in SRAM-based FPGAs
• SEU Mitigation Techniques in SRAM-based FPGAs
• Frame-level Redundancy Scrubbing Technique
– Performance Analysis
• Validation of FLR-Scrubbing
– Radiation Test Results
– Fault Injection Results
• Conclusions
Jorge Tonfat - Ph.D Thesis Defense 29

30. • Two complementary techniques are used:
Error Masking
Fault Correction
1
2
3
Voter
Full Reconfiguration
Non-volatile
Memory
Micro
processor
FPGA
SelectMAP
Triple Modular Redundancy
(TMR)
Jorge Tonfat - Ph.D Thesis Defense 30
SEU Mitigation Techniques in SRAM-based FPGAs

31. Memory Scrubbing
Jorge Tonfat - Ph.D Thesis Defense 31
SEU Mitigation Techniques in SRAM-based FPGAs

32. Memory Scrubbing: architectures
Non-volatile
Memory
Scrubber
FPGA
SelectMAP
Non-volatile
Memory
FPGA
ICAP
Scrubber
External Internal
BERG, M; et al. Effectiveness of Internal Versus External SEU Scrubbing Mitigation
Strategies in a Xilinx FPGA: Design, Test, and Analysis. IEEE Transactions on Nuclear
Science, 2008
Jorge Tonfat - Ph.D Thesis Defense 32
SEU Mitigation Techniques in SRAM-based FPGAs

33. Memory Scrubbing: architectures
Non-volatile
Memory
Scrubber
ASIC
FPGA
SelectMAP
Hardware Software
Non-volatile
Memory
Microprocessor
FPGA
SelectMAP
HERRERA-ALZU, I.; LOPEZ-VALLEJO, M. Design Techniques for Xilinx Virtex FPGA
Configuration Memory Scrubbers. IEEE Transactions on Nuclear Science, v. 60, n.
1, p.376–385, feb. 2013. Jorge Tonfat - Ph.D Thesis Defense 33
SEU Mitigation Techniques in SRAM-based FPGAs

34. Memory Scrubbing: methodologies
Time
Scrubbing interval
Scrubbing time
Readback interval
Readback time
Time
Scrubbing time
Soft error
detected
PREVENTIVE
(BLIND)
SCRUBBING
READBACK &
SCRUBBING
Jorge Tonfat - Ph.D Thesis Defense 34
SEU Mitigation Techniques in SRAM-based FPGAs

35. Memory Scrubbing: methodologies
Start
End
Frame
row
Module A Module B
35
SEU Mitigation Techniques in SRAM-based FPGAs

36. Jorge Tonfat - Ph.D Thesis Defense
But without
redundant
information of the
bitstream, how are
we going to correct
the upsets?
But without
redundant
information of the
bitstream, how are
we going to correct
the upsets?
Memory Scrubbing: eliminating the golden
memory
• Advantages:
Lower Power Consumption
Lower time to repair
Non-volatile
Memory
FPGA
ICAP
Scrubber
Slow Interface!
Fast Interface!
100 times faster!
YANG, E. et al. HHC: Hierarchical hardware checkpointing to accelerate fault
recovery for SRAM-based FPGAs. IOLTS, 2013 36
SEU Mitigation Techniques in SRAM-based FPGAs

37. Xilinx Approach
• Use Hamming Codes per frame and CRC codes
for all frames.
• Limitation: can only correct 1 bit-flip and
detect two bit-flips.
Jorge Tonfat - Ph.D Thesis Defense 37
SEU Mitigation Techniques in SRAM-based FPGAs

38. Ad hoc Error Detection and Correction Codes
38
RAO, P. M. B. et al.
DAC, 2014.
SEU Mitigation Techniques in SRAM-based FPGAs

39. Trade-off Area vs. Repair time
Jorge Tonfat - Ph.D Thesis Defense 39
RAO, P. M. B. et al.
DAC, 2014.
SEU Mitigation Techniques in SRAM-based FPGAs

40. Outline
• Motivation
• SRAM-based FPGA overview
• Radiation Effects in SRAM-based FPGAs
• SEU Mitigation Techniques in SRAM-based FPGAs
• Frame-level Redundancy Scrubbing Technique
– Performance Analysis
• Validation of FLR-Scrubbing
– Radiation Test Results
– Fault Injection Results
• Conclusions
Jorge Tonfat - Ph.D Thesis Defense 40

41. Memory Scrubbing: eliminating the
golden memory
• Advantages:
– Lower Power
Consumption
– Lower time to repair
Flash
Memory
FPGA
ICAP
Scrubber
Slow Interface!
Fast Interface!
100 times faster!
YANG, E. et al. HHC: Hierarchical hardware checkpointing to accelerate fault
recovery for SRAM-based FPGAs. IOLTS, 2013
But without redundant
information of the
bitstream, how are we
going to correct the
upsets?
But without redundant
information of the
bitstream, how are we
going to correct the
upsets?
Jorge Tonfat - Ph.D Thesis Defense 41
Frame-level Redundancy Scrubbing Technique

42. Frame-level Redundancy
Jorge Tonfat - Ph.D Thesis Defense 42
ICAP
Fast Interface!
Scrubber
TMR
1
TMR
2
TMR
3
1000010101010101010
0101010101010101010
1110100101010010000
0000000010101010101
1000010101010101010
0101010101010101010
1110100101010010000
0000000010101010101
1000010101010101010
0101010101010101010
1110100101010010000
0000000010101010101
Frame-level Redundancy Scrubbing Technique

43. Jorge Tonfat - Ph.D Thesis Defense
43
Original
design
Synth & PAR with
Xilinx Tools
Create a Hard Macro
Block from design
RapidSmith Tool from BYU
Xilinx
Standard
Flow
*.vhd
*.ncd
*.nmc
Hard Macro blockHard Macro Block
(HMB)
Synth & PAR of TMR + voter + scrubber
Placement
constraints
TMR design with modified bitstream
(frame redundancy)
TMR design with modified bitstream
(frame redundancy)
Voter Scrubber
*.vhd *.vhd*.nmc
*.ucf
LAVIN, C. et al. RapidSmith: Do-It-Yourself CAD Tools for Xilinx
FPGAs. In: FPL, 2011.
Proposed Design Flow
Frame-level Redundancy Scrubbing Technique

44. Frame-level Redundancy Scrubbing Technique
Scrubbing Mechanism
TMR
Domain 1
TMR
Domain 2
TMR
Domain 3
FrameXFrameXFrameX
44Jorge Tonfat - Ph.D Thesis Defense

45. Scrubbing Mechanism
TMR
Domain 1
TMR
Domain 2
TMR
Domain 3
FrameXFrameXFrameX
0
1
0
0
1
1
1
1
...
0
0
1
0
1
0
1
1
...
0
1
0
0
1
0
1
0
...
Scrubber controller
(frame buffers )
Procedure:
Read frame X from module 1
Read frame X from module 2
Read frame X from module 3
45Jorge Tonfat - Ph.D Thesis Defense
Frame-level Redundancy Scrubbing Technique

46. Scrubbing Mechanism
TMR
Domain 1
TMR
Domain 2
TMR
Domain 3
FrameXFrameXFrameX
0
1
0
0
1
1
1
1
...
0
0
1
0
1
0
1
1
...
0
1
0
0
1
0
1
0
...
1
0
0
1
Scrubber controller
(frame buffers )
Procedure:
Read frame X from module 1
Read frame X from module 2
Read frame X from module 3
Vote
46Jorge Tonfat - Ph.D Thesis Defense
Frame-level Redundancy Scrubbing Technique

47. Scrubbing Mechanism
TMR
Domain 1
TMR
Domain 2
TMR
Domain 3
0
1
0
0
1
0
1
1
...
0
1
0
0
1
0
1
1
...
0
1
0
0
1
0
1
1
...
Scrubber controller
(frame buffers )
Procedure:
Read frame X from module 1
Read frame X from module 2
Read frame X from module 3
Vote
If fault is detected in Mod. 1 then
write frame X in Module 1
If fault is detected in Mod. 2 then
write frame X in module 2
If fault is detected in Mod. 3 then
write frame X in module 3
47Jorge Tonfat - Ph.D Thesis Defense
Frame-level Redundancy Scrubbing Technique

48. Outline
• Motivation
• SRAM-based FPGA overview
• Radiation Effects in SRAM-based FPGAs
• SEU Mitigation Techniques in SRAM-based FPGAs
• Proposed Fault Injection Platform
• Frame-level Redundancy Scrubbing Technique
– Performance Analysis
• Validation of FLR-Scrubbing
– Fault Injection Results
– Radiation Test Results
• Conclusions
Jorge Tonfat - Ph.D Thesis Defense 48

49. • Evaluated parameters:
– Area overhead
– Power consumption
– Time to repair
• Device: Virtex-5 XC5VLX50T-FFG1136
– 65 nm
– VDD = 1.0 V
Jorge Tonfat - Ph.D Thesis Defense 49
Frame-level Redundancy Scrubbing Technique: Performance Analysis

50. TONFAT, J.; KASTENSMIDT, F.; REIS, R. Energy Efficient Frame-level Redundancy Scrubbing Technique for SRAM-based FPGAs.
In: 2015 NASA/ESA Conference on Adaptive Hardware and Systems, 2015.
Area Overhead
Scheme CLBs BRAMs External
Memory
Constant
Overhead
Work in (RAO et al.,
2014)a
1100
(6 %)
4
(1 %)
No No
Xilinx SEU Controller
(CHAPMAN, 2010)
98
(3 %)
1
(2 %)
No Yes
Xilinx SEM IP (XILINX,
2012a)b
108
(2 %)
3
(1 %)
Yes No
Blind Scrubbing (TMR) 341
(10 %)
12
(20 %)
Yes Yes
FLR-Scrubbing (TMR) 113
(3 %)
6
(10 %)
No Yes
aimplemented for a Virtex-6 FPGA with 50 clusters and TMR redundancy.
bimplemented for an Artix-7 FPGA with no optional features.
50
Frame-level Redundancy Scrubbing Technique: Performance Analysis

51. TONFAT, J.; KASTENSMIDT, F.; REIS, R. Energy Efficient Frame-level Redundancy Scrubbing Technique for SRAM-based FPGAs.
In: 2015 NASA/ESA Conference on Adaptive Hardware and Systems, 2015.
Energy consumption comparison
FLR-scrubbing
(TMR)
Blind scrubbing (TMR)
Parameter FPGA Core Source
(1.0 V)
FPGA Core
Source (1.0 V)
Flash Memory
Source (1.8 V)
Total
Sources
Dynamic Power
(RMS)
33.24 mW 21.2 mW 3.93 mW
25.13
mW
Scrub cycle
time
3.67 ms @ 50MHz 178.55 ms @ 50MHz
Energy per
scrub cycle
121.9 μ J 3.785 mJ 0.7 mJ 4.485 mJ
Number of
frames
1386 8376 (all device)
Energy to scrub
a frame
87.9 nJ
451.9 nJ
(84.3%)
83.89 nJ (15.7%) 535.79 nJ
51
Frame-level Redundancy Scrubbing Technique: Performance Analysis

52. TONFAT, J.; KASTENSMIDT, F.; REIS, R. Energy Efficient Frame-level Redundancy Scrubbing Technique for SRAM-based FPGAs.
In: 2015 NASA/ESA Conference on Adaptive Hardware and Systems, 2015.
Time to Repair one Frame
Scheme
Characteristics Time to repair
one frame (μs)
Work by (RAO et al., 2014)a BRAM use / various
parity frame bits
351
Xilinx SEU Controller (CHAPMAN,
2010)
Scrubber controller is
a PicoBlaze
240
Xilinx SEM IP (XILINX, 2012a)b Use external memory 12
Blind Scrubbing (TMR) Use external memory 21
FLR-Scrubbing (TMR) No BRAM, No
external memory,
FSM-based
5
aimplemented for a Virtex-6 FPGA with 50 clusters and TMR redundancy.
bimplemented for an Artix-7 FPGA with no optional features.
52
Frame-level Redundancy Scrubbing Technique: Performance Analysis

53. Outline
• Motivation
• SRAM-based FPGA overview
• Radiation Effects in SRAM-based FPGAs
• SEU Mitigation Techniques in SRAM-based FPGAs
• Proposed Fault Injection Platform
• Frame-level Redundancy Scrubbing Technique
– Performance Analysis
• Validation of FLR-Scrubbing
– Radiation Test Results
– Fault Injection Results
• Conclusions
Jorge Tonfat - Ph.D Thesis Defense 53

54. Outline
• Motivation
• SRAM-based FPGA overview
• Radiation Effects in SRAM-based FPGAs
• SEU Mitigation Techniques in SRAM-based FPGAs
• Proposed Fault Injection Platform
• Frame-level Redundancy Scrubbing Technique
– Performance Analysis
• Validation of FLR-Scrubbing
– Radiation Test Results
– Fault Injection Results
• Conclusions
Jorge Tonfat - Ph.D Thesis Defense 54

55. TMR Domain 1
TMR Domain 2
TMR Domain 3
TMR
Scrubber
ICAP
Radiation Test
Floorplann
Objective: Test the
efficiency of the
scrubbing technique to
correct multiple
accumulated SEUs and
MBUs.
Jorge Tonfat - Ph.D Thesis Defense
55
Validation of FLR-Scrubbing: Radiation Test Results

56. Test Details
Jorge Tonfat - Ph.D Thesis Defense 56
Validation of FLR-Scrubbing: Radiation Test Results
Los Alamos National Laboratory (LANL), Los Alamos
Neutron Science Center (LANSCE) in December 2014.
Average neutron flux = 1.43×106(n/cm2× s).
Protected area: 1,386 configuration frames
720 CLBs (20 %) and 24 18K BRAM blocks (20 %)
Two scrubbing rates : 30 min and 60 min

57. Graphical view of readbacks of a
correct run
Jorge Tonfat - Ph.D Thesis Defense 57
Validation of FLR-Scrubbing: Radiation Test Results

58. Graphical view of readbacks of an
incorrect run
Jorge Tonfat - Ph.D Thesis Defense 58
Validation of FLR-Scrubbing: Radiation Test Results

59. TONFAT, J.; RECH, P.; KASTENSMIDT, F.; REIS, R.; QUINN, H. Analyzing the Effectiveness of a Novel Frame-level Redundancy
Scrubbing Technique for SRAM-based FPGAs. In: 2015 Nuclear and Space Radiation Effects Conference NSREC, 2015.
Analyzing Results
avg. acc.
upsets
Total
runs
Case 1 Case 2 Case 3 Case 4
Scrub rate
1 (30 min)
145.7 33 87.8% 6.1% 6.1% 0%
Scrub rate
2 (60 min)
274 30 66.7% 26.7% 3.3% 3.3%
59
Validation of FLR-Scrubbing: Radiation Test Results
Result
Classification
Scrubber Report Readback Files
Case 1 Correct Confirm
Case 2 Wrong Confirm
Case 3 Correct Not Confirm
Case 4 Wrong Not Confirm

60. TONFAT, J.; RECH, P.; KASTENSMIDT, F.; REIS, R.; QUINN, H. Analyzing the Effectiveness of a Novel Frame-level Redundancy
Scrubbing Technique for SRAM-based FPGAs. In: 2015 Nuclear and Space Radiation Effects Conference NSREC, 2015.
Reliability of the Scrubber
Cross-section (cm2) SER (FIT)
Scrub rate 1 (30 min) 4.41×10−11 5.73
Scrub rate 2 (60 min) 6.59×10−11 8.57
Xilinx SEU Controller
(CHAPMAN, 2010)
N/A 8.6
60
Validation of FLR-Scrubbing: Radiation Test Results

61. Outline
• Motivation
• SRAM-based FPGA overview
• Radiation Effects in SRAM-based FPGAs
• SEU Mitigation Techniques in SRAM-based FPGAs
• Proposed Fault Injection Platform
• Frame-level Redundancy Scrubbing Technique
– Performance Analysis
• Validation of FLR-Scrubbing
– Radiation Test Results
– Fault Injection Results
• Conclusions
Jorge Tonfat - Ph.D Thesis Defense 61

62. Fault Injection
Floorplann
TMR Domain 1
TMR Domain 2
TMR Domain 3
Scrubber
Fault Injector
ICAP
Objective: Find the
maximum
accumulated SEUs
that the scrubbing
technique can correct
Jorge Tonfat - Ph.D Thesis Defense
62
Validation of FLR-Scrubbing: Fault Injection Results

63. Proposed Fault Injection Platform
Susceptible area
Fault Injector
ICAP ICAP
Controller
FPGA
RS-232
SEU locations
database bank
PicoBlaze
Memory
Controller
JTAG
Jorge Tonfat - Ph.D Thesis Defense 63
Validation of FLR-Scrubbing: Fault Injection Results

64. Fault Injection Methodology
Jorge Tonfat - Ph.D Thesis Defense 64
Time (s)
Accumulated
upsets
10
20
30
40
50
60
70
80
90
100
110
120
Scrubbing executed successfully
Scrubbing executed with errors
Validation of FLR-Scrubbing: Fault Injection Results

65. TONFAT, J.; RECH, P.; KASTENSMIDT, F.; REIS, R.; QUINN, H. Analyzing the Effectiveness of a Novel
Frame-level Redundancy Scrubbing Technique for SRAM-based FPGAs. In: 2015 Nuclear and
Space Radiation Effects Conference NSREC, 2015.
Fault Injection Results
65
Max. Acc. SEUs
corrected
Mean 1136.6
σ 562.5
Min 83
Max 2656
Validation of FLR-Scrubbing: Fault Injection Results

66. Example of a Fault Injection Campaign
224 injected faults in the
protected area
3 remain faults after the scrub cycle
Jorge Tonfat - Ph.D Thesis Defense 66
Validation of FLR-Scrubbing: Fault Injection Results

67. Outline
• Motivation
• SRAM-based FPGA overview
• Radiation Effects in SRAM-based FPGAs
• SEU Mitigation Techniques in SRAM-based FPGAs
• Proposed Fault Injection Platform
• Frame-level Redundancy Scrubbing Technique
– Performance Analysis
• Validation of FLR-Scrubbing
– Radiation Test Results
– Fault Injection Results
• Conclusions
Jorge Tonfat - Ph.D Thesis Defense 67

68. In the first part of this work, we observe the
influence of aging and voltage scaling to the soft
error rate in SRAM-based FPGAs.
Results have shown that the error rate can
increase more than twice when considering
aging and voltage scaling, so it is important to
add this type of measurement and discussions
when considering FPGAs for high reliable
applications.
Jorge Tonfat - Ph.D Thesis Defense 68
Conclusions

69. We have presented a novel scrubbing mechanism
that can correct accumulated SEUs and MBUs in the
configuration memory of SRAM-based FPGAs.
The correction mechanism does not need an
external memory, reducing the system energy
consumption and the time to repair the fault.
This technique offers good characteristics in terms
of area and energy overhead with low repair
latency compared with other solutions.
A comparison with a blind scrubber shows an
energy reduction of six times.
Jorge Tonfat - Ph.D Thesis Defense 69
Conclusions

70. Also, a fault injection system was developed
with the information of previous radiation tests.
Results show an approximation of the maximum
number of corrected SEUs.
Radiation test results demonstrated that the
proposed technique is suitable for correcting
accumulated SEUs and MBUs.
There is no evidence in the results that the
technique cannot detect nor correct
accumulated faults.
Jorge Tonfat - Ph.D Thesis Defense 70
Conclusions

71. • TONFAT, J.; RECH, P.; KASTENSMIDT, F.; REIS, R.; QUINN, H.
Analyzing the Effectiveness of a Novel Frame-level Redundancy
Scrubbing Technique for SRAM-based FPGAs. In: 2015 Nuclear
and Space Radiation Effects Conference NSREC, 2015. (should
be published in IEEE Transactions on Nuclear Science) .
• TONFAT, J.; KASTENSMIDT, F.; REIS, R. Energy Efficient Frame-
level Redundancy Scrubbing Technique for SRAM-based FPGAs.
In: 2015 NASA/ESA Conference on Adaptive Hardware and
Systems, 2015.
• TONFAT, J.; RECH, P.; KASTENSMIDT, F.; REIS, R.; QUINN, H. A
Novel Frame-level Redundancy Scrubbing Technique for SRAM-
based FPGAs. In: 2015 Military and Aerospace Programmable
Logic Devices (MAPLD) Workshop, 2015.
• TONFAT, J.; KASTENSMIDT, F.; REIS, R. Frame-level Redundancy
Correction Technique for SRAM-based FPGAs. In: 2015 LASCAS
Forum for Young Professionals/MSc/PhD Students, 2015.
Jorge Tonfat - Ph.D Thesis Defense
71
Publications

72. Jorge Tonfat - Ph.D Thesis Defense 72
• TARRILLO, J.; TONFAT, J.; TAMBARA, L.; KASTENSMIDT, F.; REIS, R. Multiple Fault
Injection Platform for SRAM-Based FPGA Based on Ground-Level Radiation
Experiments. In: 2015 16th Latin American Test Symposium - LATS. IEEE, 2015.
• TAMBARA, L.; TONFAT, J.; REIS, R.; KASTENSMIDT, F.; PEREIRA, E.; VAZ, R.;
GONÇALEZ, O. Soft error rate in SRAM-based FPGAs under neutron-induced and
TID effects. In: 2014 15th Latin American Test Workshop - LATW. IEEE, 2014. v. 6,
p. 1-6.
• KASTENSMIDT, F.; TONFAT, J.; BOTH, T.; RECH, P.; WIRTH, G.; REIS, R.; BRUGUIER, F.;
BENOIT, P.; TORRES, L.; FROST, C. Aging and Voltage Scaling Impacts under
Neutroninduced Soft Error Rate in SRAM-based FPGAs. In: Test Symposium (ETS),
2014 19th IEEE European. [S.l.: s.n.], 2014. p. 1-2.
• TONFAT, J.; AZAMBUJA, J.; NAZAR, G.; RECH, P.; FROST, C.; KASTENSMIDT, F.;
CARRO, L.; REIS, R.; BENFICA, J.; VARGAS, F.; BEZERRA, E. Measuring the Impact of
Voltage Scaling for Soft Errors in SRAM-based FPGAs From a Designer Perspective.
In: Mixed-Signals, Sensors and Systems TestWorkshop (IMS3TW), 2014 19th
International. [S.l.: s.n.], 2014. p. 1-6.
• TONFAT, J.; AZAMBUJA, J.; NAZAR, G.; RECH, P.; FROST, C.; KASTENSMIDT, F.;
CARRO, L.; REIS, R.; BENFICA, J.; VARGAS, F.; BEZERRA, E. Analyzing the influence of
voltage scaling for soft errors in SRAM-based FPGAs. In: Radiation and Its Effects
on Components and Systems (RADECS), 2013 14th European Conference on. [S.l.:
s.n.], 2013. p. 1-5.
Publications

73. • 2nd Prize in the 2015 LASCAS Forum for Young
Professionals/MSc/PhD Students in
Montevideo, Uruguay.
• Best Paper Award at Simpósio Sul de
Microeletrônica SIM 2015 in Santa Maria,
Brazil.
Jorge Tonfat - Ph.D Thesis Defense 73
Awards

74. Frame-Level Redundancy Scrubbing
Technique for SRAM-based FPGAs
Ph.D. Thesis Defense
By Jorge Tonfat
Advisor: Ricardo Reis
Co-advisor: Fernanda L. Kastensmidt
¡Gracias!
Obrigado!

75. Backup Slides

76. FPGA
Memory Scrubbing: methodologies
1
2
3
Voter
Scrubber
Error
Trigger
FUNCTIONAL
ERROR-DRIVEN
SCRUBBING
Jorge Tonfat - Ph.D Thesis Defense 76
SEU Mitigation Techniques in SRAM-based FPGAs

77. Memory Scrubbing: methodologies
Source: Santos , et al. Criticality-aware scrubbing mechanism for SRAM-based FPGAs, FPL 2014
TASK-DRIVEN
SCRUBBING
Jorge Tonfat - Ph.D Thesis Defense 77/64
SEU Mitigation Techniques in SRAM-based FPGAs

78. Hard Macro Blocks
Jorge Tonfat - Ph.D Thesis Defense 78/64

79. Start
Wait for frame address and bit(s)
positions
Read the selected frame
Flip the selected bit(s)
Write back the modified frame
Read back the modified frame to verify
the injection
Fault injection
successful?
Report a fault injection
error
yes
no
ICAP controller
Jorge Tonfat - Ph.D Thesis Defense 79/64

80. PicoBlazeStart
Wait for the start memory position of
SEU database
Wait for the fault injection rate
Wait for the fault-free area
definition
Start fault injection campaign
Read SEU position data from external memory:
frame address and bit(s) positions
SEU data outside
fault-free area ?
yes
no
Inject fault
Time delay defined by
fault injection rate
Jorge Tonfat - Ph.D Thesis Defense 80/64

81. PC ScriptStart
Configure the FPGA with the DUT + fault injector
Set the start memory position of SEU database
Start fault injection campaign
Set fault injection rate
Set fault-free area
DUT end condition
reached?
Max. Campaign Time
reached?
yes
no
no
Create campaign report
yes
Set max. time per campaign
Jorge Tonfat - Ph.D Thesis Defense 81/64

82. Start
Configure the FPGA with golden bitstream
Radiation experiment starts
Read FPGA bitstream
Differences between
current and last
readback?
Save bitflip(s) position(s) and
time
noyes
Procedure to static test
Jorge Tonfat - Ph.D Thesis Defense 82/64

83. Test in Los Alamos –Dec 2014
Jorge Tonfat - Ph.D Thesis Defense 83/64

84. 11.9
24.7
48.0
70.1
98.3
11.7
22.5
33.3
86.5
100.5
0
20
40
60
80
100
120
1 2 3 4 5
Averangenumberof
accumulatedupsets
Number of faulty modules
Fault Injection
ISIS
1.7%
9.8%
44.4%
19.7%
2,19%
Results
Case-study: 7MR of adder chains. Implemented in
Virtex-5 XC5VLX50T
Neutron experiment
84
Radiation test: more than 100 hours
Fault injection: Few minutes

85. Future works
Jorge Tonfat - Ph.D Thesis Defense 85/67
• Combine the FLR-scrubbing technique with
Frame ECC from Xilinx to improve the
correction capability.

86. Future works
Jorge Tonfat - Ph.D Thesis Defense 86/67