Flash memory

•

16 likes•8,790 views

Anne Nicolas

Flash memory — how it works, and how we use it from Linux, SSDs and ‘real’ flash file systems

Technology Business

Flash Memory
David Woodhouse
david@woodhou.se

Kernel Recipes
September 2012

Software and Services Group

Overview

• What is flash?
• What makes flash “special”?
• How to use flash effectively... and safely
• Flash Translation Layers
• Other approaches
• Linux support for flash

Software and Services Group 2
2

The flash cell
Diagram © Wikipedia user 'Cyferz'
CC BY-SA 3.0

Source Bit
Line Line

Word Line
Control Gate

Floating Gate

N P N

Software and Services Group 3
3

NOR flash
Diagram © Wikipedia user 'Cyferz'
CC BY-SA 3.0

Bit Line

N N, GND N N, GND N N, GND N
P

Software and Services Group 4
4

NAND flash
Diagram © Wikipedia user 'Cyferz'
CC BY-SA 3.0

Bit Line

N N N N N N N N N N N
P

Software and Services Group 5
5

NAND flash: one string

I/O

Software and Services Group 6
6

NAND flash: one eraseblock

Page

I/O

I/O
16896 bits
per page
I/O
(2048 bytes
+ 64 bytes)

I/O

64 pages per eraseblock

Software and Services Group 7
7

Data loss on flash

• Charge loss over time
• Charge builds up in insulator
• Write disturb
• Read disturb

Software and Services Group 8
8

Programming NAND flash
Other Programmed Other
Page Page Page
+10V +20V +10V

+10V I/O

0V I/O

+10V I/O

0V I/O

Programmed cells

Software and Services Group 9
9

Write disturb
Other Programmed Other
Page Page Page
+10V +20V +10V

+10V I/O

0V I/O

+10V I/O

0V I/O

Disturbed cells

Software and Services Group 10
10

Read disturb
Other Read Other
Page Page Page
+5V 0V +5V

0V I/O

0V I/O

0V I/O

0V I/O

Disturbed cells

Software and Services Group 11
11

Single Level Cell

Voltage

Distribution of 0-level cells

Vt – threshold voltage

Distribution of 1-level cells

Software and Services Group 12
12

Multi Level Cell

Voltage

Distribution of (0,0) cells

Distribution of (1,0) cells

Distribution of (0,1) cells

Distribution of (1,1) cells

Software and Services Group 13
13

NAND electrical interface

CE RE WE

Address Latch Enable (ALE)
Command Latch Enable (CLE)
NAND
Ready/Busy (R/B) chip

D0-7

Software and Services Group 14
14

Coping with flash

• Error correction (ECC)
• Bad block management
• Programming pages sequentially
• Garbage collection
• Wear levelling
• Paired pages?

Software and Services Group 15
15

Flash Translation Layers

1
2
3
4
5
6
7
8

Software and Services Group 16
16

Flash Translation Layers

1 1
2 2
3 3
4 4
5 chain 5
6 6
7 7
8 8

Write sectors 1-4

Software and Services Group 17
17

Flash Translation Layers

1 1 1
2 2 2
3 3 3
4 4 4
5 chain 5 chain 5
6 6 6
7 7 7
8 8 8

Write sectors 2, 4, 6

Software and Services Group 18
18

Flash Translation Layers

1 1 1
2 2 2
3 3 3
4 4 4
5 chain 5 chain 5
6 6 6
7 7 7
8 8 8

“Fold” chain into its final block

Software and Services Group 19
19

Flash Translation Layers

1 1
2 2
3 3
4 4
5 chain 5
6 6
7 7
8 8

Erase previous blocks in chain

Software and Services Group 20
20

Flash Translation Layers

1
2
3
4
5
6
7
8

Erase previous blocks in chain

Software and Services Group 21
21

TRIM

1 1 1
2 2 2
3 3 3
4 4 4
5 chain 5 chain 5
6 6 6
7 7 7
8 8 8

Software and Services Group 22
22

TRIM

1 1 1
2 2 2
3 3 3
4 4 4
5 chain 5 chain 5
6 6 6
7 7 7
8 8 8

TRIM sectors 1, 3

Software and Services Group 23
23

TRIM

1 1 1
2 2 2
3 3 3
4 4 4
5 chain 5 chain 5
6 6 6
7 7 7
8 8 8

“Fold” chain into its final block

Software and Services Group 24
24

TRIM

1 1
2 2
3 3
4 4
5 chain 5
6 6
7 7
8 8

Erase previous blocks in chain

Software and Services Group 25
25

TRIM

1
2
3
4
5
6
7
8

Erase previous blocks in chain

Software and Services Group 26
26

An alternative Flash Translation Layer

Sect #1
Sect #2

Software and Services Group 27
27

An alternative Flash Translation Layer

Sect #1
Sect #2
Sect #1
Sect #2
Sect #3
Sect #4

Software and Services Group 28
28

An alternative Flash Translation Layer

Sect #1
Sect #2
Sect #1
Sect #2
Sect #3
Sect #4
Sect #2
Sect #4
Sect #6

Software and Services Group 29
29

File system on top of file system

• Efficiency
– Hard to optimise for opaque lower file system
– Missed garbage collection opportunity
– Real file system knows more about data longevity
– Real file system knows about transactions

• Reliability
– Closed source file system implementation
– No access to underlying medium for fsck / recovery
– Often just implemented badly

Software and Services Group 30
30

MTD support in Linux

• Synchronous read/write
• Asynchronous erase
• ECC for NAND
• Bad Block management

Software and Services Group 31
31

UBI

http://www.linux-mtd.infradead.org/doc/ubidesign/ubidesign.pdf

• Static volumes
– Atomically overwritable
– Wear levelled
– CRC protected
• Dynamic volumes
– Writable flash
– Simple logical <-> physical mapping
– Deferred erase (unmap)
– Scrubbing

Software and Services Group 32
32

Ideal hardware wish list

• Fast queued DMA transfers
• Full error correction in hardware
– With reporting of corrected errors
– Scrubbing
• Page copy

Software and Services Group 33
33

Questions?

Software and Services Group 34
34

What's hot

Semiconductor memories

SambitShreeman

DDR

Payal Dwivedi

chapter-12memory.pptx

RandomGuy90

VLSI Design(Fabrication)

Trijit Mallick

Cmos fabrication

jigyashamaru

8279 in microprocessor

Aisu

Polyteda: Power DRC/LVS, October 2016

Oleksandra Nazola

Microprocessor Presentation

alaminmasum1

Surface mount technology (smt) and smd packages

karen zou

Controller Area Network (Basic Level Presentation)

Vikas Kumar

CMOS

AWANISHKUMAR84

PCB Layout Fundamentals

Analog Devices, Inc.

Semiconductor memory

Then Murugeshwari

SOI Applications -- an overview and some examples

Adele Hars

Basics Of Semiconductor Memories

Rahul Bandhe

Pin diagram 8085

Siddhesh Palkar

Reliability and yield

rohitladdu

Controller area network -ppt

velichetiphani

Vlsi design and fabrication ppt

Manjushree Mashal

bus Can

moh2020

What's hot (20)

Semiconductor memories

DDR

chapter-12memory.pptx

VLSI Design(Fabrication)

Cmos fabrication

8279 in microprocessor

Polyteda: Power DRC/LVS, October 2016

Microprocessor Presentation

Surface mount technology (smt) and smd packages

Controller Area Network (Basic Level Presentation)

CMOS

PCB Layout Fundamentals

Semiconductor memory

SOI Applications -- an overview and some examples

Basics Of Semiconductor Memories

Pin diagram 8085

Reliability and yield

Controller area network -ppt

Vlsi design and fabrication ppt

bus Can

Similar to Flash memory

Learn OpenStack from trystack.cn ——Folsom in practice

OpenCity Community

OpenStack and OpenFlow Demos

Brent Salisbury

Networking with Linux on System z

IBM India Smarter Computing

Software Defined Data Centers - June 2012

Brent Salisbury

Xensummit2009 Io Virtualization Performance

The Linux Foundation

Windows Server 2012 Hyper-V Networking Evolved

Microsoft TechNet - Belgium and Luxembourg

Edge optical transceivers

EDGEtech

Asml Euv Use Forecast

Knowledge_Broker

Public Presentation, ASML EUV forecast Jul 2010

JVervoort

OpenStack Quantum

Sumit Naiksatam

XS Boston 2008 XenLoop

The Linux Foundation

Traffic Management, DPI, Internet Offload Gateway

Continuous Computing

C3 Citrix Cloud Center

Rui Lopes

Virtual Network Performance Challenge

Stephen Hemminger

Acceleration for big data, hadoop and memcached it168文库

Accenture

Acceleration for big data, hadoop and memcached it168文库

Accenture

Mattithyahu

Big Data Smarter Networks

DataWorks Summit

Cisco open network environment

deepers

New Networking Technology Survey & Analysis

IT Brand Pulse

Similar to Flash memory (20)

Learn OpenStack from trystack.cn ——Folsom in practice

OpenStack and OpenFlow Demos

Networking with Linux on System z

Software Defined Data Centers - June 2012

Xensummit2009 Io Virtualization Performance

Windows Server 2012 Hyper-V Networking Evolved

Edge optical transceivers

Asml Euv Use Forecast

Public Presentation, ASML EUV forecast Jul 2010

OpenStack Quantum

XS Boston 2008 XenLoop

Traffic Management, DPI, Internet Offload Gateway

C3 Citrix Cloud Center

Virtual Network Performance Challenge

Acceleration for big data, hadoop and memcached it168文库

Big Data Smarter Networks

Cisco open network environment

New Networking Technology Survey & Analysis

More from Anne Nicolas

Wanting to avoid the Android experience, Google developers always aimed to make their Chrome OS Linux kernels as close to mainline as possible. However, when Chromebooks were first created, Google was left with no choice, the mainline kernel, in some subsystems, still did not have all the functionalities needed by Chromebooks. Hence, similarly to Android, Chrome OS had to develop their own out-of-tree code for the kernel and maintain that for a few different kernel versions. Luckily, over the last few years a strong and consistent effort has been happening to bring Chromebook devices closer to mainline. It has led to significant improvements that now make it possible to run mainline on Chrome OS devices. And not only Chromebooks, as these significant strides are also improving Arm-based SOCs and other key components of the rich Chromebook hardware ecosystem. In this talk, we will look at how and why upstream support for Chromebooks improved, the current status of various models, and what we expect in the future. Enric Balletbò i Serra

Kernel Recipes 2019 - Driving the industry toward upstream first

Anne Nicolas

As the name would suggest, a Non-Maskable Interrupt (NMI) is an interrupt-like feature that is unaffected by the disabling of classic interrupts. In Linux, NMIs are involved in some features such as performance event monitoring, hard-lockup detector, on demand state dumping, etc… Their potential to fire when least expected can fill the most seasoned kernel hackers with dread. AArch64 (aka arm64 in the Linux tree) does not provide architected NMIs, a consequence being that features benefiting from NMIs see their use limited on AArch64. However, the Arm Generic Interrupt Controller (GIC) supports interrupt prioritization and masking, which, among other things, provides a way to control whether or not a set of interrupts can be signaled to a CPU. This talk will cover how, using the GIC interrupt priorities, we provide a way to configure some interrupts to behave in an NMI-like manner on AArch64. We’ll discuss the implementation, some of the complications that ensued and also some of the benefits obtained from it. Julien Thierry

Kernel Recipes 2019 - No NMI? No Problem! – Implementing Arm64 Pseudo-NMI

Anne Nicolas

At a rate of almost 9 changes per hour (24/7), the Linux kernel is definitely a scary beast. Bugs are introduced on a daily basis and, through the use of multiple code analyzers, *some* of them are detected and fixed before they hit mainline. Over the course of the last few years, Gustavo has been fixing such bugs and many different issues in every corner of the Linux kernel. Recently, he was in charge of leading the efforts to globally enable -Wimplicit-fallthrough; which appears by default in Linux v5.3. This presentation is a report on all the stuff Gustavo has found and fixed in the kernel with the support of the Core Infrastructure Initiative. Gustavo A.R. Silva

Kernel Recipes 2019 - Hunting and fixing bugs all over the Linux kernel

Anne Nicolas

Kernel Recipes 2019 - Metrics are money

Anne Nicolas

The Linux kernel project includes a huge amount of documentation, but that information has seen little in the way of care over the years. The amount of care has increased significantly recently, though, and things are improving quickly. Listen as the kernel’s documentation maintainer discusses the current state of the kernel’s docs, how we got here, where we’re trying to go, and how you can help. Jonathan Corbet

Kernel Recipes 2019 - Kernel documentation: past, present, and future

Anne Nicolas

Modern SoC designs incorporate technologies from numerous vendors, each with their own inconsistent, confusing, undocumented and even contradictory terminology. The result is a mess of acronyms and product names which have a surprising impact on the ability to develop reusable, modular code thanks to the nature of the underlying IP being obscured. This presentation will dive into some of the misnomers plaguing the Arm ecosystem, with the aim of explaining why things are like they are, how they fit together under the architectural umbrella and how you, as a developer, can decipher the baffling ingredients list of your next SoC design! Will Deacon

Embedded Recipes 2019 - Knowing your ARM from your ARSE: wading through the t...

Anne Nicolas

GNU poke is a new interactive editor for binary data. Not limited to editing basic ntities such as bits and bytes, it provides a full-fledged procedural, interactive programming language designed to describe data structures and to operate on them. Once a user has defined a structure for binary data (usually matching some file format) she can search, inspect, create, shuffle and modify abstract entities such as ELF relocations, MP3 tags, DWARF expressions, partition table entries, and so on, with primitives resembling simple editing of bits and bytes. The program comes with a library of already written descriptions (or “pickles” in poke parlance) for many binary formats. GNU poke is useful in many domains. It is very well suited to aid in the development of programs that operate on binary files, such as assemblers and linkers. This was in fact the primary inspiration that brought me to write it: easily injecting flaws into ELF files in order to reproduce toolchain bugs. Also, due to its flexibility, poke is also very useful for reverse engineering, where the real structure of the data being edited is discovered by experiment, interactively. It is also good for the fast development of prototypes for programs like linkers, compressors or filters, and it provides a convenient foundation to write other utilities such as diff and patch tools for binary files. This talk (unlike Gaul) is divided into four parts. First I will introduce the program and show what it does: from simple bits/bytes editing to user-defined structures. Then I will show some of the internals, and how poke is implemented. The third block will cover the way of using Poke to describe user data, which is to say the art of writing “pickles”. The presentation ends with a status of the project, a call for hackers, and a hint at future works. Jose E. Marchesi

Kernel Recipes 2019 - GNU poke, an extensible editor for structured binary data

Anne Nicolas

Operating system distributors often face challenges that are somewhat different from that of upstream kernel developers. For instance, some kernel updates often need to stay at least binary compatible with modules that might be “out of tree” for some time. In that context, being able to automatically detect and analyze changes to the binary interface exposed by the kernel to its module does have some noticeable value. The Libabigail framework is capable of analyzing ELF binaries along with their accompanying debug info in the DWARF format, detect and report changes in types, functions, variables and ELF symbols. It has historically supported that for user space shared libraries and application so we worked to make it understand the Linux kernel binaries. In this presentation, we are going to present the current support of ABI analysis for Linux Kernel binaries, the challenges we face, how we address them and the plans we have for the future. Dodji Seketeli, Jessica Yu, Matthias Männich

Kernel Recipes 2019 - Analyzing changes to the binary interface exposed by th...

Anne Nicolas

Different upgrade and update strategies exist when it comes to embedded Linux system. If at development time none of these strategies have been chosen, adding them afterwards can be tedious task. Even harder it gets when the system is already deployed in the field and only accessible via a 3G connection. This talk is a developer experience of putting in place exactly that. Giving a return of experience on one way of doing it on a system running Barebox and a Yocto-based distribution. Patrick Boettcher

Embedded Recipes 2019 - Remote update adventures with RAUC, Yocto and Barebox

Anne Nicolas

Traditionally graphics drivers were one of the last hold-outs of proprietary software in an embedded Linux system. This situation is changing with open-source graphics drivers showing up for almost all of the graphics acceleration peripherals on the market right now. This talk will show how open-source graphics drivers are making embedded systems less special, as well as trying to provide an overview of the Linux graphics stack, de-mystifying what is often seen as black magic GPU stuff from outside observers. Lucas Stach

Embedded Recipes 2019 - Making embedded graphics less special

Anne Nicolas

This talk will explore Open Source Hardware projects relevant to Linux, including boards like BeagleBone, Olimex OLinuXino, Giant board and more. Looking at the benefits and challenges of designing Open Source Hardware for a Linux system, along with BeagleBoard.org’s experience of working with community, manufacturers, and distributors to create an Open Source Hardware platform. In closing also looking at the future, Libre Silicon like RISC-V designs, and where this might take Linux. Drew Fustini

Embedded Recipes 2019 - Linux on Open Source Hardware and Libre Silicon

Anne Nicolas

The I2C subsystem is not the shiniest part of the Linux Kernel. For embedded devices, though, it is one of the many puzzle pieces which just have to work. Wolfram Sang has the experience of maintaining this subsystem for nearly 7 years now. This talk gives a short overview of how maintaining works in general and specifically in this subsystem. But mainly, it will highlight noteworthy points in the timeline and lessons learnt from that. It will present trends, not so much regarding I2C but more the Linux Kernel and the embedded ecosystem in general. And of course, there will be plenty of anecdotes and bits from behind the scenes for your entertainment. Wolfram Sang

Embedded Recipes 2019 - From maintaining I2C to the big (embedded) picture

Anne Nicolas

ITRenew is selling recertified OCP servers under the Sesame brand, those servers come either with their original UEFI BIOS or with LinuxBoot. The LinuxBoot project is pushing the Linux kernel inside bios flash and using userland programs as bootloader. To achieve quality on our software stack, as any project, we need to test it. Traditional BIOS are tested by hand, this is 2019 we need to do it automatically! We already presented the hardware setup behind the LinuxBoot CI, this talk will focus on the software. We use u-root for our userland bootloader; this software is written in Go so we naturally choose to use Go for our testing too. We will present how we are using and extending the Go native test framework `go test` for testing embedded systems (serial console) and improving the report format for integration to a CI. Julien Viard de Galbert

Embedded Recipes 2019 - Testing firmware the devops way

Anne Nicolas

About 60% of the Linux kernel source tree is devoted to drivers for a large variety of supported hardware components. Especially in the embedded world, the number of different SoC families, versions, and revisions, integrating a myriad of “IP cores”, keeps on growing. In this presentation, Geert will explain how to match drivers against hardware, and how to support a wide variety of (dis)similar devices, without turning platform and driver code into an entangled bowl of spaghetti.tra Starting with a brief history of driver matching in Linux, he will fast-forward to device-tree based matching. He will discuss ways to handle slight variations of the same hardware devices, and different SoC revisions, each with their own quirks and bugs. Finally, Geert will show best practices for evolving device drivers in a maintainable way, based on his experiences as an embedded Linux kernel developer and maintainer. Geert Uytterhoeven

Embedded Recipes 2019 - Herd your socs become a matchmaker

Anne Nicolas

Buildroot is a popular and easy to use embedded Linux build system. It generates, in few minutes, lightweight and customized Linux systems, including the cross-compilation toolchain, kernel and bootloader images, as well as a wide variety of userspace libraries and programs. This talk is about the integration of LLVM/clang into Buildroot. In 2018, Valentin Korenblit, supervised by Romain Naour, worked on this topic during his internship at Smile ECS. After a short introduction about llvm/clang and Buildroot, this talk will go through the numerous issues discovered while adding llvm/clang componants and how these issues were fixed. Romain will also detail the work in progress and the work to be done based on llvm/clang libraries (OpenCL, Compiler-rt, BCC. Chromium, ldd). Romain Naour

Embedded Recipes 2019 - LLVM / Clang integration

Anne Nicolas

This talk introduces JTAG debugging capabilities, both for debugging hardware and software. Marek first explains what the JTAG stands for and explains the operation of the JTAG state machine. This is followed by an introduction to free software JTAG tools, OpenOCD and urJTAG. Marek shortly explains how to debug software using those tools and how that ties into the JTAG state machine. However, JTAG was designed for testing hardware. Marek explains what boundary scan testing (BST) is, what are BSDL files and their format, and practically demonstrates how to blink an LED using BST and only free software tools. Marek Vasut

Embedded Recipes 2019 - Introduction to JTAG debugging

Anne Nicolas

PipeWire is an open source project that aims to greatly improve audio and video handling under Linux. Utilising a fresh design, it bridges use cases that have been previously addressed by different tools – or not addressed at all -, providing ground for building complex, yet secure and efficient, multimedia systems. In this talk, Julien is going to present the PipeWire project and the concepts that make up its design. In addition, he is going to give an update of the current and future work going on around PipeWire, both upstream and in Automotive Grade Linux, an early adopter that Julien is actively working on. Julian Bouzas

Embedded Recipes 2019 - Pipewire a new foundation for embedded multimedia

Anne Nicolas

Ftrace’s most powerful feature is the function tracer (and function graph tracer which is built from it). But to have this enabled on production systems, it had to have its overhead be negligible when disabled. As the function tracer uses gcc’s profiling mechanism, which adds a call to “mcount” (or more recently fentry, don’t worry if you don’t know what this is, it will all be explained) at the start of almost all functions, it had to do something about the overhead that causes. The solution was to turn those calls into “nops” (an instruction that the CPU simply ignores). But this was no easy feat. It took a lot to come up with a solution (and also turning a few network cards into bricks). This talk will explain the history of how ftrace came about implementing the function tracer, and brought with it the possibility of static branches and soon static calls! Steven Rostedt

Kernel Recipes 2019 - ftrace: Where modifying a running kernel all started

Anne Nicolas

Suricata is a network threat detection engine using network packets capture to reconstruct the traffic till the application layer and find threats on the network using rules that define behavior to detect. This task is really CPU intensive and discarding non interesting traffic is a solution to enable a scaling of Suricata to 40gbps and other. This talk will present the latest evolution of Suricata that knows uses eBPF and XDP to bypass traffic. Suricata 5.0 is supporting the hardware XDP to provide ypass with network card such as Netronome. It also takes advantage of pinned maps to get persistance of the bypassed flows. This talk will cover the different usage of XDP and eBPF in Suricata and shows how it impact performance and usability. If development time permit, the talk will also cover AF_XDP and the impact on this new capture method on Suricata. Eric Leblond

Kernel Recipes 2019 - Suricata and XDP

Anne Nicolas

System memory configuration is a transparent operation nowadays, something that we all came to expect to just work out of the box. Still, it does happen behind the scenes every single time we boot our computers. This requires the cooperation of hardware components on the mainboard and on memory modules themselves, as well as firmware code to drive these. While it is possible to just let it happen, having a deeper understanding of how it works makes it possible to access valuable information from the operating system at run-time. I will take you through the history of system memory configuration from the mid 70s to now. We will explore the different types of memory modules, how their configuration data is stored and how the firmware can access them. We will see which problems had to be solved along the way and how they were solved. Lastly we will see how Linux supports reading the memory configuration information and what you can do with that information. Jean Delvare

Kernel Recipes 2019 - Marvels of Memory Auto-configuration (SPD)

Anne Nicolas

More from Anne Nicolas (20)