The document summarizes key topics and industry talks from the China Linux Summit Forum (CLSF) 2010 conference in Shanghai. It discusses presentations on writeback optimization, the BTRFS file system, SSD challenges, VFS scalability, kernel testing frameworks, and talks from companies like Intel, EMC, Taobao, and Baidu on their storage architectures and solutions. Attendees included representatives from Intel, EMC, Fujitsu, Taobao, Novell, Oracle, Baidu, and Canonical discussing topics around file systems, storage, and kernel optimizations.

1. CLSF 2010
1

2. Agenda
• Photos
• Attendances Overview
• Key Topics
• Industry Talks
2

3. LSF 2010, Shanghai
3

4. LSF 2010 participant/companies
Company # of participant Key background
Intel 5 Kernel performance, SSD, mem mgmt
EMC 5 Storage, file system
Fujitsu 4 IO Controller, btrfs
Taobao 3 Distributed storage, taobao server
Novell 2 Suse server, HA
Oracle 2 OCFS2 dev/test
Baidu 2 Baidu kernel optimization
Canonical 2
Redhat 1 Network driver
4

5. Key topics
Topic Slides? Description
Page writeback Y Dirty page ratio limit
Control process to write pages
CFQ, IO controller Y CFQ introduction and further features
BTRFS N Memory consuming, fsck speed
SSD/Block layer Y Block layer issues with SSD
Kernel Tracing N ftrace
VFS scalability N Multi-core challenges
Kernel testing Y Intel kernel auto test framework
Industrial talk: Taobao Y TFS, Tair
Industrial talk: Baidu N The architecture of Baidu search system
Industrial talk: EMC N FSCK
5

6. Writeback - Wu Fengguang
• vmscan is a bottleneck
decrease dirty ratio under memory pressure, so vmscan can less possibly find
dirty pages on page allocation.
• pageout(page) calls wirtepage() to write to disk, which is a performance killer
since it does random writes
let flusher write. expand single 4K write to 4MB write. So more dirty pages are
reclaimed and flushed.
• balance_dirty_page() should not write: random write kills performance
let flusher write and ask process to sleep. Three proposals:
a). wait io completion: NFS bumpy completion, need smoother sleep method.
b). sleep (dirtied *3/2 / write_bandwidth)
c). sleep (dirtied / throttle_bandwidth)
• flusher default write size (4MB -> 128MB), will be dynamic in the future.
Baidu's practice: SSD random write is really bad. For sequential write, increase
wb size (4MB -> 40MB) will get 120% SSD performance
6

7. Btrfs - Coly Li
• Has too much love from linux community
two years ago > now
• Used in MeeGo Project
• Taobao plan to push industrial deployment in 2-3 years
10T per data server in TFS cluster
Use on SSD and SATA hybrid data server
Metadata will be allocated on SSD and data on SATA.
• Dynamic data relocation with hot data tracking patch.
For generic fs usage, need to deal with device mapper to get device speed
information.
• FSCK
A difficult must. Currently assigned to Fujitsu.
7

8. SSD challenges - Li Shaohua
• Throughput: same issue as network
• Disk controller gap and big locks (queue locks & scsi locks)
• Interrupt related:
a. smp affinity: single queue, one CPU to deal with irqs
b. blk_iopoll: poll more req in one req
• Need hardware multiqueue
• CFQ needs to be changed to fit multiqueue (e.g. CFQ per queue)
• Queue lock contention vs. cache lock contention
See Andi Kleen's talk in Tokyo Linux Conf
• Intel is building nextGen PCIE SSD, with many fancy features. Stay tuned
8

9. VFS scalability- Ma Tao
• With multi-cores, all global locks suck
• Globle icache/dcache can be adapted to per-CPU
• CFQ can be adapted to per-queue
• The less global locks the better
9

10. Industry talk – Baidu (Cont.)
• Service types
a). Indexing: random read, high IOPS, small IO size, read-only. 80M records per data
node, and process 8-9 K queries per second.
b). Distributed system: large files, sequential read/write.
c). cache/KV storage: between a and b
d). Web Server: CPU bound.
• For a), read() sucks. Use mmap() to read blocks adhead to void
kernel/userspace memory copy.
mmap() can not use page cache LRU. Call readahead() after each mmap() to mark
pages as read.
mmap() pagefault is expensive with mm->mmap_sem lock. Use sync_readahead()
and sync_readaheadv()
With above, memory is now the bottleneck. Doing 10G+ MB read.
• Google patch for reducing mm->mmap_sem hold time
In do_page_fault(), drop mem->sem if page not found, then read it and get the lock
again.
10

11. Industry talk – Baidu (Cont.)
• 8K filesystem block size (ext2) with 8K page size. Alloc continuous two pages
each time
Get better performance for sequential IO
OCFS2 uses 1MB fs block size and 4K page size
• PCIE compress card + ECC
11

12. Industry Talk - Taobao
• TFS
• Tair uses an update server to record updates; apply updates to production
system during mid-night
• A config server is used to minimize meta server workload
A versioned bucket table is maintained by config server and stored in each data
server. Client can manipulate data location with the bucket table returned by config
server.
• Both TFS and Tair are open source projects now
12

13. Industry Talk - EMC
• Introduce the recovery methods and check methods use in the file systems
from ext2 to btrFS
• Emphasis the importance of FSCK; Introduce the issues within FSCK when
checking a huge file system; Collect the proposals to solve this problem
• pNFS learning notes
13

14. Q&A
14