Advertisement
What every data programmer needs to know about disks
Aug. 22, 2011•0 likes•11,312 views
102 likes
Be the first to like this
Show More
views
Total views
0
On Slideshare
0
From embeds
0
Number of embeds
0
Download to read offline
Report
Technology
What every data programmer needs to know about disks presentation
iammutexFollow
Advertisement
What every data programmer needs to know about disks
- Latency: 100 nanoseconds
- Latency: 10 milliseconds
- Throughput: 768 MB/sec on SATA 3
- dirty_ratio: flush after some percent of memory is used
- Ethernet Controllers
- Southbridge
- How are the caches configured?
- How big are the caches?
- How many controllers?
- How many disks?
- Negligible seek time vs 10ms seek time
- SSD as writeback cache
- Battery-backed
- Adaptec “MaxIQ”
- 6.0 GB/sec throughput. Gigabytes.
- 30 microsecond latency (30k ns)
- Roughly $20/GB
Editor's Notes
- Note that the page is actually in memory twice. Mmaped files fix this, but it’s beyond the scope of this discussion.Also this is why read performance on a lot of memory only NoSQL databases beats disk-backed SQL. Duh.
- Equate 100 nanoseconds to about 100 seconds. Then 10 milliseconds is about 3 months.
- This is where a lot of NoSQL databases get their performance, but more on that in a few minutes.
- There are threads that wake up every now and then to flush pages to disk.
- Fsync blocks until the data has been written to disk.
- With a battery-backed RAID controller, fsync can return very quickly with little risk of data loss.
- You need to dive into your vendor’s control tool to find this out.
- VMWare server is faithful to fsync, VMWare workstation is not. Xen usually queues I/O requests after they have been issued. The point is that you have no way of knowing. Your visibility of what happens to your data after you write or fsync ends at the hypervisor.
- Newer intel chips have the northbridge controller on-die. Southbridge bandwidth is usually <= 10GB/sec, and you are sharing this with other customers’ network and disk I/O. That, and you may be sharing drive spindles.
- EBS lies about the result of fsync. This is why Reddit is down all the time. You have been warned.
- EBS lies about the result of fsync. This is why Reddit is down all the time. You have been warned.
Advertisement