Windows 8 dddd (beekelaar)
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Windows 8 dddd (beekelaar)

on

  • 1,804 views

 

Statistics

Views

Total Views
1,804
Views on SlideShare
1,804
Embed Views
0

Actions

Likes
0
Downloads
50
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Windows 8 dddd (beekelaar) Presentation Transcript

  • 1. Windows 8Disk Deduplication Deep Dive Ronald Beekelaar Virsoft Solutions ronald@beekelaar.com Schiphol, 19 jan 2012
  • 2. Introductions• Presenter – MVP Security – MVP Virtual Machine Technology – E-mail: ronald@beekelaar.com• Work – Security consultancy – Virtualization consultancy – Create many VM-based labs and demos – Software to optimize, manage and run VM – Maintain four datacenters world-wide • Running Hyper-V labs for customers (MOC, training and demo purposes)
  • 3. Objectives• Discuss one interesting new aspect of Windows 8: Disk Deduplication
  • 4. What is Disk Deduplication ?• Goal: – Use less storage space• Method: – Ensure that identical content in multiple (large) files is only stored once• Is block-based, post-process, transparant solution
  • 5. Standard deduplication modes• "Source" – Prevent transferring data, if duplicate • Used by Remote Differential Compression• "Inline" – Perform deduplication when data is written • Used by NTFS file compression • Write process is slowed down• "Post-Process" (or "Background") – Perform deduplication later, in background, when idle • Used by Windows 8 Data Deduplication
  • 6. Other methods to save disk space• SIS (single-instance-store) in Win2000 – Is file-based, not block-based• NTFS file compression – Is inline, not post-process – Much more CPU intensive• NTFS hard links – Is not transparent – Is file-based, not block-based
  • 7. NTFS Hard Links• Multiple file entries pointing to same data• Manage – Create: mklink /h link.ext target.ext – List: fsutil hardlink list file.ext• Is not transparent – Edit one hardlink file, also changes other files• Windows uses thousands of hard links (!) – Good reason not to touch C:Windowswinsxs
  • 8. Windows 8 dedup architecture• Is file-system filter driver – Coordinates between file entry, regular storage and chunk storage• Dedup service (ddpsvc) runs jobs to deduplicate files
  • 9. How does Windows 8 dedup work?• Dedup service recognizes common chunks in files, and places those in Chunk Store – In System Volume Information folder• Dedup filter driver ensures that applications read correct file content• File "size" (= content length) does not change in Explorer – Explorer reports "size-on-disk" as 4 KB
  • 10. How does Windows 8 dedup work?
  • 11. Windows 8 dedup details• Dedup works per volume – Also works on portable disks – Dedup does NOT work on C: (Windows) volume• Chunk size is 32-128 KB (average 80 KB)• By default – Chunks are compressed in chunk store • Avoids re-compressing compressed files (zip, etc) – Dedup service ignores files < 64 KB – Dedup service ignores files changed in last 30 days – Dedup service ignores NTFS encrypted files
  • 12. Savings?• Depends on file content of course• Microsoft reported averages: – General: 50-60% savings • Documents: 30-50% saving • Application library: 70-80% savings • VHD library: 80-95% savings
  • 13. Performance?• Write has no direct performance hit – Dedup operations are done post-process• Read has a ~3% performance hit (if not in cache) – Due to more disk head operations – Compare with disk fragmentation• Windows caching is dedup-aware (!) – Dedup improves caching efficience
  • 14. Reliable?• My opinion: Yes - 100%• Data is check-summed – Means: invalid data is detected• Operations are crash consistent – Means: can interrupt/crash operation at any time without losing data• Data is self-describing – Means: it can be read without external data• Popular chunks (>100x) are stored multiple times – Means: avoids creating IO hotspots on diskJanuary 20, 2012 NIC 2012
  • 15. How to enable Windows 8 dedup?• Install Data Deduplication role service• Start Data Duplication Service (ddpsvc)• Powershell – import-module Deduplication – help dedup – enable-dedupvolume D: – set-dedupvolume D: -minimumfileagedays 0 • Default is 30 days – start-dedupjob D: -type Optimization • Use Unoptimization to undo – get-dedupjob – get-dedupstatus – get-dedupmetadata
  • 16. Questions ?• Thanks for your attention