Recommended
More Related Content
Similar to Module 5a - Storage -Storage System Features.pptx
Similar to Module 5a - Storage -Storage System Features.pptx (20)
Recently uploaded
Recently uploaded (20)
Module 5a - Storage -Storage System Features.pptx
- 1. Server 2012 Support Training - Storage Storage System Features Dell Global Learning and Development Original Content by Microsoft Support services
- 2. Global Learning and Development Introduction • Windows Server 2012 has a number of storage related changes. The following information is designed to help you learn about the new functionality that is in the product. Additionally, we will cover some of the improvements made to pre-existing functionality. • This documentation assumes the reader is already familiar with storage concepts as they relate to Windows 2008 and Windows 2008 R2. Microsoft Confidential - For Internal Use Only 3 7/30/2011
- 3. Global Learning and Development Module 1: Storage System Features • Lesson 1-1: Storage Spaces • Lesson 1-2: iSCSI Software Target • Lesson 1-3: Storport Enhancements • Lesson 1-4: Thin Provisioning • Lesson 1-5: ISO/VHD Mounting • Lesson 1-6: Data Optimization (Deduplication) • Lesson 1-7: Storage Management Microsoft Confidential - For Internal Use Only 4 7/30/2011
- 4. Global Learning and Development Storage System Features • Module 1 • Windows Server 2012 & Windows 8 Storage and File Systems Microsoft Confidential
- 5. Global Learning and Development What You Will Learn After completing this module, you will be able familiar with the following: • Storage Spaces • iSCSI Software Target • StorPort Enhancements • Storage Management • Thin Provisioning • Data Optimization (Deduplication) • ISO Mounting Microsoft Confidential - For Internal Use Only 8 7/30/2011
- 6. Global Learning and Development Storage Spaces • Module 1: Lesson 1 • Windows 8 & Windows Server 2012 Storage and File Systems Microsoft Confidential
- 7. Global Learning and Development Introducing Storage Spaces • New virtualization technology • Dramatically reduces costs associated with highly available storage for hosted and virtualized deployments. • Based on a pooling model. • Targets affordable commodity hardware • Integrates well with the other features Microsoft Confidential - For Internal Use Only 10 7/30/2011
- 8. Global Learning and Development Overview • Organizes physical disks into storage pools • Storage pools can be… – Easily expanded by adding disks – Composed of different sized physical disks via different storage interconnects • Resiliency can be provided by mirroring or parity • JBOD can also be used Microsoft Confidential 11
- 9. Global Learning and Development Description • Reduces cost to acquire and manage available, reliable, and scalable storage because • Does not require an external storage subsystem. • Alleviates the burden of upfront prediction of their ultimate storage capacity requirements • Pools and spaces can be grown on demand. • Provides a variety of feature sets including: – Just-in-time provisioning – Fault-resiliency and data integrity – Integration with CSV Microsoft Confidential - For Internal Use Only 12 7/30/2011
- 10. Global Learning and Development Benefits • Pooling • Scalable • Ease of Management • Flexible Control • Hardware Compatibility • Application Compatibility • Resilient Storage • Flexible Redundancy Repair Microsoft Confidential 13
- 11. Global Learning and Development Failover Clustering • Supported by Failover Cluster* • Possible for enterprises to deploy less expensive storage hardware (SAS only) • Using Storage Spaces with Failover Clustering is covered in more detail in the Failover Clustering modules. • Storage spaces on Fibre Channel and iSCSI is an unsupported configuration – Usefulness of storage spaces limited in Failover Clustering Microsoft Confidential 14
- 12. Global Learning and Development Prerequisites • Simple Storage Pool – 1 Drive (min) • Mirror Storage Pool – 2 Drives (min) • Parity Storage Pool – 3 Drives (min) • Drives: – 10GB min – Unformatted and Unpartitioned – SATA, SCSI, SAS or USB – FC & iSCSI not supported Microsoft Confidential 15
- 13. Global Learning and Development Storage Spaces Rely on the Following Technologies • Windows Operating System Storage Stack • File and Storage Services Interface • Storage PowerShell cmdlets • Storage Management API (SMAPI) • Storage Management Provider (SMP) Microsoft Confidential - For Internal Use Only 16 7/30/2011
- 14. Global Learning and Development Installation GUI/Wizard Microsoft Confidential - For Internal Use Only 17 7/30/2011
- 15. Global Learning and Development Creating Storage Spaces • Add Storage Services Role • Create a Storage Pool via – File and Storage Services interface – Powershell cmdlet: New-StoragePool • Create a Virtual Disk (Storage Space) on the Storage Pool • Partition and Format the virtual disk like a regular disk Microsoft Confidential 18 Disks Storage Pool Virtual Disk Partition Format Drive Letter
- 16. Global Learning and Development Primordial Pool Microsoft Confidential - For Internal Use Only 19 7/30/2011 Disks that are not part of a Storage Pool
- 17. Global Learning and Development Disk Management & Device Manager Views Microsoft Confidential - For Internal Use Only 20 7/30/2011 No Partitioning required Existing drivers used for enumeration of disks Disk Management Interface Device Manager View
- 18. Global Learning and Development Storage Spaces Setup - DEMO Microsoft Confidential 21
- 19. Global Learning and Development New Pool and New Space Microsoft Confidential - For Internal Use Only 22 7/30/2011
- 20. Global Learning and Development SpacePort Driver • Port Driver for Storage Spaces • Interacts with the storage stack • Provides PnP notifications Microsoft Confidential 23
- 21. Global Learning and Development Shown in Device Manager Microsoft Confidential - For Internal Use Only 24 7/30/2011
- 22. Global Learning and Development MSInfo32 Output Microsoft Confidential - For Internal Use Only 25 7/30/2011
- 23. Global Learning and Development Spaces Protective Partition • Placed on each drive in the pool • Contains the metadata and data that manifests the storage spaces within each pool Microsoft Confidential 26
- 24. Global Learning and Development Considerations – Storage Layout • Simple – Data Striping – No Redundancy • Mirror – 2-way and 3-way mirrors – Maintains 2 or 3 data copies • Parity – Similar to Simple space – Adds parity information Microsoft Confidential 27
- 25. Global Learning and Development Considerations – Provisioning Schemes • Fixed Provisioning – Virtual Disk size equals the amount of space available • Thin Provisioning – Virtual Disk size can exceed the amount of space available Microsoft Confidential 28
- 26. Global Learning and Development Server Manager View Microsoft Confidential - For Internal Use Only 29 7/30/2011
- 27. Global Learning and Development Considerations – Sector Size • Storage Pool sector size set at creation • All drives in the storage pool must have same sector size • Storage Pool with – 512 and/or 512e drives: Defaults to 512e – At least one 4KB drive: Defaults to 4KB • Administrator can define sector size – Windows will only permit the addition of the same type of drive Microsoft Confidential 30
- 28. Global Learning and Development Considerations – Cluster Disk Requirement • For a Storage Pool to support failover – Only SAS can be used – iSCSI and Fibre Channel not supported Microsoft Confidential 31
- 29. Global Learning and Development Considerations – Allocating a Drive • Data Store – Default allocation type • Manual – Not automatically used as part of a storage space – Must be selected at the creation of that storage space • Hot-Spare – Reserve Drives – Not used in the creation of storage space Microsoft Confidential 32
- 30. Global Learning and Development Issues with Storage Spaces and Dell servers
- 31. Global Learning and Development Helpful logs • Applications and Services Logs MicrosoftWindowsSpaceportAnalytic • Applications and Services Logs MicrosoftWindowsSpaceportOperational • Applications and Services LogsMicrosoftWindowsDisk • Applications and Services Logs MicrosoftWindowsStorDiag • Applications and Services Logs MicrosoftWindowsStorPort Microsoft Confidential - For Internal Use Only 34 7/30/2011
- 32. Global Learning and Development Troubleshooting Storage Spaces • To list unhealthy storage spaces, use the following PS cmdlet: Get-VirtualDisk | Where-Object {$_.HealthStatus –ne “Healthy”} • To repair a Virtual Disk, use the following PS cmdlet: Repair-VirtualDisk -FriendlyName • To list unhealthy storage pools, use the following PS cmdlet: Get-StoragePool | Where-Object {$_.HealthStatus –ne “Healthy”} Microsoft Confidential - For Internal Use Only 35 7/30/2011
- 33. Global Learning and Development Troubleshooting Storage Spaces • To list unhealthy physical disks, use the following PS cmdlet: Get-PhysicalDisk | Where-object {$_.HealthStatus –ne “Healthy”} • To remove (or Reset) a failed physical disk from a storage pool, us the following PS cmdlet: Reset-PhysicalDisk –FriendlyName WARNING! Reset-PhyscialDisk will destroy all the Pool metadata from the disk. Microsoft Confidential 36
- 34. Global Learning and Development Dell support of Storage spaces • “It is not recommended that you configure storage spaces on your Dell hardware as it is currently not supported. Using storage spaces on unsupported hardware increases the risk of data loss because hardware failure notifications and hardware failover may not occur as expected.” • “NOTE: Dell software RAID S100 and S300 are not supported with Windows Server 2012.”
- 35. Global Learning and Development Additional Resources and Source Credits Understanding and Troubleshooting Storage Spaces in Windows http://www.microsoft.com/download/en/details.aspx?id=29002 Using PowerShell Cmdlets for Storage Management http://sharepoint/sites/StorageManagement/Storage%20Management% 20Wiki/Storage%20Management%20Wiki%20122711.aspx Microsoft Confidential - For Internal Use Only 38 7/30/2011
- 36. Global Learning and Development LAB 39 • Lab 1 Storage Pools
- 37. Global Learning and Development Thin Provisioning • Module 1: Lesson 4 • Windows Server 2012 & Windows 8 Storage and File Systems Microsoft Confidential
- 38. Global Learning and Development Thick Provisioning vs. Thin Provisioning • Thick Provisioning – Nothing New (Standard) – Virtual Disk Space = Drive Space – Each Logical Disk sector backed by physical sector • Thin Provisioning – New method of disk provisioning – Virtual Disk space >= Drive space – At time of use, Logical Disk sector mapped to a physical sector – Mapping happens ‘just-in-time’ when needed Microsoft Confidential 41
- 39. Global Learning and Development Thick Provisioning Example: • 3 LUNS exposed to OS • Each logical sector (OS) backed by a physical sector (SAN) Microsoft Confidential 42
- 40. Global Learning and Development Problems with Thick Provisioning • Volumes provisioned regardless of capacity usage • Results in unnecessary up-front costs • Before Windows Server 2012 – Storage Stack was not thin provisioning aware – No support for SBC3 Commands (SCSI Block Commands v3) – No way to process resource exhaustion events from targets • From storage partners’ customer survey: – About 54% of customers were aware that they had unused storage capacity due to inefficient provisioning methods. – About 55% of customers had between 31% and 50% of provisioned storage unused. (i.e. 10TB space: 3.1TB – 5TB unused) Microsoft Confidential 43
- 41. Global Learning and Development Thin Provisioning • Only in use sectors are backed by physical sectors – White areas represent unused sectors • When files expand – White areas fill up – SAN adds backing sectors ‘just-in-time’ Microsoft Confidential 44
- 42. Global Learning and Development Thin Provisioning at the OS Level • Problems with Hardware only Thin provisioning – No reclamation support – Thin LUNs eventually become fully mapped Thin Provisioning at the OS Level • Enhances the usage efficiency of storage space • Supports reclamation and resource re-allocation • NTFS and ClassPnP API implementations used • Notifications when resources are becoming fully mapped Microsoft Confidential 46
- 43. Global Learning and Development Windows Optimize Drives Utility Can be used by admins to: • Reclaim space via manual request • Optimizing the schedule configuration Microsoft Confidential 52 NOTE: This functionality depends on a thin provisioned LUN being correctly detected in Windows
- 44. Global Learning and Development Issues with Current Thin Provisioning Storage Some storage devices • Do not report provisioning type to Windows • Support WRITE SAME but not UNMAP (both SCSI-3 spec) • Does not report – Threshold reached events – Temporary resource exhaustion – Permanent resource exhaustion • Cannot report used and available mapping space • Overall no synchronous view for storage and system administrators Microsoft Confidential 53
- 45. Global Learning and Development Windows Based Thin Provisioning Limitations • Will not function on dynamic disks – PartMgr / VolMgr blocks dynamic partitions using thin provisioning • Feature will not support thin provisioning for: – Paging File – Hibernation File – Crash dump File Microsoft Confidential 54
- 46. Global Learning and Development LAB 55 • Lab 2 Thin Provisioning
- 47. Global Learning and Development iSCSI Software Target • Module 1: Lesson 2 • Windows Server 2012 & Windows 8 Storage and File Systems Microsoft Confidential
- 48. Global Learning and Development iSCSI Software Target • Integrated with the OS in Windows Server 2012 • Software based and hardware independent • Presents iSCSI disk subsystems in SANs • In Windows Server 2012 the iSCSI Target is available in-box • Previously shipped as part of Windows Storage Server – A version was available for Windows Server 2008 R2 Microsoft Confidential 57
- 49. Global Learning and Development iSCSI Target Improvements More detail in coming slides – quick overview: • Support for 4k sector drives • Improvements to PowerShell • Volsnap based change tracking • Addition to SMI-S support • AutoRecovery Snapshot • Best Practice Analyzer • Feature Integration • Continuous Availability Improvements NOTE: No support for VHD 2.0 Microsoft Confidential 58
- 50. Global Learning and Development Support for 4K Sector Drives • A 4k sector hard drive has 4096 bytes per sector – Previous standard in 512 bytes • Reasons for Increase in sector size – Drive Size – Reliability Microsoft Confidential 59
- 51. Global Learning and Development Improvements to PowerShell • The Microsoft iSCSI Software Target is integrated into Windows Server 2012 as a feature under the file server role. • It is managed through Server Manager, which is built on top of PowerShell cmdlets. • The iSCSI management UI focuses on the presentation of enabling end-to-end user scenarios. Microsoft Confidential - For Internal Use Only 60 7/30/2011
- 52. Global Learning and Development Volsnap Based Change Tracking • Prior to Windows Server 2012 – iSCSI target used a bitmap to track dirty blocks – Allowed snapshots for easy rollback – On large virtual disks, copies were time consuming – During rollback only dirty blocks were copied – Performance affected on writes – Tracked changes at 64KB blocks • In Windows Server 2012 – iSCSI Target snapshot leverages VSS Snapshots – Volsnap already provides bitmaps to track changes – Removes need for separate bitmap tracking – VSS Tracks changes at 16KB blocks – more granular restore – Write Performance improved – Backup times decreased Microsoft Confidential 61
- 53. Global Learning and Development Auto Recovery Snapshot • Applications that rely on databases need to ensure that all transactions are processed – Ex: Exchange database can not be in dirty shutdown • Requires a post snapshot process • Integration of Volsnap and iSCSI target completes this process during backups Microsoft Confidential 62
- 54. Global Learning and Development Best Practices Analyzer (BPA) • iSCSI Target now has its own BPA • Can be accessed via Powershell Microsoft Confidential 64 PS C:Windowssystem32> get-bpamodel |format-table id Id -- Microsoft/Windows/ADRMS Microsoft/Windows/CertificateServices Microsoft/Windows/DHCPServer ….and so on…
- 55. Global Learning and Development iSCSI Target Service • The iSCSI Target Service (WinTarget) contains – A user-mode implementation of the iSCSI Protocol – Necessary management interfaces. • Previous versions of the iSCSI Target – Creation of LUNs is based on VHDs. – VHDs in turn are associated with Targets. – A target is made available to initiators. • The new iSCSI Target service – Is fully cluster aware – Owns a custom cluster resource – Has functionality related to cluster › client-access points › cluster disk › other High-Availability related items. Microsoft Confidential - For Internal Use Only 65 7/30/2011
- 56. Global Learning and Development WinTarget Service • Processes incoming connections from iSCSI Initiators. • Implemented as a service-host based service, • Runs in a separate group, called 'BlockStorageServices'. • The service will run in the LocalSystem account. • Can be stopped / started without requiring a reboot. Microsoft Confidential - For Internal Use Only 66 7/30/2011
- 57. Global Learning and Development Continuous Availability Enhancements • Now Integrated with Failover Clustering Microsoft Confidential 68
- 58. Global Learning and Development Functionality Removed • There were two small changes to the iSCSI target as far as lost functionality. 1. Unable to create targets without a virtual disk 2. Unable to assign a virtual disk to multiple targets Microsoft Confidential - For Internal Use Only 69 7/30/2011
- 59. Global Learning and Development StorPort Enhancements • Module 1: Lesson 3 • Windows Server 2012 & Windows 8 Storage and File Systems Microsoft Confidential
- 60. Global Learning and Development StorPort Enhancements • Outstanding I/O Limits increased • 64-bit addressing scheme • Faster Device Enumeration Microsoft Confidential 71
- 61. Global Learning and Development Outstanding I/O Limits increased • Older versions of Storport Outstanding I/O Limits – 254 per LUN – 1000 per adapter • Some Current HBA’s can exceed 1000 outstanding I/O’s • New versions of Storport… – Provide a new interface to query the miniport driver – No maximum limit defined with new model – Drivers not using the new interface have old limits listed above Microsoft Confidential 72
- 62. Global Learning and Development 64-bit addressing scheme • Storage capacities increase with new storage hardware • Larger storage capacities provided with 64-bit addressing • Storport now supports 64-bit addressing Microsoft Confidential - For Internal Use Only 73 7/30/2011
- 63. Global Learning and Development Faster Device Enumeration • It is possible to have HBA’s with 100’s of drives attached • Time to enumerate these devices is becoming a boot issue • StorPort can now re-enumerate devices by target – Improves performance over older versions – Previous versions enumerated by adapter. Microsoft Confidential - For Internal Use Only 74 7/30/2011
- 64. Global Learning and Development LAB 76 • Lab 3 iSCSI
- 65. Global Learning and Development ISO & VHD Mounting • Module 1: Lesson 5 • Windows Server 2012 & Windows 8 Storage and File Systems Microsoft Confidential
- 66. Global Learning and Development Introduction • ISO is a popular CD/DVD image file format • ISO files can be used by – Burning to a disc – Mounting as a virtual optical drive • In Windows Server 2012 native ISO mounting is supported Microsoft Confidential 78
- 67. Global Learning and Development Working with ISO/VHD Files So how does this work? (Its Simple) • Mount the drive – Select from the Enhanced Explorer ribbon – Double-click or Right Click the file • Windows seamlessly creates a virtual CDROM (ISO) or a virtual hard drive (VHD) to access the data Microsoft Confidential - For Internal Use Only 79 7/30/2011
- 68. Global Learning and Development Walk Through Microsoft Confidential - For Internal Use Only 80 7/30/2011
- 69. Global Learning and Development Main PowerShell CMDLets • Mount-DiskImage – Mounts either a VHD or ISO file – Mount-DiskImage –ImagePath “C:ISOMyCD.iso” • Dismount-DiskImage – Dismounts a mounted file – Dismount-DiskImage –ImagePath “C:ISOMyCD.iso” – Dismount-DiskImage –DevicePath .CDROM1 • Get-DiskImage – Gets information about a DiskImage – Get-DiskImage –ImagePath “C:ISOMyCD.iso” – Get-DiskImage –DevicePath .CDROM1 – Get-DiskImage –DevicePath .CDROM1 | Get Volume Microsoft Confidential - For Internal Use Only 81 7/30/2011
- 70. Global Learning and Development LAB 82 • Lab 4 ISO Mounting
- 71. Global Learning and Development Data Optimization (Deduplication) • Module 1: Lesson 6 • Windows Server 2012 & Windows 8 Storage and File Systems Microsoft Confidential
- 72. Global Learning and Development Background • File-based data increasing at tremendous rates • Disk storage costs have been dropping – but not fast enough to offset data growth • Efficiencies need to be in place where data is stored • Data transfers also need to be efficient • Data Optimization is a key goal with Windows Server 2012 Microsoft Confidential 84
- 73. Global Learning and Development What is Data Optimization • Act of reducing the physical amount of bytes needed • Reduce the amount of data transmitted over a network • Can not compromise the integrity of original data Microsoft Confidential 85
- 74. Global Learning and Development Common Data Optimization Techniques Concepts covered on the next slides: • Compression • File-level Deduplication • Chunk-Level Deduplication Microsoft Confidential 86
- 75. Global Learning and Development Compression • Encoding a unit of data using fewer bits • Lossless – No data lost – ZIP – LZH • Lossy – Some data lost – JPEG – MP3 Microsoft Confidential 87
- 76. Global Learning and Development File Level Deduplication • Identifying identical files within a data set • Storing only one copy of the file • Replacing all instances of the file with a reference • Also known as Single Instance Storage (SIS) Microsoft Confidential 88
- 77. Global Learning and Development Chunk Level Deduplication • Act of breaking files into chunks • Identify identical chunks • Store only one copy of each chunk • Maintaining an in-file reference set of chunks for that file • Superset of File Level Duplication • Can occur at fixed or variable chunk sizes • Smaller, variable chunks = high optimization & overhead • Large fixed chunks = Lower optimization & overhead Microsoft Confidential 89
- 78. Global Learning and Development Windows Server 2012 Deduplication • Reduces the amount of space used – Segments Unnamed Data Streams – Uses variable-sized chunks (32-128KB) – Identifies duplicate chunks in other files – Maintains a single copy of each chunk – Redundant copies are replaced with a reference – Chunks are then compressed for further optimization • Files are no longer stored as independent streams • Replaced with stubs that point to data blocks • Data blocks are in a common chunk store Microsoft Confidential 90
- 79. Global Learning and Development Windows Storage without Deduplication • The data attribute ($DATA) defines a stream • Stream = content of data • For most files, the stream is unnamed – Unnamed Data Stream – Primary Data Stream • Can have an Alternate Data Stream (ADS) • Metadata contains Attributes Microsoft Confidential 91
- 80. Global Learning and Development Pre-Deduplication • Example shows what two files look like before deduplication Microsoft Confidential 92
- 81. Global Learning and Development Identify the Chunks that are Identical Microsoft Confidential - For Internal Use Only 93 7/30/2011 Attributes Metadata A B C Attributes Metadata M N O Unnamed Data Stream A B C X Y Z Unnamed Data Stream
- 82. Global Learning and Development Move Data Chunks into the Chunk Store Microsoft Confidential - For Internal Use Only 94 7/30/2011 Attributes Metadata Attributes Metadata Unnamed Data Stream A B C X Y Z Unnamed Data Stream A B C M N O Metadata Attributes Sparse Reparse Point Metadata Attributes Sparse Reparse Point
- 83. Global Learning and Development Post Deduplication • Data streams moved chunk store. • Reparse point is created that points to the chunk store • Stream is marked as sparse to indicate that the stream is segmented. Microsoft Confidential 95
- 84. Global Learning and Development Quick Facts 1 • Feature is not installed by default • Server only feature • Not supported on – FAT Volumes – Boot Volumes – System Volumes – Remote Drives – Cluster Shared Volumes – ReFS – 3rd Party File Systems • Does support cluster configurations Microsoft Confidential 96
- 85. Global Learning and Development Quick Facts 2 • Deduplication skips the following file types – Encrypted Files – Files with Extended Attributes – Files < 64KB – Files with reparse points • Deduplicated volumes can be – Backed up optimized – Restored optimized Microsoft Confidential 97
- 86. Global Learning and Development Deduplication Risks and Mitigations • Many files can reference the same physical sectors • Even with RAID/Backups, physical sectors can go bad • A small corruption can now affect many files – not just one Microsoft Confidential 98
- 87. Global Learning and Development Deduplication Risk Mitigation • Detection: Checksum validation on all data and metadata • Redundancy: Extra copies of all metadata and popular data chunks • Reporting: Scrubbing job inspects and reports on logged corruptions and can do additional/full scanning • Repair: – Done from redundant copies – From Storage Spaces mirror – From future data (i.e. new duplicate chunk with same hash) Microsoft Confidential 99
- 88. Global Learning and Development Chunk Store • Stored in ‘System Volume Information’ (Like VSS) • Includes – Logs – Settings – State Information Microsoft Confidential 10 0
- 89. Global Learning and Development Performance and Scalability • ZERO impact to newly written files • Copying large files 1-1.5x normal copy times • Multiple volumes processed in parallel – Uses additional CPU, memory, disk resources Microsoft Confidential 10 1
- 90. Global Learning and Development Storage Management • Module 1: Lesson 7 • Windows Server 2012 & Windows 8 Storage and File Systems Microsoft Confidential
- 91. Global Learning and Development Improvement to Storage Management • Extensible WMI-based API for management tasks • Powershell cmdlets for scripting and administration • Storage Management Provider (SMP) – New interface for storage vendors to support Windows storage • In Box SMP’s – Windows Standards-Based Storage Management Service – Storage Spaces • Microsoft intends to deprecate VDS – No new storage management tools in Win2012 utilize VDS Microsoft Confidential 10 3
- 92. Global Learning and Development Storage Management – Previous Versions • Virtual Disk Service used • Disk Management Console (GUI) • DiskPart.exe • DiskRaid.exe Microsoft Confidential 10 4
- 93. Global Learning and Development Current Storage Management Problems • DiskPart not designed as a scriptable management tool – Required separate script file to be available for deployment • VDS did not use Industry standards – Customer VDS providers had to be used • Limited powershell support Microsoft Confidential 10 5
- 94. Global Learning and Development Simplifying Storage Management Storage Vendors now have two choices with Windows SMAPI • Follow SMI-S specification – No additional software needed – SMAPI will discover storage and have ability to manage it • Create an new Storage Management Provider (SMP) – Similar to what was done with VDS with a VDS provider Microsoft Confidential 10 7
- 95. Global Learning and Development Powershell Cmdlets available • Windows Server 2012 - 2300+ new PowerShell commands • Most storage management tasks can now be completed with PowerShell • Diskpart – Get-Disk – Get-Partition – New-Partition – Initialize-Disk – Resize-Partition – Get-Volume – Format-Volume – Get-Physical Disk • CHKDSK - Repair-Volume Microsoft Confidential 10 8
- 96. Global Learning and Development LAB 11 0 • Lab 5 Deduplication (Optional)
Editor's Notes
- Original Content provided by Microsoft Support Services, Brent Caskey and Chris Davis – PFE (chdavis@microsoft.com)
- This module explains the storage system additions and changes to Windows Server 2012
- This lesson explains the new functionality in Windows Server 2012 called Storage Spaces
- Available to enterprise but definitely a mid-market and small market type of feature.
- Loosely based off Windows Home Server Drive Extender
- “Just in time” Add usb drives and add quasi-raid Disks don’t necessarily have to be there to be part of the pool
- USE: ==== Pooling Storage pool model is common, which makes it a familiar concept for admins Minimum amount of input necessary for creating a pool Possible to create pools on commodity drive arrays, also known as a JBOD (Just a Bunch of Drives) Provides sophisticated storage features that were previously available only on expensive storage hardware. One or more physical drives that will participate in the pool NOTE: Certain features, such as mirroring, require more physical drives. Scalable Aggregating physical drives into storage pools reduces the number of objects administrators have to manage Able to scale up to aggregate many physical drives Admins can choose to have as few or as many pools as they desire Ease of Management Easily to configure and manage Control panel applet in the Windows “8” File and Storage Services GUI in Windows Server 2012 PowerShell available in both the client and server SKUs Ability to list participating drives on a per pool basis as well as all drives participating in any pool and query their status. Flexible Control Administration can be defined and delegated on a per-pool basis Storage Spaces are fully integrated with Active Directory and the Windows Security model, thus providing a familiar model and tools for administration Hardware Compatibility Storage Spaces works well with this broad range of devices Storage Spaces also permits drives of different size in the same pool You can use with any physical drive that otherwise works with Windows, connected via USB, SATA, or SAS For Storage Spaces to use enclosure capabilities, the enclosure must conform to the Windows logo requirements Enclosure vendors are aware of these requirements and we anticipate increasing conformance over time. Application Compatibility Designed to be compatible with existing backup-restore and cloning tools as well as snapshotting infrastructures. Resilient Storage Storage Spaces delivers storage resiliency through a number of mechanisms including: Simple: Striping Mirror: Two-way and three-way mirroring Parity: Striping with parity Flexible Redundancy Repair Automatic redundancy repair of the affected spaces, provided the pool has one or more hot spare drives or the pool has sufficient storage capacity present Supports the ability to designate specific drives in the pool as a “hot spare”
- Deck self explanatory but: === One physical drive is required to create a storage pool; a minimum of two physical drives to create a resilient mirror storage space A minimum of three physical drives are required to create a storage space with resiliency through parity or three-way mirroring Drives have at least 10 GBs of blank, unformatted, and unpartitioned space Drives can be attached using a variety of bus interfaces including: SATA: Serial Advanced Technology Attachment SCSI: Small Computer System Interface SAS: Serial Attached SCSI (only cluster option) USB: Universal Serial Bus It was decided that storage spaces on Fibre Channel and iSCSI would be unsupported configurations
- If any of these are off or broken, pool is broken (but then so is everything else) === Windows Operating System Storage Stack: Storage must be properly presented and detected by the port driver, StorPort.SYS File and Storage Services Interface: The File and Storage Services interface is available by default because Storage Services are installed by default in Windows Server 2012 Storage Spaces: An inbox storage subsystem that allows physical drives to be pooled and managed by Windows Storage PowerShell cmdlets: An extensive set of PowerShell commands built on top of the WMIv2 that allow administrators and developers full control over storage subsystems, both Storage Spaces and 3rd-party subsystems Storage Management API (SMAPI): An API that provides storage management in Windows Server 2012. Storage vendors can plug into this model either by: Implementing the SNIA (Storage Networking Industry Association) industry standard for storage management called SMI-S (Storage Management Initiative –Specification) Implementing a new provider model called the Storage Management Provider (SMP) Storage Management Provider (SMP): An inbox capability that enables standards-based storage management in Windows through the use of industry-standard providers
- Check in the box….not much to do… This is default OOB config.
- Animated – drives to letter
- As an example, suppose a drive array (e.g. Just a Bunch of Drives (JBOD)) is connected to Windows Server 2012 and four 100 Gigabyte LUNs are presented. The Windows storage stack along with the requisite third party drivers to support connectivity to the storage solution allows for proper detection of the drives. No partitioning is required for the disks. If any disk is partitioned, Storage Spaces will only use it if there is at least 10 GBs of unpartitioned space. Opening Device Manager and expanding Disk Drives, the four disks are verified as being properly registered with the operating system.
- To create a storage pool, use either the File and Storage Services interface or PowerShell cmdlet, New-StoragePool Example consists of four 100 GB LUNs (Physical Disks) This new pool is then used to create a Virtual Disk, which is also called a Storage Space. Note: One or more storage spaces can be configured in a pool.
- When you add disks to a pool, SpacePort jumps in the middle, which is why disks disappear from diskmgmt. Spaceport adds “Disk Virtualization” and makes thin provisioning possible (among other things) === The spaceport driver communicates with the inbox hardware provider (smspace.dll) to manage the Storage Spaces subsystem.
- Listed under Storage Controllers
- Protective Partition: protective mechanism on the drive that says: “I’m part of a pool” so windows knows it when drive is plugged in Holds parity information (if part of that type of pool)
- Three way mirror may be the only new concept to most here
- Note: This topic will be covered in a later section
- Redundancy and Provisioning information is displayed in the File and Storage Services interface for each storage space that is created in a pool.
- Call Generator: all drives in a pool MUST use the same sector size So if you add a 4K drive you can’t then add a 512K drive. For instance, as 3TB drives become more prevalent, this will cause confusion “This drive type is not compatible with existing storage”
- Disk in storage spaces may not correlate to physical storage locations. So what happens if physical three reports as failed?
- The following are helpful cmdlets to have for tshooting
- Don’t use reset-physicaldisk on a drive in a pool PowerShell isn’t smart enough to stop you It will usually kill the whole pool (especially in parity)
- The official wording on Storage Spaces is, “It is not recommended that you configure storage spaces on your Dell hardware as it is currently not supported. Using storage spaces on unsupported hardware increases the risk of data loss because hardware failure notifications and hardware failover may not occur as expected.” This statement is already included in our IIG manual on dell.com/ostechsheets. Also included in that doc already is the statement on SW RAID support (or lack thereof), “NOTE: Dell software RAID S100 and S300 are not supported with Windows Server 2012.” The driver wiki also includes these statements. Our WS2012 Service portal went live on Monday. You can find it at http://support.dell.com/support/topics/global.aspx/support/windows_server_support/windows-server?c=us&cs=04&l=en&s=bsd or by clicking on the Windows link on the left hand side of the SDC main support page. In another couple of weeks, the page will also be available via a vanity URL of www.dell.com/support/Windows-Server and support.dell.com/windows-server
- EQL/EMC have been doing this for years, but it is now much cheaper to do
- Capacity planning (which we drilled into IT’s head) is now costing them. Drives sit around sometimes for years without anything to do but spin ======
- OS Level/SAN Level Example of Thin overall, not example of thin in windows =======
- This is even more critical for companies that are doing 'chargeback'. That is, they are charging their end customers for storage space. NOTE: Previous versions of Windows were unaware that Thin Provisioning at the SAN level was in play. Here is another way of looking at it. The charts below show an example of space provisioned to five servers.
- How does Windows stand up to a SAN on thin? Well, some benefits Think big VHD file, delete the VM? (next slide more info) =======
- Windows Server 2012 adopted T10 SCSI Block Command 3 spec for identifying thin provisioned LUNs. During the initial target device enumeration, Windows will gather all property parameters from the target device. Windows will attempt to identify the provisioning type and UNMAP/TRIM capability by querying the storage device. The storage device shall report its provisioning type and UNMAP/TRIM capability according to SBC3 spec. If the storage device fails to report its capabilities, users could run into some device compatibility issues. For example, the storage device reports UNMAP command is supported, but it does not support UNMAP command. A disk format hang issue could happen due to the unsupported UNMAP command. With the provisioning type, the Windows storage stack will be able to provide a better IO handling.
- Reclaim the whitespace just like on a thin VM drive or on an offline defrag for AD How? (next slide) ====
- “This is why we did it” Issues with SOME HW =======
- As a result of those: Won’t work on C: either in most cases. =====
- Also: mention block level storage, VHD host side
- Support for 4K sector drives A 4k sector drive is a hard drive that has 4096 bytes per sector rather than the long time standard of 512 bytes. There are a couple reasons for this increase in sector size. Drive Size Reliability
- Basically, VolSnap now uses VSS instead of its own technology. Faster AutoRecovery Snapshot (next slide) Less chance of corrupting AD or SQL Quiesces the app itself =========== Previous Versions Previously the iSCSI Target allowed snapshots to be taken of virtual disks and rollback to those snapshots later. When the size of a virtual disk is big, the copy operation could take a long time. The iSCSI Target used a bitmap to track dirty blocks to optimize rollback speed. When a block was first touched since a snapshot, the corresponding bit is set in the bitmap. During rollback, only dirty blocks need to be copied from snapshot. Win2012 In Windows Server 2012 the iSCSI Target snapshot is built on VSS snapshots instead. Since Volsnap already provides bitmaps to track changes between adjacent snapshots, the iSCSI Target now leverages these bitmaps. Removes the need for its own bitmap tracking. Previously, the iSCSI Target tracked changes in blocks of 64KB, while Volsnap delta bitmap tracked change in blocks of 16KB. By leveraging Volsnap delta bitmap, iSCSI Target can rollback dirty blocks at a more granular level. Block Size: Previously, the iSCSI Target tracked changes in blocks of 64KB, while Volsnap delta bitmap tracked change in blocks of 16KB. By leveraging Volsnap delta bitmap, iSCSI Target can rollback dirty blocks at a more granular level.
- Some databases/datastores (SQL, AD, VHD) must do updates on a snap after it is taken. Much less chance of corruption. === Microsoft iSCSI Software Target allows initiators to take application consistent virtual disk snapshots. Some limitations existed in the older versions. They did not support taking snapshots that require auto-recover. Some applications, such as SQL and Hyper-V, need to perform updates on a snapshot after it is taken and before it is sealed. Example: An incomplete transaction would be rolled back to bring the snapshot into a consistent state. This update phase is called auto-recovery. Without auto-recovery support, customers cannot get the full benefit of the Hyper-V host backup. The iSCSI Target plays heavily in the Hyper-V environment, so it is important to add auto-recovery support.
- Only major point here: SMSI-S is an open development platform. API Dev.
- GUI or PowerShell
- - WinTarget down? iSCSI busted (more info next slide) - For the rest: Just cover terminology and concept What is a target, what’s an initiator
- Main point here, service is restartable And it is a local system account ======== The feature requires having an always-on service, ready to process incoming connection from iSCSI Initiators. Implemented as a service-host based service, and it will be running in a separate group, called 'BlockStorageServices'. The service will run in the LocalSystem account. The high privilege is required to open storage devices. The service can be stopped and started without requiring a reboot.
- File Name Description WtLmDrv.sys This is the Local mount StorPort minport driver. The driver is a root-enumerated device that offers an IOCTL based control interface. The driver surfaces a VHD file as a disk for read/write MicrosoftiSCSITarget.psd1 The PowerShell module metadata description for the cmdlet Microsoft.Iscsi.Target.Commands.dll The managed-code binary for the PowerShell cmdlet Microsoft.Windows Server.dll A managed code helper library for PowerShell cmdlet WtClusRes.dll The cluster resource DLL for the iSCSI Target iSCSITgt.dll The Main binary implementing the WinTarget service. This is a svchost.exe based DLL which implements ServiceMain. WinTargetps.dll MIDL generated proxy stub DLL WTVDSProv.dll The virtual disk service provider WSnapshotProvider.dll The volume snapshot service provider StrgPrvdMgmt.dll The WMIv1 provider used to configure the VDS/VSS providers
- Eliminates common points of corruption No longer needed Point out difference in verbiage “unable to assign virtual DISK to multiple targets” this does not mean you can’t assign multiple initiators to one target
- breeze
- Quick: old – 254 I/O’s per LUN, 1,000 per adapter New: no limits ======== The old version of StorPort has the following limitations with outstanding i/o: 254 per LUN 1000 per adapter Some HBAs are capable of handling large number (more than 1000) of outstanding i/o. To increase IO scalability, StorPort has been enhanced to allow more outstanding i/o's on both fronts. The StorPort now provides a new interface to query miniport driver for the maximum number of outstanding i/o's and use the number reported by miniport as the limit. For the miniport that does not support this new StorPort interface, the previous limits will be used as defaults. The StorPort now provides a new interface to query miniport driver for the maximum number of outstanding i/o's and use the number reported by miniport as the limit. For the miniport that does not support this new StorPort interface, the previous limits will be used as defaults. The maximum outstanding i/o's reported by an adapter will dependent on the adapter, as there is not a well-defined new maximum value. When an adapter reports a very large (e.g. 10,000) number of outstanding IOs, the required system resource can potentially affect the overall system performance.
- Just means larger capacities
- Just means FAST (seconds instead of many seconds or minutes for enumeration)
- System worker thread interface to miniport driver As storage hardware capabilities continued to increase, vendors asked for miniport driver ability to schedule work on system worker threads. To meet these needs, StorPort will provide a system worker thread interface to the miniport driver. To increase StorPort's ability to support new HW functionality, the following was added to the StorPort interface: To enhance HW caching optimization, there are four new flags in the IRP structure. Paging Sequential IO Temporary Write Through Spinlock improvements The older version of StorPort only allowed miniport drivers the ability to simply make a request for spinlock. The new StorPort has improved synchronization support by improving miniport robustness and scalability running when using multi-threaded code. Multi-timers for miniport driver The previous versions of StorPort only had support for one timer. The latest miniport drivers need more timers for operations such as enumeration, timeout, and flush cache. Providing multi-timer to miniport driver can reduce the complexity of miniport code for multiplexing of timers. Larger CDB and Bi Direction Comand Support Current SCSI spec includes commands with more than 16 byte CDB (Command Description Blocks) and Bi-directional functionality. Larger CDBs are defined for FCP(Fibre Channel Protocol) and XOR (needed for hardware RAID6) operations by an adapter or device. Older StorPort version did not support SCSI commands with more than 16 bytes CDB. The Windows Server 2012 version removes this limitation. With HW that capable of supporting bi-direction command, user can expect higher throughput. StorPort will provide the support to pass bi-direction SCSI command down to miniport driver.
- Called data optimization, but it really is dedupe Call out the NETWORK part 2012-2012 only, has to have data optimization on both side
- Compression: Encoding a unit of data using fewer bits. Can be lossless as in ZIP or LZH formats. Or it can be lossy, as in JPEG or MP3 formats where data is actually removed. File-level deduplication: This is also known as Single Instance Storage (SIS). This is the act of identifying identical files within a data set, storing only one copy of the file and replacing all instances of the file with references to that copy. A list of the optimized files is stored in the SIS database. Chunk-level deduplication: The act of breaking files into chunks, identifying identical chunks, storing only one copy of each chunk, and maintaining in-file references to the set of chunks which comprise that file. Chunk-level deduplication is a superset of File-level deduplication and generally provides higher optimization ratios. Chunk-level deduplication can occur at fixed or variable chunk sizes. Generally, smaller and variable chunk sizes lead to higher optimization rates both coming at the cost of additional processing overhead.
- Just like DFS-R Remote Diff Compression But 32- to 128 K (DFS-R is 64-K) *more info next slide on this ====
- Key points – compressed for further optimization
- Attr – filename, acl’s, etc Stream is the data
- Attrs like acl’s. meta. File name, etc. Gets added to metadata that it is a deduped file
- CSV and ReFS will likely be in the future. Emphasize cluster. ====
- Reparse points because this is what is used to make it work Backup and restore will backup and restore the reparse points, so the backup isn’t bigger than the data size ===
- Not a perf bottlenecks, but other issues listed here. Perf is discussed in a coming slide.
- Redundant “chunk” gets created at a threshold (like 20 or so) to reduce the potential impact of losing a chunk. Can pull it back from parity data when detected. 700=36 ====
- Similar to dfs-r and vss ====
- this is usually tier-two (docs) storage, not your databases and LOB. Probably a good time to talk about SAN dedupe (cost/efficiency/get-what-you-pay-for) ===
- This lesson explains the new functionality in Windows Server 2012 called Storage Spaces
- If the hardware is built on industry standards, all API - no need for them to develop an interface. We intend to get rid of VDS More info in slide 102 (or thereabouts) ====== The improvements to storage management capabilities include: A comprehensive and extensible WMI-based API, enabling common storage management operations An associated set of PowerShell cmdlets, enabling scriptable and remote administration A new interface – the Storage Management Provider (SMP) framework – enabling storage vendors to easily support Windows based storage management for their storage subsystems An in-the-box SMP titled “Windows Standards-Based Storage Management Service” which supports those storage vendors who prefer to implement SMI-S based providers An in-the-box SMP titled “Storage Spaces” which delivers resiliency and availability with commodity storage hardware. Storage vendors who desire to ensure Windows-based management of their storage subsystems can now make use of Windows Storage Management API. Microsoft intends to deprecate VDS, associated APIs, and certain associated tools/utilities. Additionally, none of the new storage management tools/utilities in Windows Server 2012 (or subsequent versions) will utilize VDS.
- Virtual Disk Service: http://msdn.microsoft.com/en-us/library/windows/desktop/bb986750(v=vs.85).aspx Disk Management Console: http://technet.microsoft.com/en-us/library/dd163558.aspx Diskpart: http://support.microsoft.com/kb/300415 http://technet.microsoft.com/en-us/library/cc766465(v=WS.10).aspx DiskRaid: http://technet.microsoft.com/en-us/library/cc770763(WS.10).aspx
- Have to load diskpart, the execute the script. Can’t script it directly. VDS was all MSFT, not industry standards – moving to public API. ====
- Big point here is open framework, good news for storage folks and manufacturers Choice: use the open std or make your own SMP ===
- Not comprehensive. But here are a lot of things in PS replacing diskpart. Mention slides are hidden with examples. Also mention there is a review that is also hidden. This is the last slide === http://blogs.msdn.com/b/san/archive/2012/07/03/managing-storage-with-windows-powershell-on-windows-server-2012.aspx
- Hidden Section – available if needed
- This lesson explains the new functionality in Windows Server 2012 called Storage Spaces