SlideShare a Scribd company logo
1 of 12
Spotlight Series

http://www.netapp.com/tech_library/3002.html   WAFL Overview
WAFL: Write Anywhere File Layout
              Filesystem for Improved Productivity



            Berkeley Fast File System/Veritas File System/NTFS/etc. –
            Writes to pre-allocated locations (data vs. metadata)




                                                                                                                  ...

            WAFL – No pre-allocated locations (data and metadata blocks
            are treated equally). Writes go to nearest available free block.

 1-2 MB
Cylinders

                                                                                                                   ...
            Writing to nearest available free block reduces disk seeking
            (the #1 performance challenge when using disks).

                                                                                                                                                     2
 July 05                  © Network Appliance 2005 - Redistribution outside of an authorized NetApp distributor or reseller to third parties prohibited withou
WAFL uses integrated RAID4

         RAID4 is similar to better known RAID5:
          –   RAID5: parity is distributed across all disks in the RAID group
          –   RAID4: parity is contained in a single disk in the RAID group


         Tradeoffs with the single parity disk RAID model:
          –   PRO: The RAID group can be instantly expanded by adding
              (pre-formatted) data disks.
          –   CON: The parity disk is perceived to be the ‘hot spot’ in the
              RAID group, due to intensive XOR parity calculations on it.




                                                                                                                                                      3
July 05                    © Network Appliance 2005 - Redistribution outside of an authorized NetApp distributor or reseller to third parties prohibited withou
WAFL eliminates the parity hot spot


    WAFL overcomes the ‘classic’ parity-disk hotspot issue, by
      the use of flexible write allocation policies:
          –   Writes any filesystem block to any disk location (data and
              meta data)*
          –   New data does not overwrite old data
          –   Allocates disk space for many client-write operations at once
              in a single new RAID-stripe write (no parity re-calculations)
          –   Writes to stripes that are near each other
          –   Writes blocks to disk in any order




                                                                                                      * except root inode

                                                                                                                                                    4
July 05                  © Network Appliance 2005 - Redistribution outside of an authorized NetApp distributor or reseller to third parties prohibited withou
Result: Minimal seeks and no hotspot

           Typical File System                                                              WAFL

                                     Long
          file1
                                     head                                                                          Short
                                     seeks                                                                         head
                                   especially                             file1
                                                                          file2                                   seeks
                                      on                                  file3
                                                                                                                  across
          file3                      parity                                                                         all
                                      disk                                                                         disks


          file2



            1 file at a time                                          Multiple files at once

                                                                                                                                                   5
July 05                 © Network Appliance 2005 - Redistribution outside of an authorized NetApp distributor or reseller to third parties prohibited withou
WAFL Combined with NVRAM



             WAFL uses NVRAM “consistency points” (NetApp’s flavor
              of journalling), thus assuring filesystem integrity and fast
              reboots.

             CP flush to disk occurs once every 10 seconds or when
              NVRAM reaches half full.

             NVRAM placement is at the file system operation level, not
              at the (more typical) block level. This assures self-
              consistent CP flushes to disk.

             No fsck!




                                                                                                                                                      6
July 05                    © Network Appliance 2005 - Redistribution outside of an authorized NetApp distributor or reseller to third parties prohibited withou
NVRAM placement is key!

          General-purpose                                                                      NetApp
             NV-RAM                                                                            NV-RAM



             TC P / o r                                                                       TC P / o r
 File
System       U D P /I P                                                                       U D P /I P
               NFS or                                                                            NFS or
                 C IF S                                                                           C IF S
             S e ma nti                                                     File              S e ma nti                     N VR A M
                                                                           System                    c
                    c
               W r it e                                                                         W r it e
             D Asl lk c
               i o                    N VR A M                                                D Asl lk c
                                                                                                i o
             D r iv e r                                                                       D r iv e r



          NVRAM safe-stores                                                  NVRAM safe-stores
           the disk blocks                                                    the FS operation
                                                                                                                                                     7
July 05                   © Network Appliance 2005 - Redistribution outside of an authorized NetApp distributor or reseller to third parties prohibited withou
Seek Example in a SAN environment

             Assume 4K disk blocks and 5 msec for one seek+rotate

             100MB/sec FC bandwidth x .005sec = .5MB worth of data
              blocks not sent on the FC channel during that seek

             .5MB x 1 block/4KB = 128 blocks not sent

             Therefore a 5ms seek for just 1 block equates to a 128
              block penalty

             Conclusion: one seek every 128 blocks or less ( ~1%)
              wastes at least half of your FC bandwidth!

                                    (seek 1 block)               128 blocks                                 (seek 1 block)
          128 blocks


                                                                                                                                                    8
July 05                  © Network Appliance 2005 - Redistribution outside of an authorized NetApp distributor or reseller to third parties prohibited withou
The protocol overhead issue


            Isn’t NAS slower than local disk?
          • Yes, we have TCP/IP overhead.
          • Yes, we have double-buffering overhead.
          • Yes, we might well have <obscure performance gotcha>.


          • Despite all that, we're able to improve performance, even
          with databases (now over 40% of NetApp customer base).


          • Clearly, we're doing *something* sufficiently right to
          make up for the overhead.



                                                                                                                                                   9
July 05                 © Network Appliance 2005 - Redistribution outside of an authorized NetApp distributor or reseller to third parties prohibited withou
The protocol overhead issue

          Keep the timing in perspective with today’s CPU speeds!
          • TCP/IP might seem to be a massive overhead, but passing
          packets up and down the stack turn out only to consume
          microseconds per request.


          (For example: 1Ghz CPU speed == 1 nanosecond clock cycle. So 1000 extra
                                            ____second clock cycle.
          CPU cycles for TCP stack = 1000x1ns = 1 microsecond)


          • Eliminating head seeks, which WAFL does better than any
          other file system thanks to its full integration with RAID,
TCP
over      saves whole milliseconds, eg, a 1000x savings.
head


                                     (5ms seek)                128 blocks                                    (5ms seek)
          128 blocks
                           TCP overhead is small by comparison

                                                                                                                                                     10
July 05                   © Network Appliance 2005 - Redistribution outside of an authorized NetApp distributor or reseller to third parties prohibited withou
NetApp Filers
           High speed, low latency

                         SFS97_R1 Performance - NFS v3 TCP
                 http://www.spec.org/cgi-bin/osgresults?conf=sfs97r1


                                                                                 • RAID protected
                                                                                 • Single file system



          F825          F880                       FAS940                           FAS960                           FAS960c




                                                                                                                                                11
July 05              © Network Appliance 2005 - Redistribution outside of an authorized NetApp distributor or reseller to third parties prohibited withou
Summary



          • WAFL extracts more ops/sec from a single drive due to
          minimum seeks.

          • More ops/sec equates to faster overall performance

          • WAFL’s “anywhere” property makes NetApp’s RAID-4
          the performance and scalability winner.

          • Fastest File System in the world with RAID enabled




                                                                                                                                             12
July 05           © Network Appliance 2005 - Redistribution outside of an authorized NetApp distributor or reseller to third parties prohibited withou

More Related Content

What's hot

Zero to Snowflake Presentation
Zero to Snowflake Presentation Zero to Snowflake Presentation
Zero to Snowflake Presentation Brett VanderPlaats
 
Apache Kafka Streams + Machine Learning / Deep Learning
Apache Kafka Streams + Machine Learning / Deep LearningApache Kafka Streams + Machine Learning / Deep Learning
Apache Kafka Streams + Machine Learning / Deep LearningKai Wähner
 
Data Protection in Transit and at Rest
Data Protection in Transit and at RestData Protection in Transit and at Rest
Data Protection in Transit and at RestAmazon Web Services
 
Introduction SQL Analytics on Lakehouse Architecture
Introduction SQL Analytics on Lakehouse ArchitectureIntroduction SQL Analytics on Lakehouse Architecture
Introduction SQL Analytics on Lakehouse ArchitectureDatabricks
 
Delta Lake OSS: Create reliable and performant Data Lake by Quentin Ambard
Delta Lake OSS: Create reliable and performant Data Lake by Quentin AmbardDelta Lake OSS: Create reliable and performant Data Lake by Quentin Ambard
Delta Lake OSS: Create reliable and performant Data Lake by Quentin AmbardParis Data Engineers !
 
A 30 day plan to start ending your data struggle with Snowflake
A 30 day plan to start ending your data struggle with SnowflakeA 30 day plan to start ending your data struggle with Snowflake
A 30 day plan to start ending your data struggle with SnowflakeSnowflake Computing
 
Building the Data Lake with Azure Data Factory and Data Lake Analytics
Building the Data Lake with Azure Data Factory and Data Lake AnalyticsBuilding the Data Lake with Azure Data Factory and Data Lake Analytics
Building the Data Lake with Azure Data Factory and Data Lake AnalyticsKhalid Salama
 
Snowflake: Your Data. No Limits (Session sponsored by Snowflake) - AWS Summit...
Snowflake: Your Data. No Limits (Session sponsored by Snowflake) - AWS Summit...Snowflake: Your Data. No Limits (Session sponsored by Snowflake) - AWS Summit...
Snowflake: Your Data. No Limits (Session sponsored by Snowflake) - AWS Summit...Amazon Web Services
 
NEW LAUNCH! Introducing Amazon Transcribe – Now in Preview - MCL215 - re:Inve...
NEW LAUNCH! Introducing Amazon Transcribe – Now in Preview - MCL215 - re:Inve...NEW LAUNCH! Introducing Amazon Transcribe – Now in Preview - MCL215 - re:Inve...
NEW LAUNCH! Introducing Amazon Transcribe – Now in Preview - MCL215 - re:Inve...Amazon Web Services
 
Building an Effective Data Warehouse Architecture
Building an Effective Data Warehouse ArchitectureBuilding an Effective Data Warehouse Architecture
Building an Effective Data Warehouse ArchitectureJames Serra
 
Real-Time Streaming: Intro to Amazon Kinesis
Real-Time Streaming: Intro to Amazon KinesisReal-Time Streaming: Intro to Amazon Kinesis
Real-Time Streaming: Intro to Amazon KinesisAmazon Web Services
 
Hyperspace for Delta Lake
Hyperspace for Delta LakeHyperspace for Delta Lake
Hyperspace for Delta LakeDatabricks
 
Best Practices for Data Warehousing with Amazon Redshift | AWS Public Sector ...
Best Practices for Data Warehousing with Amazon Redshift | AWS Public Sector ...Best Practices for Data Warehousing with Amazon Redshift | AWS Public Sector ...
Best Practices for Data Warehousing with Amazon Redshift | AWS Public Sector ...Amazon Web Services
 
Distributed Deep Learning with Apache Spark and TensorFlow with Jim Dowling
Distributed Deep Learning with Apache Spark and TensorFlow with Jim DowlingDistributed Deep Learning with Apache Spark and TensorFlow with Jim Dowling
Distributed Deep Learning with Apache Spark and TensorFlow with Jim DowlingDatabricks
 
ABD201-Big Data Architectural Patterns and Best Practices on AWS
ABD201-Big Data Architectural Patterns and Best Practices on AWSABD201-Big Data Architectural Patterns and Best Practices on AWS
ABD201-Big Data Architectural Patterns and Best Practices on AWSAmazon Web Services
 
Data Warehouse - Incremental Migration to the Cloud
Data Warehouse - Incremental Migration to the CloudData Warehouse - Incremental Migration to the Cloud
Data Warehouse - Incremental Migration to the CloudMichael Rainey
 
[DSC Europe 22] Lakehouse architecture with Delta Lake and Databricks - Draga...
[DSC Europe 22] Lakehouse architecture with Delta Lake and Databricks - Draga...[DSC Europe 22] Lakehouse architecture with Delta Lake and Databricks - Draga...
[DSC Europe 22] Lakehouse architecture with Delta Lake and Databricks - Draga...DataScienceConferenc1
 
The Connected Consumer – Real-time Customer 360
The Connected Consumer – Real-time Customer 360The Connected Consumer – Real-time Customer 360
The Connected Consumer – Real-time Customer 360Capgemini
 

What's hot (20)

Snowflake Datawarehouse Architecturing
Snowflake Datawarehouse ArchitecturingSnowflake Datawarehouse Architecturing
Snowflake Datawarehouse Architecturing
 
Zero to Snowflake Presentation
Zero to Snowflake Presentation Zero to Snowflake Presentation
Zero to Snowflake Presentation
 
Apache Kafka Streams + Machine Learning / Deep Learning
Apache Kafka Streams + Machine Learning / Deep LearningApache Kafka Streams + Machine Learning / Deep Learning
Apache Kafka Streams + Machine Learning / Deep Learning
 
Data Protection in Transit and at Rest
Data Protection in Transit and at RestData Protection in Transit and at Rest
Data Protection in Transit and at Rest
 
Introduction SQL Analytics on Lakehouse Architecture
Introduction SQL Analytics on Lakehouse ArchitectureIntroduction SQL Analytics on Lakehouse Architecture
Introduction SQL Analytics on Lakehouse Architecture
 
Delta Lake OSS: Create reliable and performant Data Lake by Quentin Ambard
Delta Lake OSS: Create reliable and performant Data Lake by Quentin AmbardDelta Lake OSS: Create reliable and performant Data Lake by Quentin Ambard
Delta Lake OSS: Create reliable and performant Data Lake by Quentin Ambard
 
A 30 day plan to start ending your data struggle with Snowflake
A 30 day plan to start ending your data struggle with SnowflakeA 30 day plan to start ending your data struggle with Snowflake
A 30 day plan to start ending your data struggle with Snowflake
 
Building the Data Lake with Azure Data Factory and Data Lake Analytics
Building the Data Lake with Azure Data Factory and Data Lake AnalyticsBuilding the Data Lake with Azure Data Factory and Data Lake Analytics
Building the Data Lake with Azure Data Factory and Data Lake Analytics
 
Snowflake: Your Data. No Limits (Session sponsored by Snowflake) - AWS Summit...
Snowflake: Your Data. No Limits (Session sponsored by Snowflake) - AWS Summit...Snowflake: Your Data. No Limits (Session sponsored by Snowflake) - AWS Summit...
Snowflake: Your Data. No Limits (Session sponsored by Snowflake) - AWS Summit...
 
NEW LAUNCH! Introducing Amazon Transcribe – Now in Preview - MCL215 - re:Inve...
NEW LAUNCH! Introducing Amazon Transcribe – Now in Preview - MCL215 - re:Inve...NEW LAUNCH! Introducing Amazon Transcribe – Now in Preview - MCL215 - re:Inve...
NEW LAUNCH! Introducing Amazon Transcribe – Now in Preview - MCL215 - re:Inve...
 
Building an Effective Data Warehouse Architecture
Building an Effective Data Warehouse ArchitectureBuilding an Effective Data Warehouse Architecture
Building an Effective Data Warehouse Architecture
 
Real-Time Streaming: Intro to Amazon Kinesis
Real-Time Streaming: Intro to Amazon KinesisReal-Time Streaming: Intro to Amazon Kinesis
Real-Time Streaming: Intro to Amazon Kinesis
 
Hyperspace for Delta Lake
Hyperspace for Delta LakeHyperspace for Delta Lake
Hyperspace for Delta Lake
 
Best Practices for Data Warehousing with Amazon Redshift | AWS Public Sector ...
Best Practices for Data Warehousing with Amazon Redshift | AWS Public Sector ...Best Practices for Data Warehousing with Amazon Redshift | AWS Public Sector ...
Best Practices for Data Warehousing with Amazon Redshift | AWS Public Sector ...
 
Distributed Deep Learning with Apache Spark and TensorFlow with Jim Dowling
Distributed Deep Learning with Apache Spark and TensorFlow with Jim DowlingDistributed Deep Learning with Apache Spark and TensorFlow with Jim Dowling
Distributed Deep Learning with Apache Spark and TensorFlow with Jim Dowling
 
ETL Process
ETL ProcessETL Process
ETL Process
 
ABD201-Big Data Architectural Patterns and Best Practices on AWS
ABD201-Big Data Architectural Patterns and Best Practices on AWSABD201-Big Data Architectural Patterns and Best Practices on AWS
ABD201-Big Data Architectural Patterns and Best Practices on AWS
 
Data Warehouse - Incremental Migration to the Cloud
Data Warehouse - Incremental Migration to the CloudData Warehouse - Incremental Migration to the Cloud
Data Warehouse - Incremental Migration to the Cloud
 
[DSC Europe 22] Lakehouse architecture with Delta Lake and Databricks - Draga...
[DSC Europe 22] Lakehouse architecture with Delta Lake and Databricks - Draga...[DSC Europe 22] Lakehouse architecture with Delta Lake and Databricks - Draga...
[DSC Europe 22] Lakehouse architecture with Delta Lake and Databricks - Draga...
 
The Connected Consumer – Real-time Customer 360
The Connected Consumer – Real-time Customer 360The Connected Consumer – Real-time Customer 360
The Connected Consumer – Real-time Customer 360
 

Viewers also liked

NATSP Professional Exam 1
NATSP Professional Exam 1NATSP Professional Exam 1
NATSP Professional Exam 1Mamdouh Etman
 
NetApp Data Centers Use Innovative Design to Improve Efficiency
NetApp Data Centers Use Innovative Design to Improve EfficiencyNetApp Data Centers Use Innovative Design to Improve Efficiency
NetApp Data Centers Use Innovative Design to Improve EfficiencyNetApp
 
CDW: SAN vs. NAS
CDW: SAN vs. NASCDW: SAN vs. NAS
CDW: SAN vs. NASSpiceworks
 
Webinar: Simplifying the Database Experience with MongoDB Atlas
Webinar: Simplifying the Database Experience with MongoDB AtlasWebinar: Simplifying the Database Experience with MongoDB Atlas
Webinar: Simplifying the Database Experience with MongoDB AtlasMongoDB
 
C cloud organizational_impacts_big_data_on-prem_vs_off-premise_john_sing
C cloud organizational_impacts_big_data_on-prem_vs_off-premise_john_singC cloud organizational_impacts_big_data_on-prem_vs_off-premise_john_sing
C cloud organizational_impacts_big_data_on-prem_vs_off-premise_john_singJohn Sing
 
MongoDB Europe 2016 - Deploying MongoDB on NetApp storage
MongoDB Europe 2016 - Deploying MongoDB on NetApp storageMongoDB Europe 2016 - Deploying MongoDB on NetApp storage
MongoDB Europe 2016 - Deploying MongoDB on NetApp storageMongoDB
 
Nutanix overview
Nutanix overviewNutanix overview
Nutanix overviewamoreland
 
Understanding Memory Management In Spark For Fun And Profit
Understanding Memory Management In Spark For Fun And ProfitUnderstanding Memory Management In Spark For Fun And Profit
Understanding Memory Management In Spark For Fun And ProfitSpark Summit
 
Twenty Years of Data
Twenty Years of DataTwenty Years of Data
Twenty Years of DataNetApp
 
NFS(Network File System)
NFS(Network File System)NFS(Network File System)
NFS(Network File System)udamale
 
Slides: Start Small, Grow Big with a Unified Scale-Out Infrastructure
Slides: Start Small, Grow Big with a Unified Scale-Out InfrastructureSlides: Start Small, Grow Big with a Unified Scale-Out Infrastructure
Slides: Start Small, Grow Big with a Unified Scale-Out InfrastructureNetApp
 
10 Good Reasons: NetApp for Automotive
10 Good Reasons: NetApp for Automotive10 Good Reasons: NetApp for Automotive
10 Good Reasons: NetApp for AutomotiveNetApp
 
De version netapp flash infographic_rev3
De version netapp flash infographic_rev3De version netapp flash infographic_rev3
De version netapp flash infographic_rev3NetApp_Germany
 
FAQ on Dedupe NetApp
FAQ on Dedupe NetAppFAQ on Dedupe NetApp
FAQ on Dedupe NetAppAshwin Pawar
 
Network Attached Storage (NAS)
Network Attached Storage (NAS)Network Attached Storage (NAS)
Network Attached Storage (NAS)sandeepgodfather
 
Common MongoDB Use Cases
Common MongoDB Use Cases Common MongoDB Use Cases
Common MongoDB Use Cases MongoDB
 

Viewers also liked (20)

NATSP Professional Exam 1
NATSP Professional Exam 1NATSP Professional Exam 1
NATSP Professional Exam 1
 
NetApp & Storage fundamentals
NetApp & Storage fundamentalsNetApp & Storage fundamentals
NetApp & Storage fundamentals
 
NetApp Data Centers Use Innovative Design to Improve Efficiency
NetApp Data Centers Use Innovative Design to Improve EfficiencyNetApp Data Centers Use Innovative Design to Improve Efficiency
NetApp Data Centers Use Innovative Design to Improve Efficiency
 
CDW: SAN vs. NAS
CDW: SAN vs. NASCDW: SAN vs. NAS
CDW: SAN vs. NAS
 
How inodes Work
How inodes WorkHow inodes Work
How inodes Work
 
Webinar: Simplifying the Database Experience with MongoDB Atlas
Webinar: Simplifying the Database Experience with MongoDB AtlasWebinar: Simplifying the Database Experience with MongoDB Atlas
Webinar: Simplifying the Database Experience with MongoDB Atlas
 
C cloud organizational_impacts_big_data_on-prem_vs_off-premise_john_sing
C cloud organizational_impacts_big_data_on-prem_vs_off-premise_john_singC cloud organizational_impacts_big_data_on-prem_vs_off-premise_john_sing
C cloud organizational_impacts_big_data_on-prem_vs_off-premise_john_sing
 
MongoDB Europe 2016 - Deploying MongoDB on NetApp storage
MongoDB Europe 2016 - Deploying MongoDB on NetApp storageMongoDB Europe 2016 - Deploying MongoDB on NetApp storage
MongoDB Europe 2016 - Deploying MongoDB on NetApp storage
 
Nutanix overview
Nutanix overviewNutanix overview
Nutanix overview
 
Understanding Memory Management In Spark For Fun And Profit
Understanding Memory Management In Spark For Fun And ProfitUnderstanding Memory Management In Spark For Fun And Profit
Understanding Memory Management In Spark For Fun And Profit
 
Nutanix
NutanixNutanix
Nutanix
 
Twenty Years of Data
Twenty Years of DataTwenty Years of Data
Twenty Years of Data
 
NFS(Network File System)
NFS(Network File System)NFS(Network File System)
NFS(Network File System)
 
Slides: Start Small, Grow Big with a Unified Scale-Out Infrastructure
Slides: Start Small, Grow Big with a Unified Scale-Out InfrastructureSlides: Start Small, Grow Big with a Unified Scale-Out Infrastructure
Slides: Start Small, Grow Big with a Unified Scale-Out Infrastructure
 
10 Good Reasons: NetApp for Automotive
10 Good Reasons: NetApp for Automotive10 Good Reasons: NetApp for Automotive
10 Good Reasons: NetApp for Automotive
 
File system
File systemFile system
File system
 
De version netapp flash infographic_rev3
De version netapp flash infographic_rev3De version netapp flash infographic_rev3
De version netapp flash infographic_rev3
 
FAQ on Dedupe NetApp
FAQ on Dedupe NetAppFAQ on Dedupe NetApp
FAQ on Dedupe NetApp
 
Network Attached Storage (NAS)
Network Attached Storage (NAS)Network Attached Storage (NAS)
Network Attached Storage (NAS)
 
Common MongoDB Use Cases
Common MongoDB Use Cases Common MongoDB Use Cases
Common MongoDB Use Cases
 

Similar to Wafl overview

Tier 2 net app baseline design standard revised nov 2011
Tier 2 net app baseline design standard   revised nov 2011Tier 2 net app baseline design standard   revised nov 2011
Tier 2 net app baseline design standard revised nov 2011Accenture
 
Ibm spectrum scale fundamentals workshop for americas part 4 spectrum scale_r...
Ibm spectrum scale fundamentals workshop for americas part 4 spectrum scale_r...Ibm spectrum scale fundamentals workshop for americas part 4 spectrum scale_r...
Ibm spectrum scale fundamentals workshop for americas part 4 spectrum scale_r...xKinAnx
 
Storage virtualization citrix blr wide tech talk
Storage virtualization citrix blr wide tech talkStorage virtualization citrix blr wide tech talk
Storage virtualization citrix blr wide tech talkSisimon Soman
 
Ibm spectrum scale fundamentals workshop for americas part 4 Replication, Str...
Ibm spectrum scale fundamentals workshop for americas part 4 Replication, Str...Ibm spectrum scale fundamentals workshop for americas part 4 Replication, Str...
Ibm spectrum scale fundamentals workshop for americas part 4 Replication, Str...xKinAnx
 
Vancouver bug enterprise storage and zfs
Vancouver bug   enterprise storage and zfsVancouver bug   enterprise storage and zfs
Vancouver bug enterprise storage and zfsRami Jebara
 
Using ZFS file system with MySQL
Using ZFS file system with MySQLUsing ZFS file system with MySQL
Using ZFS file system with MySQLMydbops
 
RAID--16112022-093218am-16022024-061222pm.pdf
RAID--16112022-093218am-16022024-061222pm.pdfRAID--16112022-093218am-16022024-061222pm.pdf
RAID--16112022-093218am-16022024-061222pm.pdfzainm7032
 
VDI storage and storage virtualization
VDI storage and storage virtualizationVDI storage and storage virtualization
VDI storage and storage virtualizationSisimon Soman
 
Distributed File System
Distributed File SystemDistributed File System
Distributed File SystemNtu
 
pnfs status
pnfs statuspnfs status
pnfs statusbergwolf
 

Similar to Wafl overview (20)

Zfs intro v2
Zfs intro v2Zfs intro v2
Zfs intro v2
 
DAS RAID NAS SAN
DAS RAID NAS SANDAS RAID NAS SAN
DAS RAID NAS SAN
 
Tier 2 net app baseline design standard revised nov 2011
Tier 2 net app baseline design standard   revised nov 2011Tier 2 net app baseline design standard   revised nov 2011
Tier 2 net app baseline design standard revised nov 2011
 
SoNAS
SoNASSoNAS
SoNAS
 
Ibm spectrum scale fundamentals workshop for americas part 4 spectrum scale_r...
Ibm spectrum scale fundamentals workshop for americas part 4 spectrum scale_r...Ibm spectrum scale fundamentals workshop for americas part 4 spectrum scale_r...
Ibm spectrum scale fundamentals workshop for americas part 4 spectrum scale_r...
 
Storage virtualization citrix blr wide tech talk
Storage virtualization citrix blr wide tech talkStorage virtualization citrix blr wide tech talk
Storage virtualization citrix blr wide tech talk
 
A32 Database Virtulization Technologies
A32 Database Virtulization TechnologiesA32 Database Virtulization Technologies
A32 Database Virtulization Technologies
 
Ibm spectrum scale fundamentals workshop for americas part 4 Replication, Str...
Ibm spectrum scale fundamentals workshop for americas part 4 Replication, Str...Ibm spectrum scale fundamentals workshop for americas part 4 Replication, Str...
Ibm spectrum scale fundamentals workshop for americas part 4 Replication, Str...
 
Ch18 system administration
Ch18 system administration Ch18 system administration
Ch18 system administration
 
Vancouver bug enterprise storage and zfs
Vancouver bug   enterprise storage and zfsVancouver bug   enterprise storage and zfs
Vancouver bug enterprise storage and zfs
 
Linux on System z – disk I/O performance
Linux on System z – disk I/O performanceLinux on System z – disk I/O performance
Linux on System z – disk I/O performance
 
File Fragmentation
File FragmentationFile Fragmentation
File Fragmentation
 
Using ZFS file system with MySQL
Using ZFS file system with MySQLUsing ZFS file system with MySQL
Using ZFS file system with MySQL
 
RAID--16112022-093218am-16022024-061222pm.pdf
RAID--16112022-093218am-16022024-061222pm.pdfRAID--16112022-093218am-16022024-061222pm.pdf
RAID--16112022-093218am-16022024-061222pm.pdf
 
Nycbsdcon14
Nycbsdcon14Nycbsdcon14
Nycbsdcon14
 
Posscon2013
Posscon2013Posscon2013
Posscon2013
 
VDI storage and storage virtualization
VDI storage and storage virtualizationVDI storage and storage virtualization
VDI storage and storage virtualization
 
Distributed File System
Distributed File SystemDistributed File System
Distributed File System
 
ZFS
ZFSZFS
ZFS
 
pnfs status
pnfs statuspnfs status
pnfs status
 

Recently uploaded

Unlocking the Potential of the Cloud for IBM Power Systems
Unlocking the Potential of the Cloud for IBM Power SystemsUnlocking the Potential of the Cloud for IBM Power Systems
Unlocking the Potential of the Cloud for IBM Power SystemsPrecisely
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machinePadma Pradeep
 
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr LapshynFwdays
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonetsnaman860154
 
Key Features Of Token Development (1).pptx
Key  Features Of Token  Development (1).pptxKey  Features Of Token  Development (1).pptx
Key Features Of Token Development (1).pptxLBM Solutions
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreternaman860154
 
Benefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksBenefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksSoftradix Technologies
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Allon Mureinik
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationSafe Software
 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitecturePixlogix Infotech
 
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptxMaking_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptxnull - The Open Security Community
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Enterprise Knowledge
 
Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024BookNet Canada
 
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024BookNet Canada
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...shyamraj55
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls
 
Pigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping ElbowsPigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping ElbowsPigging Solutions
 
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024BookNet Canada
 

Recently uploaded (20)

Unlocking the Potential of the Cloud for IBM Power Systems
Unlocking the Potential of the Cloud for IBM Power SystemsUnlocking the Potential of the Cloud for IBM Power Systems
Unlocking the Potential of the Cloud for IBM Power Systems
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machine
 
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
 
Key Features Of Token Development (1).pptx
Key  Features Of Token  Development (1).pptxKey  Features Of Token  Development (1).pptx
Key Features Of Token Development (1).pptx
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 
Vulnerability_Management_GRC_by Sohang Sengupta.pptx
Vulnerability_Management_GRC_by Sohang Sengupta.pptxVulnerability_Management_GRC_by Sohang Sengupta.pptx
Vulnerability_Management_GRC_by Sohang Sengupta.pptx
 
Benefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksBenefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other Frameworks
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC Architecture
 
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptxMaking_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024
 
Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
 
The transition to renewables in India.pdf
The transition to renewables in India.pdfThe transition to renewables in India.pdf
The transition to renewables in India.pdf
 
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
Pigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping ElbowsPigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping Elbows
 
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
 

Wafl overview

  • 2. WAFL: Write Anywhere File Layout Filesystem for Improved Productivity Berkeley Fast File System/Veritas File System/NTFS/etc. – Writes to pre-allocated locations (data vs. metadata) ... WAFL – No pre-allocated locations (data and metadata blocks are treated equally). Writes go to nearest available free block. 1-2 MB Cylinders ... Writing to nearest available free block reduces disk seeking (the #1 performance challenge when using disks). 2 July 05 © Network Appliance 2005 - Redistribution outside of an authorized NetApp distributor or reseller to third parties prohibited withou
  • 3. WAFL uses integrated RAID4  RAID4 is similar to better known RAID5: – RAID5: parity is distributed across all disks in the RAID group – RAID4: parity is contained in a single disk in the RAID group  Tradeoffs with the single parity disk RAID model: – PRO: The RAID group can be instantly expanded by adding (pre-formatted) data disks. – CON: The parity disk is perceived to be the ‘hot spot’ in the RAID group, due to intensive XOR parity calculations on it. 3 July 05 © Network Appliance 2005 - Redistribution outside of an authorized NetApp distributor or reseller to third parties prohibited withou
  • 4. WAFL eliminates the parity hot spot WAFL overcomes the ‘classic’ parity-disk hotspot issue, by the use of flexible write allocation policies: – Writes any filesystem block to any disk location (data and meta data)* – New data does not overwrite old data – Allocates disk space for many client-write operations at once in a single new RAID-stripe write (no parity re-calculations) – Writes to stripes that are near each other – Writes blocks to disk in any order * except root inode 4 July 05 © Network Appliance 2005 - Redistribution outside of an authorized NetApp distributor or reseller to third parties prohibited withou
  • 5. Result: Minimal seeks and no hotspot Typical File System WAFL Long file1 head Short seeks head especially file1 file2 seeks on file3 across file3 parity all disk disks file2 1 file at a time Multiple files at once 5 July 05 © Network Appliance 2005 - Redistribution outside of an authorized NetApp distributor or reseller to third parties prohibited withou
  • 6. WAFL Combined with NVRAM  WAFL uses NVRAM “consistency points” (NetApp’s flavor of journalling), thus assuring filesystem integrity and fast reboots.  CP flush to disk occurs once every 10 seconds or when NVRAM reaches half full.  NVRAM placement is at the file system operation level, not at the (more typical) block level. This assures self- consistent CP flushes to disk.  No fsck! 6 July 05 © Network Appliance 2005 - Redistribution outside of an authorized NetApp distributor or reseller to third parties prohibited withou
  • 7. NVRAM placement is key! General-purpose NetApp NV-RAM NV-RAM TC P / o r TC P / o r File System U D P /I P U D P /I P NFS or NFS or C IF S C IF S S e ma nti File S e ma nti N VR A M System c c W r it e W r it e D Asl lk c i o N VR A M D Asl lk c i o D r iv e r D r iv e r NVRAM safe-stores NVRAM safe-stores the disk blocks the FS operation 7 July 05 © Network Appliance 2005 - Redistribution outside of an authorized NetApp distributor or reseller to third parties prohibited withou
  • 8. Seek Example in a SAN environment  Assume 4K disk blocks and 5 msec for one seek+rotate  100MB/sec FC bandwidth x .005sec = .5MB worth of data blocks not sent on the FC channel during that seek  .5MB x 1 block/4KB = 128 blocks not sent  Therefore a 5ms seek for just 1 block equates to a 128 block penalty  Conclusion: one seek every 128 blocks or less ( ~1%) wastes at least half of your FC bandwidth! (seek 1 block) 128 blocks (seek 1 block) 128 blocks 8 July 05 © Network Appliance 2005 - Redistribution outside of an authorized NetApp distributor or reseller to third parties prohibited withou
  • 9. The protocol overhead issue Isn’t NAS slower than local disk? • Yes, we have TCP/IP overhead. • Yes, we have double-buffering overhead. • Yes, we might well have <obscure performance gotcha>. • Despite all that, we're able to improve performance, even with databases (now over 40% of NetApp customer base). • Clearly, we're doing *something* sufficiently right to make up for the overhead. 9 July 05 © Network Appliance 2005 - Redistribution outside of an authorized NetApp distributor or reseller to third parties prohibited withou
  • 10. The protocol overhead issue Keep the timing in perspective with today’s CPU speeds! • TCP/IP might seem to be a massive overhead, but passing packets up and down the stack turn out only to consume microseconds per request. (For example: 1Ghz CPU speed == 1 nanosecond clock cycle. So 1000 extra ____second clock cycle. CPU cycles for TCP stack = 1000x1ns = 1 microsecond) • Eliminating head seeks, which WAFL does better than any other file system thanks to its full integration with RAID, TCP over saves whole milliseconds, eg, a 1000x savings. head (5ms seek) 128 blocks (5ms seek) 128 blocks TCP overhead is small by comparison 10 July 05 © Network Appliance 2005 - Redistribution outside of an authorized NetApp distributor or reseller to third parties prohibited withou
  • 11. NetApp Filers High speed, low latency SFS97_R1 Performance - NFS v3 TCP http://www.spec.org/cgi-bin/osgresults?conf=sfs97r1 • RAID protected • Single file system F825 F880 FAS940 FAS960 FAS960c 11 July 05 © Network Appliance 2005 - Redistribution outside of an authorized NetApp distributor or reseller to third parties prohibited withou
  • 12. Summary • WAFL extracts more ops/sec from a single drive due to minimum seeks. • More ops/sec equates to faster overall performance • WAFL’s “anywhere” property makes NetApp’s RAID-4 the performance and scalability winner. • Fastest File System in the world with RAID enabled 12 July 05 © Network Appliance 2005 - Redistribution outside of an authorized NetApp distributor or reseller to third parties prohibited withou

Editor's Notes

  1. March 19, 2012 Earlier we talked about disk capacity increasing but disk access times not. NetApp’s patented file system is called WAFL which stands for Write Anywhere Layout. WAFL always writes to the nearest available free block as opposed to a preallocated location on disk…If you look at conventional file systems, such as the Veritas Fast File System, NTFS, or the Berkeley Fast File System which is used in Solaris and HP/UX, they all write to pre-allocated locations on disk meaning that lot’s of disk seeking must occur. With WAFL we write out a stripe and hopefully not even move the heads to write the next stripe (if free space in the same cylinder). If not, it’s hopefully just one head click away. Quite simply, we minimize head seeking as much as possible.
  2. RAID-3 typically uses a very small stripe width, sometimes as small as one byte per disk. The result: RAID-3 accesses all the disk in the group at one time, and can only execute one I/O request at a time. With RAID4, access to each disk becomes independent. The stripe size is sufficiently large that the majority of I/Os to the group will only affect a single disk. This allows the RAID-4 group to execute multiple I/O requests simultaneously (assuming they map to different member disks).
  3. RAID-3 typically uses a very small stripe width, sometimes as small as one byte per disk. The result: RAID-3 accesses all the disk in the group at one time, and can only execute one I/O request at a time. With RAID4, access to each disk becomes independent. The stripe size is sufficiently large that the majority of I/Os to the group will only affect a single disk. This allows the RAID-4 group to execute multiple I/O requests simultaneously (assuming they map to different member disks).
  4. March 19, 2012 15 92 48 Berkeley Fast File System (FFS) Assigns blocks to fixed disk locations, as physically close together as possible on a single disk, optimized for single-file-at-time access Apply it to NFS and the disk heads fly about madly Write Anywhere File Layout (WAFL) Writes blocks anywhere it finds convenient, close to the disk heads’ current positions The previous version of a changed block is not over-written (it’s either retained or marked free) WAFL then logically threads a single file’s current blocks by updating the block pointers – it’s easy to adjust the pointers in the “inode” The result: reduced disk seek/latency time* Figure 1 from http://www.netapp.com/tech_library/3002.html A tree of blocks
  5. March 19, 2012
  6. March 19, 2012 9 86 42 Unlike general purpose file systems, WAFL has an intimate understanding of its underlying physical disk configuration. WAFL caches write operations that come in from the network, and then optimizes by performing multiple write operations all together within the same RAID array stripe. The stripe is chosen based on its physical proximity to the location of the disk heads at the time of the operation. This behavior ensures that the single parity disk does not become a bottleneck within the system as it would typically do with a general purpose file system. It also allows WAFL to achieve excellent write performance, since the disk heads never have to seek very far to write client data. Fragmentation is also not a significant issue with WAFL, as data belonging to the same file is always written to adjacent locations within the stripe.
  7. March 19, 2012 9 86 42 Unlike general purpose file systems, WAFL has an intimate understanding of its underlying physical disk configuration. WAFL caches write operations that come in from the network, and then optimizes by performing multiple write operations all together within the same RAID array stripe. The stripe is chosen based on its physical proximity to the location of the disk heads at the time of the operation. This behavior ensures that the single parity disk does not become a bottleneck within the system as it would typically do with a general purpose file system. It also allows WAFL to achieve excellent write performance, since the disk heads never have to seek very far to write client data. Fragmentation is also not a significant issue with WAFL, as data belonging to the same file is always written to adjacent locations within the stripe.