The document discusses different types of storage mediums and file systems, and how fragmentation can occur. It explains that file systems use allocation methods to manage how file data is stored in blocks across storage devices. Common allocation methods include contiguous, non-contiguous, and indexed allocation. Fragmentation happens when file blocks are not stored contiguously, slowing access. Defragmentation reorganizes blocks to improve performance.
WAN & LAN Cluster with Diagrams and OSI explanationJonathan Reid
This document contains diagrams and explanations of LAN and WAN clusters. The LAN cluster diagram shows a server cluster connected to a management server and switches connecting nodes on different floors. The WAN cluster diagram shows servers spread across different geographic locations connected through fiber wires and routers. The document also explains the seven layers of the OSI model and the protocols used in each layer.
This document discusses file management concepts including files, file attributes, file operations, file types, file structure, and access methods. Key points include:
- Files represent named collections of related information stored on secondary storage.
- File attributes include name, identifier, type, location, size, protection, and time/date information.
- Basic file operations are creating, writing, reading, repositioning, deleting, and truncating files.
- File types include ordinary files, directory files, and special files which represent devices.
- File structure and access methods like sequential, direct, and indexed access determine how information is organized and retrieved from files.
RedisConf18 - Techniques for Synchronizing In-Memory Caches with RedisRedis Labs
The document describes techniques for synchronizing in-memory caches across multiple web servers using Redis. It discusses the problems with traditional in-process caches, such as data lag and inconsistency. The proposed solution uses Redis for the source of truth, with each web server maintaining an in-process cache. Hash slots are used to partition keys and publish updates via Redis pub/sub. When a key is requested, the server checks if its cache version is stale by comparing timestamps. This approach maintains consistency while minimizing network usage.
Apache Kafka is a fast, scalable, and distributed messaging system that uses a publish-subscribe messaging protocol. It is designed for high throughput systems and can replace traditional message brokers due to its higher throughput and built-in partitioning, replication, and fault tolerance. Kafka uses topics to organize streams of messages and partitions to allow horizontal scaling and parallel processing of data. Producers publish messages to topics and consumers subscribe to topics to receive messages.
The document discusses file systems and their components. It covers directory organization, allocation schemes, file attributes, operations, structures and access methods. It also compares different file systems like FAT, FAT32 and NTFS in terms of their compatibility, volume size limits, fault tolerance and other advantages/disadvantages.
This document discusses file systems and their components. It covers topics like file processing, file organizations and access methods, directories, mounting file systems, file protection, disk space allocation, interfaces between file systems and IOCS, file sharing semantics, reliability of file systems, and journaling file systems. The document provides details on how files are organized, stored, accessed and shared in operating systems.
Apache Kafka is a distributed messaging system that allows for publishing and subscribing to streams of records, known as topics, in a fault-tolerant and scalable way. It is used for building real-time data pipelines and streaming apps. Producers write data to topics which are committed to disks across partitions and replicated for fault tolerance. Consumers read data from topics in a decoupled manner based on offsets. Kafka can process streaming data in real-time and at large volumes with low latency and high throughput.
WAN & LAN Cluster with Diagrams and OSI explanationJonathan Reid
This document contains diagrams and explanations of LAN and WAN clusters. The LAN cluster diagram shows a server cluster connected to a management server and switches connecting nodes on different floors. The WAN cluster diagram shows servers spread across different geographic locations connected through fiber wires and routers. The document also explains the seven layers of the OSI model and the protocols used in each layer.
This document discusses file management concepts including files, file attributes, file operations, file types, file structure, and access methods. Key points include:
- Files represent named collections of related information stored on secondary storage.
- File attributes include name, identifier, type, location, size, protection, and time/date information.
- Basic file operations are creating, writing, reading, repositioning, deleting, and truncating files.
- File types include ordinary files, directory files, and special files which represent devices.
- File structure and access methods like sequential, direct, and indexed access determine how information is organized and retrieved from files.
RedisConf18 - Techniques for Synchronizing In-Memory Caches with RedisRedis Labs
The document describes techniques for synchronizing in-memory caches across multiple web servers using Redis. It discusses the problems with traditional in-process caches, such as data lag and inconsistency. The proposed solution uses Redis for the source of truth, with each web server maintaining an in-process cache. Hash slots are used to partition keys and publish updates via Redis pub/sub. When a key is requested, the server checks if its cache version is stale by comparing timestamps. This approach maintains consistency while minimizing network usage.
Apache Kafka is a fast, scalable, and distributed messaging system that uses a publish-subscribe messaging protocol. It is designed for high throughput systems and can replace traditional message brokers due to its higher throughput and built-in partitioning, replication, and fault tolerance. Kafka uses topics to organize streams of messages and partitions to allow horizontal scaling and parallel processing of data. Producers publish messages to topics and consumers subscribe to topics to receive messages.
The document discusses file systems and their components. It covers directory organization, allocation schemes, file attributes, operations, structures and access methods. It also compares different file systems like FAT, FAT32 and NTFS in terms of their compatibility, volume size limits, fault tolerance and other advantages/disadvantages.
This document discusses file systems and their components. It covers topics like file processing, file organizations and access methods, directories, mounting file systems, file protection, disk space allocation, interfaces between file systems and IOCS, file sharing semantics, reliability of file systems, and journaling file systems. The document provides details on how files are organized, stored, accessed and shared in operating systems.
Apache Kafka is a distributed messaging system that allows for publishing and subscribing to streams of records, known as topics, in a fault-tolerant and scalable way. It is used for building real-time data pipelines and streaming apps. Producers write data to topics which are committed to disks across partitions and replicated for fault tolerance. Consumers read data from topics in a decoupled manner based on offsets. Kafka can process streaming data in real-time and at large volumes with low latency and high throughput.
FTP is a protocol that allows files to be transferred between computers connected to the Internet or with different operating systems. It uses two TCP ports, with port 21 for sending commands to the FTP server and port 20 for transferring data. The FTP server listens on port 21 for connections and commands from clients, then uses a separate data port to send and receive files and data.
Spring Boot Persistence Best Practices - How to effectively shape the @OneToM...Anghel Leonard
Collection of best practices for Java persistence performance in Spring Boot applications. In this item, we discuss the best practices for shaping the one-to-many association.
Kafka's basic terminologies, its architecture, its protocol and how it works.
Kafka at scale, its caveats, guarantees and use cases offered by it.
How we use it @ZaprMediaLabs.
I have described all about linux OS starting from basics.
I guess this PPT will really be very very helpful for you guys.
This was one of the most appreciable PPT in my time when i presented it in my class.
The session discusses on how companies are using Apache Kafka & also covers under the hood details like partitions, brokers, replication.
About apache kafka: Apache Kafka is a distributed a streaming platform, Apache Kafka provides low-latency, high-throughput, fault-tolerant publish and subscribe pipelines and is able to process streams of events. Kafka provides reliable, millisecond responses to support both customer-facing applications and connecting downstream systems with real-time data.
Apache Kafka is a distributed publish-subscribe messaging system that allows for high-throughput, persistent storage of messages. It provides decoupling of data pipelines by allowing producers to write messages to topics that can then be read from by multiple consumer applications in a scalable, fault-tolerant way. Key aspects of Kafka include topics for categorizing messages, partitions for scaling and parallelism, replication for redundancy, and producers and consumers for writing and reading messages.
Q&A with Confluent Professional Services: Confluent Service Meshconfluent
No matter whether you are migrating your Kafka cluster to Confluent Cloud, running a cloud-hybrid environment or are in a different situation where data protection and encryption of sensitive information is required, Confluent Service Mesh allows you to transparently encrypt your data without the need to make code changes to you existing applications.
El documento describe los permisos de NTFS en Windows. Explica que NTFS almacena listas de control de acceso (ACL) con cada archivo y carpeta que contienen permisos para usuarios, grupos y equipos. También describe cómo los permisos se heredan de carpetas padres a subcarpetas y archivos, y cómo los permisos pueden cambiar al copiar o mover archivos entre carpetas.
This document provides an introduction to Apache Kafka, an open-source distributed event streaming platform. It discusses Kafka's history as a project originally developed by LinkedIn, its use cases like messaging, activity tracking and stream processing. It describes key Kafka concepts like topics, partitions, offsets, replicas, brokers and producers/consumers. It also gives examples of how companies like Netflix, Uber and LinkedIn use Kafka in their applications and provides a comparison to Apache Spark.
Fundamentals and Architecture of Apache KafkaAngelo Cesaro
Fundamentals and Architecture of Apache Kafka.
This presentation explains Apache Kafka's architecture and internal design giving an overview of Kafka internal functions, including:
Brokers, Replication, Partitions, Producers, Consumers, Commit log, comparison over traditional message queues.
The document describes the Linux file system hierarchy. It explains that the root of the hierarchy is / and then describes the purpose and contents of important directories like /bin, /boot, /dev, /etc, /home, /media, /mnt, /opt, /proc, /root, /sbin, /tmp, /usr, and /var. For example, it states that /bin contains common commands, /dev contains device files, and /home contains user directories.
Apache Kafka is a fast, scalable, and distributed messaging system. It is designed for high throughput systems and can replace traditional message brokers due to its better throughput, built-in partitioning for scalability, replication for fault tolerance, and ability to handle large message processing applications. Kafka uses topics to organize streams of messages, partitions to distribute data, and replicas to provide redundancy and prevent data loss. It supports reliable messaging patterns including point-to-point and publish-subscribe.
Direct Attached Storage - Information Storage and Management.pptxMithun B N
This ppt contains slides on DAS.
Direct – Attached storage (DAS) is a an architecture where storage connects directly to servers. Applications access data from DAS using block-level access protocols. DAS is ideal for localized data access and sharing in environments that have a small number of servers.
This document provides an overview of Linux fundamentals, including:
- The kernel acts as an interface between hardware and software, handling processes and resource allocation.
- The userland includes standard libraries that allow programs to communicate with the kernel.
- Files are organized in a hierarchy with directories like /home for user files, /etc for configurations, and /var for variable files.
- Commands like ls, grep, and find allow viewing and searching files, while pipes, redirection, and compression utilities manage file input/output.
Apache Kafka is a distributed streaming platform used for building real-time data pipelines and streaming apps. It provides a unified, scalable, and durable platform for handling real-time data feeds. Kafka works by accepting streams of records from one or more producers and organizing them into topics. It allows both storing and forwarding of these streams to consumers. Producers write data to topics which are replicated across clusters for fault tolerance. Consumers can then read the data from the topics in the order it was produced. Major companies like LinkedIn, Yahoo, Twitter, and Netflix use Kafka for applications like metrics, logging, stream processing and more.
The document discusses file management in operating systems. It covers topics such as the basic functions of a file system, different file organization techniques like sequential, direct and indexed sequential, file structures, file allocation methods like contiguous, linked and indexed, free space management using free lists or bitmaps, file access control, and backup techniques. File systems are responsible for organizing files and managing access to data on storage devices in a way that facilitates navigation and protects data from corruption or loss.
The document discusses file systems and different types of file systems. It provides information on what a file and file system is, the main components of a file system including directories and file allocation methods. It also summarizes the key differences between common file systems like FAT32, NTFS, and exFAT and describes their ideal usage cases. Different structures for organizing files in directories like single-level, two-level, and tree-structured directories are also covered along with the advantages of maintaining directories.
FTP is a protocol that allows files to be transferred between computers connected to the Internet or with different operating systems. It uses two TCP ports, with port 21 for sending commands to the FTP server and port 20 for transferring data. The FTP server listens on port 21 for connections and commands from clients, then uses a separate data port to send and receive files and data.
Spring Boot Persistence Best Practices - How to effectively shape the @OneToM...Anghel Leonard
Collection of best practices for Java persistence performance in Spring Boot applications. In this item, we discuss the best practices for shaping the one-to-many association.
Kafka's basic terminologies, its architecture, its protocol and how it works.
Kafka at scale, its caveats, guarantees and use cases offered by it.
How we use it @ZaprMediaLabs.
I have described all about linux OS starting from basics.
I guess this PPT will really be very very helpful for you guys.
This was one of the most appreciable PPT in my time when i presented it in my class.
The session discusses on how companies are using Apache Kafka & also covers under the hood details like partitions, brokers, replication.
About apache kafka: Apache Kafka is a distributed a streaming platform, Apache Kafka provides low-latency, high-throughput, fault-tolerant publish and subscribe pipelines and is able to process streams of events. Kafka provides reliable, millisecond responses to support both customer-facing applications and connecting downstream systems with real-time data.
Apache Kafka is a distributed publish-subscribe messaging system that allows for high-throughput, persistent storage of messages. It provides decoupling of data pipelines by allowing producers to write messages to topics that can then be read from by multiple consumer applications in a scalable, fault-tolerant way. Key aspects of Kafka include topics for categorizing messages, partitions for scaling and parallelism, replication for redundancy, and producers and consumers for writing and reading messages.
Q&A with Confluent Professional Services: Confluent Service Meshconfluent
No matter whether you are migrating your Kafka cluster to Confluent Cloud, running a cloud-hybrid environment or are in a different situation where data protection and encryption of sensitive information is required, Confluent Service Mesh allows you to transparently encrypt your data without the need to make code changes to you existing applications.
El documento describe los permisos de NTFS en Windows. Explica que NTFS almacena listas de control de acceso (ACL) con cada archivo y carpeta que contienen permisos para usuarios, grupos y equipos. También describe cómo los permisos se heredan de carpetas padres a subcarpetas y archivos, y cómo los permisos pueden cambiar al copiar o mover archivos entre carpetas.
This document provides an introduction to Apache Kafka, an open-source distributed event streaming platform. It discusses Kafka's history as a project originally developed by LinkedIn, its use cases like messaging, activity tracking and stream processing. It describes key Kafka concepts like topics, partitions, offsets, replicas, brokers and producers/consumers. It also gives examples of how companies like Netflix, Uber and LinkedIn use Kafka in their applications and provides a comparison to Apache Spark.
Fundamentals and Architecture of Apache KafkaAngelo Cesaro
Fundamentals and Architecture of Apache Kafka.
This presentation explains Apache Kafka's architecture and internal design giving an overview of Kafka internal functions, including:
Brokers, Replication, Partitions, Producers, Consumers, Commit log, comparison over traditional message queues.
The document describes the Linux file system hierarchy. It explains that the root of the hierarchy is / and then describes the purpose and contents of important directories like /bin, /boot, /dev, /etc, /home, /media, /mnt, /opt, /proc, /root, /sbin, /tmp, /usr, and /var. For example, it states that /bin contains common commands, /dev contains device files, and /home contains user directories.
Apache Kafka is a fast, scalable, and distributed messaging system. It is designed for high throughput systems and can replace traditional message brokers due to its better throughput, built-in partitioning for scalability, replication for fault tolerance, and ability to handle large message processing applications. Kafka uses topics to organize streams of messages, partitions to distribute data, and replicas to provide redundancy and prevent data loss. It supports reliable messaging patterns including point-to-point and publish-subscribe.
Direct Attached Storage - Information Storage and Management.pptxMithun B N
This ppt contains slides on DAS.
Direct – Attached storage (DAS) is a an architecture where storage connects directly to servers. Applications access data from DAS using block-level access protocols. DAS is ideal for localized data access and sharing in environments that have a small number of servers.
This document provides an overview of Linux fundamentals, including:
- The kernel acts as an interface between hardware and software, handling processes and resource allocation.
- The userland includes standard libraries that allow programs to communicate with the kernel.
- Files are organized in a hierarchy with directories like /home for user files, /etc for configurations, and /var for variable files.
- Commands like ls, grep, and find allow viewing and searching files, while pipes, redirection, and compression utilities manage file input/output.
Apache Kafka is a distributed streaming platform used for building real-time data pipelines and streaming apps. It provides a unified, scalable, and durable platform for handling real-time data feeds. Kafka works by accepting streams of records from one or more producers and organizing them into topics. It allows both storing and forwarding of these streams to consumers. Producers write data to topics which are replicated across clusters for fault tolerance. Consumers can then read the data from the topics in the order it was produced. Major companies like LinkedIn, Yahoo, Twitter, and Netflix use Kafka for applications like metrics, logging, stream processing and more.
The document discusses file management in operating systems. It covers topics such as the basic functions of a file system, different file organization techniques like sequential, direct and indexed sequential, file structures, file allocation methods like contiguous, linked and indexed, free space management using free lists or bitmaps, file access control, and backup techniques. File systems are responsible for organizing files and managing access to data on storage devices in a way that facilitates navigation and protects data from corruption or loss.
The document discusses file systems and different types of file systems. It provides information on what a file and file system is, the main components of a file system including directories and file allocation methods. It also summarizes the key differences between common file systems like FAT32, NTFS, and exFAT and describes their ideal usage cases. Different structures for organizing files in directories like single-level, two-level, and tree-structured directories are also covered along with the advantages of maintaining directories.
The document discusses file systems and how they organize files on storage devices. It covers key concepts like directories, file allocation tables (FAT), file attributes, and different file systems like FAT, FAT32, NTFS, and Ext3. It compares the capabilities and limitations of different file systems and how they are used depending on factors like hardware, software, security needs and disk size.
This document summarizes different file system structures and allocation methods. It describes the logical layers of a file system from the application to the physical devices. It then explains three common allocation methods: contiguous allocation, linked allocation, and indexed allocation. Contiguous allocation stores files in contiguous disk blocks but can lead to fragmentation. Linked allocation uses pointers but is unreliable and wastes space. Indexed allocation uses an index block to point to other data blocks, reducing fragmentation and pointer space waste. It analyzes the performance of reading a block from each type of allocation.
The document discusses various topics related to file systems, partitions, and the Windows boot process. It describes the FAT and NTFS file systems, how partitions and volumes are used to organize disk space, and the steps involved in booting Windows, including the roles of the boot loader, kernel, and services.
In a computer, a file system (sometimes written filesystem) is the way in which files are named and where they are placed logically for storage and retrieval.
The document discusses secondary storage management in operating systems. It addresses issues like file allocation, preallocation vs dynamic allocation, portion size, and different file allocation methods like contiguous, chained, and indexed allocation. It also covers free space management techniques like bit tables, chained free portions, indexing, and free block lists to track available blocks.
The document discusses file systems and their components. It describes how files are organized logically and mapped to physical storage. It covers key file system concepts like directories, file allocation schemes, file attributes, and file operations. It also summarizes common file systems like FAT, FAT32, and NTFS and compares their features such as supported drive sizes, cluster sizes, and compatibility with different operating systems.
A file system is used to control how data is stored and retrieved.
A filesystem is the methods and data structures that an operating system uses to keep track of files on a disk or partition; that is, the way the files are organized on the disk.
A file allocation table (FAT) is a table that an operating system maintains on a hard disk that provides a map of the clusters (the basic units of logical storage on a hard disk) that a file has been stored in.
File Allocation Table (FAT) is a computer file system architecture and a family of industry-standard file systems utilizing it. The FAT file system is a legacy file system which is simple and robust.
Today, FAT file systems are still commonly found on floppy disks, USB sticks, flash and other solid-state memory cards and modules, and many portable and embedded devices.
This document provides an overview of file systems and storage options in Windows Server 2016. It discusses the four main file systems - FAT, FAT32, NTFS, and ReFS. NTFS and ReFS are the two file systems supported by Windows Server 2016. ReFS provides advantages like availability, scalability, data integrity and application compatibility. It allows the system to continue functioning despite errors. NTFS also provides features like disk quotas, file encryption, dynamic volumes, self-healing and security. The document concludes by covering initializing disks using MBR or GPT partition styles.
This is a self made slide covering topics related to storage systems available in the market with certain information of file systems to understand the fundamentals.
There are also some information is available related to how the whole stuff works.
This document discusses managing disks and file systems in Windows. It covers monitoring and configuring disks, volumes, and removable media. Topics include partitioning basic disks, upgrading disks to dynamic disks, using disk management, NTFS compression and encryption, disk optimization utilities, and disk quotas. The Disk Management snap-in is used to manage local and remote disks, create volumes, and format disks. Basic disks contain partitions while dynamic disks use volumes. NTFS compression and encryption are incompatible. EFS encrypts data on NTFS volumes and its recovery agent's certificate must be protected.
A partition divides a hard drive into logical sections for storing files and installing operating systems. There are three types of partitions: primary, extended, and logical. A primary partition can host an operating system, while an extended partition contains logical drives. The master boot record (MBR) stores information on partition locations and boots the system, but is limited to 4 primary partitions under 2.2TB each. The GUID partition table (GPT) replaces MBR and supports over 18 exabytes per partition. GPT uses a protective MBR for compatibility with older systems.
Can be built on top of base methods
General involve creation of an index for the file
Keep index in memory for fast determination of location of data to be operated on (consider UPC code plus record of data about that item)
If too large, index (in memory) of the index (on disk)
IBM indexed sequential-access method (ISAM)
Small master index, points to disk blocks of secondary index
File kept sorted on a defined key
All done by the OS
VMS operating system provides index and relative files as another example .
The document discusses various topics related to secondary storage and file organization in databases:
1) Secondary storage devices like magnetic disks are used to permanently store large databases and provide high storage capacity compared to main memory.
2) Files are organized on disks using various methods like heap files, sorted files, and hashing to allow efficient retrieval, insertion, and deletion of records.
3) RAID (Redundant Array of Independent Disks) technology improves disk performance using data striping across multiple disks and reliability using disk mirroring.
OS_Assignment for Disk Space & File System & File allocation table(FAT)Chinmaya M. N
The document discusses file systems and the File Allocation Table (FAT). It provides information on:
1. What a file system does, including organizing files into directories, keeping track of file locations, and assisting users.
2. How the FAT works by keeping a record of all files and pointing to the clusters that contain the files.
3. The different types of FAT (FAT12, FAT16, FAT32) and their cluster size limits and maximum volume sizes.
4. How a FAT file system is created through high-level formatting of the disk, including creating boot sectors, cluster allocation, the file allocation table, and the root directory.
Course 102: Lecture 26: FileSystems in Linux (Part 1) Ahmed El-Arabawy
This lecture introduces some concepts about FileSystems in Linux.
Video for this lecture on youtube:
http://www.youtube.com/watch?v=9jj1QOokACo
Check the other Lectures and courses in
http://Linux4EnbeddedSystems.com
or Follow our Facebook Group at
- Facebook: @LinuxforEmbeddedSystems
Lecturer Profile:
Ahmed ElArabawy
- https://www.linkedin.com/in/ahmedelarabawy
The document provides an overview of file systems, including their purpose of organizing and storing information on storage devices. It discusses key aspects of file systems such as how they separate information into individual files and directories, use metadata to store attributes about files, allocate storage space in a granular manner (which can result in unused space), become fragmented over time, and use various utilities and structures to implement these functions while maintaining integrity of data and restricting access. File systems are a critical component of operating systems that allow for efficient organization, retrieval and updating of user data on different types of storage media and devices.
File Access & File System & File Allocation TableChinmaya M. N
The document discusses file systems and the File Allocation Table (FAT). It provides details on:
1. What a file system does, including creating, organizing, and tracking files and directories on a storage device.
2. How the FAT file system works using a file allocation table to point to clusters containing files. FAT12, FAT16, and FAT32 variations use different number of bits for cluster addresses.
3. The process of creating a FAT file system including formatting, creating boot sectors, cluster allocation, and the file allocation table and root directory.
1. Running head: STORAGE MEDIUMS AND FRAGMENTATION 1
Storage Mediums and Fragmentation
Larry Reid
Kaplan University
IT332-01 Principles of Information Systems Architecture
2. STORAGE MEDIUMS AND FRAGMENTATION 2
Storage Mediums and Fragmentation
There are several types of storage mediums and each of those mediums needs a file
system to be useful. A lot of file systems suffer from the effects of fragmentation and some
storage mediums become slow overtime needing to be defragmented. Sometimes it’s difficult to
wrap our heads around concepts like this but this article is meant to make it as painless as
possible to grasp these concepts and come to an understanding.
what are file system allocation methods?
A file system is what is used on a storage device to manage, maintain, and organize all of
the data on it. File systems are things such as the New Technology File System (NTFS) used by
Windows which stores data contiguously and uses indexing allocation, the File Allocation Table
32 (FAT32) and extended FAT (exFAT) which both use the File Allocation Table with linking,
the Hierarchical File System Plus (HFS+) used by Apple which is extent-based allocation with
indexing, and the Linear Tape File System (LTFS) developed by IBM for tape drives which also
has extent-based allocation with indexing. Are you wondering what file system allocation
methods are still? Well they are what explain how blocks are divided up and distributed each
time a file is saved, each allocation method has a different way of defining how addresses to
blocks are stored, some file systems only use one type of allocation method and others take the
best from multiple methods and put them together. File systems are rules or a blueprint that the
operating system follows in order to write, read, organize, and manage files all of which use
some type of table with addresses to sectors and blocks, some keep track of how many blocks the
file takes up, and others may be links to a block and then to another until they get to one that
doesn’t have a link, which means the whole file has been found.
Now let’s go over the different types of allocation. Contiguous allocation is method used
to store data in a sequential set of blocks. When you save a file using this allocation method,
3. STORAGE MEDIUMS AND FRAGMENTATION 3
your file system has to figure out how many blocks are needed to store the file, and it then has to
find free space on the disk with the amount of blocks needed, if it can’t find the space needed
then it will fragment the file and place it in two or more sets of contiguous blocks. Using this
method is the fastest way to access files for Hard Disk Drives (HDD) since the head does not
have to move all over the disk to find all of the blocks that a file is stored on (Evett, 2015).
Non-Contiguous allocation doesn’t care about allocating sequential blocks, they can be
all over the disk and the files will have links to the next block that comprises more of the file
until the last block which doesn’t contain a link, is reached, or it may store the file block
locations in a master list. Non-Contiguous allocation does not have external fragmentation but
does contain internal fragmentation (Englander, 2014).
Indexed allocation uses index tables to store all the block addresses for each file. Each
file gets its own index table typically stored in the first block allocated to the file. Additionally,
there is a file management system that is used to maintain a directory that houses information
about the files that can include attributes, the link locations, the pointer to the file, file name, etc.
Whey a file is opened the operating system will store the files index table, that houses the links
to all of the files blocks, in memory making it extremely fast to do random accesses to the file
and also very efficient with memory usage. Typically, this is a non-contiguous allocation method
but Windows NTFS allocates contiguous blocks as much as possible to save additional memory
(Englander, 2014).
Tape allocation is simpler than disk allocation. This is meant more for sequential reads
and not really feasible for random access. Tape drives can be compacted, similar to
defragmented, but it’s usually not worth the work, it would be easier to copy the data to another
tape. The Linear Tape File System is a great method to use for storing files on a tape drive, using
extent-based allocation along with indexing it allows the tape to almost act like a hard drive,
4. STORAGE MEDIUMS AND FRAGMENTATION 4
allowing for random read access. Though if the files need to be accessed more often, it would be
more efficient to move or copy the files to a disk drive. Tape drives are better to be used for
daily, weekly, or monthly backups (Englander, 2014).
Disk allocation and storage allocation has to do with how a disk is broken up into zones,
sectors, and blocks. So disks contain zones which are the tracks that contain the same number of
blocks as adjacent tracks, so as you move closer to the center of the disk the zones change. Next
we have sectors which are a section, shaped like a pie slice, that contain blocks, and blocks are 4
or 5 kilobyte sections on the disk. There are also OS blocks that are described by the operating
system, one block for the OS can contain multiple disk blocks. What happens in disk allocation
is the contiguous, non-contiguous, or whatever allocation method you are using, will allocate
these blocks to the free disk space manager on the operating system and the OS will use the free
disk space manager to allocate the disk blocks for storage (Dittrich, 2013).
Partitions are divisions created within a disk, you can have between one and three
primary partitions and one additional extended partition on a single simple disk volume. An
extended partition can have several logical partitions created within it, which is how we can get
around the four partition rule on hard disks. Additionally, we can create a dynamic disk which
allows us to create several logical volumes which are generally the same as a partition but
instead of the space residing on a single disk the logical volume can span across multiple disks.
A physical volume refers to a physical hard disk in its entirety (Indika, 2011).
Fragmentation can be broken up into two types, internal and external. Internal
fragmentation is the space reserved by the file system to store index tables, FAT tables, and any
other unused space that is set aside within a file, such as the reserved space for file expansion.
External fragmentation refers to how a file is saved to the disk blocks. If a file must be broken up
across the disk to be saved and is not saved in a contiguous manner, then it is considered
5. STORAGE MEDIUMS AND FRAGMENTATION 5
fragmented and the disk heads have to move, possibly, several times to write or read the file
creating slower file read/write times. Defragmentation takes all of the broken up non-contiguous
files and puts them in as much of a contiguous order as it can and then compresses all of the files
together to fill in all of the little blocks of free space between files, which were created from
deleting files. Defragmentation is used to make it easier for the disk heads to find files, and
speeds up read/write operations that were slow because of fragmentation.
The table below displays several different storage mediums, file systems, allocation
methods, and various attributes about each of them.
NTSF FAT32 exFAT HFS+ LTFS
Can Produce
Fragmentation
Issues/Problems
Contiguous
Allocation
No No No No No Yes
Fragmentation, finding space for files,
knowing file size, needs to compress files
Linked
Allocation
No No No No No No
Files can be scattered everywhere, uses links
stored with each block and if one block loses
its link the whole file is lost, not efficient
Indexed
Allocation
Yes -with contiguous
block allocation
No No No No Yes
Internal fragmentation is bad because of
allocating index tables for each file, Index
tables can create wasted space
File Allocation
Table (FAT)
No
Yes -Uses links at each
file but has all links
stored in fat table
Yes -with contiguous
block allocation
No No No
Not really recommended for running full
operating systems on, better for embeded
systems, small devices, and removable storage
Extent-Based
Allocation
No No No Yes -with indexing Yes -with indexing Yes
Suffers from both internal and external
fragmentation
Has Journaling
Feature
Yes No No
No - but it can be turned
on through the
command line
No N/A N/A
Max Volume
Size
16TB 2TB
64ZB - Recommended
Max is 512TB
8EB 2.5TB N/A N/A
Max Single File
Size Read/Write
16TB 4GB 64ZB 8EB 2.5TB N/A N/A
Allocation Unit
Sizes
4096b - 64KB 512b - 4KB 16KB - 32MB 4096b - 64KB 16KB - 256KB N/A N/A
Can Produce
Fragmentation
Yes -but its decreased
with larger allocation
unit sizes when
formatting
No
Yes -but fragmentation
does not cause as many
issues unless your using
an HDD.
Yes
Yes -internal
fragmentation only, LTFS
cannot be defragmented,
unless it is reformated
N/A N/A
Flash Drive Yes Yes Yes Yes No
Yes -very little effect on
solid state storage
Low maximum write cycles,
HDD Yes Yes Yes Yes No
Yes -slows file
read/write speed over
time, needs to be
defragmented
Uses physical heads and spinning disk,
fragmentation can slow performance a lot
SSD Yes Yes Yes Yes No
Yes -very little effect on
solid state storage
SSD's offer very fast access but have a
relatively short lifespan and are very expensive
SD Card Yes Yes Yes Yes No
Yes -very little effect on
solid state storage
Limited on write cycles, slow read/write speeds
Tape Drive No No No No
Yes -lets users navigate
and interact like its a
hard disk drive
No
Cannot be accesed randomly, though they can
be very fast and easy to use for backups
Windows
Compatible
Yes
Yes -but optomize for
quich removal option
needs to be turned off to
format FAT32 drives
Yes Read Only
Yes
need special software
and hardware
N/A N/A
Mac
Compatable
Read Only Yes Yes Yes
Yes
need special software
and hardware
N/A N/A
6. STORAGE MEDIUMS AND FRAGMENTATION 6
References
Englander, Irv. (2014-01-21). The Architecture of Computer Hardware and System Software: An
Information Technology Approach, 5th Edition. [VitalSource Bookshelf Online].
Retrieved from https://kaplan.vitalsource.com/#/books/9781118803127/
Evett, M (2015). Disk Allocation. retrieved May 22 2016, from YouTube Web Site:
https://www.youtube.com/watch?v=DWG6tO2EbjI
Microsoft, (2016). Description of the exFAT file system driver update package. retrieved May 23
2016, from Microsoft Web Site: https://support.microsoft.com/en-us/kb/955704
LeGault, L (2009). HFS+: The Mac OS X File System. retrieved May 23 2016, from Dep. of
Computer Science University of Wisconsin-Madison Web Site:
http://pages.cs.wisc.edu/~legault/miniproj-736.pdf
Rouse, M (2013). LTFS (Linear Tape File System). retrieved May 23 2016, from TechTarget
Web Site: http://searchdatabackup.techtarget.com/definition/LTFS-Linear-Tape-File-
System
IBM, (2010). Linear Tape File System (LTFS) Format Specification. retrieved May 23 2016,
from IBM Web Site: http://www-
01.ibm.com/support/docview.wss?uid=ssg1S7003166&aid=3
Indika, (2011). Difference Between Partition and Volume. retrieved May 24 2016, from
Difference Between Web Site: http://www.differencebetween.com/difference-between-
partition-and-vs-volume/
Dittrich, J (2013). 14.113 Hard Disks, Sectors, Zone Bit Recording, Sectors vs Blocks, CHS,
LBA, Sparing. retrieved May 24 2016, from YouTube Web Site:
https://www.youtube.com/watch?v=Cj8-WNjaGuM