SlideShare a Scribd company logo
1 of 35
UNIX OS
Lecture VI
Simonas Kareiva
Vilnius University
Faculty of Mathematics
and Informatics
Preparation of the material was supported by the project „Increasing Internationality in Study
Programs of the Department of Computer Science II“, project number VP1–2.2–ŠMM-07-K-
02-070, funded by The European Social Fund Agency and the Government of Lithuania.
Lecture VI outline
File systems
Disks
NFS protocol
RAID arrays
2
Part I : file systems and disks
disk's structure and terminology
file system types
Linux file systems
indexing
3
a very typical hard disk
terms:
A) Track
B) Geometrical sector
C) Sector
D) Cluster
4
Clusters and sectors
Sectors make clusters
Size of a sector – 512 bytes, 2048 bytes
Sectors are grouped into clusters in order to save
addressing and number of disk requests.
Cluster is the smallest logical disk’s element which can
store information, so:
 5KB file in disk uses 8KB when cluster size is 4KB
5
FAT…
FAT… bits tell the cluster address table size:
FAT12 – maximum partition Size– 32MB 
 Max number of clusters – 2^12-12, clusters up to 8K
FAT16 – max partition size – ??? GB (exercise)
 Clusters up to 64K
FAT32 - (many nice features) – max partition still 2TB 
6
More file systems:
NTFS
UFS
EXT2
EXT3
ReiserFS
HFS
7
Disk devices (Linux):
/dev/
 /dev/hda
 /dev/hda1
 /dev/hda2
 /dev/hdb
 /dev/hdc
 /dev/sda
 /dev/sda1
 …
8
Disk devices (FreeBSD)
/dev/
 /dev/ad0
 /dev/ad0s1
 /dev/ad0s1a
 /dev/ad0s1b
 /dev/ad0s1c
 /dev/ad1
 /dev/ad1s1
 /dev/ad1s1a
 /dev/da0
 /dev/da0s1
 /dev/da0s1a
 …
9
Disk devices (Solaris)
/dev/dsk/
 /dev/dsk/c1
 /dev/dsk/c1t0
 /dev/dsk/c1t0d0
 /dev/dsk/c1t0d0p0
 /dev/dsk/c1t0d0s1
c – controller (control device)
t – SCSI target
d – LUN-id (mostly 0)
s – Solaris slice or p – primary partition
10
Working with disks (Linux)
# fdisk -l /dev/hdc
Disk /dev/hdc: 64 heads, 63 sectors, 787 cylinders
Units = cylinders of 4032 * 512 bytes
Device Boot Start End Blocks Id System
/dev/hdc1 * 1 610 1229728+ 83 Linux
/dev/hdc2 611 787 356832 5 Extended
/dev/hdc5 611 787 356800+ c Win95 FAT32 (LBA)
11
Creating a new disk
Create a partition with fdisk (RTM)
Linux:
 mkfs -t ext2 /dev/hdc1
 or
 mkfs.reiserfs /dev/hdc1
FreeBSD:
 newfs /dev/da0s1a
12
Mounting
mkdir /mnt/new
mount /dev/hdc1 /mnt/new
13
/etc/fstab
# FreeBSD fstab:
# Device Mountpoint FStype Options Dump Pass#
/dev/da0s1b none swap sw 0 0
/dev/da0s1a / ufs rw 1 1
/dev/da0s1e /tmp ufs rw 2 2
/dev/da0s1f /usr ufs rw 2 2
/dev/da0s1d /var ufs rw 2 2
/dev/da1s1a /squid ufs rw 2 2
/dev/acd0 /cdrom cd9660 ro,noauto 0 0
# Linux fstab:
/dev/rootvg/lv00 / ext3 defaults 1 1
/dev/rootvg/lv02 /vz ext3 defaults 1 2
/dev/md0 /boot ext3 defaults 1 2
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
/dev/rootvg/lv01 swap swap defaults 0 0
14
NFS protocol
Like Windows File Sharing, but not exactly...
Designed for intensive data exchange between servers
Easily configurable
From an OS point of view, its just another file system
(partition)
15
What you’ll need:
FreeBSD:
 nfsd – daemon, processing user requests
 mountd – daemon, responsible for FS mounting
 rpcbind – additional service used for chatting within the
network
/etc/exports:
 Format:
 /folder/which/shared –paremeters client1 client2…
16
/etc/exports:
/cdrom -ro host1 host2 host3
/home -alldands 10.0.0.2 10.0.0.3 10.0.0.4
/a -maproot=root host.example.com box.example.org
/usr/src /usr/ports host4
Then we execute:
# rpcbind
# nfsd -u -t -n 4
# mountd -r
17
Or use:
# showmount -e server
Exports list on server:
/usr 10.10.10.0
/a 10.10.10.0
# mkdand /mnt/a
# mount server:/a /mnt/a
# df -F nfs
Filesystem Type blocks use avail %use Mounted on
server:/a nfs 68510 55804 12706 81% /mnt/a
18
Practical uses
Most OS’s support installation through NFS, its enough to
have 1 DVD which is shared via NFS on LAN.
By using NFS you can create a couple of HTTP servers
with the same content and distribute traffic between
them by using the round-robin load balancing algorithm.
In large networks, it is useful to keep /home folders on
NFS, so that user’s files are saved centrally.
Exercise: think of more practical NFS uses!
19
RAID
terms:
 RAID – Redundant array of Inexpensive Disks
 JBOD – Just a bunch of … ?
 Hot spare – reserve player
 Stripe – data distribution in multiple devices
 Mirror – data backup in multiple devices
 Parity – protection from errors system (XOR):
 0 XOR 0 = 0
 0 XOR 1 = 1
 1 XOR 0 = 1
 1 XOR 1 = 0
 Dedicated parity – data in separate disk
20
RAID types
RAID0, RAID1, RAID2, RAID3, RAID4, RAID5, RAID6
RAID0+1, RAID1+0, RAID0+3, RAID51, …
21
Say,
We have four disks:
 Three 200GB disks
 One 300GB disk
22
RAID0
Size = n * min ( 200GB, 300GB ) = 800GB
Advantages:
 Fast reading
 Fast writing
 Lots of space
Disadvantages:
 Totally unreliable (if one of the disks
breaks down, the entire RAID stops)
23
RAID1
Size= min ( 200GB, 300GB ) = 200GB
Advantages:
 Fast reading
 Reliability and security
Disadvantages:
 Expensive writing (to both disks)
24
RAID5
Size= n-1 * min ( 200GB, 300GB ) = 200GB
Advantages:
 Fast reading
 Fast writing
 Reliability and security (parity)
Disadvantages:
 Requires at least 3 disks, often requires +1 additional
 Slow, when at least stripe size is being written
25
Parity
disk #1: -------- (Data)
disk #2: -------- (Data)
disk #3: -------- (Data)
disk #4: -------- (Data)
disk #5: -------- (Hot Spare)
disk #6: -------- (Parity)
26
Parity
disk #1: 00101010 (Data)
disk #2: 10001110 (Data)
disk #3: 11110111 (Data)
disk #4: 10110101 (Data)
disk #5: -------- (Hot Spare)
disk #6: -------- (Parity)
00101010 XOR 10001110 XOR 11110111 XOR 10110101 = 11100110
D1 XOR D2 XOR D3 XOR D4 = ( (D1 XOR D2) XOR D3) XOR D4
27
Parity
disk #1: 00101010 (Data)
disk #2: 10001110 (Data)
disk #3: 11110111 (Data)
disk #4: 10110101 (Data)
disk #5: -------- (Hot Spare)
disk #6: 11100110 (Parity)
28
Parity
disk #1: 00101010 (Data)
disk #2: 10001110 (Data)
disk #3: --Dead-- (Data)
disk #4: 10110101 (Data)
disk #5: 11110111 (Hot Spare)
disk #6: 11100110 (Parity)
00101010 XOR 10001110 XOR 11100110 XOR 10110101 = 11110111
29
RAID2, RAID3
30
31
RAID derivatives
32
Yo, dawg, I heard you like RAID…
… so we put more RAID into
your RAID so that you can have
more RAID!
33
34
Cheat-sheet of RAID configurations
35

More Related Content

What's hot

What's hot (20)

AOS Lab 1: Hello, Linux!
AOS Lab 1: Hello, Linux!AOS Lab 1: Hello, Linux!
AOS Lab 1: Hello, Linux!
 
Ext filesystem4
Ext filesystem4Ext filesystem4
Ext filesystem4
 
Linux directory structure by jitu mistry
Linux directory structure by jitu mistryLinux directory structure by jitu mistry
Linux directory structure by jitu mistry
 
Vfs
VfsVfs
Vfs
 
Linux file system
Linux file systemLinux file system
Linux file system
 
Linux Shell Scripting Presantion
Linux Shell Scripting PresantionLinux Shell Scripting Presantion
Linux Shell Scripting Presantion
 
Log
LogLog
Log
 
Archiving in linux tar
Archiving in linux tarArchiving in linux tar
Archiving in linux tar
 
Install Archlinux in 10 Steps (Sort of) :)
Install Archlinux in 10 Steps (Sort of) :)Install Archlinux in 10 Steps (Sort of) :)
Install Archlinux in 10 Steps (Sort of) :)
 
PostgreSQL on EXT4, XFS, BTRFS and ZFS
PostgreSQL on EXT4, XFS, BTRFS and ZFSPostgreSQL on EXT4, XFS, BTRFS and ZFS
PostgreSQL on EXT4, XFS, BTRFS and ZFS
 
Log
LogLog
Log
 
Linux fundamental - Chap 03 file
Linux fundamental - Chap 03 fileLinux fundamental - Chap 03 file
Linux fundamental - Chap 03 file
 
AOS Lab 2: Hello, xv6!
AOS Lab 2: Hello, xv6!AOS Lab 2: Hello, xv6!
AOS Lab 2: Hello, xv6!
 
Linux fundamental - Chap 02 perm
Linux fundamental - Chap 02 permLinux fundamental - Chap 02 perm
Linux fundamental - Chap 02 perm
 
PostgreSQL on ZFS Lightning Talk
PostgreSQL on ZFS Lightning TalkPostgreSQL on ZFS Lightning Talk
PostgreSQL on ZFS Lightning Talk
 
Linux filesystemhierarchy
Linux filesystemhierarchyLinux filesystemhierarchy
Linux filesystemhierarchy
 
Linux fundamental - Chap 04 archive
Linux fundamental - Chap 04 archiveLinux fundamental - Chap 04 archive
Linux fundamental - Chap 04 archive
 
Compression Commands in Linux
Compression Commands in LinuxCompression Commands in Linux
Compression Commands in Linux
 
101 1.2 boot the system
101 1.2 boot the system101 1.2 boot the system
101 1.2 boot the system
 
Basic Linux commands
Basic Linux commandsBasic Linux commands
Basic Linux commands
 

Similar to Unix 6 en

101 2.1 design hard disk layout
101 2.1 design hard disk layout101 2.1 design hard disk layout
101 2.1 design hard disk layoutAcácio Oliveira
 
GlusterFS Update and OpenStack Integration
GlusterFS Update and OpenStack IntegrationGlusterFS Update and OpenStack Integration
GlusterFS Update and OpenStack IntegrationEtsuji Nakai
 
Open Enea Linux workshop at the Embedded Conference Scandinavia 2014
Open Enea Linux workshop at the Embedded Conference Scandinavia 2014Open Enea Linux workshop at the Embedded Conference Scandinavia 2014
Open Enea Linux workshop at the Embedded Conference Scandinavia 2014EneaSoftware
 
101 2.1 design hard disk layout v2
101 2.1 design hard disk layout v2101 2.1 design hard disk layout v2
101 2.1 design hard disk layout v2Acácio Oliveira
 
101 4.1 create partitions and filesystems
101 4.1 create partitions and filesystems101 4.1 create partitions and filesystems
101 4.1 create partitions and filesystemsAcácio Oliveira
 
2.1 design hard disk layout v2
2.1 design hard disk layout v22.1 design hard disk layout v2
2.1 design hard disk layout v2Acácio Oliveira
 
How to install gentoo distributed
How to install gentoo distributedHow to install gentoo distributed
How to install gentoo distributedSongWang54
 
TLPI Chapter 14 File Systems
TLPI Chapter 14 File SystemsTLPI Chapter 14 File Systems
TLPI Chapter 14 File SystemsShu-Yu Fu
 
Linux lv ms step by step
Linux lv ms step by stepLinux lv ms step by step
Linux lv ms step by stepsudakarman
 
Asif Jamal disk (it)
Asif Jamal disk (it)Asif Jamal disk (it)
Asif Jamal disk (it)Asif Jamal
 
Linux or unix interview questions
Linux or unix interview questionsLinux or unix interview questions
Linux or unix interview questionsTeja Bheemanapally
 
Disk and File System Management in Linux
Disk and File System Management in LinuxDisk and File System Management in Linux
Disk and File System Management in LinuxHenry Osborne
 
Mirroring the root_disk under solaris SVM
Mirroring the root_disk under solaris SVMMirroring the root_disk under solaris SVM
Mirroring the root_disk under solaris SVMKazimal Abed Mohammed
 

Similar to Unix 6 en (20)

101 2.1 design hard disk layout
101 2.1 design hard disk layout101 2.1 design hard disk layout
101 2.1 design hard disk layout
 
FreeBSD Portscamp, Kuala Lumpur 2016
FreeBSD Portscamp, Kuala Lumpur 2016FreeBSD Portscamp, Kuala Lumpur 2016
FreeBSD Portscamp, Kuala Lumpur 2016
 
Real time systems
Real time systemsReal time systems
Real time systems
 
Mac os x mount ntfs
Mac os x mount ntfsMac os x mount ntfs
Mac os x mount ntfs
 
NFS.ppt
NFS.pptNFS.ppt
NFS.ppt
 
GlusterFS Update and OpenStack Integration
GlusterFS Update and OpenStack IntegrationGlusterFS Update and OpenStack Integration
GlusterFS Update and OpenStack Integration
 
Introduction to UNIX
Introduction to UNIXIntroduction to UNIX
Introduction to UNIX
 
Open Enea Linux workshop at the Embedded Conference Scandinavia 2014
Open Enea Linux workshop at the Embedded Conference Scandinavia 2014Open Enea Linux workshop at the Embedded Conference Scandinavia 2014
Open Enea Linux workshop at the Embedded Conference Scandinavia 2014
 
101 2.1 design hard disk layout v2
101 2.1 design hard disk layout v2101 2.1 design hard disk layout v2
101 2.1 design hard disk layout v2
 
101 4.1 create partitions and filesystems
101 4.1 create partitions and filesystems101 4.1 create partitions and filesystems
101 4.1 create partitions and filesystems
 
2.1 design hard disk layout v2
2.1 design hard disk layout v22.1 design hard disk layout v2
2.1 design hard disk layout v2
 
Booy Up
Booy UpBooy Up
Booy Up
 
How to install gentoo distributed
How to install gentoo distributedHow to install gentoo distributed
How to install gentoo distributed
 
TLPI Chapter 14 File Systems
TLPI Chapter 14 File SystemsTLPI Chapter 14 File Systems
TLPI Chapter 14 File Systems
 
Linux lv ms step by step
Linux lv ms step by stepLinux lv ms step by step
Linux lv ms step by step
 
Asif Jamal disk (it)
Asif Jamal disk (it)Asif Jamal disk (it)
Asif Jamal disk (it)
 
Linux or unix interview questions
Linux or unix interview questionsLinux or unix interview questions
Linux or unix interview questions
 
RamDisk
RamDiskRamDisk
RamDisk
 
Disk and File System Management in Linux
Disk and File System Management in LinuxDisk and File System Management in Linux
Disk and File System Management in Linux
 
Mirroring the root_disk under solaris SVM
Mirroring the root_disk under solaris SVMMirroring the root_disk under solaris SVM
Mirroring the root_disk under solaris SVM
 

Unix 6 en

  • 1. UNIX OS Lecture VI Simonas Kareiva Vilnius University Faculty of Mathematics and Informatics Preparation of the material was supported by the project „Increasing Internationality in Study Programs of the Department of Computer Science II“, project number VP1–2.2–ŠMM-07-K- 02-070, funded by The European Social Fund Agency and the Government of Lithuania.
  • 2. Lecture VI outline File systems Disks NFS protocol RAID arrays 2
  • 3. Part I : file systems and disks disk's structure and terminology file system types Linux file systems indexing 3
  • 4. a very typical hard disk terms: A) Track B) Geometrical sector C) Sector D) Cluster 4
  • 5. Clusters and sectors Sectors make clusters Size of a sector – 512 bytes, 2048 bytes Sectors are grouped into clusters in order to save addressing and number of disk requests. Cluster is the smallest logical disk’s element which can store information, so:  5KB file in disk uses 8KB when cluster size is 4KB 5
  • 6. FAT… FAT… bits tell the cluster address table size: FAT12 – maximum partition Size– 32MB   Max number of clusters – 2^12-12, clusters up to 8K FAT16 – max partition size – ??? GB (exercise)  Clusters up to 64K FAT32 - (many nice features) – max partition still 2TB  6
  • 8. Disk devices (Linux): /dev/  /dev/hda  /dev/hda1  /dev/hda2  /dev/hdb  /dev/hdc  /dev/sda  /dev/sda1  … 8
  • 9. Disk devices (FreeBSD) /dev/  /dev/ad0  /dev/ad0s1  /dev/ad0s1a  /dev/ad0s1b  /dev/ad0s1c  /dev/ad1  /dev/ad1s1  /dev/ad1s1a  /dev/da0  /dev/da0s1  /dev/da0s1a  … 9
  • 10. Disk devices (Solaris) /dev/dsk/  /dev/dsk/c1  /dev/dsk/c1t0  /dev/dsk/c1t0d0  /dev/dsk/c1t0d0p0  /dev/dsk/c1t0d0s1 c – controller (control device) t – SCSI target d – LUN-id (mostly 0) s – Solaris slice or p – primary partition 10
  • 11. Working with disks (Linux) # fdisk -l /dev/hdc Disk /dev/hdc: 64 heads, 63 sectors, 787 cylinders Units = cylinders of 4032 * 512 bytes Device Boot Start End Blocks Id System /dev/hdc1 * 1 610 1229728+ 83 Linux /dev/hdc2 611 787 356832 5 Extended /dev/hdc5 611 787 356800+ c Win95 FAT32 (LBA) 11
  • 12. Creating a new disk Create a partition with fdisk (RTM) Linux:  mkfs -t ext2 /dev/hdc1  or  mkfs.reiserfs /dev/hdc1 FreeBSD:  newfs /dev/da0s1a 12
  • 14. /etc/fstab # FreeBSD fstab: # Device Mountpoint FStype Options Dump Pass# /dev/da0s1b none swap sw 0 0 /dev/da0s1a / ufs rw 1 1 /dev/da0s1e /tmp ufs rw 2 2 /dev/da0s1f /usr ufs rw 2 2 /dev/da0s1d /var ufs rw 2 2 /dev/da1s1a /squid ufs rw 2 2 /dev/acd0 /cdrom cd9660 ro,noauto 0 0 # Linux fstab: /dev/rootvg/lv00 / ext3 defaults 1 1 /dev/rootvg/lv02 /vz ext3 defaults 1 2 /dev/md0 /boot ext3 defaults 1 2 tmpfs /dev/shm tmpfs defaults 0 0 devpts /dev/pts devpts gid=5,mode=620 0 0 sysfs /sys sysfs defaults 0 0 proc /proc proc defaults 0 0 /dev/rootvg/lv01 swap swap defaults 0 0 14
  • 15. NFS protocol Like Windows File Sharing, but not exactly... Designed for intensive data exchange between servers Easily configurable From an OS point of view, its just another file system (partition) 15
  • 16. What you’ll need: FreeBSD:  nfsd – daemon, processing user requests  mountd – daemon, responsible for FS mounting  rpcbind – additional service used for chatting within the network /etc/exports:  Format:  /folder/which/shared –paremeters client1 client2… 16
  • 17. /etc/exports: /cdrom -ro host1 host2 host3 /home -alldands 10.0.0.2 10.0.0.3 10.0.0.4 /a -maproot=root host.example.com box.example.org /usr/src /usr/ports host4 Then we execute: # rpcbind # nfsd -u -t -n 4 # mountd -r 17
  • 18. Or use: # showmount -e server Exports list on server: /usr 10.10.10.0 /a 10.10.10.0 # mkdand /mnt/a # mount server:/a /mnt/a # df -F nfs Filesystem Type blocks use avail %use Mounted on server:/a nfs 68510 55804 12706 81% /mnt/a 18
  • 19. Practical uses Most OS’s support installation through NFS, its enough to have 1 DVD which is shared via NFS on LAN. By using NFS you can create a couple of HTTP servers with the same content and distribute traffic between them by using the round-robin load balancing algorithm. In large networks, it is useful to keep /home folders on NFS, so that user’s files are saved centrally. Exercise: think of more practical NFS uses! 19
  • 20. RAID terms:  RAID – Redundant array of Inexpensive Disks  JBOD – Just a bunch of … ?  Hot spare – reserve player  Stripe – data distribution in multiple devices  Mirror – data backup in multiple devices  Parity – protection from errors system (XOR):  0 XOR 0 = 0  0 XOR 1 = 1  1 XOR 0 = 1  1 XOR 1 = 0  Dedicated parity – data in separate disk 20
  • 21. RAID types RAID0, RAID1, RAID2, RAID3, RAID4, RAID5, RAID6 RAID0+1, RAID1+0, RAID0+3, RAID51, … 21
  • 22. Say, We have four disks:  Three 200GB disks  One 300GB disk 22
  • 23. RAID0 Size = n * min ( 200GB, 300GB ) = 800GB Advantages:  Fast reading  Fast writing  Lots of space Disadvantages:  Totally unreliable (if one of the disks breaks down, the entire RAID stops) 23
  • 24. RAID1 Size= min ( 200GB, 300GB ) = 200GB Advantages:  Fast reading  Reliability and security Disadvantages:  Expensive writing (to both disks) 24
  • 25. RAID5 Size= n-1 * min ( 200GB, 300GB ) = 200GB Advantages:  Fast reading  Fast writing  Reliability and security (parity) Disadvantages:  Requires at least 3 disks, often requires +1 additional  Slow, when at least stripe size is being written 25
  • 26. Parity disk #1: -------- (Data) disk #2: -------- (Data) disk #3: -------- (Data) disk #4: -------- (Data) disk #5: -------- (Hot Spare) disk #6: -------- (Parity) 26
  • 27. Parity disk #1: 00101010 (Data) disk #2: 10001110 (Data) disk #3: 11110111 (Data) disk #4: 10110101 (Data) disk #5: -------- (Hot Spare) disk #6: -------- (Parity) 00101010 XOR 10001110 XOR 11110111 XOR 10110101 = 11100110 D1 XOR D2 XOR D3 XOR D4 = ( (D1 XOR D2) XOR D3) XOR D4 27
  • 28. Parity disk #1: 00101010 (Data) disk #2: 10001110 (Data) disk #3: 11110111 (Data) disk #4: 10110101 (Data) disk #5: -------- (Hot Spare) disk #6: 11100110 (Parity) 28
  • 29. Parity disk #1: 00101010 (Data) disk #2: 10001110 (Data) disk #3: --Dead-- (Data) disk #4: 10110101 (Data) disk #5: 11110111 (Hot Spare) disk #6: 11100110 (Parity) 00101010 XOR 10001110 XOR 11100110 XOR 10110101 = 11110111 29
  • 31. 31
  • 33. Yo, dawg, I heard you like RAID… … so we put more RAID into your RAID so that you can have more RAID! 33
  • 34. 34
  • 35. Cheat-sheet of RAID configurations 35