linux secondary storage configuration

1. Logical Volume Management

2. 2/29
Common problems
● Resize partition?
Use space on separated disks?
Use speed of multiple devices?
Write same data on more devices?
Backup used filesystem?
●
●
●
●

3. 3/29
Partitions
●
●
●
●
●

4. 4/29
Patitions vs volume
●
●
●
●
●

5. 5/29
LVM
● Flexible capacity - multiple disks/partitions
Resizeable storage pools - extend/reduce
Online data relocation
Convenient device naming - names/tags
Disk striping
Mirroring volumes
Volume Snapshots
●
●
●
●
●
●

6. Concept
6/29

7. PV
● partition, disk,
LVM label - 2n sector
0, 1 or 2 copies of ASCII metadata
More PV on one disk?
●
●
●
# pvcreate /dev/sd[a-c]1
7/29

8. VG
# vgcreate -s 4M vg_test /dev/sd[a-c]1
8/29
● Pool of PVs
extent
●
● fixed for whole VG
smallest allocable space
●
● Size of logical extent = Size of physical extent
Map logical extents to physical extents
●

9. 9/29
LV
● Linear
Striped
Mirrored
Snapshot
Virtual
●
●
●
●

10. Linear
● concatenate physical
extents to logical
volume
flexible space
allocation
●
● Example:
● 2x99 physical extents
198 logical extents
●
# lvcreate -L 50G -n lv_linear vg_test
8
# lvcreate -l 100%FR
EE
-n lv_linear vg_test
/29

11. 9
Striped
● round robin
effective for large I/O
●
# lvcreate -L 50G -i 3 -I 128 -n lv_linear vg_test
/29

12. Mirrored
●
12/29
Identical copies
Mirror log on another
device or in memory
Mirror != backup
mirror_image_fault_p
olicy
●
●
●
● allocate
remove
●

13. Mirror creation
# lvcreate -m1 -L 1G -n lv_mirror vg_test
# lvcreate -m1 -L 2T -R 2 -n lv_mirror vg_test
# lvcreate -m1 -L 1G --mirrorlog core -n lv_mirror vg_test #
lvcreate -m1 -L 1G -n lv_mirror -alloc anywhere vg_test
# lvcreate -m1 -L 1G --mirrorlog mirrored -n lv_mirror vg_test
# lvcreate -m1 -L 1G -n lv_mirror vg_test /dev/sda1 /dev/sdb1
/dev/sdc1
# lvcreate -m1 -l 5 --mirrorlog mirrored -n lv_mirror vg_test /
> /dev/sda1:0-4 /dev/sdb1:0-4 /dev/sda1:5 /dev/sdb1:5
# lvcreate -m1 -i2 --mirrorlog core -n lv_lvm vg_four
13/29

14. 12/29
Snapshot
● stores only differences
origin and snapshot, both are RW
snapshot != backup
100% full is lost completely
Use
●
●
●
●
● backup
fsck
test/rollback on production data “--merge”
virtual machines
●
●
●
lvcreate --size 100M --snapshot --name

15. 9
13/2
Virtual
● --virtualsize VirtualSize
sparse device
●
$ lvcreate --virtualsize 1T --size 100M --snapshot --name sparse vg1

16. 16/29
Device mapper
● linear
striped
mirror
snapshot and snapshot-origin
error
zero
multipath
●
●
●
●
●
●

17. $ sudo dmsetup ls --tree
vg0-lv0 (253:5)
├─vg0-lv0_mimage_1 (253:2)
│ ├
─
(7:2)
│ ├
─
(7:1)
│ └
─
(7:0)
├─vg0-lv0_mimage_0 (253:1)
│ ├
─
(7:6)
│ ├
─
(7:5)
│ └
─
(7:4)
└─vg0-lv0_mlog (253:0)
└
─
(7:3)
17/29
$ sudo dmsetup status
vg0-lv0_mlog: 0 8192 linear
vg0-lv0_mimage_1: 0 2850816 striped 3 7:0 7:1 7:2 1 AAA
vg0-lv0_mimage_0: 0 2850816 striped 3 7:4 7:5 7:6 1 AAA
vg0-lv0: 0 2850816 mirror 2 253:1 253:2 2784/2784 1 AA 3 disk 253:0 A

18. 16/29
vg0-lv0_mlog
$ sudo dmsetup info
Name:
State:
Read Ahead:
ACTIV
E 256
Tables present:
Open count:
Event number:
LIV
E 1
0
Major, minor: 253, 0
Number of targets: 1
UUID: LVM-
ggZ5NqMTOZx43akuLZxxn03JdQGNta6gWlHnr0WT1Ua8Ek56ke6X9CdH8m3OVs
Ag
Name:
State:
Read Ahead:
Tables present:
Open count:
Event number:
Major, minor:
vg0-
lv0_mimage_1
ACTIVE
768
LIV
E 1
0
253, 2
Number of targets: 1
UUID: LVM-
ggZ5NqMTOZx43akuLZxxn03JdQGNta6gKIufDqbRgOaURN83IeA2AbYfH0zwTI0d
Name:
State:
Read Ahead:
Tables present:
Open count:
Event number:
Major, minor:
vg0-
lv0_mimage_0
ACTIVE
768
LIV
E 1
0
253, 1
Number of targets: 1
UUID: LVM-
ggZ5NqMTOZx43akuLZxxn03JdQGNta6g23mlbEbEk5r5l9Hy6YgRxuu2hd4DFg0Q
Name:
State:
Read Ahead:
Tables present:
Open count:
Event number:
vg0-lv0
ACTIV
E 256
LIVE
1
1

19. 19/29
How LVM Works:
LVM has basically three terms, Physical Volume PV, Volume
Group VG, Logical Volume LV.
PV – It’s a raw hard drive that it initialized to work with LVM, such
as /dev/sdb, /dev/sdc, /dev/sdb1 etc.
VG – Many PV is combined into one VG. You can create many VGs
and each of them has a unique name.
LV – You can create many LVs from a VG. You can extend, reduce
the LV size on the fly. The LV also has unique names. You format
the LV into ext4, zfs, btrfs etc filesystems, mount it and use it as
you do other ordinary partitions.

20. $ sudo pvcreate /dev/loop[0-6]
Physical volume "/dev/loop0" successfully created
Physical volume "/dev/loop1" successfully
Physical volume "/dev/loop2" successfully
Physical volume "/dev/loop3" successfully
Physical volume "/dev/loop4" successfully
Physical volume "/dev/loop5" successfully
Physical volume "/dev/loop6" successfully
created
created
created
created
created
created
$ sudo pvs
PV VG Fmt Attr PSize PFree
20/29
/dev/loop0 lvm2 a- 500,00m 500,00
m
/dev/loop1 lvm2 a- 500,00m 500,00
m
/dev/loop2 lvm2 a- 500,00m 500,00
m
/dev/loop3 lvm2 a- 500,00m 500,00
m
/dev/loop4 lvm2 a- 500,00m 500,00
m
/dev/loop5 lvm2 a- 500,00m 500,00
m
/dev/loop6 lvm2 a- 500,00m 500,00
m
$ sudo vgcreate vg0 /dev/loop[0-6]
Volume group "vg0" successfully created
$ sudo vgs
VG #PV #LV #SN Attr
VSize VFree vg0 7 0

21. vg
0
21/29
$ sudo vgdisplay
--- Volume group ---
VG Name
System ID
Format
Metadata Areas
Metadata Sequence
No VG Access
VG Status
MAX LV
Cur LV
Open
LV
Max PV
Cur PV
Act PV
VG Size
PE Size
Total PE
Alloc PE / Size
Free PE / Size
VG
UUID
lvm
2 7
22
read/write
resizable
0
0
0
0
7
7
3,39 GiB
4,00 MiB
868
0 / 0
868 / 3,39 GiB
ggZ5Nq-MTOZ-x43a-kuLZ-xxn0-3JdQ-
GNta6g
$ sudo lvcreate -i2 -m1 -L1200m -n lv0
vg
0 Using default stripesize 64,00 KiB Logical
volume "lv0" created

22. $ sudo lvs -a -o
LV
+devices
VG Attr LSize Origin Snap% Move Log Copy% Convert Devices
lv0 vg0 mwi-a- 1,17g lv0_mlog 10,29 lv0_mimage_0(0),lv0_mimage_1(0)
[lv0_mimage_0] vg0 Iwi-ao 1,17g /dev/loop0(0),/dev/loop1(0)
[lv0_mimage_0] vg0 Iwi-ao 1,17g /dev/loop2(0),/dev/loop3(0)
[lv0_mimage_1] vg0 Iwi-ao 1,17g /dev/loop4(0),/dev/loop5(0)
[lv0_mimage_1] vg0 Iwi-ao 1,17g /dev/loop2(26),/dev/loop3(27)
[lv0_mlog] vg0 lwi-ao 4,00m /dev/loop3(26)
22/29
$ sudo mkfs.ext4 /dev/vg0/lv0
$ sudo lvconvert -m0 --stripes 3 vg0/lv0
Logical volume lv0 converted.
$ sudo mount /dev/vg0/lv0 /mnt
$ sudo lvs -a -o +devices
LV VG Attr LSize Origin Snap
%
Mov
e
Log Copy
%
Convert Devices
lv0 vg0 -wi-a- 1,17g /dev/loop0(0),/dev/loop1(0)
lv0 vg0 -wi-a- 1,17g /dev/loop2(0),/dev/loop3(0)
$ sudo lvconvert -m1 --stripes 3 vg0/lv0
Using default stripesize 64,00 KiB vg0/lv0:
Converted: 0,3%
vg0/lv0: Converted: 4,7%
.
.
vg0/lv0: Converted: 100%

23. $ sudo lvs -a -o
LV
+devices
VG Attr LSize Log Devices
lv0
[lv0_mimage_0
]
[lv0_mimage_0
]
[lv0_mimage_1
] [lv0_mlog]
vg0 mwi-a-
vg0 Iwi-ao
vg0 Iwi-ao
vg0 Iwi-ao
vg0 lwi-ao
1,17g
1,17g
1,17g
1,17g
4,00m
lv0_mlog lv0_mimage_0(0),lv0_mimage_1(0)
/dev/loop0(0),/dev/loop1(0)
/dev/loop2(0),/dev/loop3(0)
/dev/loop4(0),/dev/loop5(0),/dev/loop6(0)
/dev/loop3(26)
$ sudo lvconvert
Logical volume
--splitmirrors 1 --name lv_split vg0/lv0 lv0
converted.
$ sudo lvs
LV
-a -o
VG
+devices
Attr LSize Origin Snap
%
Move Log
Copy%
Convert Devices
lv0 vg0 -wi-a- 1,17g /dev/loop4(0),/dev/loop5(0),/dev/loop6(0)
lv_split vg0 -wi-a- 1,17g /dev/loop0(0),/dev/loop1(0)
lv_split vg0 -wi-a- 1,17g /dev/loop2(0),/dev/loop3(0)
23/29
$ sudo lvremove vg0/lv_split
Do you really want to remove active logical volume lv_split? [y/n]: y Logical
volume "lv_split" successfully removed
$ sud
o
LV
lvs -a -o +devices
VG Attr LSize Origin Snap% Move Devices
lv0 vg0 -wi-a- 1,17g /dev/loop4(0),/dev/loop5(0),/dev/loop6(0)
$ sudo lvconvert -m1 --stripes 3 vg0/lv0
Using default stripesize 64,00 KiB

24. $ sudo lvs -a -o
LV
+devices
VG Attr LSize Log Devices
lv0 vg0 mwi-a- 1,17g lv0_mlog lv0_mimage_0(0),lv0_mimage_1(0)
[lv0_mimage_0] vg0 Iwi-ao 1,17g /dev/loop4(0),/dev/loop5(0),/dev/loop6(0)
[lv0_mimage_1] vg0 Iwi-ao 1,17g /dev/loop0(0),/dev/loop1(0),/dev/loop2(0)
[lv0_mlog] vg0 lwi-ao 4,00m /dev/loop3(0)
24/29
$ fsck /dev/vg0/lv0
fsck from util-linux 2.19.1
e2fsck 1.41.14 (22-Dec-2010)
/dev/mapper/vg0-lv0 is mounted.
If you continue you ***WILL***
WARNING!!! The filesystem is mounted.
cause ***SEVERE*** filesystem damage.
Do you really want to continue (y/n)? no
check aborted.
$ sudo lvcreate --size 100M --snapshot --name lv_snap /dev/vg0/lv0
Logical volume "lv_snap" created
$ sudo fsck /dev/vg0/lv_snap
fsck from util-linux 2.19.1
e2fsck 1.41.14 (22-Dec-2010)
/dev/mapper/vg0-lv_snap: clean, 12/76800 files, 13475/307200 blocks

25. $ sudo lvs -a -o
LV
+devices
VG Attr LSize Log Devices
lv0
[lv0_mimage_0
]
[lv0_mimage_1
] [lv0_mlog]
vg0 mwi-ao 1,36g lv0_mlog
vg0 Iwi-ao 1,36g
vg0 Iwi-ao 1,36g
vg0 lwi-ao 4,00m
lv0_mimage_0(0),lv0_mimage_1(0)
/dev/loop4(0),/dev/loop5(0),/dev/loop6(0)
/dev/loop0(0),/dev/loop1(0),/dev/loop2(0)
/dev/loop3(0)
$ df -h /mnt
Súborový systém Size Used Avail Use% Pripojený na
/dev/mapper/vg0-lv0 1,2G 34M 1,1G 4% /mnt
25/29
$ sudo resize2fs /dev/vg0/lv0
resize2fs 1.41.14 (22-Dec-2010)
Filesystem at /dev/vg0/lv0 is mounted on /mnt; on-line resizing required old
desc_blocks = 1, new_desc_blocks = 1
Performing an on-line resize of /dev/vg0/lv0 to 356352 (4k) blocks.
The filesystem on /dev/vg0/lv0 is now 356352 blocks long.
$ df -h /mnt
Súborový
systém
Size Used Avail Use% Pripojený na
/dev/mapper/vg0-lv0 1,4G 34M 1,3G 3% /mnt

26. $ sudo dd if=/dev/zero of=/dev/loop1 count=10
10+0 záznamov prečítaných
10+0 záznamov zapísaných
26/29
5120 skopírovaných bajtov (5,1 kB), 0,00188767 s, 2,7
MB/s
$ sudo lvs
Couldn't
-a -o +devices
find device with uuid hmPWAT-kPoM-yHZ8-Yh5M-m11G-yDjI-WLA28J.
LV VG Attr LSize Log Devices
lv0 vg0 mwi-ao 1,36g lv0_mlog lv0_mimage_0(0),lv0_mimage_1(0)
[lv0_mimage_0] vg0 iwi-ao 1,36g /dev/loop4(0),/dev/loop5(0),/dev/loop6(0)
[lv0_mimage_1] vg0 iwi-ao 1,36g /dev/loop0(0),unknown device(0),/dev/loop2(0)
[lv0_mlog] vg0 lwi-ao 4,00m /dev/loop3(0)
$ sudo vgreduce --removemissing vg0
Couldn't find device with uuid hmPWAT-kPoM-yHZ8-Yh5M-m11G-yDjI-
WLA28J. WARNING: Partial LV lv0 needs to be repaired or removed.
WARNING: Partial LV lv0_mimage_1 needs to be repaired or removed.
WARNING: There are still partial LVs in VG vg0.
To remove them unconditionally use: vgreduce --removemissing --force.
Proceeding to remove empty missing PVs.
$ sudo vgreduce --removemissing vg0 --force
Couldn't find device with uuid hmPWAT-kPoM-yHZ8-Yh5M-m11G-yDjI-
WLA28J. WARNING: Bad device removed from mirror volume, vg0/lv0
WARNING: Mirror volume, vg0/lv0 converted to linear due to device failure.
Wrote out consistent volume group vg0

27. $ sudo lvs -a -o +devices
LV VG Attr LSize Devices
lv0 vg0 -wi-ao 1,36g /dev/loop4(0),/dev/loop5(0),/dev/loop6(0)
27/29
$ sudo pvcreate --uuid hmPWAT-kPoM-yHZ8-Yh5M-m11G-yDjI-WLA28J
/dev/loop1 Physical volume "/dev/loop1" successfully created
$ sudo vgcfgrestore -f /etc/lvm/archive/vg0_00170.vg vg0
Restored volume group vg0
$ sudo lvs -a -o +devices
LV VG Attr LSize Log Devices
lv0 vg0 mwi-ao 1,27g lv0_mlog lv0_mimage_0(0),lv0_mimage_1(0)
[lv0_mimage_0] vg0 iwi--- 1,27g /dev/loop4(0),/dev/loop5(0),/dev/loop6(0)
[lv0_mimage_1] vg0 iwi--- 1,27g /dev/loop0(0),/dev/loop1(0),/dev/loop2(0)
[lv0_mlog] vg0 lwi--- 4,00m /dev/loop3(0)
Corrupted metadata

28. 27/29
New disk
$ sudo lvs -a -o +devices
LV VG Attr LSize
lv0 vg0 -wi-ao 1,36g
Devices
/dev/loop4(0),/dev/loop5(0),/dev/loop6(0)
$ sudo pvcreate --restorefile /etc/lvm/archive/vg0_00170.vg --uuid "hmPWAT-kPoM-yHZ8-
Yh5M-m11G-yDjI-WLA28J"
Physical volume "/dev/loop1" successfully created
$ sudo pvs
PV VG Fmt Attr PSize PFree
/dev/loop0
/dev/loop1
/dev/loop2
vg0
vg0
lvm
2
lvm
2
lvm
2
a-
a-
a-
496,00
m
505,00
m
496,00
m
496,00m
505,00m
496,00m
/dev/loop3 vg0 lvm2 a- 496,00
m
496,00m
/dev/loop4 vg0 lvm2 a- 496,00
m
64,00m
/dev/loop5 vg0 lvm2 a- 496,00
m
64,00m
/dev/loop6 vg0 lvm2 a- 496,00
m
64,00m
$ sudo vgextend vg0 /dev/loop1
Volume group "vg0" successfully extended
$ sudo lvconvert -m1 --stripes 3 vg0/lv0

29. 29/29
RHEL 6
● mirror_image_fault_policy=allocate/remove
data_alignment_detection
udev integration
lvconvert --repair
lvconvert --merge
lvconvert --splitmirrors
lvcreate --mirrorlog mirrored
lvextend --alloc cling
multiple --addtag and --deltag
●
●
●
●
●
●
●
●