The document discusses file systems in Linux. It describes block devices, partitioning, filesystem types like Ext2/3/4, and tools for creating, mounting, checking, and monitoring filesystems. It also covers swap space, the /etc/fstab file, mounting options, and implementing disk quotas.
PostgreSQL and ZFS were made for each other. This talk dives downstack into the internals and way that PostgreSQL consumes disk resources and tricks that are available if you run PostgreSQL on ZFS (ZFS on Linux, ZFS on FreeBSD, or ZFS on Illumos). Topics covered will include:
* Performance and sizing considerations
* Workload estimation heuristics
* Standard administrative practices that leverage ZFS
* Recovery using ZFS
* Performing database migrations using ZFS
Let's trace Linux Lernel with KGDB @ COSCUP 2021Jian-Hong Pan
https://coscup.org/2021/en/session/39M73K
https://www.youtube.com/watch?v=L_Gyvdl_d_k
Engineers have plenty of debug tools for user space programs development, code tracing, debugging and analyzing. Except “printk”, do we have any other debug tools for Linux kernel development? The “KGDB” mentioned in Linux kernel document provides another possibility.
Will share how to experiment with the KGDB in a virtual machine. And, use GDB + OpenOCD + JTAG + Raspberry Pi in the real environment as the demo in this talk.
開發 user space 軟體時,工程師們有方便的 debug 工具進行查找、分析、除錯。但在 Linux kernel 的開發,除了 printk 外,還可以有哪些工具可以使用呢?從 Linux kernel document 可以看到 KGDB 相關的資訊,提供了在 kernel 除錯時的另一個可能性。
本次將分享,從建立最簡單環境的虛擬機機開始,到實際使用 GDB + OpenOCD + JTAG + Raspberry Pi 當作展示範例。
The tar command stands for tape achieve, which is the most commonly used tape drive backup command used by the Linux system. It allows to quickly access a collection of files and placed them into a highly compressed archive file commonly called tar, gzip and bzip in Linux.
To visit www.excavatorinfo.com
A Journey to Boot Linux on Raspberry PiJian-Hong Pan
Each processor/chip architecture has its own procedure to boot the kernel. It works with desgined partition layout and vendor specific firmwares/bootloaders in the boot partition. We can learn the related knowledge from the Raspbian image for Raspberry Pi, which is the board we can obtain easily. However, the diversity between the special booting procedures with specific firmwares/bootloaders increases the complexity for distribution maintainers. It will be great if there is a way to make it more generic that can be applied to most of the chip architectures/boards to boot up the system.
After referring to some Linux distributions, we learned U-Boot may play a role in the solution. It splits the booting procedure into hardware specific and generic system parts. This helps distribution maintainers deploy the generic system with OSTree, including device trees.
Let’s deep dive into this magic booting procedure!
Command line interface,Program arguments and data streams,Program execution,Bash built-ins,Output stream redirection,Input stream redirection,Pipelining
PostgreSQL and ZFS were made for each other. This talk dives downstack into the internals and way that PostgreSQL consumes disk resources and tricks that are available if you run PostgreSQL on ZFS (ZFS on Linux, ZFS on FreeBSD, or ZFS on Illumos). Topics covered will include:
* Performance and sizing considerations
* Workload estimation heuristics
* Standard administrative practices that leverage ZFS
* Recovery using ZFS
* Performing database migrations using ZFS
Let's trace Linux Lernel with KGDB @ COSCUP 2021Jian-Hong Pan
https://coscup.org/2021/en/session/39M73K
https://www.youtube.com/watch?v=L_Gyvdl_d_k
Engineers have plenty of debug tools for user space programs development, code tracing, debugging and analyzing. Except “printk”, do we have any other debug tools for Linux kernel development? The “KGDB” mentioned in Linux kernel document provides another possibility.
Will share how to experiment with the KGDB in a virtual machine. And, use GDB + OpenOCD + JTAG + Raspberry Pi in the real environment as the demo in this talk.
開發 user space 軟體時,工程師們有方便的 debug 工具進行查找、分析、除錯。但在 Linux kernel 的開發,除了 printk 外,還可以有哪些工具可以使用呢?從 Linux kernel document 可以看到 KGDB 相關的資訊,提供了在 kernel 除錯時的另一個可能性。
本次將分享,從建立最簡單環境的虛擬機機開始,到實際使用 GDB + OpenOCD + JTAG + Raspberry Pi 當作展示範例。
The tar command stands for tape achieve, which is the most commonly used tape drive backup command used by the Linux system. It allows to quickly access a collection of files and placed them into a highly compressed archive file commonly called tar, gzip and bzip in Linux.
To visit www.excavatorinfo.com
A Journey to Boot Linux on Raspberry PiJian-Hong Pan
Each processor/chip architecture has its own procedure to boot the kernel. It works with desgined partition layout and vendor specific firmwares/bootloaders in the boot partition. We can learn the related knowledge from the Raspbian image for Raspberry Pi, which is the board we can obtain easily. However, the diversity between the special booting procedures with specific firmwares/bootloaders increases the complexity for distribution maintainers. It will be great if there is a way to make it more generic that can be applied to most of the chip architectures/boards to boot up the system.
After referring to some Linux distributions, we learned U-Boot may play a role in the solution. It splits the booting procedure into hardware specific and generic system parts. This helps distribution maintainers deploy the generic system with OSTree, including device trees.
Let’s deep dive into this magic booting procedure!
Command line interface,Program arguments and data streams,Program execution,Bash built-ins,Output stream redirection,Input stream redirection,Pipelining
This presentation covers the general concepts about real-time systems, how Linux kernel works for preemption, the latency in Linux, rt-preempt, and Xenomai, the real-time extension as the dual kernel approach.
Part 4 of 'Introduction to Linux for bioinformatics': Managing data Joachim Jacob
This is part 4 of the training session 'Introduction to Linux for bioinformatics'. We shows basics of data management, and tips for handling big data effectively. Interested in following this training session? Please contact me at http://www.jakonix.be/contact.html
This Slide was presented as an introduction to Linux . Students with little experience in free operating systems were encouraged to take up Linux based operating systems.
While probably the most prominent, Docker is not the only tool for building and managing containers. Originally meant to be a "chroot on steroids" to help debug systemd, systemd-nspawn provides a fairly uncomplicated approach to work with containers. Being part of systemd, it is available on most recent distributions out-of-the-box and requires no additional dependencies.
This deck will introduce a few concepts involved in containers and will guide you through the steps of building a container from scratch. The payload will be a simple service, which will be automatically activated by systemd when the first request arrives.
Алексей Лесовский "Тюнинг Linux для баз данных. "Tanya Denisyuk
В любом проекте всегда есть данные которые важно не потерять. И если проект перешагнул размеры домашнего, то наверняка там уже используется что-то более надежное чем текстовые файлы на диске. Скорее всего это какая-то конкретная база данных. Учитывая важную роль баз данных к ним предъявляют жесткие требования производительности и стабильности. Отдельной особенностью является и операционная система на которой работает база данных. Опыт и практика подсказывает что это будет Linux - одна из наиболее популярных на сегодняшний день операционных систем. Прогрессивное развитие в сочетании с гибкостью и стабильностью делает Linux хорошим кандидатом в качестве платформы для баз данных (да и не только). Однако настройки по-умолчанию в большинстве дистрибутивов предназначены для широкого круга задач и потребностей. В этом докладе я расскажу о том, на какие настройки следует обратить внимание и как они влияют на работу СУБД. Как сделать работу СУБД более предсказуемой, как в простых условиях, так и в условиях непостоянной или растущей нагрузки.
Macroeconomics- Movie Location
This will be used as part of your Personal Professional Portfolio once graded.
Objective:
Prepare a presentation or a paper using research, basic comparative analysis, data organization and application of economic information. You will make an informed assessment of an economic climate outside of the United States to accomplish an entertainment industry objective.
Synthetic Fiber Construction in lab .pptxPavel ( NSTU)
Synthetic fiber production is a fascinating and complex field that blends chemistry, engineering, and environmental science. By understanding these aspects, students can gain a comprehensive view of synthetic fiber production, its impact on society and the environment, and the potential for future innovations. Synthetic fibers play a crucial role in modern society, impacting various aspects of daily life, industry, and the environment. ynthetic fibers are integral to modern life, offering a range of benefits from cost-effectiveness and versatility to innovative applications and performance characteristics. While they pose environmental challenges, ongoing research and development aim to create more sustainable and eco-friendly alternatives. Understanding the importance of synthetic fibers helps in appreciating their role in the economy, industry, and daily life, while also emphasizing the need for sustainable practices and innovation.
The French Revolution, which began in 1789, was a period of radical social and political upheaval in France. It marked the decline of absolute monarchies, the rise of secular and democratic republics, and the eventual rise of Napoleon Bonaparte. This revolutionary period is crucial in understanding the transition from feudalism to modernity in Europe.
For more information, visit-www.vavaclasses.com
Embracing GenAI - A Strategic ImperativePeter Windle
Artificial Intelligence (AI) technologies such as Generative AI, Image Generators and Large Language Models have had a dramatic impact on teaching, learning and assessment over the past 18 months. The most immediate threat AI posed was to Academic Integrity with Higher Education Institutes (HEIs) focusing their efforts on combating the use of GenAI in assessment. Guidelines were developed for staff and students, policies put in place too. Innovative educators have forged paths in the use of Generative AI for teaching, learning and assessments leading to pockets of transformation springing up across HEIs, often with little or no top-down guidance, support or direction.
This Gasta posits a strategic approach to integrating AI into HEIs to prepare staff, students and the curriculum for an evolving world and workplace. We will highlight the advantages of working with these technologies beyond the realm of teaching, learning and assessment by considering prompt engineering skills, industry impact, curriculum changes, and the need for staff upskilling. In contrast, not engaging strategically with Generative AI poses risks, including falling behind peers, missed opportunities and failing to ensure our graduates remain employable. The rapid evolution of AI technologies necessitates a proactive and strategic approach if we are to remain relevant.
4. Hard Disk PartitionHard Disk Partition
●
Partitioning rules:Partitioning rules:
– Divided by cylinderDivided by cylinder
– Maximum of four Primary PartitionsMaximum of four Primary Partitions
– Only one of primary partitions may beOnly one of primary partitions may be
used as an Extended Partitionused as an Extended Partition
– An extended partition contains numbersAn extended partition contains numbers
of Logical Partitionof Logical Partition
– Only primary and logical partitions can beOnly primary and logical partitions can be
populated with file systemspopulated with file systems
Ref. Pge. 130
7. UsingUsing fdiskfdisk
●
InternalInternal fdiskfdisk commandscommands
– mm : help: help
– pp : prints partition table: prints partition table
– nn : creates new partition: creates new partition
– ll : lists partition type ID: lists partition type ID
– tt : changes partition type ID: changes partition type ID
– dd : deletes partition: deletes partition
– ww : save and exit: save and exit
– qq : quit without saving: quit without saving
8. Loop DeviceLoop Device
●
Access a file as a block deviceAccess a file as a block device
# dd if=/dev/zero of=mydisk bs=1M count=32# dd if=/dev/zero of=mydisk bs=1M count=32
9. File SystemFile System
●
Linux supports many types of fileLinux supports many types of file
system:system:
– Ext2, Ext3, Ext4Ext2, Ext3, Ext4
– ReiserfsReiserfs
– JFSJFS
– XFSXFS
– FAT, NTFSFAT, NTFS
– ISO9660ISO9660
– ......
Ref. Pge. 142
10. Ext2 File SystemExt2 File System
●
Partition divided into blocks withPartition divided into blocks with
fixed block size (1k, 2k, 4k)fixed block size (1k, 2k, 4k)
●
Block usages:Block usages:
– SuperblockSuperblock
– Inode BlocksInode Blocks
– Data BlocksData Blocks
11. Super BlockSuper Block
●
The first block of a file systemThe first block of a file system
●
May have a couple of copiesMay have a couple of copies
●
Contains general information:Contains general information:
– Block SizeBlock Size
– Pointers to free inodesPointers to free inodes
– Pointers to free blocksPointers to free blocks
– Pointer to root directoryPointer to root directory
– Mounting timeMounting time
– ......
12. Inode (Index Node)Inode (Index Node)
●
128 bytes for each128 bytes for each
●
Contains file information:Contains file information:
– OwnershipOwnership
– PermissionPermission
– File TypeFile Type
– Time StampsTime Stamps
– Pointers to blocksPointers to blocks
13. Data BlockData Block
●
Regular File:Regular File:
– The actual dataThe actual data
●
Directory:Directory:
– File NameFile Name
– Inode NumberInode Number
15. File System FeaturesFile System Features
●
Journaling:Journaling:
– Keeps a journal of file operationKeeps a journal of file operation
– Makes recovery much fasterMakes recovery much faster
●
ACL:ACL:
– More flexible control above traditionalMore flexible control above traditional
permission schemepermission scheme
●
Label:Label:
– Allows to use label name and regardlessAllows to use label name and regardless
changing of device namechanging of device name
16. Creating File SystemCreating File System
●
mkfs partitionmkfs partition
– Makes file system on partitionMakes file system on partition
– Common options:Common options:
●
t t : file system type: file system type
●
b b : block size: block size
●
i i : inode ratio to block: inode ratio to block
●
j j : turn on journal: turn on journal
●
c c : check bad blocks: check bad blocks
Ref. Pge. 145
17. Mounting File SystemMounting File System
●
mount device /mount/pointmount device /mount/point
– The device must be formatedThe device must be formated
– The mount point must be an empty directoryThe mount point must be an empty directory
– Displays current mounting file systems if noDisplays current mounting file systems if no
argument givenargument given
– Common options:Common options:
●
t t : file system type: file system type
●
bind bind : mounts directory instead of device: mounts directory instead of device
●
o m_opts o m_opts : mount options: mount options
Ref. Pge. 160
18. Common Mount OptionsCommon Mount Options
●
ro/rw : read and ro/rw : read and writewrite
●
sync/async : sync/async : synchronizationsynchronization
●
atime/noatim : atime/noatim : access time updatingaccess time updating
●
dev/nodev : dev/nodev : device accessdevice access
●
exec/noexec : exec/noexec : program runningprogram running
●
suid/nosuid : suid/nosuid : suid/sgid functionsuid/sgid function
●
remount,m_opt : remount,m_opt : changes mount options on achanges mount options on a
mounted file systemmounted file system
19. TheThe /etc/fstab/etc/fstab
●
Mount files systems on startupMount files systems on startup
●
While runningWhile running mount amount a
●
Fields:Fields:
– Device Name/ID (shown by runningDevice Name/ID (shown by running blkidblkid))
– Mount pointMount point
– File system typeFile system type
– Mount optionsMount options
– dumpdump settings (0 for off and 1 for on)settings (0 for off and 1 for on)
– fsckfsck settings (1 for root and 2 for others, 0 for off)settings (1 for root and 2 for others, 0 for off)
Ref. Pge. 164
21. Unmounting File SystemUnmounting File System
●
umount <device|/mount/point>umount <device|/mount/point>
– Device must be unmounted before detachingDevice must be unmounted before detaching
– Device is not being used (check withDevice is not being used (check with fuserfuser):):
●
Opening filesOpening files
●
Running programRunning program
●
Working directoryWorking directory
Ref. Pge. 163
22. Swap SpaceSwap Space
●
Frees out real memory by swapping outFrees out real memory by swapping out
unused pages to diskunused pages to disk
●
Swap pages in when reuseSwap pages in when reuse
●
Impacting performance by high swappingImpacting performance by high swapping
●
Partition ID is 82Partition ID is 82
Ref. Pge. 147
23. Swap CommandsSwap Commands
●
mkswap <partition>mkswap <partition>
– Formats partition as swap spaceFormats partition as swap space
●
swapon <partition>swapon <partition>
– Turns on swap spaceTurns on swap space
●
swapoff <partition>swapoff <partition>
– Turns off swap spaceTurns off swap space
●
freefree
– Displays memory usagesDisplays memory usages
24. Checking FilesystemChecking Filesystem
●
fsck <device>fsck <device>
– Examines file system and fixes errorsExamines file system and fixes errors
– Common optionsCommon options
●
A A : checks all file system: checks all file system
●
N N : test only: test only
●
y y : always answer YES: always answer YES
25. Monitor File System UseMonitor File System Use
●
dfdf
– Displays summary of disk spaceDisplays summary of disk space
– Common optionsCommon options
●
h h : human readable format: human readable format
●
dudu
– Displays directory consummation on diskDisplays directory consummation on disk
– Common optionsCommon options
●
h h : human readable format: human readable format
●
s s : summary of directories only: summary of directories only
Ref. Pge. 155/156
26. Quota ConceptQuota Concept
●
Limitation of data consumption for aLimitation of data consumption for a
user/group in a file systemuser/group in a file system
●
Based on block or inode usageBased on block or inode usage
●
Two types of limit:Two types of limit:
– Soft Limit: warning onlySoft Limit: warning only
– Hard Limit: errorHard Limit: error
– Grace Period: how long the soft limit may beGrace Period: how long the soft limit may be
exceeded, rechargeableexceeded, rechargeable
Ref. Pge. 205
27. Quota ImplementationQuota Implementation
●
Modify the option field ofModify the option field of /etc/fstab/etc/fstab
# defaults# defaults,usrquota,grpquota,usrquota,grpquota
●
Create record files:Create record files:
# touch /mount/point/aquota.{user,group}# touch /mount/point/aquota.{user,group}
●
Remount file systemRemount file system
# mount o remount /mount/point# mount o remount /mount/point
●
Calculate current useCalculate current use
# quotacheck /mount/point# quotacheck /mount/point
●
Turn on quotaTurn on quota
# quotaon /mount/point# quotaon /mount/point
29. Quota InformationQuota Information
●
quotaquota
– Reports quota usageReports quota usage
– A regular user can only views her/his own quotaA regular user can only views her/his own quota
●
repquota <device>repquota <device>
– Root onlyRoot only
– Reports quotas for all users and groupsReports quotas for all users and groups
– Common optionsCommon options
●
-u : displays user quota report-u : displays user quota report
●
-g : displays group quota report-g : displays group quota report
●
-a : all devices if no device specified-a : all devices if no device specified