Lenz Grimmer - Product Manager openATTIC - talks about "The Evolution of Storage on Linux" at FrOSCon 2015 in Germany.
Here comes a short description:
Linux and Open Source Software have always played a crucial role in data centers to provide storage in various ways. In this lecture, Lenz will give an overview of how storage on Linux has evolved over the years: - from local file systems to scalable file systems, logical volume managers and cluster file systems to today's modern file systems - distributed, parallel and fault-tolerant file systems.
Linux is an open-source operating system based on UNIX with a modular kernel. It uses processes, memory management and file systems similar to UNIX. The Linux kernel supports features like symmetric multiprocessing, virtual memory and loading of kernel modules. Popular Linux distributions package and distribute the Linux system along with utilities and applications.
Basic Information About Linux. This helps you to know about the basic details of linux, such as architecture, kernel design, process management, file management and etc.
This document provides an overview of the Linux operating system, including its history and design principles. It describes key components like the Linux kernel, kernel modules, process management, scheduling, and memory management. It discusses how Linux implements features like file systems, input/output, and interprocess communication. The document also covers Linux distributions and licensing. It provides details on the evolution of the Linux kernel from early versions to version 2.0 and beyond, which added support for new architectures and multiprocessor systems.
The document summarizes the architecture of the Linux operating system. It discusses the main components of Linux including the kernel, process management, memory management, file systems, device drivers, network stack, and architecture-dependent code. The kernel is at the core and acts as a resource manager. It uses a monolithic design. Process and memory management are handled via data structures like task_struct and buddy allocation. Virtual memory is implemented using page tables. File systems organize files in a hierarchy with inodes. Device drivers interface with hardware. The network stack follows a layered model. Architecture code is separated by subdirectory.
Linux uses a unified, hierarchical file system to organize and store data on disk partitions. It places all partitions under the root directory by mounting them at specific points. The file system is case sensitive. The Linux kernel manages hardware resources and the file system, while users interact through commands interpreted by the shell. Journaling file systems like ext3 and ReiserFS were developed to improve robustness over ext2 by logging file system changes to reduce the need for integrity checks after crashes. Ext4 further improved on this with features like larger maximum file sizes and delayed allocation.
Linux is an open-source operating system based on UNIX with a modular kernel. It uses processes, memory management and file systems similar to UNIX. The Linux kernel supports features like symmetric multiprocessing, virtual memory and loading of kernel modules. Popular Linux distributions package and distribute the Linux system along with utilities and applications.
Basic Information About Linux. This helps you to know about the basic details of linux, such as architecture, kernel design, process management, file management and etc.
This document provides an overview of the Linux operating system, including its history and design principles. It describes key components like the Linux kernel, kernel modules, process management, scheduling, and memory management. It discusses how Linux implements features like file systems, input/output, and interprocess communication. The document also covers Linux distributions and licensing. It provides details on the evolution of the Linux kernel from early versions to version 2.0 and beyond, which added support for new architectures and multiprocessor systems.
The document summarizes the architecture of the Linux operating system. It discusses the main components of Linux including the kernel, process management, memory management, file systems, device drivers, network stack, and architecture-dependent code. The kernel is at the core and acts as a resource manager. It uses a monolithic design. Process and memory management are handled via data structures like task_struct and buddy allocation. Virtual memory is implemented using page tables. File systems organize files in a hierarchy with inodes. Device drivers interface with hardware. The network stack follows a layered model. Architecture code is separated by subdirectory.
Linux uses a unified, hierarchical file system to organize and store data on disk partitions. It places all partitions under the root directory by mounting them at specific points. The file system is case sensitive. The Linux kernel manages hardware resources and the file system, while users interact through commands interpreted by the shell. Journaling file systems like ext3 and ReiserFS were developed to improve robustness over ext2 by logging file system changes to reduce the need for integrity checks after crashes. Ext4 further improved on this with features like larger maximum file sizes and delayed allocation.
Unix and shell programming | Unix File System | Unix File Permission | BlocksLOKESH KUMAR
This document provides an introduction to the Unix operating system, including its history, components, features, and file system organization. It discusses the kernel and shell components, types of shells like Bourn shell and C shell, and features such as multi-user capability, multitasking, security, and portability. It also describes the Unix file system structure, types of files, directories, and permission blocks.
The document provides an overview of the Linux architecture including:
1) It discusses the history of Linux from its origins as a free UNIX-like operating system developed by Linus Torvalds to the over 18 million lines of code it contains today.
2) It describes the key components of the Linux system architecture including the hardware layer, kernel, shell, and utilities. The kernel acts as the core of the OS and interacts with hardware to perform low-level services.
3) It outlines several important kernel functions including file system management, process management and scheduling, memory management, and device drivers which allow communication with I/O devices through device files.
The document provides an introduction to Unix presented by Ananthi Murugesan. It covers:
1. The history and origins of Unix from its development at Bell Labs in the 1960s-1970s.
2. An overview of what Unix is, including its portability, multi-user capabilities, and hierarchical file structure.
3. Details about the core components of Unix including the kernel, shell, and file management system.
The document provides an overview of the Ubuntu operating system. It discusses Ubuntu's history as a Debian-based Linux distribution first released in 2004. It covers Ubuntu's design principles including its use of the Linux kernel for process management, memory management, and file systems. It also addresses security topics like hacking threats and strategies for hardening Ubuntu systems. Basic commands and utilities included in Ubuntu are outlined.
Linux kernel Architecture and PropertiesSaadi Rahman
This document discusses the key components and architecture of the Linux kernel. It begins by defining the kernel as the central module of an operating system that loads first and remains in memory, providing essential services. It then describes the major subsystems of Linux, including process management, memory management, virtual file systems, network stacks, and device drivers. It concludes that the modular design of the Linux kernel has supported its growth and success through independent and extensible development of these subsystems.
Unix was created in 1969 by Ken Thompson at Bell Labs to allow multiple users to access a computer simultaneously. It features a multi-user design, hierarchical file system, and shell interface. The kernel handles memory management, process scheduling, and device interactions to enable these features. Common Unix commands like cat, ls, cp and rm allow users to work with files and directories from the shell. File permissions and ownership are managed through inodes to control access across users.
Linux is a free, open-source operating system based on UNIX with a modular kernel. It uses processes, threads, virtual memory, and files systems. Device drivers allow access to hardware via the block I/O system. Interprocess communication includes signals, pipes, shared memory, and semaphores. Security features authentication via PAM and access controls permissions via user and group IDs.
This document provides an overview of topics covered in a Red Hat System Administration training course, including:
1) Managing files graphically with Nautilus and getting help in both graphical and textual environments.
2) Managing physical storage by creating and formatting partitions, and managing logical volumes using LVM concepts.
3) Monitoring system resources such as processes, disk usage, and software packages.
4) Configuring networking and managing users, groups, and permissions.
The document provides information about the Ubuntu operating system. It discusses Ubuntu's history as a fork of Debian Linux that was created to be more user-friendly. It was founded by Mark Shuttleworth in 2004. The document also covers Ubuntu's design principles, use of the Linux kernel for processes, memory management, file systems, security features, and graphical user interface.
The document provides an overview of the Linux kernel internals including:
1) The Linux kernel architecture adopts a monolithic design that provides stability, safety, and high performance while maintaining advantages of both monolithic and microkernel approaches.
2) Key kernel components include processes and threads management, memory management using paging and segmentation, file systems, networking, and device drivers.
3) The kernel uses inodes and blocks to organize file system data on disk and supports various file systems like Ext2 through a virtual file system interface.
The document summarizes a presentation on the history and usage of Linux. It discusses:
- The dominance of proprietary operating systems in the 1960s-1970s and the motivation to create a free and open-source alternative called UNIX.
- How Linus Torvalds began developing Linux in 1991 based on UNIX to create a free academic version, gradually adding features over several years.
- Key advantages of Linux including being free, portable, scalable, and having short debug times. Some perceived disadvantages are too many distributions and being difficult to learn for newcomers.
- An overview of common Linux installation methods, partitioning disks, hardware configuration, and bootloaders like LILO that help Linux systems start
The document discusses the architecture of the Linux kernel. It describes the user space and kernel space components. In user space are the user applications, glibc library, and each process's virtual address space. In kernel space are the system call interface, architecture-independent kernel code, and architecture-dependent code. It then covers several kernel subsystems like process management, memory management, virtual file system, network stack, and device drivers.
The document provides an overview of the history and components of the Linux operating system. It discusses how Linux originated as a small kernel developed by Linus Torvalds in 1991 and has since evolved through collaborations. The core components of Linux include the kernel, system libraries, system utilities, and kernel modules. It also describes key aspects of Linux such as process management, scheduling, memory management, and file systems.
This document provides an introduction to using UNIX. It outlines the course aims of explaining what UNIX is used for and how to navigate files and directories. The agenda covers a brief history of UNIX, logging on, working with files and directories, commands like vi editor, and customizing the login environment. It describes the kernel, commands/utilities, and shell that make up the UNIX operating system model. It also explains the file structure, special characters, and transferring files between Windows and UNIX.
This document contains Linux commands for filesystem management, network management, system management, jobs and processes. It lists commands like df, du, fsck, mount, umount for managing filesystems and commands like arp, dig, finger, ftp, ifconfig, netstat, ping for network management. It also lists commands for managing system processes, users, time and shutdown.
Case study of windows a product of microsoft including the history and related to operating system with MS-DOS its scheduling, networking, performance, etc. It also contains the windows architecture, it's system components like kernel, and scheduling through threads in windows.
- The document provides an introduction and overview of the history and structure of the Unix operating system
- It describes how Unix was developed in the 1960s and 1970s at Bell Labs and others and became widely adopted, especially in academia
- The key aspects of Unix covered include its layered design, file system structure, use of directories, files and inodes to organize data, and how programs interface with the kernel through system calls
The document discusses the history and characteristics of UNIX operating systems. It describes how UNIX was developed in the 1960s and has since evolved into many different versions. Some key aspects of UNIX include its multi-user and multi-tasking capabilities, large number of free and commercial applications, and importance in internet development. The document also covers UNIX system calls, memory management, process management, and file systems.
This document discusses Ubuntu file systems. It begins with an overview of Ubuntu's history and pros and cons. It then covers the basics of file systems, including what they are, common types (e.g. ext2, ext3), and how they are structured and mounted. It also discusses commands used to create, modify, and manage file systems.
Unix is a multi-user and multi-tasking operating system developed in the 1960s and 1970s at Bell Labs. It was influenced by the MULTICS project and initially developed by Ken Thompson on a PDP-7 computer. Dennis Ritchie further developed Unix and created the C programming language. Unix became widely adopted on university campuses and later had several commercial releases from Bell Labs. Linux was later developed by Linus Torvalds in 1991 as a free Unix-like operating system and has become widely popular and distributed through different Linux distributions.
Assignment On Linux Unix Life Cycle And Its Commands Course Title System Pro...Robin Beregovska
The document provides an overview of the Linux operating system, including its history, components, design principles, and licensing. It discusses how Linux originated in 1991 as a hobby project and has since grown into a full-fledged operating system through collaboration. It describes the main components of Linux, including the Linux kernel, system libraries and tools from GNU and other open source projects, and various Linux distributions that package these components together.
The document summarizes Linux file systems and input/output. It discusses:
1) Linux file systems arrange files on disk storage in a structured collection. Common file systems include Ext2, Ext3, Ext4, JFS, ReiserFS, XFS, and Btrfs.
2) Linux uses two caches for input/output - a page cache that is unified with virtual memory, and a buffer cache for metadata.
3) Devices are classified as block, character, or network. Block devices allow random access to fixed blocks, while character devices don't need to support regular file functionality. Network devices use the kernel's networking subsystem.
Unix and shell programming | Unix File System | Unix File Permission | BlocksLOKESH KUMAR
This document provides an introduction to the Unix operating system, including its history, components, features, and file system organization. It discusses the kernel and shell components, types of shells like Bourn shell and C shell, and features such as multi-user capability, multitasking, security, and portability. It also describes the Unix file system structure, types of files, directories, and permission blocks.
The document provides an overview of the Linux architecture including:
1) It discusses the history of Linux from its origins as a free UNIX-like operating system developed by Linus Torvalds to the over 18 million lines of code it contains today.
2) It describes the key components of the Linux system architecture including the hardware layer, kernel, shell, and utilities. The kernel acts as the core of the OS and interacts with hardware to perform low-level services.
3) It outlines several important kernel functions including file system management, process management and scheduling, memory management, and device drivers which allow communication with I/O devices through device files.
The document provides an introduction to Unix presented by Ananthi Murugesan. It covers:
1. The history and origins of Unix from its development at Bell Labs in the 1960s-1970s.
2. An overview of what Unix is, including its portability, multi-user capabilities, and hierarchical file structure.
3. Details about the core components of Unix including the kernel, shell, and file management system.
The document provides an overview of the Ubuntu operating system. It discusses Ubuntu's history as a Debian-based Linux distribution first released in 2004. It covers Ubuntu's design principles including its use of the Linux kernel for process management, memory management, and file systems. It also addresses security topics like hacking threats and strategies for hardening Ubuntu systems. Basic commands and utilities included in Ubuntu are outlined.
Linux kernel Architecture and PropertiesSaadi Rahman
This document discusses the key components and architecture of the Linux kernel. It begins by defining the kernel as the central module of an operating system that loads first and remains in memory, providing essential services. It then describes the major subsystems of Linux, including process management, memory management, virtual file systems, network stacks, and device drivers. It concludes that the modular design of the Linux kernel has supported its growth and success through independent and extensible development of these subsystems.
Unix was created in 1969 by Ken Thompson at Bell Labs to allow multiple users to access a computer simultaneously. It features a multi-user design, hierarchical file system, and shell interface. The kernel handles memory management, process scheduling, and device interactions to enable these features. Common Unix commands like cat, ls, cp and rm allow users to work with files and directories from the shell. File permissions and ownership are managed through inodes to control access across users.
Linux is a free, open-source operating system based on UNIX with a modular kernel. It uses processes, threads, virtual memory, and files systems. Device drivers allow access to hardware via the block I/O system. Interprocess communication includes signals, pipes, shared memory, and semaphores. Security features authentication via PAM and access controls permissions via user and group IDs.
This document provides an overview of topics covered in a Red Hat System Administration training course, including:
1) Managing files graphically with Nautilus and getting help in both graphical and textual environments.
2) Managing physical storage by creating and formatting partitions, and managing logical volumes using LVM concepts.
3) Monitoring system resources such as processes, disk usage, and software packages.
4) Configuring networking and managing users, groups, and permissions.
The document provides information about the Ubuntu operating system. It discusses Ubuntu's history as a fork of Debian Linux that was created to be more user-friendly. It was founded by Mark Shuttleworth in 2004. The document also covers Ubuntu's design principles, use of the Linux kernel for processes, memory management, file systems, security features, and graphical user interface.
The document provides an overview of the Linux kernel internals including:
1) The Linux kernel architecture adopts a monolithic design that provides stability, safety, and high performance while maintaining advantages of both monolithic and microkernel approaches.
2) Key kernel components include processes and threads management, memory management using paging and segmentation, file systems, networking, and device drivers.
3) The kernel uses inodes and blocks to organize file system data on disk and supports various file systems like Ext2 through a virtual file system interface.
The document summarizes a presentation on the history and usage of Linux. It discusses:
- The dominance of proprietary operating systems in the 1960s-1970s and the motivation to create a free and open-source alternative called UNIX.
- How Linus Torvalds began developing Linux in 1991 based on UNIX to create a free academic version, gradually adding features over several years.
- Key advantages of Linux including being free, portable, scalable, and having short debug times. Some perceived disadvantages are too many distributions and being difficult to learn for newcomers.
- An overview of common Linux installation methods, partitioning disks, hardware configuration, and bootloaders like LILO that help Linux systems start
The document discusses the architecture of the Linux kernel. It describes the user space and kernel space components. In user space are the user applications, glibc library, and each process's virtual address space. In kernel space are the system call interface, architecture-independent kernel code, and architecture-dependent code. It then covers several kernel subsystems like process management, memory management, virtual file system, network stack, and device drivers.
The document provides an overview of the history and components of the Linux operating system. It discusses how Linux originated as a small kernel developed by Linus Torvalds in 1991 and has since evolved through collaborations. The core components of Linux include the kernel, system libraries, system utilities, and kernel modules. It also describes key aspects of Linux such as process management, scheduling, memory management, and file systems.
This document provides an introduction to using UNIX. It outlines the course aims of explaining what UNIX is used for and how to navigate files and directories. The agenda covers a brief history of UNIX, logging on, working with files and directories, commands like vi editor, and customizing the login environment. It describes the kernel, commands/utilities, and shell that make up the UNIX operating system model. It also explains the file structure, special characters, and transferring files between Windows and UNIX.
This document contains Linux commands for filesystem management, network management, system management, jobs and processes. It lists commands like df, du, fsck, mount, umount for managing filesystems and commands like arp, dig, finger, ftp, ifconfig, netstat, ping for network management. It also lists commands for managing system processes, users, time and shutdown.
Case study of windows a product of microsoft including the history and related to operating system with MS-DOS its scheduling, networking, performance, etc. It also contains the windows architecture, it's system components like kernel, and scheduling through threads in windows.
- The document provides an introduction and overview of the history and structure of the Unix operating system
- It describes how Unix was developed in the 1960s and 1970s at Bell Labs and others and became widely adopted, especially in academia
- The key aspects of Unix covered include its layered design, file system structure, use of directories, files and inodes to organize data, and how programs interface with the kernel through system calls
The document discusses the history and characteristics of UNIX operating systems. It describes how UNIX was developed in the 1960s and has since evolved into many different versions. Some key aspects of UNIX include its multi-user and multi-tasking capabilities, large number of free and commercial applications, and importance in internet development. The document also covers UNIX system calls, memory management, process management, and file systems.
This document discusses Ubuntu file systems. It begins with an overview of Ubuntu's history and pros and cons. It then covers the basics of file systems, including what they are, common types (e.g. ext2, ext3), and how they are structured and mounted. It also discusses commands used to create, modify, and manage file systems.
Unix is a multi-user and multi-tasking operating system developed in the 1960s and 1970s at Bell Labs. It was influenced by the MULTICS project and initially developed by Ken Thompson on a PDP-7 computer. Dennis Ritchie further developed Unix and created the C programming language. Unix became widely adopted on university campuses and later had several commercial releases from Bell Labs. Linux was later developed by Linus Torvalds in 1991 as a free Unix-like operating system and has become widely popular and distributed through different Linux distributions.
Assignment On Linux Unix Life Cycle And Its Commands Course Title System Pro...Robin Beregovska
The document provides an overview of the Linux operating system, including its history, components, design principles, and licensing. It discusses how Linux originated in 1991 as a hobby project and has since grown into a full-fledged operating system through collaboration. It describes the main components of Linux, including the Linux kernel, system libraries and tools from GNU and other open source projects, and various Linux distributions that package these components together.
The document summarizes Linux file systems and input/output. It discusses:
1) Linux file systems arrange files on disk storage in a structured collection. Common file systems include Ext2, Ext3, Ext4, JFS, ReiserFS, XFS, and Btrfs.
2) Linux uses two caches for input/output - a page cache that is unified with virtual memory, and a buffer cache for metadata.
3) Devices are classified as block, character, or network. Block devices allow random access to fixed blocks, while character devices don't need to support regular file functionality. Network devices use the kernel's networking subsystem.
I have described all about linux OS starting from basics.
I guess this PPT will really be very very helpful for you guys.
This was one of the most appreciable PPT in my time when i presented it in my class.
This document provides an overview of the XFS file system for Linux. Some key points:
- XFS was designed from the start for large storage systems with many CPUs and large disk arrays, focusing on large files and streaming I/O performance.
- It uses B+ trees and extents to efficiently manage free space, file data blocks, and inode metadata, allowing it to scale to much larger file systems and files than ext3.
- Features like delayed allocation help achieve large contiguous block allocations, improving sequential I/O performance compared to ext3 and ext4. However, XFS sees less benefit for random I/O workloads.
- XFS supports direct I/O to
This document provides an overview of the CSC 539 Operating Systems Structure and Design course. It discusses influential early operating systems like Atlas, CTSS, MULTICS, OS/360, UNIX, Alto and Mach. It then focuses on case studies of the Linux and Windows XP operating systems, describing their histories, design principles, process management, memory management, virtual memory, file systems and more.
This document provides an overview of the Ubuntu operating system. It discusses Ubuntu's history as a Debian-based Linux distribution first released in 2004. It also covers Ubuntu's key features, components, and principles such as its use of the Linux kernel, GNOME desktop environment, and Ext4 file system. The document discusses Ubuntu's security, processes, memory management, networking, and compares its robustness to other operating systems. It provides examples of basic Ubuntu commands and how to password protect the GRUB boot loader for added security.
This document provides an overview of the development of Linux and open source operating systems. It describes how Linux originated from earlier systems like UNIX and how Linus Torvalds released the first version in 1991. It also lists some of the major Linux distributions like Debian, Ubuntu, Fedora and compares characteristics of the Linux kernel to Windows.
The document provides an introduction to the Linux operating system, including:
- A brief history of UNIX and Linux, describing their origins in the 1960s-1990s.
- An overview of Linux distributions, kernels, features, and structure, explaining concepts like monolithic vs. microkernel designs.
- Descriptions of key Linux components like modules, eBPF, and the roles of processes, user mode, kernel mode, and context switches.
- Discussions of ongoing developments like extended BPF which allow more dynamic programmability of the Linux kernel.
This document provides an overview and summary of the Ubuntu operating system. It discusses Ubuntu's history beginning with the creation of Linux by Linus Torvalds in 1991. It then covers key aspects of Ubuntu including its design principles, process and memory management, security features, networking capabilities, and graphical user interface. The document also provides examples of basic commands and important directories in Ubuntu. It concludes with recommendations for securing the boot loader to help harden the Ubuntu system.
The document discusses Linux operating system design principles, kernel modules, process management, scheduling, memory management, and input/output management. It covers Linux history and key versions. It also describes the components of a Linux system including the kernel, system libraries, and system utilities. It provides details on Linux process management, kernel modules, scheduling, and memory management.
The document discusses the Linux operating system, including its history, design principles, kernel modules, process management, scheduling, memory management, input/output management, file systems, and inter-process communication. It also briefly covers the architectures and frameworks of two popular mobile operating systems, iOS and Android. The document provides details on Linux kernel versions and distributions, and explains concepts like kernel synchronization, interrupt handling, and the Completely Fair Scheduler algorithm used in Linux.
This document provides an overview of the history and development of the Linux operating system. It discusses how Linux originated from earlier operating systems like Unix and was completed using software from the GNU project. Key advantages of Linux mentioned include low cost, stability, performance, flexibility and security. The document also describes the file hierarchy and culture of free software in Linux. It provides examples of commands used in Linux and explains the use of sudo for privilege escalation.
The document provides an overview of the UNIX operating system through a seminar presentation. It discusses the history of UNIX from the 1970s to the 2000s, defines what UNIX is, describes common UNIX commands and the file system structure, and covers topics like memory management, interrupts, reasons for using UNIX, and some applications of UNIX like storage consulting and middleware/database administration. The presentation is intended to educate about the key aspects and functionality of the UNIX operating system.
The document summarizes key aspects of the Linux operating system as described in Chapter 16 of the 9th Edition of Operating System Concepts by Silberschatz, Galvin and Gagne. It covers Linux history and design principles, kernel modules, process management, scheduling, and memory management. The core topics of the chapter are explained, including the Linux kernel, distributions, licensing, and the major components that make up the Linux system.
The document provides an overview of the history and design of the Linux operating system in 3 paragraphs:
Linux was first developed in 1991 by Linus Torvalds as a small kernel for compatibility with UNIX. It has since grown through collaboration over the internet to run on various hardware platforms while remaining free and open source. Early versions only supported 386 processors and basic functionality, while later versions added support for new hardware, file systems, and networking.
The core components of Linux include the kernel, system libraries, and system utilities. The kernel provides core system functions and resource management. Libraries and utilities are developed separately but work together to provide a full UNIX-compatible system. Device drivers, file systems, and network protocols can
This are the slides from Lenz Grimmer, Team Lead for Product Management and Development talking about openATTIC and Ceph Management at Suse Monthly Open Source Talks in July 2016.
openATTIC Ceph Management @ OpenSuse Con - 2016-06-23it-novum
This is the presentation of Lenz Grimmer, Team Lead for Product Management and Development, at OpenSuse Con 2016 regarding Ceph and Storage Management with openATTIC.
This is the presentation of Lenz Grimmer, Team Lead for Product Management and Development, at Ceph Tech Talks 2016 regarding Ceph and Storage Management with openATTIC.
An overview about the openATTIC open source storage management system and the state of the Ceph support. Presented by Lenz Grimmer at the Vault Conference 2016 in Raleigh, NC.
Tweets und Aktienkurse? Wertvolle Erkenntnisse durch Data Blending gewinnenit-novum
Stefan Müller/ Director Business Intelligence & Big Data erklärt in seinem Vortrag wie die Kombination von Datenquellen aus klassischen Big Data Analytics-Systemen mit real-time Informationen erfasst und von der Open Source Business Intelligence-Lösung von Pentaho dargestellt werden können. Der BI- und Big Data-Spezialist zeigt anhand eines Best Practice-Beispiels auf wie bei der Gegenüberstellung von Aktienkursanalysen und aktuellen Daten aus dem Nachrichtenkurzdienst Twitter entscheidungsrelevante Zusammenhänge ersichtlich werden.
Flexible storage management with Linux and openATTICit-novum
At this years Kieler Open Source and Linux Days Lenz Grimmer (Product Manager of openATTIC) has presented how easy and flexible Storage Management with Linux and openATTIC can be. See here the complete slides.
Closing the Storage gap - presentation from OpenStack Summit in Vancouver 2015it-novum
This are the original slides from Michael Kienle´s presentation at OpenStack Summit in Vanouver, dated May 20th 2015 regarding Closing the storage gap - how to administrate Ceph storage resources without deep-diving into the technical details
Open Cloud Storage @ OpenStack Summit Parisit-novum
This slides are the original slides from Michael Kienle @ OpenStack Summit in Paris November 2014 focusing on Open Cloud Storage - Building a flexible and large - scale software-defined storage platform for OpenStack
OpenStack Day Italy: openATTC as an open storage platform for OpenStackit-novum
The first OpenStack Day in Italy took place in Milan on Friday, May 30. This presentation shows how the open source storage project openATTIC can be used as storage platform for cloud systems like OpenStack.
openATTIC is a storage project started in 2012 in Germany. It can be downloaded at www.openattic.org
Building an open source cloud storage platform for OpenStack - openATTICit-novum
Although OpenStack is purposely open it partly relies on proprietary storage products. To build your cloud upon a truly open software-defined storage platform (SDS), the storage project openATTIC might be worth a try. openATTIC is a cloud storage platform based on 100% open source. It is optimized for OpenStack's Cinder component and for openQRM.
The openATTIC project has been initiated to support organizations in getting the best return on investment in their data center operations while achieving greatest flexibility. openATTIC is based on open source software and extended with intelligent storage functions. Building software-defined storage platforms with openATTIC is easy as it gives you complete access to all its functionalities via a single central API making integration and extension both simple and inexpensive. openATTIC is hardware-independent and includes intelligent SDS functions such as
- consistent snapshots of VMs, databases and applications
- high availability and high performance
- the comprehensive variety of protocols typical of a unified storage system.
- variety of standard hardware can be used
- secure & enterprise-class reliable
More about the openATTIC project at http://openattic.org/en
This presentation by Thibault Schrepel, Associate Professor of Law at Vrije Universiteit Amsterdam University, was made during the discussion “Artificial Intelligence, Data and Competition” held at the 143rd meeting of the OECD Competition Committee on 12 June 2024. More papers and presentations on the topic can be found at oe.cd/aicomp.
This presentation was uploaded with the author’s consent.
Collapsing Narratives: Exploring Non-Linearity • a micro report by Rosie WellsRosie Wells
Insight: In a landscape where traditional narrative structures are giving way to fragmented and non-linear forms of storytelling, there lies immense potential for creativity and exploration.
'Collapsing Narratives: Exploring Non-Linearity' is a micro report from Rosie Wells.
Rosie Wells is an Arts & Cultural Strategist uniquely positioned at the intersection of grassroots and mainstream storytelling.
Their work is focused on developing meaningful and lasting connections that can drive social change.
Please download this presentation to enjoy the hyperlinks!
This presentation by OECD, OECD Secretariat, was made during the discussion “Pro-competitive Industrial Policy” held at the 143rd meeting of the OECD Competition Committee on 12 June 2024. More papers and presentations on the topic can be found at oe.cd/pcip.
This presentation was uploaded with the author’s consent.
Mastering the Concepts Tested in the Databricks Certified Data Engineer Assoc...SkillCertProExams
• For a full set of 760+ questions. Go to
https://skillcertpro.com/product/databricks-certified-data-engineer-associate-exam-questions/
• SkillCertPro offers detailed explanations to each question which helps to understand the concepts better.
• It is recommended to score above 85% in SkillCertPro exams before attempting a real exam.
• SkillCertPro updates exam questions every 2 weeks.
• You will get life time access and life time free updates
• SkillCertPro assures 100% pass guarantee in first attempt.
This presentation by Nathaniel Lane, Associate Professor in Economics at Oxford University, was made during the discussion “Pro-competitive Industrial Policy” held at the 143rd meeting of the OECD Competition Committee on 12 June 2024. More papers and presentations on the topic can be found at oe.cd/pcip.
This presentation was uploaded with the author’s consent.
This presentation by Professor Alex Robson, Deputy Chair of Australia’s Productivity Commission, was made during the discussion “Competition and Regulation in Professions and Occupations” held at the 77th meeting of the OECD Working Party No. 2 on Competition and Regulation on 10 June 2024. More papers and presentations on the topic can be found at oe.cd/crps.
This presentation was uploaded with the author’s consent.
This presentation by OECD, OECD Secretariat, was made during the discussion “Artificial Intelligence, Data and Competition” held at the 143rd meeting of the OECD Competition Committee on 12 June 2024. More papers and presentations on the topic can be found at oe.cd/aicomp.
This presentation was uploaded with the author’s consent.
Suzanne Lagerweij - Influence Without Power - Why Empathy is Your Best Friend...Suzanne Lagerweij
This is a workshop about communication and collaboration. We will experience how we can analyze the reasons for resistance to change (exercise 1) and practice how to improve our conversation style and be more in control and effective in the way we communicate (exercise 2).
This session will use Dave Gray’s Empathy Mapping, Argyris’ Ladder of Inference and The Four Rs from Agile Conversations (Squirrel and Fredrick).
Abstract:
Let’s talk about powerful conversations! We all know how to lead a constructive conversation, right? Then why is it so difficult to have those conversations with people at work, especially those in powerful positions that show resistance to change?
Learning to control and direct conversations takes understanding and practice.
We can combine our innate empathy with our analytical skills to gain a deeper understanding of complex situations at work. Join this session to learn how to prepare for difficult conversations and how to improve our agile conversations in order to be more influential without power. We will use Dave Gray’s Empathy Mapping, Argyris’ Ladder of Inference and The Four Rs from Agile Conversations (Squirrel and Fredrick).
In the session you will experience how preparing and reflecting on your conversation can help you be more influential at work. You will learn how to communicate more effectively with the people needed to achieve positive change. You will leave with a self-revised version of a difficult conversation and a practical model to use when you get back to work.
Come learn more on how to become a real influencer!
This presentation by Juraj Čorba, Chair of OECD Working Party on Artificial Intelligence Governance (AIGO), was made during the discussion “Artificial Intelligence, Data and Competition” held at the 143rd meeting of the OECD Competition Committee on 12 June 2024. More papers and presentations on the topic can be found at oe.cd/aicomp.
This presentation was uploaded with the author’s consent.
This presentation by OECD, OECD Secretariat, was made during the discussion “Competition and Regulation in Professions and Occupations” held at the 77th meeting of the OECD Working Party No. 2 on Competition and Regulation on 10 June 2024. More papers and presentations on the topic can be found at oe.cd/crps.
This presentation was uploaded with the author’s consent.
Competition and Regulation in Professions and Occupations – OECD – June 2024 ...
The evolution of storage on Linux
1. The Evolution of Storage
on Linux
Lenz Grimmer <lenz.grimmer@it-novum.com>
FrOSCON 2015, Sankt Augustin
22. August 2015
2. 2
Agenda
A trip down memory lane (pun intended)
Overview of how storage on Linux has evolved
Local file systems and related concepts/technologies
Network Services
Distributed / Cluster filesystems
3. 3
Introduction
40+ file systems in /fs/
Focus on the most popular/widely used systems
Primary focus on the software side
High-level Descriptions only
4. 4
Noteworthy Observations / Conclusions
The role of SourceForge.net today
Distribution kernels vs. mainline Linux
Honorable mention: Christoph Hellwig
Don‘t miss his talk about the Linux Storage Stack tomorrow (14:00, HS6)
Big Thanks to: LWN, Kernelnewbies.org, Thorsten Leemhuis
(Heise) and Wikipedia
6. 6
MINIX file system
While developing Linux in 1991, Linus required some form of
persistent storage
A Minix-compatible file system was the canonical choice:
Well-documented, robust
Exchange data with the host OS (and vice versa)
Severely limited
Max. file/filesystem size: 64MB (16bit block addresses)
14 char file names
Only one time stamp (mtime)
7. 7
Virtual File System Switch (VFS)
Abstraction / indirection layer to route file oriented system calls to
necessary functions in the physical filesystem code to do the I/O
Eased the addition of new file systems
Initially written by Chris Provenzano
Integrated into Linux 0.96
Defines a set of functions that every filesystem has to implement
Three kinds of objects: filesystems, inodes, and open files
8. 8
Extended File System (ext)
Designed by Rémy Card
Max. file/filesystem size: 2 GB, max. file name size was 255 chars
Metadata structure inspired by the traditional Unix File System
(UFS)
Added to Linux 0.96c in April 1992
Issues remained (bad performance, missing time stamps,
fragmentation)
9. 9
Second Extended File System (ext2)
Also implemented by Rémy Card
Introduced in Linux Kernel 0.99 (January 1993)
Designed with extensibility in mind
Adopted advanced ideas from other file systems (e.g. BSD Fast File System),
e.g. mtime/ctime/atime, file attributes, BSD/SysV semantics, different block
sizes, immutable/append-only files
Initially supported file/file systems sizes up to 2TB (limitation of the block
device layer)
Kernel version 2.6.17 (March 2006) extended max. file system size to 32TB
(using 8kB Blocks)
10. 10
FAT/MSDOS
Added to Linux in 1992/1993 by Werner Almesberger
VFAT support was later developed by Gordon Chaffee
VFAT filesystem is compatible with Windows 95/NT long filenames on the
FAT filesystem
Initially called xmsdos
Patches for Linux 1.2.x and 1.3.x.
As of Linux 1.3.60, the vfat filesystem is part of the Linux kernel distribution
Mtools as a userland-only alternative
11. 11
NTFS
NTFS driver for Linux by Martin von Löwis (started around 1996)
Legato Systems later sponsored Anton Altaparmakov to further
develop NTFS on Linux since June 2001
Read-only mode only, with no fault-tolerance supported
NFTS-TNG replaced old NTFS driver in Linux 2.5.11 (April 29th,
2002)
NTFS-3G (FUSE-based) by Tuxera (read-write support)
13. 13
Fsck vs. Journaling
Unclean unmounts, too many mount counts, or remounts after
a long time period triggered file system checks
Disk drives got bigger
A Journaling file system keeps track of changes not yet
committed to the file system's main part in a Journal
Keep track of just metadata changes or data as well
Several file systems were developed in parallel, to alleviate this
shortcoming of ext2, namely ext3, XFS, JFS and ReiserFS.
14. 14
Journaling Block Device layer (JBD)
JBD established as a filesystem-independent service, to be used
by any file system
First incarnation of JBD developed by Stephen C. Tweedie
together with the ext3 file system
OCFS2 and later ext4 also used JBD and it’s successor JBD2
15. 15
Third extended filesystem (ext3)
Originally released in September 1999
Written by Stephen Tweedie for the 2.2 branch
Ported to 2.4 kernels by Peter Braam, Andreas Dilger, Andrew
Morton, Alexander Viro, Ted Ts'o and Stephen Tweedie
Merged with the mainline Linux kernel 2.4.15 (November 2001)
Basically ext2 with journaling capabilities, easy conversion
Max filesystem size: 8TB, Max 32k subdirs/directory
16. 16
IBM JFS
Rooted in AIX and OS/2 Warp Server (new design in 1995)
Port to Linux started in December 1999 (Dave Kleikamp, Steve Best)
Uses own journaling implementation (metadata only)
Max volume size: 32PB, Max file size: 4PB
Later ported to AIX 5L as JFS2 (April 2001)
JFS 0.0.1 released in Feb. 2000., 0.1.0 (Beta) in August 2000
Version 1.0.0 was released in June 2001
Kernel module since 2.4.18pre9-ac4, Version 1.1.0 was included by Marcelo
Tosatti in Linux 2.4.20.
17. 17
ReiserFS
Early supported by SuSE, Introduced in version 2.4.1 (2001)
The first journaling file system to be included in mainline
Max volume size: 16TB
Based on B+ trees
Metadata-only journaling (block journaling since 2.6.8)
Online resizing
Tail packing block suballocation
Reiser4 still under active development (Edward Shishkin)
18. 18
SGI XFS
64-bit journaling file system created by Silicon Graphics
SGI IRIX since 1994, GPLed in 2000
Version 1.0 for Linux in May 2001 as Patch against 2.4.2
Merged in 2.6.x and 2.4.25 (Feb 2004)
Steve Lord, Russell Cattelan, Nathan Scott, Jim Mostek
Advanced features, high performance
Max volume size: 16EB
20. 20
The need for Logical Volume Management
Initially, Linux could only address disks/partitions
Changes to the layout required downtime and shuffling of data
Logical Volume Management abstracts physical disk drives
First incarnation of Linux LVM was introduced in Kernel version
2.4
Heinz Mauelshagen wrote the original LVM code in 1998,
inspired by HP-UX's volume manager.
21. 21
Device Mapper (DM)
A kernel framework for mapping physical block devices onto higher-
level virtual block devices
Added in Linux 2.6
Passes data from a virtual block device, which is provided by the
device mapper itself, to another block device
Pluggable design
Data can be also modified in transition
Forms the foundation of LVM2/EVMS, RAID and dm-crypt disk
encryption and many other useful features
22. 22
DM Multipath (DM-MPIO)
Consists of kernel components and user-space components
Provides input-output (I/O) fail-over and load-balancing within Linux
for block devices
Handles the rerouting of block I/O to an alternate path in the event of
a path failure
Can also balance the I/O load across all of the available paths in Fibre
Channel (FC) or iSCSI SAN environments
Started as part of a patchset created by Joe Thornber, later
maintained by Alasdair G Kergon at Red Hat. Christophe Varoqui
maintains the userland multipath tools
23. 23
DM-Cache
Allows a fast device (e.g. an SSD) to be used as a cache for a slower device
(e.g. a rotating disk)
Different policy plugins can be used to change the algorithms used to select
which blocks are promoted, demoted, cleaned etc.
Supports writeback and writethrough modes
Requires three physical storage devices to separately store actual data,
cache data and required metadata
Joe Thornber, Heinz Mauelshagen and Mike Snitzer
Inclusion into the Linux mainline kernel version 3.9, released on April 28,
2013
24. 24
LVM2
Based on DM
Flexible storage management
Add/remove disks
Resize/move logical volumes
Move LVs between PVs
Span volumes across multiple physical devices
RAID
Thin provisioning
Cluster Volume Manager
25. 25
IBM EVMS
IBM-sponsored effort to provide volume management services for
Linux
A single, unified system for handling all storage management tasks
Despite many of the features and GUI management tools found in
EVMS, LVM2 was preferred
As a result, IBM dropped their kernel driver and reworked their tools
to work with LVM2 instead
Development stopped in 2006
27. 27
NFS
Rick Sladkey original author of the NFS client and also ported the NFS server
and the RPC library code. Doug Quale helped extending the kernel to
support networking filesystems
NFS Version 2 since 1.2 kernel series
Kernel 2.2.18 a major milestone: mixing Linux NFS with other operating
systems' NFS, use file locking reliably over NFS, and NFS Version 3.
NFS Versions 2, 3, and 4 are supported on 2.6 and later kernels. Version 4.1
(Client) at least kernel 2.6.31
NFSv4 for Linux has been under development at CITI and NetApp since 2001
28. 28
Samba
A free-software re-implementation of the SMB/CIFS networking protocol
Andrew Tridgell started development of Samba in 1992, Jeremy Allison
joined early on
Volker Lendecke founded SerNet in 1997, to provide commercial support
Version 3 (2003): file and print services for Microsoft Windows clients and can
integrate with a Windows NT 4.0 server domain, either as a Primary Domain
Controller (PDC) or as a domain member
Samba4 installations can act as an Active Directory domain controller or
member server, at Windows 2008 domain and forest functional levels.
29. 29
SMB vs.CIFS
SMB "server message block" and CIFS "common internet file system"
are protocols. CIFS is the extension of the SMB protocol
“smbfs” was an older FS originated from the Samba project, heavily
coupled with the Samba tools (smb.conf, smbmount, etc.). Removed
in Linux 2.6.27
CIFS VFS was added to mainline Linux kernels in 2.5.42 Supports
advanced network file system features such as locking, Unicode
(advanced internationalization), hardlinks, dfs (hierarchical,
replicated name space), distributed caching and uses native TCP
names. All key network functions implemented in kernel
31. 31
Fourth Extended Filesystem (ext4)
Advanced version of ext3, led by Ted Tso et al
Incorporated scalability and reliability enhancements for supporting
large filesystems up to 1EB.
First experimental support for ext4 was merged into Linux 2.6.19,
which was released on 29 November 2006.
Ext4 was marked as experimental until Linux 2.6.27
Starting with 2.6.28 (December 2008), ext4 was marked as stable
New extent format reduced metadata overhead (RAM, IO for access,
transactions)
32. 32
Btrfs
Chris Mason (Oracle) in 2007
COW (Snapshots)
Checksums, Compression
RAID, Volume management
Conversion of ext3/4 file systems
Merged into mainline Linux 2.6.29 (March 2009)
Florian Winkler talks about Btrfs today (11:15, HS7)
33. 33
ZFS
Filesystem and logical volume manager combined
Designed and implemented at Sun Microsystems (Jeff Bonwick, Matthew
Ahrens)
Development started in 2001,officially announced in 2004
128bit, COW, Snapshots, Deduplication, RAID
OpenSolaris (CDDL)
Early port based on FUSE
Kernel modules based OpenZFS (2013)
Not included in mainline Linux due to license incompatibilities
35. 35
Network Block Device (NBD)
Remotely access a block device attached to another system
Userspace Server/Client, Client kernel module
Issues arise if network goes down or server crashes
Markus Pargmann talks about NBD on Sunday (16:30, HS6)
36. 36
Distributed Replicated Block Device (DRBD)
A shared-nothing, synchronously replicated block device
“RAID1 over Network”
Writes to the primary node are transferred to the lower-level block device and
simultaneously propagated to the secondary node
The secondary node then transfers data to its corresponding lower-level block
device. All read I/O is performed locally
Fail-over capabilities (Secondary/Primary)
Lars Ellenberg and Philipp Reisner originally submitted code in July 2007
DRBD was merged on 8 December 2009 during the "merge window" for Linux
kernel version 2.6.33
38. 38
OCFS/OCFS2
Shared disk file system by Oracle
Main focus of OCFS was to accommodate Oracle clustered databases,
not POSIX-compliant
OCFS2 designed as a Linux filesystem from scratch
On-disk filesystem implementation heavily inspired by ext3, uses JBD
for journaling
OCFS2 integrated into version 2.6.16 of mainline Linux
Max Volume/File Size 4PB (currently limited to 16TB)
Trivia question: what feature do OCFS2 and Btrfs have in common?
39. 39
GFS/GFS2
Shared disk filesystem, allows concurrent access to the same block storage
Development of GFS began in 1995 and was originally developed by
University of Minnesota professor Matthew O'Keefe and a group of students
Originally for SGI IRIX, ported to Linux in 1998
Acquired by Sistina in 2000, turned into proprietary product
OpenGFS fork
Red Hat acquired Sistina in 2003 and released GFS2 under GPL in June 2004
GFS2 and the DLM merged into Linux 2.6.19 (29 November 2006)
40. 40
Storage Requirements and Challenges
Amount of data to be stored grows exponentially
Today, Storage has to be:
Fault tolerant, reliable
Scalable without limitations or service interruptions
Distributable
Easy to manage / automate
Previous approaches do not address these requirements
42. 42
GlusterFS
Aggregates various storage servers over Ethernet or Infiniband RDMA
interconnect into one large parallel network file system
Storage bricks export local file systems as volumes
GlusterFS clients create composite virtual volumes from multiple remote
servers using stackable „translators“
Translators provide Mirroring, Replication, Striping, etc.
Final volume mounted by client host using its own native protocol via FUSE,
using NFS v3 protocol (via built-in server translator)
Originally developed by Gluster, Inc., which was acquired by Red Hat in 2011
43. 43
Ceph
Initially created by Sage Weil, founded Inktank in 2012
First release in July 2012
Object, block, and file storage from a single distributed computer cluster
Reliable autonomic distributed object store (RADOS)
RADOS Block Device (RBD), Snapshots
RadosGW provides REST API (Amazon S3/OpenStack Swift)
Completely distributed without a single point of failure
Replicates data for fault tolerance (CRUSH)
Ceph client code was merged into mainling Linux version 2.6.34
Red Hat acquired Inktank in April 2014
44. 44
Lustre
Parallel distributed file system, generally used for large-scale cluster computing
Widely used in TOP500 supercomputers
Max. volume size: 100 PB (production), over 16 EB (theoretical)
Max. file size: 2.5 PB (ext4), 16 EB (ZFS)
Started as a research project in 1999 by Peter Braam at CMU, who founded Cluster Filesystems Inc. in
2001 to work on Intermezzo, Coda and Lustre
First installed in March 2003 on the MCR Linux Cluster (Lawrence Livermore National Laboratory).
Lustre 1.0.0 was released in December 2003.
Acquired by Sun Microsystems in 2007
Oracle acquired Sun in 2010 and discontinued the development
Whamcloud->Intel, OpenScalabaleFilesystems Inc. (OpenSFS), Xyratex Inc.
45. 45
Shameless plug: openATTIC
Unified Storage: manage XFS, ZFS, Btrfs, NFS, Samba
Modern GUI (AngularJS/Boostrap)
REST API
Built-in Monitoring
Clustering (Pacemaker/Corosync, DRBD)
http://www.openattic.org/
Find us in the exhibition hall
46. 46
PHP-ENTWICKLER (M/W) mit
Linux Know-how
Sie entwickeln leidenschaftlich gerne und fühlen sich im
Open Source-Umfeld Zuhause?
Dann sollten wir uns kennenlernen!
Diese Aufgaben erwarten Sie bei uns…
• Entwicklung unseres Systemmonitoring-Tools
openITCOCKPIT für Frontend und/oder Backend
• Konzeption und Realisierung von Projekten in
Teamarbeit
• Testing der entwickelten Anwendungen
• Pflege und Ausbau der bestehenden Entwicklungs- und
Testumgebung
Weitere Informationen finden Sie unter:
www.it-novum.com/karriere
Gesucht: PHP-Entwickler (m/w) mit Linux Know-How