PFcache (Linuxcon, Seattle, 2015)

The document provides an overview of log structured file systems. It discusses how log structured file systems work by writing all data and metadata sequentially to a circular buffer called a log to improve write performance. It also describes how log structured file systems address issues like limited disk space through garbage collection and provide simpler crash recovery without requiring a file system check.

Glusterfs session #1 disk filesystems

Pranith Karampuri

This document provides an overview of disk filesystems and network filesystems from the perspective of GlusterFS. It discusses the basic data structures of files and directories, including inodes, data blocks, and representation of different file types. It also outlines the main Linux system calls used to manipulate filesystem metadata and data, such as read, write, truncate, and directory operations. These calls can operate on files via paths, file descriptors, or directory file descriptors.

Building a userspace filesystem in node.js

Clay Smith

Introduction to file system and OCFS2

This document provides an introduction to file systems and the OCFS2 file system. It begins with basic concepts of how data can be stored using block devices, databases, and file systems. It then discusses file system interfaces, I/O models, and classifications. It provides an overview of the virtual file system (VFS) layer and its key data structures. It describes the EXT3 and OCFS2 file systems in detail, covering their layouts, journaling, mounting, and space management.

Redis is an in-memory data structure store that can be used as a database, cache, or message broker. It supports various data structures like strings, hashes, lists, sets, and sorted sets. Data can be persisted to disk for durability and replicated across multiple servers for high availability. Redis also implements features like expiration of keys, master-slave replication, clustering, and bloom filters.

RDM#2- The Distributed Web

Node.js Interactive

The document discusses module management and the InterPlanetary File System (IPFS). It covers topics like how modules are currently managed through tools like npm, and some of the limitations of existing systems. It then introduces IPFS as a new protocol that could be used to upgrade the web by allowing modules and other content to be permanently stored, distributed and accessed in a decentralized manner. Key components of IPFS discussed include its use of content addressing, distributed hash tables, the merkle dag data structure, IPNS for mutable naming, and how these pieces could provide benefits like discovery, integrity, transport and updating of modules in a more robust way compared to existing systems.

Glusterfs session #2 1 layer above disk filesystems

Pranith Karampuri

redis basics

Manoj Kumar

B tree file system

Dinesh Gupta

The evolution of linux file system

The document summarizes the evolution of Linux file systems from local to cluster to distributed systems. It discusses Ext2, Ext3, and Ext4 local file systems and improvements made to support larger file systems and reduce filesystem check times. It introduces cluster file systems used with shared storage for high availability and scaling compute and storage. Distributed file systems are described as scaling to unified storage across commodity hardware, with examples like HDFS based on the Google File System model with separate metadata and data servers. Current trends include further scaling out, flash technology use, and unified object/block/file storage.

Non-Framework MVC sites with PHP

This document discusses building a Twitter clone without using a framework. It demonstrates using PHP, Simple-ORM, MySQL initially, then switching to MongoDB. It argues that for many small and medium websites, a relational database is not needed and MongoDB provides a simpler alternative. The document also outlines strategies for scaling the application by using fast front-end servers, client caching, and DNS round-robin load balancing across front-end servers.

Containers from scratch

Rakuten Group, Inc.

Storage with ceph (osdc 2013)

Martin Loschwitz

This document discusses the scale-out storage capabilities of Ceph. It explains that Ceph uses an object store model called RADOS to allow for scaling storage horizontally across commodity hardware. Ceph uses a technique called CRUSH to automatically replicate and distribute data across its object storage daemons and monitor daemons for redundancy and high availability as more nodes are added. It also describes how Ceph provides block storage, file system, and cloud storage interfaces to stored data through its RADOS Block Device, CephFS, and RADOS Gateway components.

OSBConf 2015 | Scale out backups with bareos and gluster by niels de vos

NETWAYS

MongoDB

jsterce

Document-oriented databases store data in collections of documents rather than tables with a predefined schema. MongoDB is an open-source, document-oriented database that is easy to install and use. It supports many programming languages and operating systems. Documents in MongoDB can have unique field sets and storing non-normalized data can provide faster speeds than relational databases for heavy use cases.

Filesystem Comparison: NFS vs GFS2 vs OCFS2

Giuseppe Paterno'

The document compares the performance of NFS, GFS2, and OCFS2 filesystems on a high-performance computing cluster with nodes split across two datacenters. Generic load testing showed that NFS performance declined significantly with more than 6 nodes, while GFS2 maintained higher throughput. Further testing of GFS2 and OCFS2 using workload simulations modeling researcher usage found that OCFS2 outperformed GFS2 on small file operations and maintained high performance across nodes, making it the best choice for the shared filesystem needs of the project.

Redis, a 2 minutes introduction

Mirko Calvaresi

The Care + Feeding of a Mongodb Cluster

Chris Henry

TopDB data transfer

Chonpin HSU

This document discusses transferring data between Redis and MongoDB using TopDB. Redis is an open source, advanced key-value store that can store different data structures like strings, hashes, lists, and sets under keys. MongoDB is an open source document database and leading NoSQL database written in C++. TopDB implements a Jedis-like interface to support Redis key types and mappings between Redis databases and MongoDB databases to enable hot/cold data transfer between the two databases.

Ext filesystem4

Neha Kulkarni

Thinking in documents

The document discusses scaling databases and moving from relational databases to NoSQL document databases. It provides an overview of several NoSQL databases like Redis, Tokyo Cabinet, Cassandra, CouchDB and MongoDB. It focuses on how MongoDB stores data as documents rather than tables and rows. It provides examples of common queries and operations in MongoDB like counts, filters, pagination and more to demonstrate how to model data and think in documents rather than tables and rows.

OmniXtend

singh.gurjeet

OmniXtend is an open, heterogeneous architecture that allows main memory to be shared equally across systems. It enhances TileLink to serialize cache coherence requests over Ethernet for shared access to memory without context switching or software overhead. Initial measurements show OmniXtend providing global memory access between systems with no local DRAM. The CHIPS Alliance is working to integrate OmniXtend support into RISC-V based systems-on-chip to validate the architecture in FPGAs and drive further collaboration on next-generation memory fabrics.

HDFS Deep Dive

Zoltan C. Toth

This document provides an overview of HDFS (Hadoop Distributed File System), including its design goals, architecture, key components, and some limitations. The main points are: HDFS is a distributed file system designed for large files and streaming data access across commodity hardware. It uses a master-slave architecture with a NameNode managing the file system metadata and DataNodes storing file data in blocks. Files are replicated across multiple DataNodes for fault tolerance. The NameNode controls permissions, file-block mappings, and DataNode locations and balances the cluster as needed.

Oracle Open World 2014: Lies, Damned Lies, and I/O Statistics [ CON3671]

Kyle Hailey

The document discusses analyzing I/O performance and summarizing lessons learned. It describes common tools used to measure I/O like moats.sh, strace, and ioh.sh. It also summarizes the top 10 anomalies encountered like caching effects, shared drives, connection limits, I/O request consolidation and fragmentation over NFS, and tiered storage migration. Solutions provided focus on avoiding caching, isolating workloads, proper sizing of NFS parameters, and direct I/O.

002-Storage Basics and Application Environments V1.0.pptx

DrewMe1

Storage Basics and Application Environments is a document that discusses storage concepts, hardware, protocols, and data protection basics. It begins by defining storage and describing different types including block storage, file storage, and object storage. It then covers basic concepts of storage hardware such as disks, disk arrays, controllers, enclosures, and I/O modules. Storage protocols like SCSI, NVMe, iSCSI, and Fibre Channel are also introduced. Additional concepts like RAID, LUNs, multipathing, and file systems are explained. The document provides a high-level overview of fundamental storage topics.

What's hot

Redis database

Ñáwrás Ñzár

RDM#2- The Distributed Web

Node.js Interactive

Glusterfs session #2 1 layer above disk filesystems

Pranith Karampuri

redis basics

Manoj Kumar

B tree file system

Dinesh Gupta

The evolution of linux file system

Non-Framework MVC sites with PHP

Containers from scratch

Rakuten Group, Inc.

Storage with ceph (osdc 2013)

Martin Loschwitz

OSBConf 2015 | Scale out backups with bareos and gluster by niels de vos

NETWAYS

MongoDB

jsterce

Filesystem Comparison: NFS vs GFS2 vs OCFS2

Giuseppe Paterno'

Redis, a 2 minutes introduction

Mirko Calvaresi

The Care + Feeding of a Mongodb Cluster

Chris Henry

TopDB data transfer

Chonpin HSU

Ext filesystem4

Neha Kulkarni

Thinking in documents