Why flash storage caching
•
1 like•453 views
Tim Fletcher from Brighter Connections discusses ways to improve storage area network (SAN) performance through caching. Hardware approaches include adding spindles, RAM, and flash drives. Software solutions like copy on write, quality of service, and caching frequently accessed data in faster flash memory can optimize workload performance. Open source options for caching include Flashcache, Bcache and ZFS, while some vendors offer proprietary caching software. Caching leverages faster flash to improve SAN speeds without replacing existing storage arrays.
Report
Share
Report
Share
Download to read offline
Recommended
IMCSummit 2016 Keynote - Benzi Galili - More Memory for In-Memory Easy
In-memory computing is a reality. So are the limits of memory capacity. Data size constantly increases, while application developers and IT staff push for in-memory efficiencies; the conclusion is inevitable: we need to be able to access more memory than the DRAM capacity that the server provides. ScaleMP’s Software Defined Memory (SDM) technology allows for more system memory to be available per server, far beyond the hardware limits, by utilizing memory from other nodes (over fabric) or from locally installed non-volatile memory (NVM) such as NAND Flash or 3D XPoint – transparently and without any changes to operating system or applications. We shall present the benefits of SDM, discuss the relevant use-cases, and share performance data.
StarWind Virtual SAN Overview
StarWind Virtual SAN is entirely software-based, hypervisor-centric virtual machine storage. It creates a fully fault-tolerant and high-performing storage pool that is built for the virtualization workload “from scratch”. StarWind Virtual SAN basically “mirrors” inexpensive internal storage between hosts. Virtual SAN completely eliminates any need for an expensive SAN or NAS or other physical shared storage. It seamlessly integrates into the hypervisor for unbeatable performance and exceptional simplicity of use.
StarWind_V_SAN_product_presentation
StarWind Virtual SAN is software that eliminates the need for physical shared storage infrastructure by mirroring internal hard disks and flash between servers. It reduces capital and operational costs for SMBs, ROBOs, and cloud/hosting providers by starting with just two nodes and scaling out infinitely without dedicated SAN/NAS hardware. Performance is maintained through proprietary technologies like adaptive caching and deduplication that reduce I/O paths compared to traditional storage networks.
OSDC 2013 | Scale-Out made easy: Petabyte storage with Ceph by Martin Gerhard...
Ceph is what you are looking for if you need seamlessly scalable storage: While traditional storage solutions like SANs or SAN drop-in replacements are scale-up only, Ceph allows you to turn every off-the-shelf computer into a member of a distributed, centrally managed storage network. With Ceph in place, you will never again have to worry about adding new disks to existing SANs or replacing existing disks with bigger ones -- just add new nodes to the cluster and see your storage grow. And as if that wasn't enough already, Ceph comes with inherent HA and replication capabilities, taking the burden of securing your data against outages off your shoulder.
This presentation gives an insight into the basic ideas behind the Ceph object storage solution. It also elaborates on the existing front-ends for Ceph (CephFS, RBD, radosgw) and explains how they work. Examples will demonstrate how to use Ceph in production. A live demo completes the talk.
Offloading for Databases - Deep Dive
A deep five into offloading techniques for Oracle database servers, that takes both hardware and software solution into consideration. The focus is clearly to boost the efficiency of your already paid licenses.
Sql saturday powerpoint dc_san
The document discusses storage area networks (SANs) for database administrators. It covers SAN components like fiber optic cards, switches, and disk arrays. RAID levels like 0, 1, 5, and 10 are explained along with their benefits and drawbacks. Solid state drives offer faster performance but at a higher cost. The document recommends placing tempdb and log files on separate high-performance disks. SANs provide benefits like easy capacity expansion, high availability, and disaster recovery through replication. DBAs should work with their SAN administrators to test performance and isolate databases and their files.
Hybrid Storage Pools (Now with the benefit of hindsight!)
This document discusses the history and current state of hybrid storage pools that combine disk and flash storage using ZFS. It notes that while flash was initially seen as a replacement for disk, disk still plays an important role. ZFS introduced the hybrid storage pool concept in 2007 to take advantage of flash's speed between DRAM and disk. The L2ARC feature caches data from disk on flash to improve performance but has issues with persistence and warmup times that need further work. Overall, hybrid storage pools that integrate disk and flash can provide benefits but developing the L2ARC and optimizing for real workloads remains an area for improvement.
SOUG_SDM_OracleDB_V3
Software Defined Memory (SDM) uses new technologies like non-volatile RAM and flash storage to treat memory and storage as a unified persistent resource without traditional performance tiers. This can optimize Oracle database I/O performance by bypassing buffer caches and using fast kernel threads. Benchmarks showed a Plexistor SDM solution outperforming a traditional two-node Oracle RAC cluster. The best approach is to use fast storage like 3D XPoint as the secondary tier to maintain high performance even with cache misses. Combining SDM with solutions like FlashGrid and Oracle RAC could provide extremely high performance.
Recommended
IMCSummit 2016 Keynote - Benzi Galili - More Memory for In-Memory Easy
In-memory computing is a reality. So are the limits of memory capacity. Data size constantly increases, while application developers and IT staff push for in-memory efficiencies; the conclusion is inevitable: we need to be able to access more memory than the DRAM capacity that the server provides. ScaleMP’s Software Defined Memory (SDM) technology allows for more system memory to be available per server, far beyond the hardware limits, by utilizing memory from other nodes (over fabric) or from locally installed non-volatile memory (NVM) such as NAND Flash or 3D XPoint – transparently and without any changes to operating system or applications. We shall present the benefits of SDM, discuss the relevant use-cases, and share performance data.
StarWind Virtual SAN Overview
StarWind Virtual SAN is entirely software-based, hypervisor-centric virtual machine storage. It creates a fully fault-tolerant and high-performing storage pool that is built for the virtualization workload “from scratch”. StarWind Virtual SAN basically “mirrors” inexpensive internal storage between hosts. Virtual SAN completely eliminates any need for an expensive SAN or NAS or other physical shared storage. It seamlessly integrates into the hypervisor for unbeatable performance and exceptional simplicity of use.
StarWind_V_SAN_product_presentation
StarWind Virtual SAN is software that eliminates the need for physical shared storage infrastructure by mirroring internal hard disks and flash between servers. It reduces capital and operational costs for SMBs, ROBOs, and cloud/hosting providers by starting with just two nodes and scaling out infinitely without dedicated SAN/NAS hardware. Performance is maintained through proprietary technologies like adaptive caching and deduplication that reduce I/O paths compared to traditional storage networks.
OSDC 2013 | Scale-Out made easy: Petabyte storage with Ceph by Martin Gerhard...
Ceph is what you are looking for if you need seamlessly scalable storage: While traditional storage solutions like SANs or SAN drop-in replacements are scale-up only, Ceph allows you to turn every off-the-shelf computer into a member of a distributed, centrally managed storage network. With Ceph in place, you will never again have to worry about adding new disks to existing SANs or replacing existing disks with bigger ones -- just add new nodes to the cluster and see your storage grow. And as if that wasn't enough already, Ceph comes with inherent HA and replication capabilities, taking the burden of securing your data against outages off your shoulder.
This presentation gives an insight into the basic ideas behind the Ceph object storage solution. It also elaborates on the existing front-ends for Ceph (CephFS, RBD, radosgw) and explains how they work. Examples will demonstrate how to use Ceph in production. A live demo completes the talk.
Offloading for Databases - Deep Dive
A deep five into offloading techniques for Oracle database servers, that takes both hardware and software solution into consideration. The focus is clearly to boost the efficiency of your already paid licenses.
Sql saturday powerpoint dc_san
The document discusses storage area networks (SANs) for database administrators. It covers SAN components like fiber optic cards, switches, and disk arrays. RAID levels like 0, 1, 5, and 10 are explained along with their benefits and drawbacks. Solid state drives offer faster performance but at a higher cost. The document recommends placing tempdb and log files on separate high-performance disks. SANs provide benefits like easy capacity expansion, high availability, and disaster recovery through replication. DBAs should work with their SAN administrators to test performance and isolate databases and their files.
Hybrid Storage Pools (Now with the benefit of hindsight!)
This document discusses the history and current state of hybrid storage pools that combine disk and flash storage using ZFS. It notes that while flash was initially seen as a replacement for disk, disk still plays an important role. ZFS introduced the hybrid storage pool concept in 2007 to take advantage of flash's speed between DRAM and disk. The L2ARC feature caches data from disk on flash to improve performance but has issues with persistence and warmup times that need further work. Overall, hybrid storage pools that integrate disk and flash can provide benefits but developing the L2ARC and optimizing for real workloads remains an area for improvement.
SOUG_SDM_OracleDB_V3
Software Defined Memory (SDM) uses new technologies like non-volatile RAM and flash storage to treat memory and storage as a unified persistent resource without traditional performance tiers. This can optimize Oracle database I/O performance by bypassing buffer caches and using fast kernel threads. Benchmarks showed a Plexistor SDM solution outperforming a traditional two-node Oracle RAC cluster. The best approach is to use fast storage like 3D XPoint as the secondary tier to maintain high performance even with cache misses. Combining SDM with solutions like FlashGrid and Oracle RAC could provide extremely high performance.
SSD для вашей базы данных, Петр Зайцев (Percona)
The document discusses flash technology and its implications for database performance. It provides an overview of flash properties like fast random access times and endurance limits. It suggests that databases can benefit from flash through faster warmup times, higher I/O performance, and optimization techniques like bypassing caching, using direct I/O, and accounting for the different costs of sequential vs random I/O. The document concludes that flash is well-suited to replace HDDs for OLTP databases with the right configuration.
Webinar: Where is the All-Flash Data Center?
Find it With Multi-tiered Flash!
The all-flash data center was to solve all our problems. We’ve had all-flash arrays for ½ a decade. Where are the all-flash data centers? The problem is that performance and cost have been opposing forces in data storage systems for decades, especially in the initial all-flash era. Recent advancements in flash technology are finally breaking this tension and are enabling a new class of applications while driving costs down dramatically.
UniPlex vScaleDB pat. pending
vScaleDB is a universal database acceleration solution that provides in-memory read path optimization, access to data in the fastest tier using software-defined memory, and universal I/O offloading to save CPU cycles and licenses on the database host. It allows optimization of data center rack design through addition of hardware accelerators using PCIe and includes OptaneGRID I/O modules for maximum performance.
Webinar: Overcoming the Storage Challenges Cassandra and Couchbase Create
NoSQL databases like Cassandra and Couchbase are quickly becoming key components of the modern IT infrastructure. But this modernization creates new challenges – especially for storage. Storage in the broad sense. In-memory databases perform well when there is enough memory available. However, when data sets get too large and they need to access storage, application performance degrades dramatically. Moreover, even if enough memory is available, persistent client requests can bring the servers to their knees.
Join Storage Switzerland and Plexistor where you will learn:
1. What is Cassandra and Couchbase?
2. Why organizations are adopting them?
3. What are the storage challenges they create?
4. How organizations attempt to workaround these challenges.
5. How to design a solution to these challenges instead of a workaround.
Storage with ceph (osdc 2013)
This document discusses the scale-out storage capabilities of Ceph. It explains that Ceph uses an object store model called RADOS to allow for scaling storage horizontally across commodity hardware. Ceph uses a technique called CRUSH to automatically replicate and distribute data across its object storage daemons and monitor daemons for redundancy and high availability as more nodes are added. It also describes how Ceph provides block storage, file system, and cloud storage interfaces to stored data through its RADOS Block Device, CephFS, and RADOS Gateway components.
Ceph Day San Jose - Ceph at Salesforce
Salesforce uses Ceph for various storage needs including block storage, replacing some SAN scenarios, and as a general purpose blob store. They are experimenting with multiple small Ceph clusters across different availability zones. Performance testing shows good random read and write speeds for SSD-only pools. Challenges include scaling to meet their needs, ensuring security and isolation across multiple tenants, and managing clusters across many data centers.
Firebird and RAID
Firebird and RAID document discusses choosing the right RAID configuration for a Firebird server. It covers the basics of different RAID levels like mirrored RAID, parity RAID, JBOD, and RAID 0. It provides examples of performance comparisons between hardware RAID, software RAID, and SSD using inserts, updates, and selects. The conclusion is that for a database server, a mirrored RAID implementation will generally outperform a parity RAID configuration with the same specifications due to lower write penalties.
Ssd And Enteprise Storage
SSD uses NAND flash memory instead of spinning disks, providing much faster read and write speeds. SLC NAND flash is commonly used in enterprise environments due to its excellent performance and endurance. However, SSDs have higher costs and more limited endurance than HDDs. Emerging technologies like MLC flash, new controller designs, and tiered storage systems aim to improve SSD performance, capacity, endurance and lower costs to make SSDs more viable for enterprise storage.
Building an open memory-centric computing architecture using intel optane
OOW 2017 presentation showcasing Fabric Attached Memory with 2 node RAC system based on two standard x86 servers running 200 GB/s with a per licensed cpu core data rate of 25 GB/s.
UniPlex T1 Storage Supercharger
The document describes the UniPlex T1 Storage Supercharger, a solution that adds Tier 1 in-memory performance using NVDIMM to an existing storage array design. It builds on the proven Enigma III SDM Storage Appliance and uses software-defined memory spanning across NVDIMM (Tier 1), UniPlex NVMe PCIe JBODs (Tier 2), and an organization's existing storage (Tier 3 capacity tier). This provides a cost-efficient upgrade for high-performance workloads starting at $10,000 per "T1 brick".
UniPlex 1000 Series PCIe NVMe JBOF
The UniPlex 1000 is a 24-bay NVMe JBOF that connects to hosts via PCIe, supporting up to 64 GB/s throughput and 16 million IOPS. It comes in locked and unlocked versions, with the locked version containing pre-installed drives and the unlocked allowing any drives. The JBOF can expand into an appliance using I/O drawers and storage controllers, and start small with a single JBOF then grow by connecting multiple JBOFs and tiers of storage.
Ficha Tecnica EMC VNX5400
The document describes the EMC VNX5400 unified storage system. It delivers high performance through its MCx multicore optimization and supports up to 1 PB of storage capacity. Features include block, file, and object support along with data reduction technologies. It offers strong integration with virtualization platforms and automated storage tiering through FAST Suite to optimize performance and costs. A range of services and support options are available as well.
NVMe over Fabric
NVMe over Fabrics (NVMe-oF) allows NVMe-based storage to be shared across multiple servers over a network. It provides better utilization of resources and scalability compared to directly attached storage. NVMe-oF maintains NVMe performance by transferring commands and data end-to-end over the fabric using technologies like RDMA that bypass legacy storage stacks. It enables applications like composable infrastructure with remote direct memory access (RDMA) providing near-local performance. While NVMe-oF can use different transports, RDMA has been most common due to low latency it provides.
Building scalable and reliable websites
The document summarizes best practices for building scalable and reliable websites. It discusses techniques for caching, load balancing, storage, databases, monitoring, and application architecture. Specific recommendations include caching everything possible, reducing DNS lookups and requests, using content delivery networks, sharding databases, and scaling services horizontally.
Skalowalna architektura na przykładzie soccerway.com
This document discusses scalable web application architecture. It describes using load balancing and database replication across multiple servers to handle high traffic volumes. It emphasizes using caching at various levels (PHP, databases, CDNs) and preprocessing data to improve performance. A CDN is recommended for distributing static resources globally to reduce latency and failure risk.
11g r2 flashcache_Tips
The Oracle Database Smart Flash Cache is a read-only cache that stores clean database blocks from the buffer cache on flash memory devices to improve performance. As blocks age out of the buffer cache, they can be moved to the Flash Cache for quick retrieval back into the buffer cache without needing slower I/O from conventional storage. The Flash Cache provides a cost-effective way to expand the buffer cache beyond memory limits by caching data on faster flash storage.
Bridging Big - Small, Fast - Slow with Campaign Storage
Campaign Storage was invented at Los Alamos National Laboratory. Peter Braam and Nathan Thompson founded Campaign Storage, LLC in 2016 to deliver software-defined storage products in this space. Campaign Storage is a file system that focuses on staging and archiving data using industry standard object stores and existing metadata stores. It provides 10-100x higher bandwidth than archives but 10x lower than parallel file systems, making it a new tier between these storage solutions.
San presentation nov 2012 central pa
Overview of Storage and SAN topics for DBAs. Presented to Central Pennsylvania SQL Server Users group on November 13, 2012.
Making clouds: turning opennebula into a product
A summary of our experiences and why we choose OpenNebula over competing open source cloud stacks for our CloudWeavers product.
Spinning Brown Donuts
David Pechon is an IT professional with experience in virtualization, storage, and identity management. He has worked for the US Army, banks, and consulting companies. While RAID provides data redundancy, it does not replace the need for backups, as RAID cannot protect against accidental or intentional deletion of data. Storage technologies have evolved from drum memory and floppy disks to today's hybrid arrays, all-flash arrays, and converged infrastructure solutions. The differences between SANs and NAS depend on factors like latency requirements, data access patterns, distance between servers and storage, and administrative needs. Newer solutions aim to provide the benefits of both SAN and NAS through unified storage and software-defined storage.
Spinning Brown Donuts: Why Storage Still Counts
Storage, next to server hardware, is pretty commoditized and probably the least exciting thing in your datacenter. However, not properly assessing your storage needs and requirements can be the difference between a great app or resume generating event. This session will cover topics such as: Why you may not need all flash, SAN is not just NAS spelled backwards, leveraging cloud storage, why RAID is not a sound backup solution, and cutting through the marketing to make sense of it all.
More Related Content
What's hot
SSD для вашей базы данных, Петр Зайцев (Percona)
The document discusses flash technology and its implications for database performance. It provides an overview of flash properties like fast random access times and endurance limits. It suggests that databases can benefit from flash through faster warmup times, higher I/O performance, and optimization techniques like bypassing caching, using direct I/O, and accounting for the different costs of sequential vs random I/O. The document concludes that flash is well-suited to replace HDDs for OLTP databases with the right configuration.
Webinar: Where is the All-Flash Data Center?
Find it With Multi-tiered Flash!
The all-flash data center was to solve all our problems. We’ve had all-flash arrays for ½ a decade. Where are the all-flash data centers? The problem is that performance and cost have been opposing forces in data storage systems for decades, especially in the initial all-flash era. Recent advancements in flash technology are finally breaking this tension and are enabling a new class of applications while driving costs down dramatically.
UniPlex vScaleDB pat. pending
vScaleDB is a universal database acceleration solution that provides in-memory read path optimization, access to data in the fastest tier using software-defined memory, and universal I/O offloading to save CPU cycles and licenses on the database host. It allows optimization of data center rack design through addition of hardware accelerators using PCIe and includes OptaneGRID I/O modules for maximum performance.
Webinar: Overcoming the Storage Challenges Cassandra and Couchbase Create
NoSQL databases like Cassandra and Couchbase are quickly becoming key components of the modern IT infrastructure. But this modernization creates new challenges – especially for storage. Storage in the broad sense. In-memory databases perform well when there is enough memory available. However, when data sets get too large and they need to access storage, application performance degrades dramatically. Moreover, even if enough memory is available, persistent client requests can bring the servers to their knees.
Join Storage Switzerland and Plexistor where you will learn:
1. What is Cassandra and Couchbase?
2. Why organizations are adopting them?
3. What are the storage challenges they create?
4. How organizations attempt to workaround these challenges.
5. How to design a solution to these challenges instead of a workaround.
Storage with ceph (osdc 2013)
This document discusses the scale-out storage capabilities of Ceph. It explains that Ceph uses an object store model called RADOS to allow for scaling storage horizontally across commodity hardware. Ceph uses a technique called CRUSH to automatically replicate and distribute data across its object storage daemons and monitor daemons for redundancy and high availability as more nodes are added. It also describes how Ceph provides block storage, file system, and cloud storage interfaces to stored data through its RADOS Block Device, CephFS, and RADOS Gateway components.
Ceph Day San Jose - Ceph at Salesforce
Salesforce uses Ceph for various storage needs including block storage, replacing some SAN scenarios, and as a general purpose blob store. They are experimenting with multiple small Ceph clusters across different availability zones. Performance testing shows good random read and write speeds for SSD-only pools. Challenges include scaling to meet their needs, ensuring security and isolation across multiple tenants, and managing clusters across many data centers.
Firebird and RAID
Firebird and RAID document discusses choosing the right RAID configuration for a Firebird server. It covers the basics of different RAID levels like mirrored RAID, parity RAID, JBOD, and RAID 0. It provides examples of performance comparisons between hardware RAID, software RAID, and SSD using inserts, updates, and selects. The conclusion is that for a database server, a mirrored RAID implementation will generally outperform a parity RAID configuration with the same specifications due to lower write penalties.
Ssd And Enteprise Storage
SSD uses NAND flash memory instead of spinning disks, providing much faster read and write speeds. SLC NAND flash is commonly used in enterprise environments due to its excellent performance and endurance. However, SSDs have higher costs and more limited endurance than HDDs. Emerging technologies like MLC flash, new controller designs, and tiered storage systems aim to improve SSD performance, capacity, endurance and lower costs to make SSDs more viable for enterprise storage.
Building an open memory-centric computing architecture using intel optane
OOW 2017 presentation showcasing Fabric Attached Memory with 2 node RAC system based on two standard x86 servers running 200 GB/s with a per licensed cpu core data rate of 25 GB/s.
UniPlex T1 Storage Supercharger
The document describes the UniPlex T1 Storage Supercharger, a solution that adds Tier 1 in-memory performance using NVDIMM to an existing storage array design. It builds on the proven Enigma III SDM Storage Appliance and uses software-defined memory spanning across NVDIMM (Tier 1), UniPlex NVMe PCIe JBODs (Tier 2), and an organization's existing storage (Tier 3 capacity tier). This provides a cost-efficient upgrade for high-performance workloads starting at $10,000 per "T1 brick".
UniPlex 1000 Series PCIe NVMe JBOF
The UniPlex 1000 is a 24-bay NVMe JBOF that connects to hosts via PCIe, supporting up to 64 GB/s throughput and 16 million IOPS. It comes in locked and unlocked versions, with the locked version containing pre-installed drives and the unlocked allowing any drives. The JBOF can expand into an appliance using I/O drawers and storage controllers, and start small with a single JBOF then grow by connecting multiple JBOFs and tiers of storage.
Ficha Tecnica EMC VNX5400
The document describes the EMC VNX5400 unified storage system. It delivers high performance through its MCx multicore optimization and supports up to 1 PB of storage capacity. Features include block, file, and object support along with data reduction technologies. It offers strong integration with virtualization platforms and automated storage tiering through FAST Suite to optimize performance and costs. A range of services and support options are available as well.
NVMe over Fabric
NVMe over Fabrics (NVMe-oF) allows NVMe-based storage to be shared across multiple servers over a network. It provides better utilization of resources and scalability compared to directly attached storage. NVMe-oF maintains NVMe performance by transferring commands and data end-to-end over the fabric using technologies like RDMA that bypass legacy storage stacks. It enables applications like composable infrastructure with remote direct memory access (RDMA) providing near-local performance. While NVMe-oF can use different transports, RDMA has been most common due to low latency it provides.
Building scalable and reliable websites
The document summarizes best practices for building scalable and reliable websites. It discusses techniques for caching, load balancing, storage, databases, monitoring, and application architecture. Specific recommendations include caching everything possible, reducing DNS lookups and requests, using content delivery networks, sharding databases, and scaling services horizontally.
Skalowalna architektura na przykładzie soccerway.com
This document discusses scalable web application architecture. It describes using load balancing and database replication across multiple servers to handle high traffic volumes. It emphasizes using caching at various levels (PHP, databases, CDNs) and preprocessing data to improve performance. A CDN is recommended for distributing static resources globally to reduce latency and failure risk.
11g r2 flashcache_Tips
The Oracle Database Smart Flash Cache is a read-only cache that stores clean database blocks from the buffer cache on flash memory devices to improve performance. As blocks age out of the buffer cache, they can be moved to the Flash Cache for quick retrieval back into the buffer cache without needing slower I/O from conventional storage. The Flash Cache provides a cost-effective way to expand the buffer cache beyond memory limits by caching data on faster flash storage.
Bridging Big - Small, Fast - Slow with Campaign Storage
Campaign Storage was invented at Los Alamos National Laboratory. Peter Braam and Nathan Thompson founded Campaign Storage, LLC in 2016 to deliver software-defined storage products in this space. Campaign Storage is a file system that focuses on staging and archiving data using industry standard object stores and existing metadata stores. It provides 10-100x higher bandwidth than archives but 10x lower than parallel file systems, making it a new tier between these storage solutions.
San presentation nov 2012 central pa
Overview of Storage and SAN topics for DBAs. Presented to Central Pennsylvania SQL Server Users group on November 13, 2012.
Making clouds: turning opennebula into a product
A summary of our experiences and why we choose OpenNebula over competing open source cloud stacks for our CloudWeavers product.
What's hot (20)
Webinar: Overcoming the Storage Challenges Cassandra and Couchbase Create
Webinar: Overcoming the Storage Challenges Cassandra and Couchbase Create
Building an open memory-centric computing architecture using intel optane
Building an open memory-centric computing architecture using intel optane
Skalowalna architektura na przykładzie soccerway.com
Skalowalna architektura na przykładzie soccerway.com
Bridging Big - Small, Fast - Slow with Campaign Storage
Bridging Big - Small, Fast - Slow with Campaign Storage
Similar to Why flash storage caching
Spinning Brown Donuts
David Pechon is an IT professional with experience in virtualization, storage, and identity management. He has worked for the US Army, banks, and consulting companies. While RAID provides data redundancy, it does not replace the need for backups, as RAID cannot protect against accidental or intentional deletion of data. Storage technologies have evolved from drum memory and floppy disks to today's hybrid arrays, all-flash arrays, and converged infrastructure solutions. The differences between SANs and NAS depend on factors like latency requirements, data access patterns, distance between servers and storage, and administrative needs. Newer solutions aim to provide the benefits of both SAN and NAS through unified storage and software-defined storage.
Spinning Brown Donuts: Why Storage Still Counts
Storage, next to server hardware, is pretty commoditized and probably the least exciting thing in your datacenter. However, not properly assessing your storage needs and requirements can be the difference between a great app or resume generating event. This session will cover topics such as: Why you may not need all flash, SAN is not just NAS spelled backwards, leveraging cloud storage, why RAID is not a sound backup solution, and cutting through the marketing to make sense of it all.
Filesystems
The document discusses designing large-scale file storage and serving systems. It covers requirements like scalability, reliability, and cost-effectiveness. It describes various storage architectures like network attached storage (NAS), storage area networks (SAN), and caching strategies to improve performance. It also discusses redundancy, high availability, and business continuity planning to ensure reliability in the face of failures.
Beyond the File System - Designing Large Scale File Storage and Serving
The document discusses designing large-scale file storage and serving systems. It covers requirements like scalability, reliability, and cost-effectiveness. It describes various storage architectures like network attached storage (NAS), storage area networks (SAN), and caching strategies to improve performance. It also discusses redundancy, high availability, and business continuity planning to ensure reliability in the face of failures.
Storage Networks
The document provides an overview of different types of storage networks: direct attached storage (DAS), network attached storage (NAS), and storage area networks (SAN). It discusses the key differences between these networks in terms of interface technologies, file systems, communication models, and features. The document also lists some major players and core technologies in the storage network market, including vendors that provide storage equipment, switches, backup solutions, and other related products and services.
It's the End of Data Storage As We Know It (And I Feel Fine)
Technological change is finally coming to storage, and it will wipe away the architecture we've come to know over the last few decades. Say goodbye to the "do it all" Fibre Channel SAN storage array and get ready for converged infrastructure, distributed storage, alternative attachments like PCIe, and top-of-rack flash! In this session, Stephen Foskett will explain why this change is inevitable and how it will shake out. You won't recognize what's coming, but it will be faster, cheaper, and more integrated than ever! Delivered at
Web20expo Filesystems
The document discusses designing large scale file storage and serving systems. It covers key requirements like scalability, reliability, and cost-effectiveness. It then describes various approaches to storage, such as network attached storage (NAS), storage area networks (SAN), and caching. Specific examples like Google File System, MogileFS, and Flickr's file system are examined. Business continuity planning (BCP) through redundancy and replication is also discussed.
Web20expo Filesystems
The document discusses designing large scale file storage and serving systems. It covers requirements like scalability, reliability, and cost effectiveness. It then discusses various approaches to storage, such as network attached storage (NAS), storage area networks (SAN), and caching strategies to improve performance. Examples of real systems like Google File System, Amazon S3, and content delivery networks are provided. Business continuity planning through redundancy and replication is also covered.
Beyond the File System: Designing Large-Scale File Storage and Serving
Beyond the File System: Designing Large-Scale File Storage and Serving, Web 2.0 Expo, San Francisco
Web20expo Filesystems
The document discusses designing large scale file storage and serving systems. It covers key requirements like scalability, reliability, and cost-effectiveness. It then describes various approaches to storage, such as network attached storage (NAS), storage area networks (SAN), and caching. Specific examples like Google File System, MogileFS, and Flickr's file system are analyzed. Business continuity planning (BCP) through redundancy and replication is also discussed.
Web20expo Filesystems
The document discusses designing large scale file storage and serving systems. It covers key requirements like scalability, reliability, and cost-effectiveness. It then describes various approaches to storage, such as network attached storage (NAS), storage area networks (SAN), and caching. Specific examples like Google File System, MogileFS, and Flickr's file system are examined. Business continuity planning through redundancy and replication is also discussed.
Fulcrum Group Storage And Storage Virtualization Presentation
The document discusses storage solutions and SANs. Exponential data growth is expected to continue challenging data protection efforts. Different storage types fit different business needs. By understanding storage design and an organization's needs, storage virtualization may be a good fit. SANs can help with general server needs, virtualization, and disaster recovery/backup needs. Planning is key to deploying storage in a centralized way.
Zettabyte File Storage System
ZFS is an innovative file system that provides immense storage capacity, data integrity, and simplified administration. It was developed by Sun Microsystems in 2000 and released in 2005. Some key features of ZFS include its ability to detect and correct errors, provide end-to-end data integrity checks, flexibly pool storage devices, and scale to exabytes of data. While it has limitations like a lack of boot support and encryption, ZFS is widely used with Solaris and is being ported to other platforms like Linux and FreeBSD.
Zettabyte File Storage System
ZFS is an innovative file system that provides immense storage capacity, data integrity, and simplified administration. It was developed by Sun Microsystems and introduced in 2005. Key features of ZFS include its ability to detect and self-heal data errors, use copy-on-write transactions, and support up to 256 quadrillion zettabytes of storage in a single pool. While powerful, ZFS has some limitations around boot support and encryption. It is supported on Solaris and is being ported to other platforms like Linux and FreeBSD.
Posscon2013
This document provides an introduction and overview of FreeNAS 8.3.1. It outlines the core features of FreeNAS including the ability to create storage volumes using UFS or the ZFS filesystem. It describes how ZFS provides data integrity and flexibility through features like RAIDZ, snapshots, and clones. The document also covers installing and configuring plugins to extend FreeNAS functionality, and an overview of the new GELI full disk encryption support. Additional resources for FreeNAS support are provided at the end.
Varrow datacenter storage today and tomorrow
The document summarizes changes in datacenter storage technologies. It discusses typical storage types used today like DAS, SAN, and NAS and how new technologies are changing them. Technologies discussed include PCIe flash, all-flash arrays, denser drives, InfiniBand, and cloud storage. It suggests storage architectures may move away from RAID with new flash-based solutions and caching algorithms optimized for flash performance rather than spinning disks.
Storage School 1
This document provides an introduction to storage concepts for those new to the topic. It begins with a brief history of storage and how it evolved from being inside computers to outside networks and arrays. It then covers five key concepts: that storage exists outside computers, the differences between blocks and files, the importance of SCSI, different RAID levels, and the three types of storage arrays. The document aims to explain the basic foundations of modern storage systems and how all the pieces fit together.
Storage, San And Business Continuity Overview
The document provides an overview of storage systems and business continuity options. It discusses various types of storage including DAS, NAS and SAN. It then covers business continuity and disaster recovery strategies like replication, snapshots and mirroring. It also discusses how server virtualization can help improve disaster recovery.
What is the average rotational latency of this disk drive What seek.docx
SSDs have advantages over HDDs like faster speeds without seek times, but are more expensive. Various caching methods leverage SSD speed while retaining HDD capacity. DM-cache, Flashcache, Bcache, and EnhanceIO all use SSDs to cache hot data for faster access without extra storage management. Each has advantages like transparency or ability to cache partitions, but DM-cache may have metadata limits while Bcache uses system memory. A hybrid system provides the best features of both SSDs and HDDs.
Olf2013
ZFS is a filesystem and volume manager that provides data integrity and efficiency features. It can be used with FreeNAS and PC-BSD to create storage pools by combining disks. Pools can be logically divided into datasets or zvols. Key ZFS features include RAIDZ parity protection, snapshots, clones, compression, and self-healing capabilities. ZFS is well-suited for network storage and provides advanced features for optimizing storage and protecting data.
Similar to Why flash storage caching (20)
Beyond the File System - Designing Large Scale File Storage and Serving
Beyond the File System - Designing Large Scale File Storage and Serving
It's the End of Data Storage As We Know It (And I Feel Fine)
It's the End of Data Storage As We Know It (And I Feel Fine)
Beyond the File System: Designing Large-Scale File Storage and Serving
Beyond the File System: Designing Large-Scale File Storage and Serving
Fulcrum Group Storage And Storage Virtualization Presentation
Fulcrum Group Storage And Storage Virtualization Presentation
What is the average rotational latency of this disk drive What seek.docx
What is the average rotational latency of this disk drive What seek.docx
Recently uploaded
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
Imagine a world where machines not only perform tasks but also learn, adapt, and make decisions. This is the promise of Artificial Intelligence (AI), a technology that's not just enhancing our lives but revolutionizing entire industries.
みなさんこんにちはこれ何文字まで入るの？40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの？えこ...
ここ3000字までしか入らないけどタイトルの方がたくさん文字入ると思います。
20240609 QFM020 Irresponsible AI Reading List May 2024
Everything I found interesting about the irresponsible use of machine intelligence in May 2024
Climate Impact of Software Testing at Nordic Testing Days
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Are you ready to revolutionize how you handle data? Join us for a webinar where we’ll bring you up to speed with the latest advancements in Generative AI technology and discover how leveraging FME with tools from giants like Google Gemini, Amazon, and Microsoft OpenAI can supercharge your workflow efficiency.
During the hour, we’ll take you through:
Guest Speaker Segment with Hannah Barrington: Dive into the world of dynamic real estate marketing with Hannah, the Marketing Manager at Workspace Group. Hear firsthand how their team generates engaging descriptions for thousands of office units by integrating diverse data sources—from PDF floorplans to web pages—using FME transformers, like OpenAIVisionConnector and AnthropicVisionConnector. This use case will show you how GenAI can streamline content creation for marketing across the board.
Ollama Use Case: Learn how Scenario Specialist Dmitri Bagh has utilized Ollama within FME to input data, create custom models, and enhance security protocols. This segment will include demos to illustrate the full capabilities of FME in AI-driven processes.
Custom AI Models: Discover how to leverage FME to build personalized AI models using your data. Whether it’s populating a model with local data for added security or integrating public AI tools, find out how FME facilitates a versatile and secure approach to AI.
We’ll wrap up with a live Q&A session where you can engage with our experts on your specific use cases, and learn more about optimizing your data workflows with AI.
This webinar is ideal for professionals seeking to harness the power of AI within their data management systems while ensuring high levels of customization and security. Whether you're a novice or an expert, gain actionable insights and strategies to elevate your data processes. Join us to see how FME and AI can revolutionize how you work with data!
UiPath Test Automation using UiPath Test Suite series, part 5
Welcome to UiPath Test Automation using UiPath Test Suite series part 5. In this session, we will cover CI/CD with devops.
Topics covered:
CI/CD with in UiPath
End-to-end overview of CI/CD pipeline with Azure devops
Speaker:
Lyndsey Byblow, Test Suite Sales Engineer @ UiPath, Inc.
20240605 QFM017 Machine Intelligence Reading List May 2024
Everything I found interesting about machines behaving intelligently during May 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
Maruthi Prithivirajan, Head of ASEAN & IN Solution Architecture, Neo4j
Get an inside look at the latest Neo4j innovations that enable relationship-driven intelligence at scale. Learn more about the newest cloud integrations and product enhancements that make Neo4j an essential choice for developers building apps with interconnected data and generative AI.
Pushing the limits of ePRTC: 100ns holdover for 100 days
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
Neha Bajwa, Vice President of Product Marketing, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
20240607 QFM018 Elixir Reading List May 2024
Everything I found interesting about the Elixir programming ecosystem in May 2024
HCL Notes and Domino License Cost Reduction in the World of DLAU
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-and-domino-license-cost-reduction-in-the-world-of-dlau/
The introduction of DLAU and the CCB & CCX licensing model caused quite a stir in the HCL community. As a Notes and Domino customer, you may have faced challenges with unexpected user counts and license costs. You probably have questions on how this new licensing approach works and how to benefit from it. Most importantly, you likely have budget constraints and want to save money where possible. Don’t worry, we can help with all of this!
We’ll show you how to fix common misconfigurations that cause higher-than-expected user counts, and how to identify accounts which you can deactivate to save money. There are also frequent patterns that can cause unnecessary cost, like using a person document instead of a mail-in for shared mailboxes. We’ll provide examples and solutions for those as well. And naturally we’ll explain the new licensing model.
Join HCL Ambassador Marc Thomas in this webinar with a special guest appearance from Franz Walder. It will give you the tools and know-how to stay on top of what is going on with Domino licensing. You will be able lower your cost through an optimized configuration and keep it low going forward.
These topics will be covered
- Reducing license cost by finding and fixing misconfigurations and superfluous accounts
- How do CCB and CCX licenses really work?
- Understanding the DLAU tool and how to best utilize it
- Tips for common problem areas, like team mailboxes, functional/test users, etc
- Practical examples and best practices to implement right away
Microsoft - Power Platform_G.Aspiotis.pdf
Revolutionizing Application Development
with AI-powered low-code, presentation by George Aspiotis, Sr. Partner Development Manager, Microsoft
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
ABSTRACT: A prima vista, un mattoncino Lego e la backdoor XZ potrebbero avere in comune il fatto di essere entrambi blocchi di costruzione, o dipendenze di progetti creativi e software. La realtà è che un mattoncino Lego e il caso della backdoor XZ hanno molto di più di tutto ciò in comune.
Partecipate alla presentazione per immergervi in una storia di interoperabilità, standard e formati aperti, per poi discutere del ruolo importante che i contributori hanno in una comunità open source sostenibile.
BIO: Sostenitrice del software libero e dei formati standard e aperti. È stata un membro attivo dei progetti Fedora e openSUSE e ha co-fondato l'Associazione LibreItalia dove è stata coinvolta in diversi eventi, migrazioni e formazione relativi a LibreOffice. In precedenza ha lavorato a migrazioni e corsi di formazione su LibreOffice per diverse amministrazioni pubbliche e privati. Da gennaio 2020 lavora in SUSE come Software Release Engineer per Uyuni e SUSE Manager e quando non segue la sua passione per i computer e per Geeko coltiva la sua curiosità per l'astronomia (da cui deriva il suo nickname deneb_alpha).
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Discover the seamless integration of RPA (Robotic Process Automation), COMPOSER, and APM with AWS IDP enhanced with Slack notifications. Explore how these technologies converge to streamline workflows, optimize performance, and ensure secure access, all while leveraging the power of AWS IDP and real-time communication via Slack notifications.
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-und-domino-lizenzkostenreduzierung-in-der-welt-von-dlau/
DLAU und die Lizenzen nach dem CCB- und CCX-Modell sind für viele in der HCL-Community seit letztem Jahr ein heißes Thema. Als Notes- oder Domino-Kunde haben Sie vielleicht mit unerwartet hohen Benutzerzahlen und Lizenzgebühren zu kämpfen. Sie fragen sich vielleicht, wie diese neue Art der Lizenzierung funktioniert und welchen Nutzen sie Ihnen bringt. Vor allem wollen Sie sicherlich Ihr Budget einhalten und Kosten sparen, wo immer möglich. Das verstehen wir und wir möchten Ihnen dabei helfen!
Wir erklären Ihnen, wie Sie häufige Konfigurationsprobleme lösen können, die dazu führen können, dass mehr Benutzer gezählt werden als nötig, und wie Sie überflüssige oder ungenutzte Konten identifizieren und entfernen können, um Geld zu sparen. Es gibt auch einige Ansätze, die zu unnötigen Ausgaben führen können, z. B. wenn ein Personendokument anstelle eines Mail-Ins für geteilte Mailboxen verwendet wird. Wir zeigen Ihnen solche Fälle und deren Lösungen. Und natürlich erklären wir Ihnen das neue Lizenzmodell.
Nehmen Sie an diesem Webinar teil, bei dem HCL-Ambassador Marc Thomas und Gastredner Franz Walder Ihnen diese neue Welt näherbringen. Es vermittelt Ihnen die Tools und das Know-how, um den Überblick zu bewahren. Sie werden in der Lage sein, Ihre Kosten durch eine optimierte Domino-Konfiguration zu reduzieren und auch in Zukunft gering zu halten.
Diese Themen werden behandelt
- Reduzierung der Lizenzkosten durch Auffinden und Beheben von Fehlkonfigurationen und überflüssigen Konten
- Wie funktionieren CCB- und CCX-Lizenzen wirklich?
- Verstehen des DLAU-Tools und wie man es am besten nutzt
- Tipps für häufige Problembereiche, wie z. B. Team-Postfächer, Funktions-/Testbenutzer usw.
- Praxisbeispiele und Best Practices zum sofortigen Umsetzen
Video Streaming: Then, Now, and in the Future
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
Recently uploaded (20)
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
みなさんこんにちはこれ何文字まで入るの？40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの？えこ...
みなさんこんにちはこれ何文字まで入るの？40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの？えこ...
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
Why flash storage caching
- 1. Storage Caching Tim Fletcher Brighter Connections
- 2. Who are we? 15+ year Linux experience 13 years work in education RHCE since 1999 (RedHat 6.1) Joined Brighter Connections in October IT Practise in Huddersfield Specialise in innovative solutions
- 3. Why your SAN is slow Centralised storage, good for cost bad for performance Lots of different competing workloads mean disks "thrash"
- 4. Making your SAN faster Hardware Spindles RAM Software Copy on Write Quality of Service
- 5. Hard Drives Technology basically the same for over 30 years, known as "Spinning Rust". Good for cheap bulk storage Good at linear access Bad at random access Power hungry Fragile
- 6. Memory Very fast access Very good random access Directly connected to the CPU Expensive Volatile Power Hungry
- 7. Flash Very fast, especially at random access Can be connected directly to the PCI-e bus Non volatile Low power Massive drops in price, but still expensive Cells can wear out Includes firmware
- 9. PCIe vs Hard disk emulation Hard disk emulation Easy to deploy Well understood Cheap Many layers of translation PCIe Expensive, or is it? Higher power draw Directly attached to the CPU bus
- 10. Caching - Why does it matter Flash is expensive and wears out The closer the data is to the CPU the better Keep your SAN array Leverages the best of both worlds
- 11. Caching - Where do your blocks go?
- 12. Flashcache Fully open source, released by Facebook Kernel module, based in the device mapper layer Starting to appear in distros Error resistant Cache can be added and removed from storage Used by Facebook in production
- 13. Bcache Fully Open Source Designed from the ground up to cache Designed with an understanding of flash Higher performance Requires devices to be rebuilt as "bcache" devices Requires full kernel rebuild
- 14. ZFS Not just a filesystem Includes device management, RAID and filesystem management. Open Source but license incompatible with GPL Written by Sun, available in FreeBSD, Solaris derivatives, and Linux Can use SSDs as L2ARC and as SLOG Cache devices can be live added and removed Cache does not persist over reboots
- 15. Closed Source Options Hook in to VMware FusionIO - ioTurbine SanDisk - FlashSoft