Resume boda liao-storage
•
0 likes•60 views
Boda Liao has over 10 years of experience as a senior Linux administrator and system engineer for companies such as BGI-Shenzhen, Complete Genomics, and EMC Isilon. He has expert knowledge of large-scale Linux clusters, parallel storage systems, data lifecycle management, and networking solutions. Liao maintains HPC and storage infrastructure supporting thousands of users and petabytes of genomic data per day. He is proficient in Linux administration, storage performance and maintenance, cluster deployment, and data center design.
Report
Share
Report
Share
Download to read offline
Recommended
Resume-Boda_Liao-20151204
Boda Liao has over 10 years of experience as a senior Linux administrator and data center builder. He is currently a senior Linux administrator at Complete Genomics where he maintains their automatic sequencing IT systems. Previously, he was a system engineer and data center builder at BGI-Shenzhen where he designed, built, deployed and maintained HPC clusters supporting thousands of bioinformaticians analyzing over 10TB of genomic data per day. He has expert knowledge of large-scale HPC clusters, parallel storage systems, data lifecycle management, sequencer IT solutions, automation, and Linux.
Resume boda liao-20170911
Boda Liao has over 10 years of experience as a senior Linux administrator and data center builder. He has extensive expertise designing, building, and maintaining large-scale high performance computing clusters and storage systems for genomic sequencing companies. His skills include solutions for DNA sequencers, bioinformatics automation, large storage systems, high-speed networks, and cluster management using tools like Puppet.
SUSE Storage: Sizing and Performance (Ceph)
The document discusses Ceph, an open-source distributed storage system. It provides an overview of Ceph's architecture and components, how it works, and considerations for setting up a Ceph cluster. Key points include: Ceph provides unified block, file and object storage interfaces and can scale exponentially. It uses CRUSH to deterministically map data across a cluster for redundancy. Setup choices like network, storage nodes, disks, caching and placement groups impact performance and must be tuned for the workload.
Red Hat Storage Server Administration Deep Dive
"In this session for administrators of all skill levels, you’ll get a deep technical dive into Red Hat Storage Server and GlusterFS administration.
We’ll start with the basics of what scale-out storage is, and learn about the unique implementation of Red Hat Storage Server and its advantages over legacy and competing technologies. From the basic knowledge and design principles, we’ll move to a live start-to-finish demonstration. Your experience will include:
Building a cluster.
Allocating resources.
Creating and modifying volumes of different types.
Accessing data via multiple client protocols.
A resiliency demonstration.
Expanding and contracting volumes.
Implementing directory quotas.
Recovering from and preventing split-brain.
Asynchronous parallel geo-replication.
Behind-the-curtain views of configuration files and logs.
Extended attributes used by GlusterFS.
Performance tuning basics.
New and upcoming feature demonstrations.
Those new to the scale-out product will leave this session with the knowledge and confidence to set up their first Red Hat Storage Server environment. Experienced administrators will sharpen their skills and gain insights into the newest features. IT executives and managers will gain a valuable overview to help fuel the drive for next-generation infrastructures."
Gluster for Geeks: Performance Tuning Tips & Tricks
This document summarizes a webinar on performance tuning tips and tricks for GlusterFS. The webinar covered planning cluster hardware configuration to meet performance requirements, choosing the correct volume type for workloads, key tuning parameters, benchmarking techniques, and the top 5 causes of performance issues. The webinar provided guidance on optimizing GlusterFS performance through hardware sizing, configuration, implementation best practices, and tuning.
Hadoop course content
The Hadoop course covers hardware and software requirements, installing and configuring Linux, Hadoop, HDFS, MapReduce, Pig, Hive, HBase, and MongoDB. It also covers ETL processes using Pentaho Data Integration to load and transform data between relational databases and Hadoop. The course content includes tutorials on virtual machines, Linux commands, Hadoop processes and architecture, developing MapReduce applications, Pig commands, Hive basics, and connecting data warehouse tools to relational databases and Hadoop.
librados
- Librados is a C/C++ programming interface that provides applications access to the Ceph distributed object store (RADOS) and hides the complexity of networking, data distribution, replication and failure recovery.
- It can be used by Ceph components like RADOS Gateway and tools like rados, as well as third party applications that want to use Ceph for storage. Examples include providing storage for mail systems, Hadoop, and building custom applications.
- The interface handles configuration, connections, I/O operations on objects, extended attributes and more so applications can easily integrate scalable reliable storage via Ceph without having to implement these functions themselves.
Quantcast File System (QFS) - Alternative to HDFS
The document summarizes Quantcast File System (QFS), an alternative to HDFS that provides petabyte storage at half the disk space of HDFS. QFS offers significantly faster I/O than HDFS through the use of Reed-Solomon encoding, requiring only 1.5x disk space compared to HDFS's 3x. It has been production hardened at Quantcast under massive processing loads and is fully compatible with Apache Hadoop. Benchmark results show QFS writes are half the disk I/O of HDFS writes and reads require accessing the network versus HDFS's focus on data locality.
Recommended
Resume-Boda_Liao-20151204
Boda Liao has over 10 years of experience as a senior Linux administrator and data center builder. He is currently a senior Linux administrator at Complete Genomics where he maintains their automatic sequencing IT systems. Previously, he was a system engineer and data center builder at BGI-Shenzhen where he designed, built, deployed and maintained HPC clusters supporting thousands of bioinformaticians analyzing over 10TB of genomic data per day. He has expert knowledge of large-scale HPC clusters, parallel storage systems, data lifecycle management, sequencer IT solutions, automation, and Linux.
Resume boda liao-20170911
Boda Liao has over 10 years of experience as a senior Linux administrator and data center builder. He has extensive expertise designing, building, and maintaining large-scale high performance computing clusters and storage systems for genomic sequencing companies. His skills include solutions for DNA sequencers, bioinformatics automation, large storage systems, high-speed networks, and cluster management using tools like Puppet.
SUSE Storage: Sizing and Performance (Ceph)
The document discusses Ceph, an open-source distributed storage system. It provides an overview of Ceph's architecture and components, how it works, and considerations for setting up a Ceph cluster. Key points include: Ceph provides unified block, file and object storage interfaces and can scale exponentially. It uses CRUSH to deterministically map data across a cluster for redundancy. Setup choices like network, storage nodes, disks, caching and placement groups impact performance and must be tuned for the workload.
Red Hat Storage Server Administration Deep Dive
"In this session for administrators of all skill levels, you’ll get a deep technical dive into Red Hat Storage Server and GlusterFS administration.
We’ll start with the basics of what scale-out storage is, and learn about the unique implementation of Red Hat Storage Server and its advantages over legacy and competing technologies. From the basic knowledge and design principles, we’ll move to a live start-to-finish demonstration. Your experience will include:
Building a cluster.
Allocating resources.
Creating and modifying volumes of different types.
Accessing data via multiple client protocols.
A resiliency demonstration.
Expanding and contracting volumes.
Implementing directory quotas.
Recovering from and preventing split-brain.
Asynchronous parallel geo-replication.
Behind-the-curtain views of configuration files and logs.
Extended attributes used by GlusterFS.
Performance tuning basics.
New and upcoming feature demonstrations.
Those new to the scale-out product will leave this session with the knowledge and confidence to set up their first Red Hat Storage Server environment. Experienced administrators will sharpen their skills and gain insights into the newest features. IT executives and managers will gain a valuable overview to help fuel the drive for next-generation infrastructures."
Gluster for Geeks: Performance Tuning Tips & Tricks
This document summarizes a webinar on performance tuning tips and tricks for GlusterFS. The webinar covered planning cluster hardware configuration to meet performance requirements, choosing the correct volume type for workloads, key tuning parameters, benchmarking techniques, and the top 5 causes of performance issues. The webinar provided guidance on optimizing GlusterFS performance through hardware sizing, configuration, implementation best practices, and tuning.
Hadoop course content
The Hadoop course covers hardware and software requirements, installing and configuring Linux, Hadoop, HDFS, MapReduce, Pig, Hive, HBase, and MongoDB. It also covers ETL processes using Pentaho Data Integration to load and transform data between relational databases and Hadoop. The course content includes tutorials on virtual machines, Linux commands, Hadoop processes and architecture, developing MapReduce applications, Pig commands, Hive basics, and connecting data warehouse tools to relational databases and Hadoop.
librados
- Librados is a C/C++ programming interface that provides applications access to the Ceph distributed object store (RADOS) and hides the complexity of networking, data distribution, replication and failure recovery.
- It can be used by Ceph components like RADOS Gateway and tools like rados, as well as third party applications that want to use Ceph for storage. Examples include providing storage for mail systems, Hadoop, and building custom applications.
- The interface handles configuration, connections, I/O operations on objects, extended attributes and more so applications can easily integrate scalable reliable storage via Ceph without having to implement these functions themselves.
Quantcast File System (QFS) - Alternative to HDFS
The document summarizes Quantcast File System (QFS), an alternative to HDFS that provides petabyte storage at half the disk space of HDFS. QFS offers significantly faster I/O than HDFS through the use of Reed-Solomon encoding, requiring only 1.5x disk space compared to HDFS's 3x. It has been production hardened at Quantcast under massive processing loads and is fully compatible with Apache Hadoop. Benchmark results show QFS writes are half the disk I/O of HDFS writes and reads require accessing the network versus HDFS's focus on data locality.
Quick-and-Easy Deployment of a Ceph Storage Cluster
Quick & Easy Deployment of a Ceph Storage Cluster with SUSE Enterprise Storage
The document discusses deploying a Ceph storage cluster using SUSE Enterprise Storage. It begins with an introduction to Ceph and how it works as a distributed object storage system. It then covers designing Ceph clusters based on workload needs and measuring performance. The document concludes with step-by-step instructions for deploying a basic three node Ceph cluster with monitoring using SUSE Enterprise Storage.
Performance comparison of Distributed File Systems on 1Gbit networks
Compare the performance of a few Distributed File Systems on 1Gbit networks
- GlusterFS
- XtremeFS
- FhgFS
Ceph Day San Jose - Enable Fast Big Data Analytics on Ceph with Alluxio
Adit Madan from Alluxio presented how to enable fast big data analytics on Ceph object storage using Alluxio. Alluxio acts as a virtual distributed file system that caches data in memory to accelerate access to data stored in Ceph. This provides orders of magnitude faster performance for Spark queries on large datasets. A demo on EC2 showed Spark counting a 60GB dataset was 20x faster when using Alluxio to cache data from Ceph compared to directly accessing Ceph.
Ceph Day San Jose - Ceph at Salesforce
Salesforce uses Ceph for various storage needs including block storage, replacing some SAN scenarios, and as a general purpose blob store. They are experimenting with multiple small Ceph clusters across different availability zones. Performance testing shows good random read and write speeds for SSD-only pools. Challenges include scaling to meet their needs, ensuring security and isolation across multiple tenants, and managing clusters across many data centers.
Performance characterization in large distributed file system with gluster fs
This document discusses the key features and capabilities of the Gluster distributed file system including its ability to provide highly scalable storage across multiple petabytes using scale-out clusters. It can replicate data across geographic locations for disaster recovery and uses an algorithmic approach to avoid metadata bottlenecks. Gluster leverages commodity hardware and is software-only with no SAN requirements. It supports various deployment models including physical, virtual, cloud, and hybrid and uses open protocols like NFS, CIFS, and REST.
Ceph - High Performance Without High Costs
Ceph is a high-performance storage platform that provides storage without high costs. The presentation discusses BlueStore, a redesign of Ceph's object store to improve performance and efficiency. BlueStore preserves wire compatibility but uses an incompatible storage format. It aims to double write performance and match or exceed read performance of the previous FileStore design. BlueStore simplifies the architecture and uses algorithms tailored for different hardware like flash. It was in a tech preview in the Jewel release and aims to be default in the Luminous release next year.
Unlock Bigdata Analytic Efficiency with Ceph Data Lake - Zhang Jian, Fu Yong
Cephalocon APAC 2018
March 22-23, 2018 - Beijing, China
Zhang Jian, Intel Software Engineer Manager
Fu Yong, Intel Software Engineer Manager
Comparison of foss distributed storage
This document summarizes Marian Marinov's testing and experience with different distributed filesystems at his company SiteGround. He tested CephFS, GlusterFS, MooseFS, OrangeFS, and BeeGFS. CephFS required a lot of resources but lacked redundancy. GlusterFS was relatively easy to set up but had high CPU usage. MooseFS and OrangeFS were also easy to set up. Ultimately, they settled on Ceph RBD with NFS and caching for performance and simplicity. File creation performance tests showed MooseFS and NFS+Ceph RBD outperformed OrangeFS and GlusterFS. Tuning settings like MTU, congestion control, and caching helped optimize performance.
Ceph Day San Jose - Object Storage for Big Data
This document discusses using object storage for big data. It outlines key stakeholders in big data projects and what they want from object storage solutions. It then discusses using the Ceph object store to provide an elastic data lake that can disaggregate compute resources from storage. This allows analytics to be performed directly on the object store without expensive ETL processes. It also describes testing various analytics use cases and workloads with the Ceph object store.
Kfs presentation
KFS aka Kosmos FS is a distributed file system written in C++ that is modeled after HDFS. It was originally developed by Kosmix, which was later acquired by Walmart. Some key points:
- KFS uses a master/chunkserver architecture where the metadata is stored on a master node and file data is stored in chunks on chunkservers.
- It supports features like replication, data integrity checks, and rebalancing of chunks.
- While still in early stages, it provides alternatives to the Hadoop ecosystem through its C++ implementation and bindings for other languages like Java and Python.
- The documentation provides instructions for building, deploying, and accessing KFS, though some functionality
Ceph optimized Storage / Global HW solutions for SDS, David Alvarez
This document discusses Supermicro's portfolio of scale-out optimized storage nodes and Ceph-ready hardware solutions. It presents several models of storage nodes that support high density and ultra dense storage and are optimized for the Red Hat Ceph storage platform. The document also covers Supermicro's modular LAN switching I/O modules that provide flexible networking connectivity including 10GbE, 25GbE, and InfiniBand options.
Trying and evaluating the new features of GlusterFS 3.5
My presentation in LinuxCon/CloudOpen Japan 2014.
It has passed few days since GlusterFS 3.5 released so feel free to correct me if you find my mistakes or misunderstandings. Thanks.
Bluestore
This document summarizes BlueStore, a new storage backend for Ceph that provides faster performance compared to the existing FileStore backend. BlueStore manages metadata and data separately, with metadata stored in a key-value database (RocksDB) and data written directly to block devices. This avoids issues with POSIX filesystem transactions and enables more efficient features like checksumming, compression, and cloning. BlueStore addresses consistency and performance problems that arose with previous approaches like FileStore and NewStore.
QCT Ceph Solution - Design Consideration and Reference Architecture
This document discusses QCT's Ceph storage solutions, including an overview of Ceph architecture, QCT hardware platforms, Red Hat Ceph software, workload considerations, reference architectures, test results and a QCT/Red Hat whitepaper. It provides technical details on QCT's throughput-optimized and capacity-optimized solutions and shows how they address different storage needs through workload-driven design. Hands-on testing and a test drive lab are offered to explore Ceph features and configurations.
Hadoop_content_by_sasidhar2
The document outlines the content to be covered in a Hadoop training course by Mr. Sasidhar, a Cloudera certified Hadoop professional. The training will cover topics including Hadoop basics and installation, HDFS, MapReduce, Pig, Hive, Impala, Sqoop, Flume, HBase, MongoDB, ZooKeeper, Spark, and Pentaho Data Integration. It will include both theoretical concepts and hands-on practical sessions using Hadoop clusters. Real-life use cases will also be discussed along with working with Apache and Cloudera distributions. The goal is to provide a comprehensive overview of Hadoop and its ecosystem.
Case Studies on PostgreSQL
Agenda
• Technical cases in PostgreSQL
• Database Monitoring Methods
By Rohit Vyas at India PostgreSQL UserGroup Meetup, Bangalore at InMobi.
http://technology.inmobi.com/events/india-postgresql-usergroup-meetup-bangalore
redis basics
This document summarizes Redis, including where to get it, how it compares to Memcached, common Redis commands, Redis data types, and simple Redis applications. It discusses using Redis for cohort analysis using bitmaps, offloading logic and computing using Lua scripts, and publishing notifications using Pub/Sub. The document provides an overview of Redis capabilities and use cases.
Ceph Day Taipei - Bring Ceph to Enterprise
i. SUSE Enterprise Storage 3 provides iSCSI access to connect remotely to ceph storage over TCP/IP, allowing any iSCSI initiator to access the storage over a network. The iSCSI target driver sits on top of RBD (RADOS block device) to enable this access.
ii. Configuring the lrbd package simplifies setting up an iSCSI gateway to Ceph. Multiple gateways can be configured for high availability using targetcli utility.
iii. Optimizations have been made to the iSCSI gateway to efficiently handle certain SCSI operations like atomic compare and write by offloading work to OSDs to avoid locking on gateway nodes.
Comparison of-foss-distributed-storage
This document summarizes Marian Marinov's testing and experience with various distributed filesystems including CephFS, GlusterFS, MooseFS, OrangeFS, and BeeGFS. Some key findings are:
- CephFS requires significant resources but lacks redundancy for small clusters. GlusterFS offers redundancy but can have high CPU usage.
- MooseFS and OrangeFS were easy to setup but MooseFS offered better reliability and stats.
- Performance testing found MooseFS and NFS+Ceph to have better small file creation times than GlusterFS and OrangeFS. Network latency was identified as a major factor impacting distributed filesystem performance.
- Tuning efforts focused on NFS
Red Hat Gluster Storage Performance
This session will cover performance-related developments in Red Hat Gluster Storage 3 and share best practices for testing, sizing, configuration, and tuning.
Join us to learn about:
Current features in Red Hat Gluster Storage, including 3-way replication, JBOD support, and thin-provisioning.
Features that are in development, including network file system (NFS) support with Ganesha, erasure coding, and cache tiering.
New performance enhancements related to the area of remote directory memory access (RDMA), small-file performance, FUSE caching, and solid state disks (SSD) readiness.
Building a Distributed File System for the Cloud-Native Era
Big Data Bellevue Meetup
May 19, 2022
For more Alluxio events: https://alluxio.io/events/
Speaker: Bin Fan (Founding Engineer & VP of Open Source, Alluxio)
Today, data engineering in modern enterprises has become increasingly more complex and resource-consuming, particularly because (1) the rich amount of organizational data is often distributed across data centers, cloud regions, or even cloud providers, and (2) the complexity of the big data stack has been quickly increasing over the past few years with an explosion in big-data analytics and machine-learning engines (like MapReduce, Hive, Spark, Presto, Tensorflow, PyTorch to name a few).
To address these challenges, it is critical to provide a single and logical namespace to federate different storage services, on-prem or cloud-native, to abstract away the data heterogeneity, while providing data locality to improve the computation performance. [Bin Fan] will share his observation and lessons learned in designing, architecting, and implementing such a system – Alluxio open-source project — since 2015.
Alluxio originated from UC Berkeley AMPLab (used to be called Tachyon) and was initially proposed as a daemon service to enable Spark to share RDDs across jobs for performance and fault tolerance. Today, it has become a general-purpose, high-performance, and highly available distributed file system to provide generic data service to abstract away complexity in data and I/O. Many companies and organizations today like Uber, Meta, Tencent, Tiktok, Shopee are using Alluxio in production, as a building block in their data platform to create a data abstraction and access layer. We will talk about the journey of this open source project, especially in its design challenges in tiered metadata storage (based on RocksDB), embedded state-replicate machine (based on RAFT) for HA, and evolution in RPC framework (based on gRPC) and etc.
Desktop as a Service supporting Environmental ‘omics
Within the Environmental 'omics community Bio-Linux is a widely used tool. This has the advantage of providing in a single deliverable package all necessary software and tools to support common analyses. With the growth in data volumes within the community and increasing
constraints on user access and control over their own desktops an alternative delivery method of Bio-Linux and, in future, the Docker container environment is necessary.
Within the EOS Cloud project we have constructed a Desktop as a Service system to centrally host virtual machines with these tools preconfigured and maintained. To enable efficient use of the resources we have enabled user controlled resource scaling so that users are able to utilise small scale VMs for task configuration and data manipulation and boost to a larger scale to run analysis applications all the while
maintaining the user environment in a consistent manner. Alongside this within the project we have been developed tools to simplify the increasingly popular Docker software usage model. This includes ensure uniformity of behaviour between the host system and the running Docker container.
Within the invitation only trial user community we identify two different exemplars groups and explain their usage and how the products and services developed within the project are useful for them. We conclude discussing the useful nature of Desktop as a Service, how it is of great benefit to the bioinformatics community but could also be of great use elsewhere, where the need for a stable user environment with applications already available that do not rely on local ICT
support.
More Related Content
What's hot
Quick-and-Easy Deployment of a Ceph Storage Cluster
Quick & Easy Deployment of a Ceph Storage Cluster with SUSE Enterprise Storage
The document discusses deploying a Ceph storage cluster using SUSE Enterprise Storage. It begins with an introduction to Ceph and how it works as a distributed object storage system. It then covers designing Ceph clusters based on workload needs and measuring performance. The document concludes with step-by-step instructions for deploying a basic three node Ceph cluster with monitoring using SUSE Enterprise Storage.
Performance comparison of Distributed File Systems on 1Gbit networks
Compare the performance of a few Distributed File Systems on 1Gbit networks
- GlusterFS
- XtremeFS
- FhgFS
Ceph Day San Jose - Enable Fast Big Data Analytics on Ceph with Alluxio
Adit Madan from Alluxio presented how to enable fast big data analytics on Ceph object storage using Alluxio. Alluxio acts as a virtual distributed file system that caches data in memory to accelerate access to data stored in Ceph. This provides orders of magnitude faster performance for Spark queries on large datasets. A demo on EC2 showed Spark counting a 60GB dataset was 20x faster when using Alluxio to cache data from Ceph compared to directly accessing Ceph.
Ceph Day San Jose - Ceph at Salesforce
Salesforce uses Ceph for various storage needs including block storage, replacing some SAN scenarios, and as a general purpose blob store. They are experimenting with multiple small Ceph clusters across different availability zones. Performance testing shows good random read and write speeds for SSD-only pools. Challenges include scaling to meet their needs, ensuring security and isolation across multiple tenants, and managing clusters across many data centers.
Performance characterization in large distributed file system with gluster fs
This document discusses the key features and capabilities of the Gluster distributed file system including its ability to provide highly scalable storage across multiple petabytes using scale-out clusters. It can replicate data across geographic locations for disaster recovery and uses an algorithmic approach to avoid metadata bottlenecks. Gluster leverages commodity hardware and is software-only with no SAN requirements. It supports various deployment models including physical, virtual, cloud, and hybrid and uses open protocols like NFS, CIFS, and REST.
Ceph - High Performance Without High Costs
Ceph is a high-performance storage platform that provides storage without high costs. The presentation discusses BlueStore, a redesign of Ceph's object store to improve performance and efficiency. BlueStore preserves wire compatibility but uses an incompatible storage format. It aims to double write performance and match or exceed read performance of the previous FileStore design. BlueStore simplifies the architecture and uses algorithms tailored for different hardware like flash. It was in a tech preview in the Jewel release and aims to be default in the Luminous release next year.
Unlock Bigdata Analytic Efficiency with Ceph Data Lake - Zhang Jian, Fu Yong
Cephalocon APAC 2018
March 22-23, 2018 - Beijing, China
Zhang Jian, Intel Software Engineer Manager
Fu Yong, Intel Software Engineer Manager
Comparison of foss distributed storage
This document summarizes Marian Marinov's testing and experience with different distributed filesystems at his company SiteGround. He tested CephFS, GlusterFS, MooseFS, OrangeFS, and BeeGFS. CephFS required a lot of resources but lacked redundancy. GlusterFS was relatively easy to set up but had high CPU usage. MooseFS and OrangeFS were also easy to set up. Ultimately, they settled on Ceph RBD with NFS and caching for performance and simplicity. File creation performance tests showed MooseFS and NFS+Ceph RBD outperformed OrangeFS and GlusterFS. Tuning settings like MTU, congestion control, and caching helped optimize performance.
Ceph Day San Jose - Object Storage for Big Data
This document discusses using object storage for big data. It outlines key stakeholders in big data projects and what they want from object storage solutions. It then discusses using the Ceph object store to provide an elastic data lake that can disaggregate compute resources from storage. This allows analytics to be performed directly on the object store without expensive ETL processes. It also describes testing various analytics use cases and workloads with the Ceph object store.
Kfs presentation
KFS aka Kosmos FS is a distributed file system written in C++ that is modeled after HDFS. It was originally developed by Kosmix, which was later acquired by Walmart. Some key points:
- KFS uses a master/chunkserver architecture where the metadata is stored on a master node and file data is stored in chunks on chunkservers.
- It supports features like replication, data integrity checks, and rebalancing of chunks.
- While still in early stages, it provides alternatives to the Hadoop ecosystem through its C++ implementation and bindings for other languages like Java and Python.
- The documentation provides instructions for building, deploying, and accessing KFS, though some functionality
Ceph optimized Storage / Global HW solutions for SDS, David Alvarez
This document discusses Supermicro's portfolio of scale-out optimized storage nodes and Ceph-ready hardware solutions. It presents several models of storage nodes that support high density and ultra dense storage and are optimized for the Red Hat Ceph storage platform. The document also covers Supermicro's modular LAN switching I/O modules that provide flexible networking connectivity including 10GbE, 25GbE, and InfiniBand options.
Trying and evaluating the new features of GlusterFS 3.5
My presentation in LinuxCon/CloudOpen Japan 2014.
It has passed few days since GlusterFS 3.5 released so feel free to correct me if you find my mistakes or misunderstandings. Thanks.
Bluestore
This document summarizes BlueStore, a new storage backend for Ceph that provides faster performance compared to the existing FileStore backend. BlueStore manages metadata and data separately, with metadata stored in a key-value database (RocksDB) and data written directly to block devices. This avoids issues with POSIX filesystem transactions and enables more efficient features like checksumming, compression, and cloning. BlueStore addresses consistency and performance problems that arose with previous approaches like FileStore and NewStore.
QCT Ceph Solution - Design Consideration and Reference Architecture
This document discusses QCT's Ceph storage solutions, including an overview of Ceph architecture, QCT hardware platforms, Red Hat Ceph software, workload considerations, reference architectures, test results and a QCT/Red Hat whitepaper. It provides technical details on QCT's throughput-optimized and capacity-optimized solutions and shows how they address different storage needs through workload-driven design. Hands-on testing and a test drive lab are offered to explore Ceph features and configurations.
Hadoop_content_by_sasidhar2
The document outlines the content to be covered in a Hadoop training course by Mr. Sasidhar, a Cloudera certified Hadoop professional. The training will cover topics including Hadoop basics and installation, HDFS, MapReduce, Pig, Hive, Impala, Sqoop, Flume, HBase, MongoDB, ZooKeeper, Spark, and Pentaho Data Integration. It will include both theoretical concepts and hands-on practical sessions using Hadoop clusters. Real-life use cases will also be discussed along with working with Apache and Cloudera distributions. The goal is to provide a comprehensive overview of Hadoop and its ecosystem.
Case Studies on PostgreSQL
Agenda
• Technical cases in PostgreSQL
• Database Monitoring Methods
By Rohit Vyas at India PostgreSQL UserGroup Meetup, Bangalore at InMobi.
http://technology.inmobi.com/events/india-postgresql-usergroup-meetup-bangalore
redis basics
This document summarizes Redis, including where to get it, how it compares to Memcached, common Redis commands, Redis data types, and simple Redis applications. It discusses using Redis for cohort analysis using bitmaps, offloading logic and computing using Lua scripts, and publishing notifications using Pub/Sub. The document provides an overview of Redis capabilities and use cases.
Ceph Day Taipei - Bring Ceph to Enterprise
i. SUSE Enterprise Storage 3 provides iSCSI access to connect remotely to ceph storage over TCP/IP, allowing any iSCSI initiator to access the storage over a network. The iSCSI target driver sits on top of RBD (RADOS block device) to enable this access.
ii. Configuring the lrbd package simplifies setting up an iSCSI gateway to Ceph. Multiple gateways can be configured for high availability using targetcli utility.
iii. Optimizations have been made to the iSCSI gateway to efficiently handle certain SCSI operations like atomic compare and write by offloading work to OSDs to avoid locking on gateway nodes.
Comparison of-foss-distributed-storage
This document summarizes Marian Marinov's testing and experience with various distributed filesystems including CephFS, GlusterFS, MooseFS, OrangeFS, and BeeGFS. Some key findings are:
- CephFS requires significant resources but lacks redundancy for small clusters. GlusterFS offers redundancy but can have high CPU usage.
- MooseFS and OrangeFS were easy to setup but MooseFS offered better reliability and stats.
- Performance testing found MooseFS and NFS+Ceph to have better small file creation times than GlusterFS and OrangeFS. Network latency was identified as a major factor impacting distributed filesystem performance.
- Tuning efforts focused on NFS
Red Hat Gluster Storage Performance
This session will cover performance-related developments in Red Hat Gluster Storage 3 and share best practices for testing, sizing, configuration, and tuning.
Join us to learn about:
Current features in Red Hat Gluster Storage, including 3-way replication, JBOD support, and thin-provisioning.
Features that are in development, including network file system (NFS) support with Ganesha, erasure coding, and cache tiering.
New performance enhancements related to the area of remote directory memory access (RDMA), small-file performance, FUSE caching, and solid state disks (SSD) readiness.
What's hot (20)
Quick-and-Easy Deployment of a Ceph Storage Cluster
Quick-and-Easy Deployment of a Ceph Storage Cluster
Performance comparison of Distributed File Systems on 1Gbit networks
Performance comparison of Distributed File Systems on 1Gbit networks
Ceph Day San Jose - Enable Fast Big Data Analytics on Ceph with Alluxio
Ceph Day San Jose - Enable Fast Big Data Analytics on Ceph with Alluxio
Performance characterization in large distributed file system with gluster fs
Performance characterization in large distributed file system with gluster fs
Unlock Bigdata Analytic Efficiency with Ceph Data Lake - Zhang Jian, Fu Yong
Unlock Bigdata Analytic Efficiency with Ceph Data Lake - Zhang Jian, Fu Yong
Ceph optimized Storage / Global HW solutions for SDS, David Alvarez
Ceph optimized Storage / Global HW solutions for SDS, David Alvarez
Trying and evaluating the new features of GlusterFS 3.5
Trying and evaluating the new features of GlusterFS 3.5
QCT Ceph Solution - Design Consideration and Reference Architecture
QCT Ceph Solution - Design Consideration and Reference Architecture
Similar to Resume boda liao-storage
Building a Distributed File System for the Cloud-Native Era
Big Data Bellevue Meetup
May 19, 2022
For more Alluxio events: https://alluxio.io/events/
Speaker: Bin Fan (Founding Engineer & VP of Open Source, Alluxio)
Today, data engineering in modern enterprises has become increasingly more complex and resource-consuming, particularly because (1) the rich amount of organizational data is often distributed across data centers, cloud regions, or even cloud providers, and (2) the complexity of the big data stack has been quickly increasing over the past few years with an explosion in big-data analytics and machine-learning engines (like MapReduce, Hive, Spark, Presto, Tensorflow, PyTorch to name a few).
To address these challenges, it is critical to provide a single and logical namespace to federate different storage services, on-prem or cloud-native, to abstract away the data heterogeneity, while providing data locality to improve the computation performance. [Bin Fan] will share his observation and lessons learned in designing, architecting, and implementing such a system – Alluxio open-source project — since 2015.
Alluxio originated from UC Berkeley AMPLab (used to be called Tachyon) and was initially proposed as a daemon service to enable Spark to share RDDs across jobs for performance and fault tolerance. Today, it has become a general-purpose, high-performance, and highly available distributed file system to provide generic data service to abstract away complexity in data and I/O. Many companies and organizations today like Uber, Meta, Tencent, Tiktok, Shopee are using Alluxio in production, as a building block in their data platform to create a data abstraction and access layer. We will talk about the journey of this open source project, especially in its design challenges in tiered metadata storage (based on RocksDB), embedded state-replicate machine (based on RAFT) for HA, and evolution in RPC framework (based on gRPC) and etc.
Desktop as a Service supporting Environmental ‘omics
Within the Environmental 'omics community Bio-Linux is a widely used tool. This has the advantage of providing in a single deliverable package all necessary software and tools to support common analyses. With the growth in data volumes within the community and increasing
constraints on user access and control over their own desktops an alternative delivery method of Bio-Linux and, in future, the Docker container environment is necessary.
Within the EOS Cloud project we have constructed a Desktop as a Service system to centrally host virtual machines with these tools preconfigured and maintained. To enable efficient use of the resources we have enabled user controlled resource scaling so that users are able to utilise small scale VMs for task configuration and data manipulation and boost to a larger scale to run analysis applications all the while
maintaining the user environment in a consistent manner. Alongside this within the project we have been developed tools to simplify the increasingly popular Docker software usage model. This includes ensure uniformity of behaviour between the host system and the running Docker container.
Within the invitation only trial user community we identify two different exemplars groups and explain their usage and how the products and services developed within the project are useful for them. We conclude discussing the useful nature of Desktop as a Service, how it is of great benefit to the bioinformatics community but could also be of great use elsewhere, where the need for a stable user environment with applications already available that do not rely on local ICT
support.
Accelerating Apache Hadoop through High-Performance Networking and I/O Techno...
Accelerating Apache Hadoop through High-Performance Networking and I/O Techno...DataWorks Summit/Hadoop Summit
The document discusses accelerating Apache Hadoop through high-performance networking and I/O technologies. It describes how technologies like InfiniBand, RoCE, SSDs, and NVMe can benefit big data applications by alleviating bottlenecks. It outlines projects from the High-Performance Big Data project that implement RDMA for Hadoop, Spark, HBase and Memcached to improve performance. Evaluation results demonstrate significant acceleration of HDFS, MapReduce, and other workloads through the high-performance designs.Storage, Backup und Business Continuity mit Open-E
Produkte von Open-E sind seit vielen Jahren etabliert im Storage-Markt. Im Webinar geben Hr. Krzysztof Franek (CEO) und Janusz Bak (CTO) einen Überblick zur aktuellen Funktionalität der auf ZFS basierenden Software Open-E JovianDSS. Sie werden verschiedene High-Availablilty-Konfigurationen vorstellen, sowie auf On- und Offsite-Data-Protection eingehen. Die Präsentation wird unterstützt mit zahlreichen Live-Demos.
The Proto-Burst Buffer: Experience with the flash-based file system on SDSC's...
Glenn K. Lockwood's document summarizes his professional background and experience with data-intensive computing systems. It then discusses the Gordon supercomputer deployed at SDSC in 2012, which was one of the world's first systems to use flash storage. The document analyzes Gordon's architecture using burst buffers and SSDs, experiences using the flash file system, and lessons learned. It also compares Gordon's proto-burst buffer approach to the dedicated burst buffer nodes on the Cori supercomputer.
Ncar globally accessible user environment
In this video from the DDN User Group at SC16, Pamela Hill, Computational & Information Systems Laboratory, NCAR/UCAR, presents: NCAR’s Evolving Infrastructure for Weather and Climate Research.
Watch the video presentation: http://wp.me/p3RLHQ-g4a
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Big data with hadoop Setup on Ubuntu 12.04
This document provides an overview of Hadoop and how to set it up. It first defines big data and describes Hadoop's advantages over traditional systems, such as its ability to handle large datasets across commodity hardware. It then outlines Hadoop's components like HDFS and MapReduce. The document concludes by detailing the steps to install Hadoop, including setting up Linux prerequisites, configuring files, and starting the processes.
NSCC Training Introductory Class
The document provides an introduction to the National Supercomputing Centre (NSCC) high performance computing cluster. It describes the 1 petaflop system consisting of 1300 nodes and 13 petabytes of storage. The system uses PBS Pro for job scheduling and includes compilers, libraries, developer tools, and applications for engineering, science, and industry users from organizations such as A*STAR, NUS, and NTU.
NSCC Training - Introductory Class
The National Supercomputing Centre (NSCC) operates a 1 petaflop supercomputing cluster with 1300 nodes for research and industry users. It has 13 petabytes of storage including a 265TB burst buffer and uses a Mellanox 100Gbps network. The cluster scheduler is PBS Pro and supports applications in areas like engineering, science, life sciences and more.
Big Data Meets HPC - Exploiting HPC Technologies for Accelerating Big Data Pr...
In this deck from the Stanford HPC Conference, DK Panda from Ohio State University presents: Big Data Meets HPC - Exploiting HPC Technologies for Accelerating Big Data Processing.
"This talk will provide an overview of challenges in accelerating Hadoop, Spark and Memcached on modern HPC clusters. An overview of RDMA-based designs for Hadoop (HDFS, MapReduce, RPC and HBase), Spark, Memcached, Swift, and Kafka using native RDMA support for InfiniBand and RoCE will be presented. Enhanced designs for these components to exploit NVM-based in-memory technology and parallel file systems (such as Lustre) will also be presented. Benefits of these designs on various cluster configurations using the publicly available RDMA-enabled packages from the OSU HiBD project (http://hibd.cse.ohio-state.edu) will be shown."
Watch the video: https://youtu.be/iLTYkTandEA
Learn more: http://web.cse.ohio-state.edu/~panda.2/
and
http://hpcadvisorycouncil.com
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
BeeGFS Enterprise Deployment
The document summarizes Fred Hutch's transition to using BeeGFS for their file storage needs over recent years:
1) They initially deployed a scale-out NAS in 2012 to consolidate storage for hundreds of terabytes of data across multiple research groups.
2) In 2014, they deployed BeeGFS for scratch storage for up to 500TB of data across 10 research groups with 100% uptime for 3 years.
3) By 2019, they migrated all 150 research groups and over 700 million files totaling 2 petabytes to a redundant BeeGFS deployment for higher reliability, with backup to the cloud.
LAN, WAN, SAN upgrades: hyperconverged vs traditional vs cloud
A presentation for Networkshop47 by Simon Palmer, head of IT development, Coleg Sir Gar.
Experiences of two merged colleges in Wales.
Hadoop for Scientific Workloads__HadoopSummit2010
The document discusses using Hadoop for scientific workloads and summarizes early results from benchmarking Hadoop. It explores using Hadoop and MapReduce for data-intensive scientific applications like BLAST sequence analysis. Performance results show that Hadoop can provide comparable performance to existing parallel file systems. Challenges include lack of turn-key solutions, managing data formats, and performance tuning. The research aims to understand the unique needs of science clouds and how to effectively support data-intensive scientific applications on cloud platforms.
Deploying Big-Data-as-a-Service (BDaaS) in the Enterprise
This document summarizes a presentation about deploying Big Data as a Service (BDaaS) in the enterprise. It discusses how BDaaS can address conflicting needs of data scientists wanting flexibility and IT wanting control. It defines different types of BDaaS and requirements for enterprise deployment such as multi-tenancy, security, and application support. The presentation covers design decisions for BDaaS including running Hadoop/Spark unmodified using containers for isolation. It provides details on the implementation including network architecture, storage, and image management. It also discusses performance testing results and demos the BDaaS platform.
Big Data Benchmarking with RDMA solutions
This document discusses how Mellanox technologies can accelerate big data solutions using RDMA. It summarizes that Mellanox provides end-to-end interconnect solutions including adapters, switches, and cables. It also discusses three key areas for acceleration: data analytics, storage, and distributed storage. The document presents the Unstructured Data Accelerator plugin which can double MapReduce performance using RDMA for efficient data shuffling. It also discusses using RDMA and SSDs to unlock higher throughput in HDFS and overcome bandwidth limitations of 1GbE and 10GbE networks.
Foss evolution cos-boudnik
This document discusses the evolution of open source software (FOSS) for data, specifically focusing on the Hadoop ecosystem. It notes that FOSS has evolved through an iterative and collaborative process, driven by user demands for simplicity, data preservation, consistency, and high availability. It describes how the Hadoop ecosystem has rapidly advanced through many releases and over 18 components to meet these demands. However, challenges remain around operational complexity and achieving strong consistency and high availability. The document proposes consensus-based replication as an alternative to provide preventative fault tolerance rather than reactive solutions. It introduces WANdisco as a company that provides patented active-active replication technology to solve availability problems for enterprises.
Ceph Day New York 2014: Best Practices for Ceph-Powered Implementations of St...
This document discusses best practices for implementing Ceph-powered storage as a service. It covers planning a Ceph implementation based on business and technical requirements. Various use cases for Ceph are described, including OpenStack, cloud storage, web-scale applications, high performance block storage, archive/cold storage, databases and Hadoop. Architectural considerations for redundancy, servers, networking are also discussed. The document concludes with a case study of a university implementing a Ceph-based storage cloud to address storage needs for cancer and genomic research data.
MyCloud for $100k
A simple setup to build a private or public cloud.
A cloud at the IaaS layer is simply a cluster of hypervisors with some added storage infrastructure and software to orchestrate everything. In this presentation we show some straightfoward DELL hardware that could be purchased to build a single rack as the basic for a private or public cloud. It totals $100k and coupled with open source software: cloudstack, ceph, glusterfs, nfs etc is the basis for your cloud.
You will get a AWS compatible cloud in no-time and with limited acquisition cost.
Best Practices for Virtualizing Apache Hadoop
Join this webinar to discuss best practices for designing and building a solid, robust and flexible Hadoop platform on an enterprise virtual infrastructure. Attendees will learn the flexibility and operational advantages of Virtual Machines such as fast provisioning, cloning, high levels of standardization, hybrid storage, vMotioning, increased stabilization of the entire software stack, High Availability and Fault Tolerance. This is a can`t miss presentation for anyone wanting to understand design, configuration and deployment of Hadoop in virtual infrastructures.
New Ceph capabilities and Reference Architectures
Have you heard about Inktank Ceph and are interested to learn some tips and tricks for getting started quickly and efficiently with Ceph? Then this is the session for you!
In this two part session you learn details of:
• the very latest enhancements and capabilities delivered in Inktank Ceph Enterprise such as a new erasure coded storage back-end, support for tiering, and the introduction of user quotas.
• best practices, lessons learned and architecture considerations founded in real customer deployments of Dell and Inktank Ceph solutions that will help accelerate your Ceph deployment.
Similar to Resume boda liao-storage (20)
Building a Distributed File System for the Cloud-Native Era
Building a Distributed File System for the Cloud-Native Era
Desktop as a Service supporting Environmental ‘omics
Desktop as a Service supporting Environmental ‘omics
Accelerating Apache Hadoop through High-Performance Networking and I/O Techno...
Accelerating Apache Hadoop through High-Performance Networking and I/O Techno...
Storage, Backup und Business Continuity mit Open-E
Storage, Backup und Business Continuity mit Open-E
The Proto-Burst Buffer: Experience with the flash-based file system on SDSC's...
The Proto-Burst Buffer: Experience with the flash-based file system on SDSC's...
Big Data Meets HPC - Exploiting HPC Technologies for Accelerating Big Data Pr...
Big Data Meets HPC - Exploiting HPC Technologies for Accelerating Big Data Pr...
LAN, WAN, SAN upgrades: hyperconverged vs traditional vs cloud
LAN, WAN, SAN upgrades: hyperconverged vs traditional vs cloud
Deploying Big-Data-as-a-Service (BDaaS) in the Enterprise
Deploying Big-Data-as-a-Service (BDaaS) in the Enterprise
Ceph Day New York 2014: Best Practices for Ceph-Powered Implementations of St...
Ceph Day New York 2014: Best Practices for Ceph-Powered Implementations of St...
Recently uploaded
Blood finder application project report (1).pdf
Blood Finder is an emergency time app where a user can search for the blood banks as
well as the registered blood donors around Mumbai. This application also provide an
opportunity for the user of this application to become a registered donor for this user have
to enroll for the donor request from the application itself. If the admin wish to make user
a registered donor, with some of the formalities with the organization it can be done.
Specialization of this application is that the user will not have to register on sign-in for
searching the blood banks and blood donors it can be just done by installing the
application to the mobile.
The purpose of making this application is to save the user’s time for searching blood of
needed blood group during the time of the emergency.
This is an android application developed in Java and XML with the connectivity of
SQLite database. This application will provide most of basic functionality required for an
emergency time application. All the details of Blood banks and Blood donors are stored
in the database i.e. SQLite.
This application allowed the user to get all the information regarding blood banks and
blood donors such as Name, Number, Address, Blood Group, rather than searching it on
the different websites and wasting the precious time. This application is effective and
user friendly.
Levelised Cost of Hydrogen (LCOH) Calculator Manual
The aim of this manual is to explain the
methodology behind the Levelized Cost of
Hydrogen (LCOH) calculator. Moreover, this
manual also demonstrates how the calculator
can be used for estimating the expenses associated with hydrogen production in Europe
using low-temperature electrolysis considering different sources of electricity
一比一原版(爱大毕业证书)爱荷华大学毕业证如何办理
原版一模一样【微信：741003700 】【(爱大毕业证书)爱荷华大学毕业证成绩单】【微信：741003700 】学位证，留信认证（真实可查，永久存档）原件一模一样纸张工艺/offer、雅思、外壳等材料/诚信可靠,可直接看成品样本，帮您解决无法毕业带来的各种难题！外壳，原版制作，诚信可靠，可直接看成品样本。行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备。十五年致力于帮助留学生解决难题，包您满意。
本公司拥有海外各大学样板无数，能完美还原。
1:1完美还原海外各大学毕业材料上的工艺：水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠。文字图案浮雕、激光镭射、紫外荧光、温感、复印防伪等防伪工艺。材料咨询办理、认证咨询办理请加学历顾问Q/微741003700
【主营项目】
一.毕业证【q微741003700】成绩单、使馆认证、教育部认证、雅思托福成绩单、学生卡等！
二.真实使馆公证(即留学回国人员证明,不成功不收费)
三.真实教育部学历学位认证（教育部存档！教育部留服网站永久可查）
四.办理各国各大学文凭(一对一专业服务,可全程监控跟踪进度)
如果您处于以下几种情况：
◇在校期间，因各种原因未能顺利毕业……拿不到官方毕业证【q/微741003700】
◇面对父母的压力，希望尽快拿到；
◇不清楚认证流程以及材料该如何准备；
◇回国时间很长，忘记办理；
◇回国马上就要找工作，办给用人单位看；
◇企事业单位必须要求办理的
◇需要报考公务员、购买免税车、落转户口
◇申请留学生创业基金
留信网认证的作用:
1:该专业认证可证明留学生真实身份
2:同时对留学生所学专业登记给予评定
3:国家专业人才认证中心颁发入库证书
4:这个认证书并且可以归档倒地方
5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息
6:个人职称评审加20分
7:个人信誉贷款加10分
8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才
办理(爱大毕业证书)爱荷华大学毕业证【微信：741003700 】外观非常简单，由纸质材料制成，上面印有校徽、校名、毕业生姓名、专业等信息。
办理(爱大毕业证书)爱荷华大学毕业证【微信：741003700 】格式相对统一，各专业都有相应的模板。通常包括以下部分：
校徽：象征着学校的荣誉和传承。
校名:学校英文全称
授予学位：本部分将注明获得的具体学位名称。
毕业生姓名：这是最重要的信息之一，标志着该证书是由特定人员获得的。
颁发日期：这是毕业正式生效的时间，也代表着毕业生学业的结束。
其他信息：根据不同的专业和学位，可能会有一些特定的信息或章节。
办理(爱大毕业证书)爱荷华大学毕业证【微信：741003700 】价值很高，需要妥善保管。一般来说，应放置在安全、干燥、防潮的地方，避免长时间暴露在阳光下。如需使用，最好使用复印件而不是原件，以免丢失。
综上所述，办理(爱大毕业证书)爱荷华大学毕业证【微信：741003700 】是证明身份和学历的高价值文件。外观简单庄重，格式统一，包括重要的个人信息和发布日期。对持有人来说，妥善保管是非常重要的。
Object Oriented Analysis and Design - OOAD
This ppt gives detailed description of Object Oriented Analysis and design.
Prediction of Electrical Energy Efficiency Using Information on Consumer's Ac...
Energy efficiency has been important since the latter part of the last century. The main object of this survey is to determine the energy efficiency knowledge among consumers. Two separate districts in Bangladesh are selected to conduct the survey on households and showrooms about the energy and seller also. The survey uses the data to find some regression equations from which it is easy to predict energy efficiency knowledge. The data is analyzed and calculated based on five important criteria. The initial target was to find some factors that help predict a person's energy efficiency knowledge. From the survey, it is found that the energy efficiency awareness among the people of our country is very low. Relationships between household energy use behaviors are estimated using a unique dataset of about 40 households and 20 showrooms in Bangladesh's Chapainawabganj and Bagerhat districts. Knowledge of energy consumption and energy efficiency technology options is found to be associated with household use of energy conservation practices. Household characteristics also influence household energy use behavior. Younger household cohorts are more likely to adopt energy-efficient technologies and energy conservation practices and place primary importance on energy saving for environmental reasons. Education also influences attitudes toward energy conservation in Bangladesh. Low-education households indicate they primarily save electricity for the environment while high-education households indicate they are motivated by environmental concerns.
Unit -II Spectroscopy - EC I B.Tech.pdf
Chemical engineering and electronics engineering and chemical development of the day of the almighty paw paw paw
Tools & Techniques for Commissioning and Maintaining PV Systems W-Animations ...
Join us for this solutions-based webinar on the tools and techniques for commissioning and maintaining PV Systems. In this session, we'll review the process of building and maintaining a solar array, starting with installation and commissioning, then reviewing operations and maintenance of the system. This course will review insulation resistance testing, I-V curve testing, earth-bond continuity, ground resistance testing, performance tests, visual inspections, ground and arc fault testing procedures, and power quality analysis.
Fluke Solar Application Specialist Will White is presenting on this engaging topic:
Will has worked in the renewable energy industry since 2005, first as an installer for a small east coast solar integrator before adding sales, design, and project management to his skillset. In 2022, Will joined Fluke as a solar application specialist, where he supports their renewable energy testing equipment like IV-curve tracers, electrical meters, and thermal imaging cameras. Experienced in wind power, solar thermal, energy storage, and all scales of PV, Will has primarily focused on residential and small commercial systems. He is passionate about implementing high-quality, code-compliant installation techniques.
一比一原版(CalArts毕业证)加利福尼亚艺术学院毕业证如何办理
CalArts毕业证学历书【微信95270640】CalArts毕业证’圣力嘉学院毕业证《Q微信95270640》办理CalArts毕业证√文凭学历制作{CalArts文凭}购买学历学位证书本科硕士,CalArts毕业证学历学位证【实体公司】办毕业证、成绩单、学历认证、学位证、文凭认证、办留信网认证、（网上可查，实体公司，专业可靠）
(诚招代理)办理国外高校毕业证成绩单文凭学位证,真实使馆公证（留学回国人员证明）真实留信网认证国外学历学位认证雅思代考国外学校代申请名校保录开请假条改GPA改成绩ID卡
1.高仿业务:【本科硕士】毕业证,成绩单（GPA修改）,学历认证（教育部认证）,大学Offer,,ID,留信认证,使馆认证,雅思,语言证书等高仿类证书；
2.认证服务: 学历认证（教育部认证）,大使馆认证（回国人员证明）,留信认证（可查有编号证书）,大学保录取,雅思保分成绩单。
3.技术服务：钢印水印烫金激光防伪凹凸版设计印刷激凸温感光标底纹镭射速度快。
办理加利福尼亚艺术学院加利福尼亚艺术学院毕业证文凭证书流程：
1客户提供办理信息：姓名生日专业学位毕业时间等（如信息不确定可以咨询顾问：我们有专业老师帮你查询）；
2开始安排制作毕业证成绩单电子图；
3毕业证成绩单电子版做好以后发送给您确认；
4毕业证成绩单电子版您确认信息无误之后安排制作成品；
5成品做好拍照或者视频给您确认；
6快递给客户（国内顺丰国外DHLUPS等快读邮寄）
-办理真实使馆公证（即留学回国人员证明）
-办理各国各大学文凭（世界名校一对一专业服务,可全程监控跟踪进度）
-全套服务：毕业证成绩单真实使馆公证真实教育部认证。让您回国发展信心十足！
（详情请加一下 文凭顾问+微信:95270640）欢迎咨询！子小伍玩小伍比山娃小一岁虎头虎脑的很霸气父亲让山娃跟小伍去夏令营听课山娃很高兴夏令营就设在附近一所小学山娃发现那所小学比自己的学校更大更美操场上还铺有塑胶跑道呢里面很多小朋友一班一班的快快乐乐原来城里娃都藏这儿来了怪不得平时见不到他们山娃恍然大悟起来吹拉弹唱琴棋书画山娃都不懂却什么都想学山娃怨自己太笨什么都不会斟酌再三山娃终于选定了学美术当听说每月要交元时父亲犹豫了山娃也说爸算了吧咱学校一学期才转
一比一原版(USF毕业证)旧金山大学毕业证如何办理
原件一模一样【微信：95270640】【旧金山大学毕业证USF学位证成绩单】【微信：95270640】（留信学历认证永久存档查询）采用学校原版纸张、特殊工艺完全按照原版一比一制作（包括：隐形水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠，文字图案浮雕，激光镭射，紫外荧光，温感，复印防伪）行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备，十五年致力于帮助留学生解决难题，业务范围有加拿大、英国、澳洲、韩国、美国、新加坡，新西兰等学历材料，包您满意。
【业务选择办理准则】
一、工作未确定，回国需先给父母、亲戚朋友看下文凭的情况，办理一份就读学校的毕业证【微信：95270640】文凭即可
二、回国进私企、外企、自己做生意的情况，这些单位是不查询毕业证真伪的，而且国内没有渠道去查询国外文凭的真假，也不需要提供真实教育部认证。鉴于此，办理一份毕业证【微信：95270640】即可
三、进国企，银行，事业单位，考公务员等等，这些单位是必需要提供真实教育部认证的，办理教育部认证所需资料众多且烦琐，所有材料您都必须提供原件，我们凭借丰富的经验，快捷的绿色通道帮您快速整合材料，让您少走弯路。
留信网认证的作用:
1:该专业认证可证明留学生真实身份【微信：95270640】
2:同时对留学生所学专业登记给予评定
3:国家专业人才认证中心颁发入库证书
4:这个认证书并且可以归档倒地方
5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息
6:个人职称评审加20分
7:个人信誉贷款加10分
8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才
→ 【关于价格问题（保证一手价格）
我们所定的价格是非常合理的，而且我们现在做得单子大多数都是代理和回头客户介绍的所以一般现在有新的单子 我给客户的都是第一手的代理价格，因为我想坦诚对待大家 不想跟大家在价格方面浪费时间
对于老客户或者被老客户介绍过来的朋友，我们都会适当给一些优惠。
选择实体注册公司办理，更放心，更安全！我们的承诺：可来公司面谈，可签订合同，会陪同客户一起到教育部认证窗口递交认证材料，客户在教育部官方认证查询网站查询到认证通过结果后付款，不成功不收费！
办理旧金山大学毕业证毕业证学位证USF学位证【微信：95270640 】外观非常精致，由特殊纸质材料制成，上面印有校徽、校名、毕业生姓名、专业等信息。
办理旧金山大学毕业证USF学位证毕业证学位证【微信：95270640 】格式相对统一，各专业都有相应的模板。通常包括以下部分：
校徽：象征着学校的荣誉和传承。
校名:学校英文全称
授予学位：本部分将注明获得的具体学位名称。
毕业生姓名：这是最重要的信息之一，标志着该证书是由特定人员获得的。
颁发日期：这是毕业正式生效的时间，也代表着毕业生学业的结束。
其他信息：根据不同的专业和学位，可能会有一些特定的信息或章节。
办理旧金山大学毕业证毕业证学位证USF学位证【微信：95270640 】价值很高，需要妥善保管。一般来说，应放置在安全、干燥、防潮的地方，避免长时间暴露在阳光下。如需使用，最好使用复印件而不是原件，以免丢失。
综上所述，办理旧金山大学毕业证毕业证学位证USF学位证【微信：95270640 】是证明身份和学历的高价值文件。外观简单庄重，格式统一，包括重要的个人信息和发布日期。对持有人来说，妥善保管是非常重要的。
Generative AI Use cases applications solutions and implementation.pdf
Generative AI solutions encompass a range of capabilities from content creation to complex problem-solving across industries. Implementing generative AI involves identifying specific business needs, developing tailored AI models using techniques like GANs and VAEs, and integrating these models into existing workflows. Data quality and continuous model refinement are crucial for effective implementation. Businesses must also consider ethical implications and ensure transparency in AI decision-making. Generative AI's implementation aims to enhance efficiency, creativity, and innovation by leveraging autonomous generation and sophisticated learning algorithms to meet diverse business challenges.
https://www.leewayhertz.com/generative-ai-use-cases-and-applications/
OOPS_Lab_Manual - programs using C++ programming language
This manual contains programs on object oriented programming concepts using C++ language.
smart pill dispenser is designed to improve medication adherence and safety f...
Smart Pill Dispenser that boosts medication adherence, empowers patients, enables remote monitoring, enhances safety, reduces healthcare costs, and contributes to data-driven healthcare improvements
Applications of artificial Intelligence in Mechanical Engineering.pdf
Historically, mechanical engineering has relied heavily on human expertise and empirical methods to solve complex problems. With the introduction of computer-aided design (CAD) and finite element analysis (FEA), the field took its first steps towards digitization. These tools allowed engineers to simulate and analyze mechanical systems with greater accuracy and efficiency. However, the sheer volume of data generated by modern engineering systems and the increasing complexity of these systems have necessitated more advanced analytical tools, paving the way for AI.
AI offers the capability to process vast amounts of data, identify patterns, and make predictions with a level of speed and accuracy unattainable by traditional methods. This has profound implications for mechanical engineering, enabling more efficient design processes, predictive maintenance strategies, and optimized manufacturing operations. AI-driven tools can learn from historical data, adapt to new information, and continuously improve their performance, making them invaluable in tackling the multifaceted challenges of modern mechanical engineering.
Null Bangalore | Pentesters Approach to AWS IAM
#Abstract:
- Learn more about the real-world methods for auditing AWS IAM (Identity and Access Management) as a pentester. So let us proceed with a brief discussion of IAM as well as some typical misconfigurations and their potential exploits in order to reinforce the understanding of IAM security best practices.
- Gain actionable insights into AWS IAM policies and roles, using hands on approach.
#Prerequisites:
- Basic understanding of AWS services and architecture
- Familiarity with cloud security concepts
- Experience using the AWS Management Console or AWS CLI.
- For hands on lab create account on [killercoda.com](https://killercoda.com/cloudsecurity-scenario/)
# Scenario Covered:
- Basics of IAM in AWS
- Implementing IAM Policies with Least Privilege to Manage S3 Bucket
- Objective: Create an S3 bucket with least privilege IAM policy and validate access.
- Steps:
- Create S3 bucket.
- Attach least privilege policy to IAM user.
- Validate access.
- Exploiting IAM PassRole Misconfiguration
-Allows a user to pass a specific IAM role to an AWS service (ec2), typically used for service access delegation. Then exploit PassRole Misconfiguration granting unauthorized access to sensitive resources.
- Objective: Demonstrate how a PassRole misconfiguration can grant unauthorized access.
- Steps:
- Allow user to pass IAM role to EC2.
- Exploit misconfiguration for unauthorized access.
- Access sensitive resources.
- Exploiting IAM AssumeRole Misconfiguration with Overly Permissive Role
- An overly permissive IAM role configuration can lead to privilege escalation by creating a role with administrative privileges and allow a user to assume this role.
- Objective: Show how overly permissive IAM roles can lead to privilege escalation.
- Steps:
- Create role with administrative privileges.
- Allow user to assume the role.
- Perform administrative actions.
- Differentiation between PassRole vs AssumeRole
Try at [killercoda.com](https://killercoda.com/cloudsecurity-scenario/)
Power Electronics- AC -AC Converters.pptx
This PPT is about AC-AC Converters, including AC Voltage controllers and Cycloconverters, their types, and control.
Recently uploaded (20)
Levelised Cost of Hydrogen (LCOH) Calculator Manual
Levelised Cost of Hydrogen (LCOH) Calculator Manual
Zener Diode and its V-I Characteristics and Applications
Zener Diode and its V-I Characteristics and Applications
Prediction of Electrical Energy Efficiency Using Information on Consumer's Ac...
Prediction of Electrical Energy Efficiency Using Information on Consumer's Ac...
Tools & Techniques for Commissioning and Maintaining PV Systems W-Animations ...
Tools & Techniques for Commissioning and Maintaining PV Systems W-Animations ...
Generative AI Use cases applications solutions and implementation.pdf
Generative AI Use cases applications solutions and implementation.pdf
OOPS_Lab_Manual - programs using C++ programming language
OOPS_Lab_Manual - programs using C++ programming language
smart pill dispenser is designed to improve medication adherence and safety f...
smart pill dispenser is designed to improve medication adherence and safety f...
Applications of artificial Intelligence in Mechanical Engineering.pdf
Applications of artificial Intelligence in Mechanical Engineering.pdf
5G Radio Network Througput Problem Analysis HCIA.pdf
5G Radio Network Througput Problem Analysis HCIA.pdf
Resume boda liao-storage
- 1. BODA LIAO (C): 650-284-9752; E-mail: liaoboda@gmail.com; Linkedin: https://www.linkedin.com/in/boda-liao-00bb7a33 ; Address: 1921 California Street, Apt 10, Mountain View, 94040, CA Work Experience 02/05/2016 ~ 01/01/2017 BGI-shenzhen (part time) • Senior Linux administrator Maintain HPC and storage system for high-throughput sequencing centers of BGI @Children’s Hospital of Philadelphia(USA) and BGI@Denmark: 10 linux nfs storage servers and one HDS HUS100 san storage. Storage Solution consulting for China GeneBank: JBOD+zfs+lustre 4*90*10TB SATA driver in each JBOD, ZFS 12+3 zpool as the OST, Total 3.6PB capacity. 4*4 port 12Gb sas3 cable connect to 2 OSS servers. The lustre system throughput is upto 20GBps.(Each OSS have 4 mellanox 25Gb Ethernet ) The total cost is $0.3/GB. 09/08/2014 ~ 01/22/2016 Complete genomics (CGI) Mountain View, USA • Senior Linux administrator Linux administration Maintain the automatic sequencing IT system at CGI and transfer them to BGI-Shenzhen. Installation with DHCP+Pxe+Kickstart. Configuration with puppet. Monitoring with Ganglia and nagious system. Remote control with Ipmitool (remote screen redirect) and Megcli (command line make raid) Job schedule software Scrat (writen by CGI) and SGE to manage the jobs. Bash script for crontab and monitor. Storage work experience: Isilon, Huawei 9000, glusterfs. Storage POC experience: Mapr , Isilon , Huawei 9000, glusterfs, lustre, moosefs. 05/12/2007 ~ 09/07/2014 BGI-Shenzhen Shenzhen, China • Senior system engineer Configure, deploy, admin the platform with ~22,000 + CPU cores and ~20+PB storage. Support thousands of bioinformaticians analyzing ~10+TB data per day from hundreds of high-throughput genomic sequencers. Page ! of !1 3
- 2. Boda Liao (liaoboda@gmail.com) Storage system experience: Experience of 10 PB isilon, 1PB panasas, 2PB lustre, 1PB huawei 9000, 500TB Oracle 7410, Oracle SL 3000 Tap system+ NBU Storage system POC experience: DDN, GPFS, gluster, HP ibrix Data center infrastructure work experience: Build all of the six BGI Data-Centers around the world. In charge of the data centers of BGI@UC-Davis, BGI@Children’s Hospital of Philadelphia and BGI@Demark. 06/30/2006 ~ 05/11/2007 BGI (Beijing Genomics Institute) Beijing, China •Linux system engineer BLC (Bioinformatics Linux Cluster) deploy Hardware and software test Skills • Expert knowledge of large-scale Linux HPC Cluster solution. • Expert knowledge of large-scale Parallel Storage system: Isilon, Panasas, Lustre, Gluster, Mapr, HDS. I also have tested and have performance data of the gpfs, blue arch, Oracle zfs (7410), HP Ibirix systems. • Expert knowledge of data life cycle and archive.Experienced use Sun SL3000 tape system(2PB LTO4) and NBU, SAMFS+QFS, Storenex, Luster HSM and iRods to archive files. • Experienced IT solution for DNA Sequencers: Complete Genomics’ Private sequencers, Illumina Hiseq, Miseq ABI Solid, Thermo Fisher Ion Torrent, Roche 454. I can design different IT solutions for different bioinformatics requests. • Expert knowledge of sequencer and bioinformatics system automation. • Expert knowledge of software management: puppet. • Expert knowledge of storage skill: performance test, monitor and maintain. • Expert knowledge of Linux skill: kickstart, dhcp, nfs, samba, http, ssh, partition, acl, bash. • Expert knowledge of Rocks cluster: most of the clusters are based on Rocks and SGE job schedule system. • Extensive experience in building data centers including the power and UPS system, the cooling-water system and air condition, the rack and cable management. • Experience remote control system such as ipmitool. • Experience installing and compiling main different kernels, and making the rpm install package. • Experience installing and compiling different bioinformatics software and libraries including yum source build. • Expert knowledge of network solution with 10 Gb, 40Gb network and 56Gb Infiniband. • Experience configuration of switch such as VLAN. Page ! of !2 3
- 3. Boda Liao (liaoboda@gmail.com) • Knowledge of HADOOP and Mapr. • Knowledge of GPU system. We support another team to develop bioinformatics software on the GPU system. • Knowledge of Virtualization: vmware, openstack. Education 09/2002~06/2006 East China Jiaotong University (Nanchang, China) |Bachelor of engineering |Major: Computer science and technology Certification and Training 01/2014 Red Hat certified system administrator (Red Hat Enterprise Linux 6) 11/2011 Isilon senior engineer training (EMC Isilon, Seattle, USA) 09/2011 Project Management Professional training (Shenzhen, China) Page ! of !3 3