This document describes a system called Disconnected Distributed YFS that allows clients to operate offline by caching file system data locally. The system supports two modes - normal mode where clients are connected, and disconnected mode where offline operations are simulated using cached data. Conflicts are detected on reconnection and resolved automatically for directories when possible, while a repair agent is used for manual resolution of file conflicts. The key components - YFS client, lock server, extent server, and repair agent - are implemented to support disconnected operation, caching, synchronization on reconnection, and conflict handling.
Virtualizing the Next Generation of Server Workloads with AMD™James Price
In this white paper, we discuss the potential bottlenecks that organizations typically encounter: high memory
utilization, high processor utilization, and high I/O traffic. We look at the performance characteristics of server
workloads that can be successfully virtualized, and we discuss how an awareness of the performance
characteristics of a particular workload can help inform an intelligent virtualization strategy. We also examine the
improvements in virtualization hardware that are making it possible to virtualize an increasingly wide range of
workloads.
The document discusses creating personas for accessible user experience (UX) design that incorporate people with disabilities. It provides examples of 8 personas with different abilities, aptitudes, attitudes, assistive technologies, and demographic information. The personas are meant to help designers consider a diverse range of users and their specific needs during the design process to create more inclusive and accessible experiences.
NUMA (Non-Uniform Memory Access) is a computer memory design that allows for multiprocessor systems where the memory access time depends on the location of the memory relative to the processor. With NUMA, accessing some regions of memory will take longer than others. The document discusses the background of NUMA, how it impacts operating system policies and programming approaches, and provides performance comparisons between UMA (Uniform Memory Access) and NUMA architectures.
Each month, join us as we highlight and discuss hot topics ranging from the future of higher education to wearable technology, best productivity hacks and secrets to hiring top talent. Upload your SlideShares, and share your expertise with the world!
Not sure what to share on SlideShare?
SlideShares that inform, inspire and educate attract the most views. Beyond that, ideas for what you can upload are limitless. We’ve selected a few popular examples to get your creative juices flowing.
How to Make Awesome SlideShares: Tips & TricksSlideShare
Turbocharge your online presence with SlideShare. We provide the best tips and tricks for succeeding on SlideShare. Get ideas for what to upload, tips for designing your deck and more.
SlideShare is a global platform for sharing presentations, infographics, videos and documents. It has over 18 million pieces of professional content uploaded by experts like Eric Schmidt and Guy Kawasaki. The document provides tips for setting up an account on SlideShare, uploading content, optimizing it for searchability, and sharing it on social media to build an audience and reputation as a subject matter expert.
- The Andrew File System (AFS) is a distributed file system that provides transparent access to shared files across a network like NFS. It aims to be scalable and efficient by caching entire files on client machines to reduce server load.
- AFS distinguishes between client machines that access files and dedicated server machines that store files. It achieves scalability through caching whole files on clients to reduce load on servers.
- AFS servers keep track of open files to inform clients of any updates through callbacks, providing location independence and transparency not present in NFS.
Virtualizing the Next Generation of Server Workloads with AMD™James Price
In this white paper, we discuss the potential bottlenecks that organizations typically encounter: high memory
utilization, high processor utilization, and high I/O traffic. We look at the performance characteristics of server
workloads that can be successfully virtualized, and we discuss how an awareness of the performance
characteristics of a particular workload can help inform an intelligent virtualization strategy. We also examine the
improvements in virtualization hardware that are making it possible to virtualize an increasingly wide range of
workloads.
The document discusses creating personas for accessible user experience (UX) design that incorporate people with disabilities. It provides examples of 8 personas with different abilities, aptitudes, attitudes, assistive technologies, and demographic information. The personas are meant to help designers consider a diverse range of users and their specific needs during the design process to create more inclusive and accessible experiences.
NUMA (Non-Uniform Memory Access) is a computer memory design that allows for multiprocessor systems where the memory access time depends on the location of the memory relative to the processor. With NUMA, accessing some regions of memory will take longer than others. The document discusses the background of NUMA, how it impacts operating system policies and programming approaches, and provides performance comparisons between UMA (Uniform Memory Access) and NUMA architectures.
Each month, join us as we highlight and discuss hot topics ranging from the future of higher education to wearable technology, best productivity hacks and secrets to hiring top talent. Upload your SlideShares, and share your expertise with the world!
Not sure what to share on SlideShare?
SlideShares that inform, inspire and educate attract the most views. Beyond that, ideas for what you can upload are limitless. We’ve selected a few popular examples to get your creative juices flowing.
How to Make Awesome SlideShares: Tips & TricksSlideShare
Turbocharge your online presence with SlideShare. We provide the best tips and tricks for succeeding on SlideShare. Get ideas for what to upload, tips for designing your deck and more.
SlideShare is a global platform for sharing presentations, infographics, videos and documents. It has over 18 million pieces of professional content uploaded by experts like Eric Schmidt and Guy Kawasaki. The document provides tips for setting up an account on SlideShare, uploading content, optimizing it for searchability, and sharing it on social media to build an audience and reputation as a subject matter expert.
- The Andrew File System (AFS) is a distributed file system that provides transparent access to shared files across a network like NFS. It aims to be scalable and efficient by caching entire files on client machines to reduce server load.
- AFS distinguishes between client machines that access files and dedicated server machines that store files. It achieves scalability through caching whole files on clients to reduce load on servers.
- AFS servers keep track of open files to inform clients of any updates through callbacks, providing location independence and transparency not present in NFS.
This document provides an overview of distributed file systems (DFS), including their structure, naming and transparency, remote file access, caching, and example systems. Some key points:
- A DFS manages dispersed storage devices across a network to provide shared files and storage. It provides location transparency so the physical location of files is hidden from users.
- Remote file access in DFS is enabled through caching, where frequently accessed data is cached locally to reduce network traffic. This raises cache consistency issues in keeping cached copies up-to-date.
- The Andrew distributed computing environment example presented a DFS with a shared name space spanning over 5,000 workstations using whole-file caching for remote access.
This document provides an overview of distributed file systems (DFS), including naming and transparency, remote file access, caching techniques, and example systems such as the Andrew file system. Some key points are:
- A DFS manages dispersed storage devices across a network to provide a shared file space for multiple users. It provides location transparency so file locations are hidden from clients.
- Caching is used to improve performance of remote file access by retaining frequently used data locally. Consistency must be maintained between cached and master copies.
- Systems can use stateful or stateless file service. Stateful requires maintaining client session data while stateless makes each request self-contained.
- The Andrew file system example illustrates
This document discusses distributed file systems (DFS), including their structure, naming and transparency, remote file access, caching, consistency issues, stateful vs stateless service, file replication, and examples like the Andrew distributed computing environment. Key points are that a DFS manages dispersed storage across a network, provides location and naming transparency, uses caching to improve performance of remote access, and must address consistency between cached and master copies of files.
Linux server virtualization involves masking server resources like physical servers, processors, and operating systems from users. This increases resource sharing and utilization while simplifying management. There are several approaches to virtualization including paravirtualization which modifies guest OS kernels, hardware virtualization which allows virtual machines to run on host hardware, and operating-system level virtualization using containers to segregate user environments while running under one kernel.
Introduction to High Availability for WordPress Developers (and not only!) v1.0Pressidium®
The document provides an introduction to high availability design for WordPress developers. It begins with an informal definition of high availability as the ability of a system to remain operational despite failures. It then discusses how high availability mitigates failures through component redundancy, eliminating single points of failure, and failure detection/response. The document provides examples of how WordPress behaves in highly available systems when installing plugins/themes or handling user sessions. It closes by discussing factors to consider in deciding if a business needs high availability, such as revenue loss from downtime versus costs of reducing downtime.
WebSphere Application Server is a platform on which Java-based business applications run. WebSphere Application Server Is an implementation of the Java 2 Enterprise Edition(J2ee) Specification.
The document discusses the architecture of Liferay Portal. It describes the key layers including the enterprise layer, extensions framework, logical architecture, service layer, persistence layer, and portal architecture. The service layer uses Service Builder to generate services and is divided into local and remote services. The persistence layer relies on Hibernate for database access. The portal architecture manages users, user groups, roles, organizations, sites, and teams to structure user access.
Network File System (NFS) is a distributed file system protocol that allows users to access and share files located on remote computers as if they were local. NFS runs on top of RPC and supports operations like file reads, writes, lookups and locking. It uses a stateless client-server model where clients make requests to NFS servers, which are responsible for file storage and operations. NFS provides mechanisms for file sharing, locking, caching and replication to enable reliable access and performance across a network.
In computing, a distributed file system (DFS) or network file system is any file system that allows access to files from multiple hosts sharing via a computer network. This makes it possible for multiple users on multiple machines to share files and storage resources.
This document summarizes new features added to FreeNAS 8.x since July 2012, including the Plugins Jail, ZFSv28, and the ability to encrypt ZFS pools using GELI. The Plugins Jail allows installing additional applications like Firefly within a jail from the FreeNAS GUI. ZFSv28 added RAIDZ3, replaceable ZIL, zpool split, autoexpand, ZLE, and deduplication. Encryption with GELI provides full disk encryption of ZFS pools with AES-256 and management of encryption keys.
A server is a computer that manages network resources and makes them available to authorized users. Kerberos is an authentication protocol that provides encryption during authentication. Active Directory installation can be verified by checking SRV records in DNS, verifying the SYSVOL folder, and database and log files, and using commands like Dcdiag and Net share.
What Is DFS In Windows Server 2016 2.pdfEIHEducation
Distributed File System (DFS) is a file system that allows sharing of files across multiple servers. It provides greater storage capacity and bandwidth than a standard file system. DFS can be configured in Windows Server 2016 by opening Server Manager, clicking Manage, then Features and Roles. This allows selecting the DFS role and configuring file sharing across multiple servers.
The document discusses clustering Subversion for performance, scalability, availability and reliability. Active/active clustering uses replication and load balancing to ensure all servers are continuously in sync over LANs or WANs. This allows planned and unplanned outages without disruption, provides 24/7 access globally, and enables distributed development teams to work together seamlessly across locations.
The document discusses distributed file systems. It defines a distributed file system as a classical model of a file system distributed across multiple machines to promote sharing of dispersed files. Key aspects discussed include:
- Files are accessed using the same operations (create, read, etc.) regardless of physical location.
- Systems aim to make file locations transparent to clients through techniques like replication and unique file identifiers.
- Caching is used to improve performance by retaining recently accessed data locally to reduce remote access.
- Consistency must be maintained when copies are updated.
FUSE and beyond: bridging filesystems paper by Emmanuel Dreyfuseurobsdcon
Abstract
Everything started with the desire to have a distributed filesystem better than NFS, which would be elastic and replicated with no SPOF. glusterFS was among the filesystems that could do that, but it uses The FUSE interface to the kernel, which was not available on NetBSD.
We will therefore discuss FUSE implementation and its NetBSD counterpart which is called PUFFS. A first but isufficient bridging attmpt called reFUSE was done, but porting real filesystems required the more complete PERFUSE bridge, which we will present here.
In a third part, we will look at the various unexpected roadblocks that had to be handled during PERFUSE implementation, in order to get a fully functionnal glusterFS on NetBSD.
Speaker bio
Emmanuel Dreyfus works as an IT manager at ESPCI ParisTech in France. He has been contributing for 13 years to NetBSD, in areas such as binary compatibility, and FUSE implementation. More at http://hcpnet.free.fr/pubz/
This document provides an overview of how XenServer manages storage supplied to virtual machines. It describes how Linux views physical disks and partitions, as well as logical volume management and network file systems. The key concepts covered include storage repositories, virtual disk images, storage attributes, and different storage types like local, NFS, Fibre Channel, and appliance-based storage. The target audience is IT specialists who need to understand XenServer's storage concepts and terminology for planning virtualized infrastructures.
The document provides an overview of Linux interview essentials related to operating system concepts, system calls, inter-process communication, and threads. It discusses topics such as the role and components of an operating system, multi-tasking and scheduling policies, differences between function calls and system calls, static and dynamic linking, common code and stack errors, memory leaks, kernel modes, monolithic and micro kernels, interrupts, exceptions, system calls implementation in Linux, and synchronous vs asynchronous communication methods.
Consolidating Applications With Solaris Containersvaishal
This document discusses how Solaris containers can be used to consolidate applications by providing isolation and allocating resources. It explains that Solaris containers use zones and resource pools to partition a single Solaris operating system instance. Zones isolate applications by providing separate namespaces, while resource pools allocate CPU resources. The document demonstrates how an email server can be isolated with its own dedicated resource pool, while two web servers share another pool and are further isolated using shares allocated by the fair share scheduler.
Low power architecture of logic gates using adiabatic techniquesnooriasukmaningtyas
The growing significance of portable systems to limit power consumption in ultra-large-scale-integration chips of very high density, has recently led to rapid and inventive progresses in low-power design. The most effective technique is adiabatic logic circuit design in energy-efficient hardware. This paper presents two adiabatic approaches for the design of low power circuits, modified positive feedback adiabatic logic (modified PFAL) and the other is direct current diode based positive feedback adiabatic logic (DC-DB PFAL). Logic gates are the preliminary components in any digital circuit design. By improving the performance of basic gates, one can improvise the whole system performance. In this paper proposed circuit design of the low power architecture of OR/NOR, AND/NAND, and XOR/XNOR gates are presented using the said approaches and their results are analyzed for powerdissipation, delay, power-delay-product and rise time and compared with the other adiabatic techniques along with the conventional complementary metal oxide semiconductor (CMOS) designs reported in the literature. It has been found that the designs with DC-DB PFAL technique outperform with the percentage improvement of 65% for NOR gate and 7% for NAND gate and 34% for XNOR gate over the modified PFAL techniques at 10 MHz respectively.
This document provides an overview of distributed file systems (DFS), including their structure, naming and transparency, remote file access, caching, and example systems. Some key points:
- A DFS manages dispersed storage devices across a network to provide shared files and storage. It provides location transparency so the physical location of files is hidden from users.
- Remote file access in DFS is enabled through caching, where frequently accessed data is cached locally to reduce network traffic. This raises cache consistency issues in keeping cached copies up-to-date.
- The Andrew distributed computing environment example presented a DFS with a shared name space spanning over 5,000 workstations using whole-file caching for remote access.
This document provides an overview of distributed file systems (DFS), including naming and transparency, remote file access, caching techniques, and example systems such as the Andrew file system. Some key points are:
- A DFS manages dispersed storage devices across a network to provide a shared file space for multiple users. It provides location transparency so file locations are hidden from clients.
- Caching is used to improve performance of remote file access by retaining frequently used data locally. Consistency must be maintained between cached and master copies.
- Systems can use stateful or stateless file service. Stateful requires maintaining client session data while stateless makes each request self-contained.
- The Andrew file system example illustrates
This document discusses distributed file systems (DFS), including their structure, naming and transparency, remote file access, caching, consistency issues, stateful vs stateless service, file replication, and examples like the Andrew distributed computing environment. Key points are that a DFS manages dispersed storage across a network, provides location and naming transparency, uses caching to improve performance of remote access, and must address consistency between cached and master copies of files.
Linux server virtualization involves masking server resources like physical servers, processors, and operating systems from users. This increases resource sharing and utilization while simplifying management. There are several approaches to virtualization including paravirtualization which modifies guest OS kernels, hardware virtualization which allows virtual machines to run on host hardware, and operating-system level virtualization using containers to segregate user environments while running under one kernel.
Introduction to High Availability for WordPress Developers (and not only!) v1.0Pressidium®
The document provides an introduction to high availability design for WordPress developers. It begins with an informal definition of high availability as the ability of a system to remain operational despite failures. It then discusses how high availability mitigates failures through component redundancy, eliminating single points of failure, and failure detection/response. The document provides examples of how WordPress behaves in highly available systems when installing plugins/themes or handling user sessions. It closes by discussing factors to consider in deciding if a business needs high availability, such as revenue loss from downtime versus costs of reducing downtime.
WebSphere Application Server is a platform on which Java-based business applications run. WebSphere Application Server Is an implementation of the Java 2 Enterprise Edition(J2ee) Specification.
The document discusses the architecture of Liferay Portal. It describes the key layers including the enterprise layer, extensions framework, logical architecture, service layer, persistence layer, and portal architecture. The service layer uses Service Builder to generate services and is divided into local and remote services. The persistence layer relies on Hibernate for database access. The portal architecture manages users, user groups, roles, organizations, sites, and teams to structure user access.
Network File System (NFS) is a distributed file system protocol that allows users to access and share files located on remote computers as if they were local. NFS runs on top of RPC and supports operations like file reads, writes, lookups and locking. It uses a stateless client-server model where clients make requests to NFS servers, which are responsible for file storage and operations. NFS provides mechanisms for file sharing, locking, caching and replication to enable reliable access and performance across a network.
In computing, a distributed file system (DFS) or network file system is any file system that allows access to files from multiple hosts sharing via a computer network. This makes it possible for multiple users on multiple machines to share files and storage resources.
This document summarizes new features added to FreeNAS 8.x since July 2012, including the Plugins Jail, ZFSv28, and the ability to encrypt ZFS pools using GELI. The Plugins Jail allows installing additional applications like Firefly within a jail from the FreeNAS GUI. ZFSv28 added RAIDZ3, replaceable ZIL, zpool split, autoexpand, ZLE, and deduplication. Encryption with GELI provides full disk encryption of ZFS pools with AES-256 and management of encryption keys.
A server is a computer that manages network resources and makes them available to authorized users. Kerberos is an authentication protocol that provides encryption during authentication. Active Directory installation can be verified by checking SRV records in DNS, verifying the SYSVOL folder, and database and log files, and using commands like Dcdiag and Net share.
What Is DFS In Windows Server 2016 2.pdfEIHEducation
Distributed File System (DFS) is a file system that allows sharing of files across multiple servers. It provides greater storage capacity and bandwidth than a standard file system. DFS can be configured in Windows Server 2016 by opening Server Manager, clicking Manage, then Features and Roles. This allows selecting the DFS role and configuring file sharing across multiple servers.
The document discusses clustering Subversion for performance, scalability, availability and reliability. Active/active clustering uses replication and load balancing to ensure all servers are continuously in sync over LANs or WANs. This allows planned and unplanned outages without disruption, provides 24/7 access globally, and enables distributed development teams to work together seamlessly across locations.
The document discusses distributed file systems. It defines a distributed file system as a classical model of a file system distributed across multiple machines to promote sharing of dispersed files. Key aspects discussed include:
- Files are accessed using the same operations (create, read, etc.) regardless of physical location.
- Systems aim to make file locations transparent to clients through techniques like replication and unique file identifiers.
- Caching is used to improve performance by retaining recently accessed data locally to reduce remote access.
- Consistency must be maintained when copies are updated.
FUSE and beyond: bridging filesystems paper by Emmanuel Dreyfuseurobsdcon
Abstract
Everything started with the desire to have a distributed filesystem better than NFS, which would be elastic and replicated with no SPOF. glusterFS was among the filesystems that could do that, but it uses The FUSE interface to the kernel, which was not available on NetBSD.
We will therefore discuss FUSE implementation and its NetBSD counterpart which is called PUFFS. A first but isufficient bridging attmpt called reFUSE was done, but porting real filesystems required the more complete PERFUSE bridge, which we will present here.
In a third part, we will look at the various unexpected roadblocks that had to be handled during PERFUSE implementation, in order to get a fully functionnal glusterFS on NetBSD.
Speaker bio
Emmanuel Dreyfus works as an IT manager at ESPCI ParisTech in France. He has been contributing for 13 years to NetBSD, in areas such as binary compatibility, and FUSE implementation. More at http://hcpnet.free.fr/pubz/
This document provides an overview of how XenServer manages storage supplied to virtual machines. It describes how Linux views physical disks and partitions, as well as logical volume management and network file systems. The key concepts covered include storage repositories, virtual disk images, storage attributes, and different storage types like local, NFS, Fibre Channel, and appliance-based storage. The target audience is IT specialists who need to understand XenServer's storage concepts and terminology for planning virtualized infrastructures.
The document provides an overview of Linux interview essentials related to operating system concepts, system calls, inter-process communication, and threads. It discusses topics such as the role and components of an operating system, multi-tasking and scheduling policies, differences between function calls and system calls, static and dynamic linking, common code and stack errors, memory leaks, kernel modes, monolithic and micro kernels, interrupts, exceptions, system calls implementation in Linux, and synchronous vs asynchronous communication methods.
Consolidating Applications With Solaris Containersvaishal
This document discusses how Solaris containers can be used to consolidate applications by providing isolation and allocating resources. It explains that Solaris containers use zones and resource pools to partition a single Solaris operating system instance. Zones isolate applications by providing separate namespaces, while resource pools allocate CPU resources. The document demonstrates how an email server can be isolated with its own dedicated resource pool, while two web servers share another pool and are further isolated using shares allocated by the fair share scheduler.
Similar to Disconnected Distributed File System (20)
Low power architecture of logic gates using adiabatic techniquesnooriasukmaningtyas
The growing significance of portable systems to limit power consumption in ultra-large-scale-integration chips of very high density, has recently led to rapid and inventive progresses in low-power design. The most effective technique is adiabatic logic circuit design in energy-efficient hardware. This paper presents two adiabatic approaches for the design of low power circuits, modified positive feedback adiabatic logic (modified PFAL) and the other is direct current diode based positive feedback adiabatic logic (DC-DB PFAL). Logic gates are the preliminary components in any digital circuit design. By improving the performance of basic gates, one can improvise the whole system performance. In this paper proposed circuit design of the low power architecture of OR/NOR, AND/NAND, and XOR/XNOR gates are presented using the said approaches and their results are analyzed for powerdissipation, delay, power-delay-product and rise time and compared with the other adiabatic techniques along with the conventional complementary metal oxide semiconductor (CMOS) designs reported in the literature. It has been found that the designs with DC-DB PFAL technique outperform with the percentage improvement of 65% for NOR gate and 7% for NAND gate and 34% for XNOR gate over the modified PFAL techniques at 10 MHz respectively.
CHINA’S GEO-ECONOMIC OUTREACH IN CENTRAL ASIAN COUNTRIES AND FUTURE PROSPECTjpsjournal1
The rivalry between prominent international actors for dominance over Central Asia's hydrocarbon
reserves and the ancient silk trade route, along with China's diplomatic endeavours in the area, has been
referred to as the "New Great Game." This research centres on the power struggle, considering
geopolitical, geostrategic, and geoeconomic variables. Topics including trade, political hegemony, oil
politics, and conventional and nontraditional security are all explored and explained by the researcher.
Using Mackinder's Heartland, Spykman Rimland, and Hegemonic Stability theories, examines China's role
in Central Asia. This study adheres to the empirical epistemological method and has taken care of
objectivity. This study analyze primary and secondary research documents critically to elaborate role of
china’s geo economic outreach in central Asian countries and its future prospect. China is thriving in trade,
pipeline politics, and winning states, according to this study, thanks to important instruments like the
Shanghai Cooperation Organisation and the Belt and Road Economic Initiative. According to this study,
China is seeing significant success in commerce, pipeline politics, and gaining influence on other
governments. This success may be attributed to the effective utilisation of key tools such as the Shanghai
Cooperation Organisation and the Belt and Road Economic Initiative.
We have compiled the most important slides from each speaker's presentation. This year’s compilation, available for free, captures the key insights and contributions shared during the DfMAy 2024 conference.
A SYSTEMATIC RISK ASSESSMENT APPROACH FOR SECURING THE SMART IRRIGATION SYSTEMSIJNSA Journal
The smart irrigation system represents an innovative approach to optimize water usage in agricultural and landscaping practices. The integration of cutting-edge technologies, including sensors, actuators, and data analysis, empowers this system to provide accurate monitoring and control of irrigation processes by leveraging real-time environmental conditions. The main objective of a smart irrigation system is to optimize water efficiency, minimize expenses, and foster the adoption of sustainable water management methods. This paper conducts a systematic risk assessment by exploring the key components/assets and their functionalities in the smart irrigation system. The crucial role of sensors in gathering data on soil moisture, weather patterns, and plant well-being is emphasized in this system. These sensors enable intelligent decision-making in irrigation scheduling and water distribution, leading to enhanced water efficiency and sustainable water management practices. Actuators enable automated control of irrigation devices, ensuring precise and targeted water delivery to plants. Additionally, the paper addresses the potential threat and vulnerabilities associated with smart irrigation systems. It discusses limitations of the system, such as power constraints and computational capabilities, and calculates the potential security risks. The paper suggests possible risk treatment methods for effective secure system operation. In conclusion, the paper emphasizes the significant benefits of implementing smart irrigation systems, including improved water conservation, increased crop yield, and reduced environmental impact. Additionally, based on the security analysis conducted, the paper recommends the implementation of countermeasures and security approaches to address vulnerabilities and ensure the integrity and reliability of the system. By incorporating these measures, smart irrigation technology can revolutionize water management practices in agriculture, promoting sustainability, resource efficiency, and safeguarding against potential security threats.
Presentation of IEEE Slovenia CIS (Computational Intelligence Society) Chapte...University of Maribor
Slides from talk presenting:
Aleš Zamuda: Presentation of IEEE Slovenia CIS (Computational Intelligence Society) Chapter and Networking.
Presentation at IcETRAN 2024 session:
"Inter-Society Networking Panel GRSS/MTT-S/CIS
Panel Session: Promoting Connection and Cooperation"
IEEE Slovenia GRSS
IEEE Serbia and Montenegro MTT-S
IEEE Slovenia CIS
11TH INTERNATIONAL CONFERENCE ON ELECTRICAL, ELECTRONIC AND COMPUTING ENGINEERING
3-6 June 2024, Niš, Serbia
Understanding Inductive Bias in Machine LearningSUTEJAS
This presentation explores the concept of inductive bias in machine learning. It explains how algorithms come with built-in assumptions and preferences that guide the learning process. You'll learn about the different types of inductive bias and how they can impact the performance and generalizability of machine learning models.
The presentation also covers the positive and negative aspects of inductive bias, along with strategies for mitigating potential drawbacks. We'll explore examples of how bias manifests in algorithms like neural networks and decision trees.
By understanding inductive bias, you can gain valuable insights into how machine learning models work and make informed decisions when building and deploying them.
1. ABSTRACT
Disconnected operation refers to the ability of a distributed
system client to operate despite server inaccessibility by
emulating services locally. This work aims at extending YFS and
making it highly available by supporting disconnected
operation at YFS clients. YFS clients work in two modes. One
is called normal mode where clients are connected to the
network and other is called disconnected mode where the clients
voluntarily or due to network failure are disconnected from the
network. We allow all YFS client's operations in disconnected
mode as long as files are found in cache. An operation fails if the
file being worked on does not exist in client's cache. Thus all
disconnected operations are transparent to a user unless a cache
miss occurs. Similarly, return to normal operation is also
transparent, unless a conflict is detected. We attempt to resolve
conflicts on directories with an exception of same file name
conflict. We don't resolve conflicts on the files, instead we
provide a repair agent, which user uses to resolve conflicts
manually.
DESIGN & IMPLEMENTATION
In the following sections we first describe the design and
implementation of our YFS client, Lock server, Extent server, and
Repair agent and then we conclude.
YFS Client
Writeback cache
We implement a writeback cache on YFS client. If client is
connected then a particular dirty cache line is flushed whenever
client releases the lock on the extent sitting in that cache line. If
client is disconnected and gets connected, sync thread flushes all the
dirty cache lines. Note that after flushing a cache line we don't
erase the line from the cache instead we register a callback update
request with the extent server.
Hoarding
We implement the concept of hoarding in disconnected YFS. A
user can indicate that a certain set of files is critical for working in
the disconnected mode by adding them to an input file for YFS
client. During startup YFS client launches a thread, which wakes
up every three minutes, reads the files to hoard from the input file
and caches them if they are not present in the cache. This feature
gives users control over what is necessary to be there when they get
disconnected.
Simulation in Disconnected mode
We want user to continue working in disconnected mode. Thus
on YFS client we modified lock client cache and extent client
cache to simulate the lock server and the extent server locally.
This gives user permission to modify files in disconnected mode
in a hope that they can reintegrate their changes when they get
reconnected. This includes granting of even those locks for
which the client was not the owner when he got disconnected. In
disconnected mode, we also allow user to create new files or
directories provided that the parent directory's content is present
in the cache.
Heartbeater and sync thread
YFS client assumes that extent server is always up and keeps
pinging the server to maintain its connected or disconnected
state. This thread also takes care of syncing YFS client with lock
server and the extent server. Upon a connection after
disconnection it is the responsibility of YFS client to learn the
current state of lock server and update the extent server with its
changes. As soon as YFS client sees that a reconnection is
established, it starts syncing its state with the lock server and the
extent server. During syncing we don't permit further operations
on YFS client for example if a new thread gets started on YFS
client and tries to read a file we put that thread into wait and
wake it up once the sync process gets over. This is important
because the new thread might get the lock from cache and while
the new thread is working on file, sync thread might mark that
particular lock invalid, which clearly breaks the semantics. As a
first step during sync, we first validate the lock cache and then
we flush extent cache, which might definitely lead to conflicts. If
any of the flush extent causes the conflict (which is detected by
extent server) then that particular extent immediately becomes
inaccessible and marked inconsistent on extent server and erased
from client's cache. Note that during flushing if extent server
accepts the extents then we don't erase the extent from client's
cache rather we register a callback to get the cache line updated
by server. Once syncing is over normal operation takes place.
Update client's cache
We implement the functionality to enable YFS client receive
update for a cache line from extent server. Whenever YFS client
fetches a missing extent or flushes dirty extent from its cache it
optionally registers a callback request with the extent server.
Currently we register a callback for all fetched/flushed extents.
One can limit such registration only for hoarded files. After
registration, extent server guarantees to send one update request
to the client on modification to the extent. Note that the server
sends an update request and not invalidation request.
Lock Server
We modified YFS lock server to grant locks to other clients in
Disconnected Distributed YFS (Yet another File System)
Nakul Manchanda, Uma Balasubramanian, and Arvind Kumar
{nm1157, ub263, ak2603} @cs.nyu.edu
2. case the owner of the lock has got disconnected. Revoker thread on
the lock server assumes that the client got disconnected if a revoke
RPC to that client times out. Our current implementation of lock
server also provides an interface to let the client know if he is the
current owner of the lock. This interface is used by an YFS client
during syncing to learn the current state of locks.
Extent Server
Update Callback Registration
Our extent server exposes an interface to YFS client to let client
register an update callback. Once registered, server sends an update
notification to the client on first modification to the extent after the
registration. This is one time registration i.e. after sending an update
request server forgets about that client. We implement one time
notification policy to avoid the burden of sending update
notification again and again to the disconnected clients. Also client
might not be interested in getting the notification for a particular
extent after a period of time so it makes sense to have one time
notification.
Conflict Detection
Conflict detection is done by the extent server. We augmented
the information about the extent with the version number. Whenever
YFS client fetches the extent, it also gets its current version number.
Whenever YFS client modifies and sends the extent to the extent
server, it increments the version number by one. The extent server
compares the version of received extent with that of the existing
content with the same extent id. If the extent server finds the
received version to be less than or equal to the version of existing
content, it first tries to solve the conflict automatically and in case of
failure it marks the content as inconsistent and returns conflict error
code to the client. YFS client deletes the extent from its cache if
extent server returns conflict error code. Consider an example when
two disconnected clients are working on the same file i.e. with the
same version say four. When these clients get connected they both
will send the file with version 5 so whose content comes first will
get accepted and others will conflict with the existing content.
Conflict Resolution
Upon conflict detection, resolution happens on the extent server.
We try to resolve conflicts only on directories. All conflicts on files
are reported to the user. On extent server we maintain a version
history for each directory. We need version history to know the
content of directory at a version immediately before the conflicting
one. We need this immediate version because if we see a newly
added file it is ambiguous if the file was added by the client who
sent the conflicting version or deleted by the client who sent the
latest existing version. To keep our conflict resolution algorithm
simple, we don't consider complicated scenarios where a
disconnected user and also a connected user deletes and creates the
same file multiple times. Thus on directories we successfully resolve
most of the conflicts except the same file name conflicts. All
conflicting copies, which cannot be resolved without user
intervention are moved to a separate covolume, which is used by
the repair agent.
Repair Agent
Manual Conflict resolution Tool
We implement a repair agent, which is used to resolve
conflicting extent(s). The agent talks to the extent server and
acquires a lock from the extent server itself. Once acquired it
gets all conflicting extents and prompts user to select the desired
version for each extent one by one. The agent releases the lock
after resolving the conflicts. We make agent acquire a lock to
make sure that only one user is resolving the conflicts otherwise
conflict resolution process might lead to further conflicts. Once
a particular conflict is resolved manually the extent immediately
becomes accessible to YFS clients.
CONCLUSION
We think that the disconnected distributed file system is a
powerful and useful concept. We enjoyed throughout this work
and learned various possible issues and scenarios, which might
arise in a disconnected distributed file system. Distributed
deadlocks are difficult to debug, keeping the cache of connected
clients in sync is highly important. We think that our conflict
resolution algorithm can be further improved to include
complicated delete and create scenarios. We have not tested it for
such complicated scenarios. Also as an optimization to reduce
space complexity of version history of a directory we can
truncate the very old contents of the version history of a
directory but then extent server needs to know the latest version
of different directories on different clients. Thus it was not clear
how much such optimization could pay so we didn't implement
that. In short the project was as interesting as our class and we
had fun in doing it.
REFERENCES
[1] Mahadev Satyanarayanan, James J. , Puneet Kumar, Maria
E. Okasaki, Ellen H. Siegal, and David C. Steere. “Coda: A
Highly Available File System for a Distributed Workstation
Environment.” Proceedings of the Second IEEE Workshop
on Workstation Operating Systems Sep. 1989, Pacific Grove,
CA.
[2] James J. Kistler and M. Satyanarayanan. “Disconnected
Operation in the Coda File System”. ACM Transactions on
Computer Systems Feb. 1992, Vol. 10, No. 1, pp. 325.