The document discusses the on-disk structures used by AdvFS, including bitfiles, mcells, extent maps, tags, directory files, and fragments. It describes how AdvFS uses a two-level implementation with the File Access System (FAS) and Bitfile Access System (BAS). The BAS uses bitfiles, mcells, extent maps, and tag files to store and locate file data and metadata on disk.
It is the File system that is contained on the same partition on which the "Root directory" is located. It is the File system on which all the other file systems are mounted
It is the File system that is contained on the same partition on which the "Root directory" is located. It is the File system on which all the other file systems are mounted
OSDC 2011 | Enterprise Linux Server Filesystems by Remo RickliNETWAYS
Der Einsatz des richtigen Filesystems ist noch immer eine grundlegende Entscheidung mit großem Einfluss auf spätere Performance und Management. Mit Ext4 und BTRFS gibt es nun zwei neue Alternativen die der Erwartungshaltung bisherige Schwachstellen auszumerzen standhalten müssen. Während Ext4 bereits stable ist und den Sprung in die gängigen Enterprise Distributionen geschafft hat, steht das von Oracle BTRFS hier noch eher am Anfang. Trotz des scheinbaren Nachteils sehen vielen in BTRFS das nächste Standardfilesystem für Linux, da es im Vergleich bestehenden Limitierungen von Ext4 auflöst und als die Linux-Alternative zu ZFS gesehen wird.
Der Vortrag erläutert Architektur und Charakteristiken beider Dateisysteme, bewertet sie aus Sicht eines Systemadministrators für den Einsatz im Rechenzentrum und beschreibt mögliche Migrationspfade.
OSDC 2011 | Enterprise Linux Server Filesystems by Remo RickliNETWAYS
Der Einsatz des richtigen Filesystems ist noch immer eine grundlegende Entscheidung mit großem Einfluss auf spätere Performance und Management. Mit Ext4 und BTRFS gibt es nun zwei neue Alternativen die der Erwartungshaltung bisherige Schwachstellen auszumerzen standhalten müssen. Während Ext4 bereits stable ist und den Sprung in die gängigen Enterprise Distributionen geschafft hat, steht das von Oracle BTRFS hier noch eher am Anfang. Trotz des scheinbaren Nachteils sehen vielen in BTRFS das nächste Standardfilesystem für Linux, da es im Vergleich bestehenden Limitierungen von Ext4 auflöst und als die Linux-Alternative zu ZFS gesehen wird.
Der Vortrag erläutert Architektur und Charakteristiken beider Dateisysteme, bewertet sie aus Sicht eines Systemadministrators für den Einsatz im Rechenzentrum und beschreibt mögliche Migrationspfade.
SF Big Analytics & SF Machine Learning Meetup: Machine Learning at the Limit ...Chester Chen
Machine Learning at the Limit
John Canny, UC Berkeley
How fast can machine learning and graph algorithms be? In "roofline" design, every kernel is driven toward the limits imposed by CPU, memory, network etc. This can lead to dramatic improvements: BIDMach is a toolkit for machine learning that uses rooflined design and GPUs to achieve two- to three-orders of magnitude improvements over other toolkits on single machines. These speedups are larger than have been reported for *cluster* systems (e.g. Spark/MLLib, Powergraph) running on hundreds of nodes, and BIDMach with a GPU outperforms these systems for most common machine learning tasks. For algorithms (e.g. graph algorithms) which do require cluster computing, we have developed a rooflined network primitive called "Kylix". We can show that Kylix approaches the rooline limits for sparse Allreduce, and empirically holds the record for distributed Pagerank. Beyond rooflining, we believe there are great opportunities from deep algorithm/hardware codesign. Gibbs Sampling (GS) is a very general tool for inference, but is typically much slower than alternatives. SAME (State Augmentation for Marginal Estimation) is a variation of GS which was developed for marginal parameter estimation. We show that it has high parallelism, and a fast GPU implementation. Using SAME, we developed a GS implementation of Latent Dirichlet Allocation whose running time is 100x faster than other samplers, and within 3x of the fastest symbolic methods. We are extending this approach to general graphical models, an area where there is currently a void of (practically) fast tools. It seems at least plausible that a general-purpose solution based on these techniques can closely approach the performance of custom algorithms.
Bio
John Canny is a professor in computer science at UC Berkeley. He is an ACM dissertation award winner and a Packard Fellow. He is currently a Data Science Senior Fellow in Berkeley's new Institute for Data Science and holds a INRIA (France) International Chair. Since 2002, he has been developing and deploying large-scale behavioral modeling systems. He designed and protyped production systems for Overstock.com, Yahoo, Ebay, Quantcast and Microsoft. He currently works on several applications of data mining for human learning (MOOCs and early language learning), health and well-being, and applications in the sciences.
Developing Realtime Data Pipelines With Apache KafkaJoe Stein
Developing Realtime Data Pipelines With Apache Kafka. Apache Kafka is publish-subscribe messaging rethought as a distributed commit log. A single Kafka broker can handle hundreds of megabytes of reads and writes per second from thousands of clients. Kafka is designed to allow a single cluster to serve as the central data backbone for a large organization. It can be elastically and transparently expanded without downtime. Data streams are partitioned and spread over a cluster of machines to allow data streams larger than the capability of any single machine and to allow clusters of co-ordinated consumers. Messages are persisted on disk and replicated within the cluster to prevent data loss. Each broker can handle terabytes of messages without performance impact. Kafka has a modern cluster-centric design that offers strong durability and fault-tolerance guarantees.
In-depth forensic analysis of Windows registry filesMaxim Suhanov
Uncovering the details of how a registry file is organized, how to locate & recover deleted data, and why third-party offline registry editors & viewers are failing to do their job well.
Errata.
- Page 8: "Zero-based", should be: "Zero-based, unset bits not counted".
- Page 12: "multiple delete records (entities)", should be: "multiple deleted records (entities)".
ITC 360Professor John CovingtonSystem Administration And Managemen.docxchristiandean12115
ITC 360Professor John CovingtonSystem Administration And Management Homework # 1
1) (5 pts) Fill in the blanks…
• The smallest addressable unit of disk space is called
• The smallest allocatable unit of disk space is called
• A subset of a hard disk drive that can contain a separate file system is called
2) (5 pts) In the FAT file system, how many entries are there in the FAT itself (hint: how is it indexed) and what does each entry contain? Hint: I don’t want an actual size, I want what the size is based on.
For problems #3 – #8 , consider a 500 GB hard disk drive with 512 byte sectors and 4 KB (4096 byte) clusters. Consider an ASCII text document midterm.txt storing the answers to this exam. The file midterm.txt is
42500 characters. Recall: every ASCII character consumes 8 bits (i.e. 1 byte). Answer every question in short-answer form. Show calculations where necessary.
3) (5 pts) How many clusters will be allocated to midterm.txt by the Operating System?
4) (5 pts) In the last cluster it is given, how many bytes does midterm.txt use?
5) (5 pts) In the last cluster it is given, how many sectors does midterm.txt use?
6) (5 pts) What is the RAM slack of midterm.txt?
7) (5 pts) What is the file slack of midterm.txt?
8) (5 pts) What is the drive slack of midterm.txt?
9) (20 pts) The following tables contain a simplified layout for a FAT file system. Describe the hierarchical arrangement of files and directories and show their content. Do this in a clean, clear text-format way using tabbing and newlines in your answer.
Root (\) starts at Cluster: 1
FAT
Cluster
Status
Cluster
Status
Cluster
Status
1
EOF
7
8
13
EOF
2
17
8
EOF
14
EOF
3
EOF
9
EOF
15
EOF
4
EOF
10
14
16
EOF
5
EOF
11
EOF
17
EOF
6
7
12
EOF
(
Cluster
Conten
t
(
D
fo
r
directory
,
F
fo
r
regula
r
file)
1
D
:
prog
s
11
doc
s
12
user
s
13
README.tx
t
6
2
F
:
lette
r
t
o
th
e
department
3
F
:
lette
r
t
o
m
y
student
4
F
:
us
e
emacs
5
D
:
<empty>
6
F
:
t
o
install
,
downloa
d
IS
O
image
7
next
,
unpac
k
int
o
C:\temp
8
9
F
:
918%%%429!!!
10
F
:
123%%%986xxx
11
D
:
p.ex
e
9
q.ex
e
10
12
D
:
a.tx
t
16
b.tx
t
3
c.tx
t
2
13
D
:
Bo
b
5
Rya
n
15
14
pqr
***888,,,
15
D
:
profile.tx
t
4
16
F
:
t
o
d
o
list
17
th
e
end
)Clusters
For Questions #10 and #11, consider a disk whose blocks are sequentially labeled A, B, C, D, E, F, G, and H.
10) (10 pts) A RAID organization of this disk that looks like this…
is using a technique known as… (indicate the single best answer)
a) parity
b) striping
c) mirroring
d) fragmenting
e) clustering
11) (10 pts) A RAID organization of this disk that looks like this…
is u.
Covered:
1. Databases and Schemas
2. Tablespaces
3. Data Type
4. Exploring Databases
5. Locating the database server's message log
6. Locating the database's system identifier
7. Listing databases on this database server
8. How much disk space does a table use?
9. Which are my biggest tables?
10. How many rows are there in a table?
11. Quickly estimating the number of rows in a table
12. Understanding object dependencies
The Anatomy Of The Google Architecture Fina Lv1.1Hassy Veldstra
A comprehensive overview of Google's architecture - starting from the search page and all the way to its internal networks.
By Ed Austin, talk given at Edinburgh Techmeetup in December 2009
http://techmeetup.co.uk
Streaming AllWorx Call Detail Records To External DevicesJustin Goldberg
The Allworx server will listen on a TCP port on the LAN interface for connections. While a connection exists to the port, any new CDR records will be sent to the connecting client.
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
UiPath Test Automation using UiPath Test Suite series, part 6DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 6. In this session, we will cover Test Automation with generative AI and Open AI.
UiPath Test Automation with generative AI and Open AI webinar offers an in-depth exploration of leveraging cutting-edge technologies for test automation within the UiPath platform. Attendees will delve into the integration of generative AI, a test automation solution, with Open AI advanced natural language processing capabilities.
Throughout the session, participants will discover how this synergy empowers testers to automate repetitive tasks, enhance testing accuracy, and expedite the software testing life cycle. Topics covered include the seamless integration process, practical use cases, and the benefits of harnessing AI-driven automation for UiPath testing initiatives. By attending this webinar, testers, and automation professionals can gain valuable insights into harnessing the power of AI to optimize their test automation workflows within the UiPath ecosystem, ultimately driving efficiency and quality in software development processes.
What will you get from this session?
1. Insights into integrating generative AI.
2. Understanding how this integration enhances test automation within the UiPath platform
3. Practical demonstrations
4. Exploration of real-world use cases illustrating the benefits of AI-driven test automation for UiPath
Topics covered:
What is generative AI
Test Automation with generative AI and Open AI.
UiPath integration with generative AI
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex ProofsAlex Pruden
This paper presents Reef, a system for generating publicly verifiable succinct non-interactive zero-knowledge proofs that a committed document matches or does not match a regular expression. We describe applications such as proving the strength of passwords, the provenance of email despite redactions, the validity of oblivious DNS queries, and the existence of mutations in DNA. Reef supports the Perl Compatible Regular Expression syntax, including wildcards, alternation, ranges, capture groups, Kleene star, negations, and lookarounds. Reef introduces a new type of automata, Skipping Alternating Finite Automata (SAFA), that skips irrelevant parts of a document when producing proofs without undermining soundness, and instantiates SAFA with a lookup argument. Our experimental evaluation confirms that Reef can generate proofs for documents with 32M characters; the proofs are small and cheap to verify (under a second).
Paper: https://eprint.iacr.org/2023/1886
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
UiPath Test Automation using UiPath Test Suite series, part 5DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 5. In this session, we will cover CI/CD with devops.
Topics covered:
CI/CD with in UiPath
End-to-end overview of CI/CD pipeline with Azure devops
Speaker:
Lyndsey Byblow, Test Suite Sales Engineer @ UiPath, Inc.
Maruthi Prithivirajan, Head of ASEAN & IN Solution Architecture, Neo4j
Get an inside look at the latest Neo4j innovations that enable relationship-driven intelligence at scale. Learn more about the newest cloud integrations and product enhancements that make Neo4j an essential choice for developers building apps with interconnected data and generative AI.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...Neo4j
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
Sudheer Mechineni, Head of Application Frameworks, Standard Chartered Bank
Discover how Standard Chartered Bank harnessed the power of Neo4j to transform complex data access challenges into a dynamic, scalable graph database solution. This keynote will cover their journey from initial adoption to deploying a fully automated, enterprise-grade causal cluster, highlighting key strategies for modelling organisational changes and ensuring robust disaster recovery. Learn how these innovations have not only enhanced Standard Chartered Bank’s data infrastructure but also positioned them as pioneers in the banking sector’s adoption of graph technology.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
2. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Objectives
To explain AdvFS on-disk structures, you should be
able to describe:
• Bitfiles
• Mcells
• Extent maps
• Tags
• Directory files and directory index files
• Fragments
• Storage bitmap bitfiles
• Quota files
3. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Two-level implementation of AdvFS (1 of 2)
• File Access System (FAS)
– the higher level of AdvFS
– transforms bitfiles into normal UNIX files
– the client layer
• Bitfile Access System (BAS)
– the lowest level of AdvFS providing storage support
4. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Two-level implementation of AdvFS (2 of 2)
5. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
.tags directory
• Each AdvFS file system has a .tags directory
• Allows files to be accessed by tag, sequence number
– /Advfs_mount_point/.tags/15374
– /Advfs_mount_point/.tags/0x3c0e.8001
6. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Tag number can access any file
# ls -li
total 31
22894 -rwxr-xr-x 1 root system 31114 Jun 24 15:20
ob_1
#
# tail -3 ob_1
:of#169728:pf#35135:bf#8192:ff#1024:
:og#99458:pg#149368:bg#8192:fg#1024:
:oh#0:ph#0:bh#8192:fh#1024:
#
# tail -3 /usr/.tags/22894
:of#169728:pf#35135:bf#8192:ff#1024:
:og#99458:pg#149368:bg#8192:fg#1024:
:oh#0:ph#0:bh#8192:fh#1024:
#
7. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
BAS on-disk format: Everything is a bitfile
• Bitfiles are arrays of 8K disk pages holding user data or
metadata
• A series of contiguous 8K pages in a bitfile is stored as an
extent
• Each bitfile is identified by its tag
– Tag consists of a tag number.sequence number pair
• Tag number can be used to locate the extents of a file
Note that tag2name has a new format for V5.
8. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Tag number 22894 being
translated by tag2name
# /sbin/advfs/tag2name /usr/.tags/22894
/usr/bruden/ob_1
#
# echo $PATH
/sbin:/usr/sbin:/usr/bin:/usr/ccs/bin:/usr/bin/X11:/usr
/local
#
# PATH=$PATH:/sbin/advfs
#
# tag2name usr_domain -S usr 22894
open_vol: open for volume "/dev/disk/dsk2g" failed:
Device busy
#
# tag2name -r usr_domain -S usr 22894 <- Uses raw
device (-r)
bruden/ob_1
9. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Using AdvFS metadata
to translate FAS to BAS
10. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
BAS on-disk format
Logical File
On Disk
owner
group
size
mod bits
....
extent 1 extent 2
(Primary) mcell
292 bytes
Contains variable
sized records such as;
POSIX attributes
extent map records
Additional mcell(s)
optional
can contain more extent
map records if needed
8K Pages
11. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Bitfiles
• Array of 8K pages
• Stored as extents
– groups of on-disk contiguous 8K pages
– managed by extent maps
• Identified by a tag
– tag.sequence, for example 4714.8001
– tag number is similar to an inode number
– sequence number functions as a generation number
• All sectors are free or in a bitfile
• Managed by mcell chains
12. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Mcells
• AdvFS locates extents by finding the file's Primary Metadata
cell (Mcell) and stepping through the mcells to find the
extent information
– tag number is like an inode number, but an mcell functions like
an inode
– holds permissions, size, extent info, link count, and so forth
• Each mcell is 292 bytes
– can fit 28 on an 8K page (plus a 16-byte header)
• Where are the mcells located?
13. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Tag number to BMT mcell
to logical blocks.
Massaged tag #
Birfile Metadata Table (BMT)
.
.
.
lots of mcells
.
.
.
Primary mcell describing and
locating file (will be chained).
.
.
.
14. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
AdvFS file addresses
AdvFS files have a means of identification:
• Tag
– very similar to the UFS inode number
– can be discovered with the ls -i command
• Primary mcell ID
– component of the lower BAS layer
– start of a linked list of one or more mcells
– well hidden from users and system administration
15. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Bitfile-sets
• FAS fileset represents a BAS bitfile-set
• Identified by numbers
• Bitfiles are known by:
– domain ID
– fileset ID
– tag, sequence number
16. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Reusing tags
• Tag numbers can be reused
– with file creation and deletion
– like inode numbers
• Sequence number identifies various lives of the tag
– tags have initial sequence number of 8001 hexadecimal or
32769 decimal
– sequence number is incremented when tag number is reused
– when sequence number overflows, tag is discarded
– leftmost bit indicates ‘tag in use’, remaining 15 bits are used for
sequencing
• hexadecimal 8001 is binary 1000000000000001
18. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
BAS on-disk metadata bitfiles
ReservedBitfile
MetadataTable
Storage Bitmap
Root Tag File
On-Disk Log
MISC Bit File
ReservedBitfile
MetadataTable
Storage Bitmap
MISC Bit File
Tag File: filesetA
Fragment File:
filesetA
per volume
per domain
per fileset
BitfileMetadataTable BitfileMetadataTable
19. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Per domain bitfiles
• On-disk log
– contains the transaction log
– usually 4MB in size
• Root tag file
– lists filesets
– one page in size
20. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Displaying the root tag file
# nvtagpg -r usr_domain
==========================================================================
DOMAIN "usr_domain" VDI 1 (/dev/rdisk/dsk2g) lbn 96 root TAG page 0
--------------------------------------------------------------------------
currPage 0
numAllocTMaps 3 numDeadTMaps 0 nextFreePage 0 nextFreeMap 5
tMapA[1] tag 1 seqNo 1 primary mcell (vol,page,cell) 1 0 1 usr
tMapA[2] tag 2 seqNo 1 primary mcell (vol,page,cell) 1 0 13 var
tMapA[3] tag 3 seqNo 1 primary mcell (vol,page,cell) 2 2 4
ob_fset
#
21. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Per volume bitfiles
Each volume is supported by these bitfiles:
• Reserved Bitfile Metadata Table (RBMT)
– contains mcells for reserved bitfiles
– last mcell in each RBMT page will link to the next page
– eliminates BMT fragmentation problems
22. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Displaying RBMT summary information
(1 of 2)
# nvbmtpg -r -R usr_domain
=============================================================
=
DOMAIN "usr_domain" VDI 1 (/dev/rdisk/dsk2g) lbn 32 RBMT
page 0
-------------------------------------------------------------
-
There is 1 page in the RBMT on this volume.
There are 19 free mcells in the RBMT on this volume.
=============================================================
=
DOMAIN "usr_domain" VDI 2 (/dev/rdisk/dsk4b) lbn 32 RBMT
page 0
-------------------------------------------------------------
-
There is 1 page in the RBMT on this volume.
There are 19 free mcells in the RBMT on this volume.
23. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Displaying RBMT summary information
(2 of 2)
• Bitfile metadata table (BMT)
– contains all the mcells for nonreserved bitfiles
– grows as new files are created
• Storage bitmap (SBM)
– contains 1 bit for every 8K bytes
(1 bit per 1K in Tru64 UNIX V4.0)
• Misc bitfile
– contains bootblocks, and so forth
– four pages in size
24. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Displaying BMT for volume 1
of usr_domain
# nvbmtpg -r usr_domain 1
=================================================================
=
DOMAIN "usr_domain" VDI 1 (/dev/rdisk/dsk2g) lbn 48 BMT page
0
-----------------------------------------------------------------
-
There are 1025 pages in the BMT on this volume.
BMT uses 2 extents (out of 33) in 2 mcells.
25. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Per fileset bitfiles (1 of 2)
• Tag file (not .tags)
– translates tag # into location of primary mcell (within BMT) for
the appropriate file
– formerly called the ‘Tag Directory File’
• Tags can be reused as files are deleted
– limited to size of associated sequence number
– 8001 is a typical sequence number showing that this tag is in
use (left bit is set) and it is in use for the first time (001)
– limits tag reuse to ~4k times before tag is dead
26. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Per fileset bitfiles (2 of 2)
Tag file consists of 8K pages with 1022 tagmap
entries
• (8 bytes each) preceded by a 16-byte header
Tagmap entry contains:
• Sequence number
• Volume index
• Primary Mcell ID (BMT page # and cell # within page)
27. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Fileset tag directory locating primary mcell
29. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Finding primary mcell thru tag directory
using tag number
# ls -li bigfile
2324 -rwxr-xr-x 1 root system 15429536 Nov 24 14:54
bigfile
# df .
Filesystem 512-blocks Used Available Capacity
Mounted on
var_domain#var 524288 273858 238416 54% /var
# ntagpg -r var_domain var -t 2324
========================================================================
DOMAIN "var_domain" VDI 1 (/dev/rdisk/dsk1h) lbn 144 "var" TAG page 2
--------------------------------------------------------------------------
numAllocTMaps 350 numDeadTMaps 0 nextFreePage 0 nextFreeMap 337
tMapA[280] tag 2324 seqNo 49 primary mcell (vol,page,cell) 1 104 20
30. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Finding primary mcell thru tag directory
using file name
# ls -li bigfile
2324 -rwxr-xr-x 1 root system 15429536 Nov 24 14:54
bigfile
# pwd
/var/cluster/members/member0/tmp
# df .
Filesystem 512-blocks Used Available Capacity Mounted on
var_domain#var 524288 273858 238416 54% /var
# ntagpg -r var_domain var nvtagpg -r var_domain var
cluster/members/member0/tmp/bigfile
========================================================================
DOMAIN "var_domain" VDI 1 (/dev/rdisk/dsk1h) lbn 144 "var" TAG page 2
--------------------------------------------------------------------------
numAllocTMaps 350 numDeadTMaps 0 nextFreePage 0 nextFreeMap 337
tMapA[280] tag 2324 seqNo 49 primary mcell (vol,page,cell) 1 104
20
31. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Fragment bitfile
• Contains small files including the last parts of small files
• Size varies with number of small files
• Has tag number 1
32. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Reserved bitfile special names (1 of 3)
All accessible under .tags directory
Name Function One Per
M-6, M-12, ... Reserved Bitfile Metadata Table Volume
M-7, M-13, ... Storage Bitmap Volume
M-8, M-14, ... Root Tag File Domain
M-9, M-15, ... Transaction Log Domain
M-10, M-16, ...Bitfile Metadata Table Volume
M-11, M-17, ...Miscellaneous Bitfile Volume
33. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Reserved bitfile special names (2 of 3)
Name Function One Per
M1 Fileset tag file (not .tags) for fileset #1. Fileset
M2 Fileset tag file (not .tags) for fileset #2. Fileset
M3 Fileset tag file (not .tags) for fileset #3. Fileset
(...)
Mn Fileset tag file (not .tags) for fileset #n. Fileset
1 Fragment Bitfile Fileset
34. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Reserved bitfile special names (3 of 3)
Name Function One Per
2 Fileset's Root Directory Fileset
3 .tags Directory Fileset
4 User Quota File Fileset
5 Group Quota File Fileset
6 User File with Tag # 6 Fileset
7 User File with Tag # 7 Fileset
(...)
n User File with tag # n Fileset
35. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Metadata bitfile tags
Tags for reserved bitfiles of virtual disk i are:
• tag = - (magic-number-shown-below + (vol_index * 6))
Metadata Bitfile Tags Example
Reserved File Formula Disk Disk 2
RBMT - (0 + (vol * 6)) -6 -12
SBM - (1 + (vol * 6)) -7 -13
Root tag file - (2 + (vol * 6)) -8 -14
Log - (3 + (vol * 6)) -9 -15
BMT - (4 + (vol * 6)) -10 -16
Misc Bitfile - (5 + (vol * 6)) -11 -17
• May be printed in weird tags which effectively translate
to negative numbers
36. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
.tags for directory entries
for metadata bitfiles
Start with an M
• For virtual disk specific files, use the negative number
• For fileset tagfiles, use the fileset ID
For example, if /usr is an AdvFS fileset:
File Description
/usr/.tags/1 Fragment bitfile
/usr/.tags/M-6 RBMT of disk 1
/usr/.tags/-6 RBMT of disk 1 also
/usr/.tags/M-15 Log of disk 2
/usr/.tags/M2 tag file for 2nd fileset
37. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Bitfile metadata table (1 of 2)
• BMT is represented by a file found under.tags directory
– special file name is M-10 (more later)
– contains a series of 8K pages just like any other AdvFS file
– pages contain mcells (292 bytes each) and header information
• Each mcell contains one or more variable-length
records
– describe various file attributes, extents, permissions, frag info, etc
• BMT can grow just as any file can grow - just adds
another extent
– starts with slightly more than 1M (can be tailored)
• BMT is created when the mkfdmn command is issued
– one BMT for each volume in the domain
38. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Bitfile metadata table (2 of 2)
• Contains all mcells for all files other than the reserved
bitfiles
– user files
– user directories
• Mcells for the reserved bitfiles are in RBMT
• RBMT and BMT:
– first mcell describes itself
– as more mcells are needed, it grows using extents
– RBMT reserves the last mcell on each page to chain to other
pages of mcells
39. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Reserved bitfiles on disk layout
Miscellaneous Bitfile (M-11)
Pages 0, 1
Sectors 0-31
RBMT (M-6)
Page 0
Sectors 32-47
BMT (M-10)
Page 0
Sectors 48-63
Miscellaneous Bitfile (continued)
Pages 2,3
Sector 64-95
Root Tag Directory (M-8)
Page 1
Sectors 96-111
Storage Bitmap (M-7)
(1 bit per 8k cluster)
Sectors 112-?
Transaction Log (M-9)
512 Pages
Sectors ?
Fileset Tag Directory File (M1)
8 Pages
Sectors ?
Mount Point Directory for Fileset (2)
1 Page
Sector ?
.tags Directory (3)
1 Page
Sector ?
Quota.user (4)
1 Page
Sector ?
Quota.Group (5)
1 Page
Sector ?
40. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Mcell records (1 of 2)
• The inodes of AdvFS
• 28 Fixed-size (292 byte) mcells are packed into 8K
pages
• One or more linked mcells describe bitfiles
– First mcell in list is primary mcell
• Each mcell contains variably sized records describing
attributes of the bitfile
• Contained in the Bitfile Metadata Table (BMT) and the
RBMT
41. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Mcell records (2 of 2)
Record Types include:
• Extent Maps (of various kinds)
• Bitfile Attributes (clone, original, etc.)
• Domain Attributes
• Virtual Disk Attributes (disk ID, disk index, etc.)
• Fragment Attributes - more later
• POSIX file stats (permissions, size, link count, etc.)
• Symbolic link targets
42. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Mcell page structure
mcell header
record
record
record
mcell header
record
record
record
mcell header
record
record
record
page header
mcell
mcell
mcell
Page
28 mcells
per page
variable sized
records
43. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
RBMT page 0
RBMT Page 0:
• Starts at sector 32 (LBN 32)
• Contains these primary mcells:
– mcell 0 Reserved Bitfile Metadata Table (RBMT)
– mcell 1 Storage Bitmap (SBM)
– mcell 2 Root Tag File - Optional, one per domain
– mcell 3 Log - Optional, one per domain
– mcell 4 Bitfile Metadata Table (BMT)
– mcell 5 Misc bitfile
44. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
BMT page 0
• Starts at Sector 48
• mcell 0 is head of the BMT page free list
• Contains mcells for nonreserved bitfiles, such as user
files and directories
• All other BMT pages are found via the RBMT
45. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
BMT page format
Each 8192-byte page is composed of:
• 16-byte header followed by
• 28 292-byte mcells
BMT header consists of:
• Pointer to next free mcell on page
• Pointer to next page with free mcells
• Number of free mcells on the page
• Page number (within BMT)
• AdvFS version (now 4)
46. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Mcell addresses
• Mcells are addressed by a 32-bit mcell ID, bfMCIdT
– 27 bits give the mcell’s BMT page number
– 5 bits give the mcell’s position within its page
• Every bitfile has a primary mcell
• Tagfiles map (tag file)
– tag numbers can be mapped to primary mcell locations
47. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Reserved mcell addresses
Within RBMT page 0, the slot numbers listed are used
to calculate the tag numbers for the reserve bitfiles:
• 0: the RBMT itself
• 1: SBM, storage bitmap
• 2: root tag file
• 3: transaction log
• 4: BMT
• 5: misc bitfile
48. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Mcell format
• Begins with 24-byte header
– 32-bit ID of the next mcell in the chain and virtual disk
containing next mcell
– position of this mcell within the chain
– tag number of the bitfile
– tag number of the fileset
• Has 268 bytes remaining for mcell records
49. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Mcell records
• Vary in size and type
• Begin with a 4-byte header
– 2-byte count of record size
– 1-byte type field (There are currently about 20 record types)
– 1-byte version number for different versions of a type
• Have a null record whose size is 4 and type is 0
– this is the end-of-records indicator
50. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Utilities for viewing records
•nvbmtpg
•nvfragpg
•nvlogpg
•nvtagpg
•vsbmpg
•vfilepg
•savemeta
51. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Extent maps for nonreserved files
For nonreserved files:
• Primary extent map record
– within the primary mcell
– allocated when file gets a page
– if full, points to an extra extent map
• Extra extent map record
– allocated as the file grows
For striped files, extent map records use a shadow
extent map and may point to more than one disk.
52. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Extent maps for reserved files
For reserved bitfiles, for example, the RBMT
• Primary extent map
– if full, points to extra extent map
• Extra extent map
– usually only needed for the BMT itself
All records for reserved files are in the RBMT
53. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Encoding of extents
• Extents information is captured in two fields
– page number within bitfile
– block number within virtual disks
• Size of one extent is inferred from the next
– compute difference between the page numbers
– place -1 in block number to indicate a hole (-2 for clone hole)
• Try the following:
– page = 0, block = 8000
– page = 100, block = -1
– page = 200, block = 9600
– page = 300, block = -1
54. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Tag file characteristics
• Bitfiles are identified by tags
• Tag files are:
– arrays of tagmap entries
– indexed by tag number
• Tag File entries contain:
– sequence number (high bit set if in use)
– volume index
– mcell ID within volume
• Justifying Tag Files
– tag files introduce an extra level of overhead in accessing file
data
55. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
File access through tag file
File System Directory Fileset Tag File (M1)
(…)
file1 tag 623
file2 tag 51
file3 tag 893
(…)
. . .
tag 893 –
Sequence # 3,
Volume # 1,
BMT page 811,
mcell 7
. . .
BMT
. . .
page 811, mcell 7
Extent 80334, 50 pages
. . .
File on disk
LBN 80334
56. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Tag file allowing transparent data move
File System Directory Fileset Tag File (M1)
(…)
file1 tag 623
file2 tag 51
file3 tag 893
(…)
. . .
tag 893 –
Sequence # 3,
Volume # 1,
BMT page 811,
mcell 7
. . .
BMT
. . .
page 811, mcell 7
Extent 88526, 50 pages
. . .
File on disk
LBN 88526
57. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Tag file page
• Starts with a five-field, 16-byte header
– number of this page (for sanity checking)
– next page with free tagmap entries
– next free tagmap within this page
– number of of allocated tagmaps
– number of dead tagmap entries
• Followed by 1022 tagmap entries
58. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Tagmap entries
• Head of free tagmap list
– stored in first slot of page 0
– points to:
• first page with a free tagmap entries
• first uninitialized page
• Element of free tagmap list
– sequence number
– pointer to next free entry on this page
• Allocated tagmap entry
– sequence number
– virtual disk of bitfile
– mcell ID of bitfile
59. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Root tag file
• Entries for each fileset of the domain
• Contains mcell IDs of fileset tagfiles
• Can be used to find the list of domain filesets
60. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Fileset tag file
• Each fileset has its own tag file
• Maps fileset’s bitfiles to primary mcells
• Has these special fileset tags
– 1 Fragment bitfile
– 2 Root directory
– 3 .tags
– 4 User quota file
– 5 Group quota file
61. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Cloning through fileset tag
• Clone fileset is a read-only virtual copy of the data as it
existed at the time the clone was created
– should be used as source for a backup operation and then
removed
– creates a copy of the Fileset Tag File (M1)
– still points to the same BMT
• As original data changes while clone exists:
– copy of the original data must be created
– new mcell must be allocated in the BMT
– appropriate entry in new copy of Fileset Tag file must be
updated to point to cloned page's new mcell in BMT
62. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Fileset tag file before and after cloning
No Clone After clonefset command
Fileset Tag File (M1)
. . . . . tag 85, mcell 14 . . . . . . .
BMT
mcell 14, LBN 919
original data
(LBN 919)
Fileset Tag File (M2)
(clone)
. . . . . tag 85, mcell 14 . . . . . . .
Fileset Tag File (M1)
. . . . . tag 85, mcell 14 . . . . . . .
63. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Fileset tag file before and after cloning
• No data is copied unless a change is made to original
data
• A copy is only made of data that is about to be changed
• Clone is effectively a 'snapshot' of the data at a known
time
64. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Clone structures after data write
No Clone After Data has been Changed
Fileset Tag File (M1)
. . . . . tag 85, mcell 14 . . . . . . .
BMT
mcell 14, LBN 919
mcell 22, LBN 1214
original data
(LBN 919)
including
changes
Fileset Tag File (M2)
(clone)
. . . . . tag 85, mcell 22 . . . . . . .
Fileset Tag File (M1)
. . . . . tag 85, mcell 14 . . . . . . .
original data
(LBN 1214)
copied before
changes were
made in LBN
919
65. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Utility for viewing tag files
The nvtagpg utility:
• Prints formatted pages of a root tag file or a fileset tag
file
66. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Displaying root tag file information
# nvtagpg -r usr_domain
=================================================================
DOMAIN "usr_domain" VDI 1 (/dev/rdisk/dsk2g) lbn 96 root TAG page 0
-----------------------------------------------------------------
currPage 0
numAllocTMaps 3 numDeadTMaps 0 nextFreePage 0 nextFreeMap 5
tMapA[1] tag 1 seqNo 1 primary mcell (vol,page,cell) 1 0 1
usr
tMapA[2] tag 2 seqNo 1 primary mcell (vol,page,cell) 1 0 13
var
tMapA[3] tag 3 seqNo 1 primary mcell (vol,page,cell) 2 2 4
ob_fset
67. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Displaying fileset tag file
Given a particular tag number, divide by 1022:
• Quotient is the page
• Remainder is tagmap slot
# nvtagpg -r usr_domain usr
================================================================
DOMAIN "usr_domain" VDI 1 (/dev/rdisk/dsk2g) lbn 1047552 "usr" FRAG page 0
----------------------------------------------------------------
currPage 0
numAllocTMaps 1021 numDeadTMaps 0 nextFreePage 23 nextFreeMap 0
tMapA[1] tag 1 seqNo 1 primary mcell (vol,page,cell) 1 0 3
tMapA[2] tag 2 seqNo 1 primary mcell (vol,page,cell) 1 0 4
tMapA[3] tag 3 seqNo 1 primary mcell (vol,page,cell) 1 0 6
tMapA[4] tag 4 seqNo 1 primary mcell (vol,page,cell) 1 0 7
tMapA[5] tag 5 seqNo 1 primary mcell (vol,page,cell) 1 0 8
(...)
69. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
UNIX directories
• UNIX directories are contained in standard bitfiles
– Directories are a series of directory entries
• POSIX directory entry format
– inode number: 32 bits
– entry length: 16 bits
– name length: 16 bits
– name: variable-length, zero-padded to fill the entry
• AdvFS directory information hidden in POSIX entry
– tag numbers replace inode numbers
– the 64-bit tag.sequence ID is placed in the name “padding”
70. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
AdvFS directory index files
File name lookup through large directory files is very slow
• On average, one-half of the directory entries must be searched
AdvFS uses a B-tree to index files to speed the search
• Each page of the index file has 510 entries
• Filenames are “hashed” using ELF
– b-tree maps hash keys to directory entry locations
• Mcell chain of large directory
– contains tag.sequence number of the index file
• Mcell chain of the index file
– identifies the height and root of the B-tree
71. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Displaying the location of the index file
# showfile -i LotsOfFiles
Id Vol PgSz Pages XtntType Segs SegSz I/O
Perf File
713.8001 1 16 4 simple ** ** ftx
33% index ()
# ls -ld LotsOfFiles
drwxr-xr-x 2 root system 32768 Nov 22 18:20
LotsOfFiles
72. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
AdvFS directory index b-tree
Page header
Page header
991007
950305
ELF hash function maps file name FileName to 950305
FileName
Root node of B-tree
Leaf node of B-tree
Directory file
73. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Relationship to POSIX files
Directory File
Fileset Tag File
BMT
File Data
74. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
AdvFS tagfiles and migration
The ability to move:
• A bitfile's data transparently - this is supported by the
buffer cache and I/O scheduling algorithms
• A bitfile's metadata to another volume - tag files enable
this
75. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Fragment bitfile
• One per fileset
• Contains small (< 8K) ends of files
• Bitfiles are allocated in 8K (16 sector) units
• If file is less than 100K, a fragment is used if needed.
• Fragment bitfile is broken into a series of 128KB
fragment groups
76. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Fragment groups
• Each group consists of fragments of a particular size
– free (0K), 1K, 2K,...7K
• Each fragment group has the form
– one page fragment header
– fragments sized for the group
• Fragment addresses are usually
– in 1K pages
– relative to start of fragment bitfile
• Fragments are not Extents!!
– extent is a contiguous range of 8K pages
– fragment is a chunk of disk space between 1K and 7K in size
78. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Fragment header
• Pointer to next fragment group of this type with free
space
• Page number of this page (a sanity check)
• Type of this group
• Number of free fragments
• Fileset ID
• Version
• List of free fragments in the group
83. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Fragments and files
• POSIX stats record of mcells contains
– fragId field
• fragId.frag is the page offset of fragment
• fragId.type is the size of fragment
• Use showfile to determine if there is a fragment
– does number of pages match file size?
• Use nvbmtpg to find fragment location
84. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Tracking down a fragment (1 of 2)
# ls -i /etc/disktab
1931 disktab
# nvbmtpg -r root_domain root -t 1931 -c
======================================================================
DOMAIN "root_domain" VDI 1 (/dev/rdisk/dsk1a) lbn 222464 BMT page 82
----------------------------------------------------------------------
pageId 82 megaVersion 4
freeMcellCnt 0 nextFreePg -1 nextfreeMCId page,cell 0,0
CELL 25 next mcell volume page cell 0 0 0 bfSetTag,tag 1,1931
………………………………
RECORD 2 bCnt 92 BMTR_FS_STAT
st_mode 100755 (S_IFREG) st_uid 3 st_gid 4 st_size 30910
st_nlink 1 dir_tag 13 st_mtime Wed May 29 07:06:42 2002
fragId.type BF_FRAG_7K fragId.frag 2396
85. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Tracking down a fragment (2 of 2)
# dd if=/.tags/1 of=/tmp/dtfrag bs=1024
iseek=2396 count=7
7+0 records in
7+0 records out
# head /tmp/dtfrag
#8192:fc#1024:
:od#393216:pd#324698:bd#8192:fd#1024:
:oe#717914:pe#324698:be#8192:fe#1024:
:of#1042612:pf#324698:bf#8192:ff#1024:
:og#393216:pg#819200:bg#8192:fg#1024:
86. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Storage bitmap bitfile characteristics
• Storage bitmap file represents each 8K of on-disk
storage within a volume with 1 bit (1K per bit in Tru64
UNIX V4.0)
• Basically a bunch of bits representing whether storage
is in use or not
• SBM file is .tags/M-7.
• One per volume
• All storage is either free or in a bitfile
• AdvFS storage is allocated in pages
• On-disk SBM
– little more than an array of bits (1 bit per page)
87. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
SBM format
• SBM page header
– sequence number, now unused (32 bits)
– XOR field for rest of page (32 bits)
• Bitmap
– 65472 bits
– enough for 8184 pages (8k each)
89. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
SBM display through .tags/M-7 (2 of 2)
# showfile -x /facusers/.tags/M-7
Id Vol PgSz Pages XtntType Segs SegSz I/O Perf File
fffffff9.0000 1 16 9 simple ** ** ftx 100% M-7
extentMap: 1
pageOff pageCnt vol volBlock blockCnt
0 4 1 112 64
extentCnt: 1
#
# ls -l /facusers/.tags/M-7
---------- 0 root system 73728 Dec 31 1969 /facusers/.tags/M-7
90. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
SBM page allocation status display
using vsbmpg
# vsbmpg -r facusers_dmn 1 -B 40000
==========================================================================
DOMAIN "facusers_dmn" VDI 1 (/dev/rdisk/dsk6c) lbn 112 SBM page 0
--------------------------------------------------------------------------
blocks 40000 - 40015 (0x9c40 - 0x9c4f) are mapped by SBM map entry 78, bit
4
mapInt[78] 00000000 00000000 00000000 00000000
block 40000
91. Copyright (C) 2008 Hewlett-Packard Development Company, L.P.
Miscellaneous bitfile
Characteristics of the misc bitfile includes:
• One per volume
• Holds pages for
– primary and Secondary Boot Block
– partition Table (Disk Label)
– fake UFS Super Block with AdvFS Magic Number