Your SlideShare is downloading. ×
0
A journey of file system
on JOS
Name :wayling
Date: 2012/09/11
Who am I?
●JuluOSDev 星系001
● virtual file system (2011-12-20 )
● http://www.juluos.org/home/files
● 出沒於TOSSUG & H4
●waylin...
Introduction JOS
●MIT 6.828 Operating System Engineering
●JOS是一個有基本雛形的作業系統,擁有傳統unix-like
functiom(fork, exec…) ,但是實作方式是exo...
JOS File System
●Part1 : File system struct
● In-memory and On-disk struct
●Part 2: Client/Server file system access
● Rem...
JOS File System
●Part1 : File system struct
● In-memory and On-disk struct
●Part 2: Client/Server File System Access
● rem...
The File System
●File system是memory跟disk操作結合
而成的,意及我們在存取檔案時,memory
跟disk都需要有對應的struct來存放表示。
●static + dynamic組成file system...
JOS file system item(1/2)
●Sectors and Blocks
● Sector :Disk 實體磁區 (512 B) 。
● Block : file system 存取單位 (4KB) 。
●Superblock...
JOS file system item(2/2)
●File Meta-data
● 保存”檔案”的必要資訊,檔名、大小、 以及
block pointer(direct/indirect) 。
●Directories versus Reg...
JOS file system struct overview
File/Directory struct level
Block struct level
Disk struct (sector) level
User access
File/directory
data blocks
Free Block
Bitmap
Spuerblock
BootSector Block 0
Block 1
Block 2
Totaldisksize:NBlocks
Bitmapsiz...
struct File
(256 bytes)
Name : “foo”
Size : 54321
Direct block pointers:
(10)
Indirect block pointer :
Indirect block
(409...
JOS file system
●Disk access
● user-level file system - IDE disk driver 實做於user
space 。
● polling vs interrupt-driven 。
●T...
Format a file system (1/2)
#fsformat fs.img 1024 init newmotd motd
struct File
struct File
struct File
struct File
struct ...
struct File
Name : “/”
Size : 54321
Direct block pointers:
11
12
13
14
struct File
struct File
Name : “init”
Size : 54321
...
JOS file system service
●Part1 : File system struct
● In-memory and On-disk struct
●Part 2: Client/Server file system acce...
JOS’s file overview
●Server
● server.c
● Server 端主體,處理clinet請求。(JOS定義八種檔案操作請求)
● FSREQ_OPEN, 、FSREQ_SET_SIZE 、 FSREQ_READ
...
Client/Server file system access
●JOS使用IPC機制實作client/server架構的file
system service。
●File system Server (FSServ) :
● 作為user...
FSServ Block cache
3GB
0x10000000
0xD000000
FSServ address space
1
2
1.申請實體記憶體page。
2.讀取block大小到記憶體中。
(4096byte = 512 byte...
JOS IPC (Inter-process
communication ) (1/2)
●void ipc_send(envid_t to_env, uint32_t val,
void *pg, int perm)
→static int ...
JOS IPC (Inter-process
communication ) (2/2)
●int32_t ipc_recv(envid_t *from_env_store,
void *pg, int *perm_store)
→static...
Client (fs.c)
// Send an inter-environment request to the file server, and wait for
// a reply.
static int fsipc(unsigned ...
FSServ (serv.c)
●初始OpenFile table。 //用於管理FSServ所開啟的檔案。
●struct OpenFile opentab[MAXOPEN] = {
{ 0, 0, 1, 0 }
};
●struct Ope...
FSServ Client
fd
File
fd
FSREQ_OPEN
ipc_send
ipc_send
ipc_recv ipc_recv
1
1
2
3
3
0xD0000000
0xD0000000
0x0ffff000
1.Clien...
FSServ Client
fd
File
FSREQ_WRITE
ipc_send
ipc_sendipc_recv ipc_recv
1
1
2
3 3
0xD0000000
0x0ffff000
1.Client傳送一個IPC reque...
JOS file system RPC
●RPC base on IPC (ex : call file system server read
a file)
read( )
lib/fd.c
devfile_read ( )
lib/file...
Q&A
Reference
●[mit 6.828 lab 5]
http://pdos.csail.mit.edu/6.
828/2011/labs/lab5/
Upcoming SlideShare
Loading in...5
×

0911 juluosdev a_journey_of_filesystem_on_jos

49

Published on

Published in: Engineering, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
49
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "0911 juluosdev a_journey_of_filesystem_on_jos"

  1. 1. A journey of file system on JOS Name :wayling Date: 2012/09/11
  2. 2. Who am I? ●JuluOSDev 星系001 ● virtual file system (2011-12-20 ) ● http://www.juluos.org/home/files ● 出沒於TOSSUG & H4 ●waylingII@gmail.com wayling
  3. 3. Introduction JOS ●MIT 6.828 Operating System Engineering ●JOS是一個有基本雛形的作業系統,擁有傳統unix-like functiom(fork, exec…) ,但是實作方式是exokernel style(實作 的unix-like functiom大部份皆為user level的函式,不實作於 kernel level) 。 ●主要部份: ● Booting ● Memory management ● User-level environments ● Preemptive multitasking ● File system and spawn ● A shell Lectures: Frans Kaashoek Robert Morris
  4. 4. JOS File System ●Part1 : File system struct ● In-memory and On-disk struct ●Part 2: Client/Server file system access ● Remote procedure call (RPC) base on IPC + process
  5. 5. JOS File System ●Part1 : File system struct ● In-memory and On-disk struct ●Part 2: Client/Server File System Access ● remote procedure call (RPC) base on IPC + process
  6. 6. The File System ●File system是memory跟disk操作結合 而成的,意及我們在存取檔案時,memory 跟disk都需要有對應的struct來存放表示。 ●static + dynamic組成file system ● Static : 格式化硬碟成特有檔案系統格式 ex: ext2、ext3。 ● Dynamic :runtime時作業系統保存對應的資料 結構,藉此操作disk,提供檔案操作給使用者。
  7. 7. JOS file system item(1/2) ●Sectors and Blocks ● Sector :Disk 實體磁區 (512 B) 。 ● Block : file system 存取單位 (4KB) 。 ●Superblocks (disk & memory) ● 保存block size、 disk size 、 以及其他meta-data 用於尋找到root directory 。 ●Block Bitmap (disk & memory) ● 管理 free disk blocks 。
  8. 8. JOS file system item(2/2) ●File Meta-data ● 保存”檔案”的必要資訊,檔名、大小、 以及 block pointer(direct/indirect) 。 ●Directories versus Regular Files ● Directories主要保存多個file struct 。 ● Regular Files保存data block 。
  9. 9. JOS file system struct overview File/Directory struct level Block struct level Disk struct (sector) level User access
  10. 10. File/directory data blocks Free Block Bitmap Spuerblock BootSector Block 0 Block 1 Block 2 Totaldisksize:NBlocks Bitmapsize: EnoughblocktoholdNbits Block N -1 Block struct level
  11. 11. struct File (256 bytes) Name : “foo” Size : 54321 Direct block pointers: (10) Indirect block pointer : Indirect block (4096 bytes) (1024) Block 0 Block 1 Block 2 Block 3 Block 4 Block 5 Block 6 Block 9 Block 10 Block 11 Block 12 ….. File data blocks (4096 bytes each) File/Directory struct level
  12. 12. JOS file system ●Disk access ● user-level file system - IDE disk driver 實做於user space 。 ● polling vs interrupt-driven 。 ●The Block Cache ● 運用虛擬記憶體的特性,當程式存取檔案時轉換為對其 地址空間做存取 (0x10000000 - 0xD000000, 3GB) 。 ●File operations ● Open/Read/Write ,基本檔案操作。
  13. 13. Format a file system (1/2) #fsformat fs.img 1024 init newmotd motd struct File struct File struct File struct File struct File / init newmode mode init dir reg reg reg reg data block data block …… =>請參閱source code
  14. 14. struct File Name : “/” Size : 54321 Direct block pointers: 11 12 13 14 struct File struct File Name : “init” Size : 54321 Name : “newmode” Size : 54321 struct File Name : “init” Size : 54321 21 22 23 35 36 37 38 52 53 54 55 Format a file system (2/2) #fs.img (block view) 使用block bitmap紀錄block 分配狀況(不一定連續分配) 。
  15. 15. JOS file system service ●Part1 : File system struct ● In-memory and On-disk struct ●Part 2: Client/Server file system access ● remote procedure call (RPC) base on IPC + process
  16. 16. JOS’s file overview ●Server ● server.c ● Server 端主體,處理clinet請求。(JOS定義八種檔案操作請求) ● FSREQ_OPEN, 、FSREQ_SET_SIZE 、 FSREQ_READ ● FSREQ_WRITE 、 FSREQ_STAT 、 FSREQ_FLUSH ● FSREQ_REMOVE 、 FSREQ_SYNC ● fs.c ● File system主體,處理block、File操作。 ● bc.c ● Block cache,類似copy on write 的機制。 ●Client ● file.c ● Client端主要函式,對server發起IPC操作。 ● fd.c ● file description (fd)是server跟client間的橋樑。
  17. 17. Client/Server file system access ●JOS使用IPC機制實作client/server架構的file system service。 ●File system Server (FSServ) : ● 作為user space porcess,提供檔案操作服務, 藉由IPC 提供其他process所需之請求。 ●Client : ● 透過FSServ來存取檔案。 ●FSServ 特徵: ● Process id 1。 ● Block cache。 ● 回應client的IPC request。
  18. 18. FSServ Block cache 3GB 0x10000000 0xD000000 FSServ address space 1 2 1.申請實體記憶體page。 2.讀取block大小到記憶體中。 (4096byte = 512 byte *8)
  19. 19. JOS IPC (Inter-process communication ) (1/2) ●void ipc_send(envid_t to_env, uint32_t val, void *pg, int perm) →static int sys_ipc_try_send(envid_t envid, uint32_t value, void *srcva, unsigned perm) 對某個process傳送資料(32bit) ,也可 指定一個地址(小於UTOP) ,跟對方 process共享該page。 =>請參閱source code (ipc.c &
  20. 20. JOS IPC (Inter-process communication ) (2/2) ●int32_t ipc_recv(envid_t *from_env_store, void *pg, int *perm_store) →static int sys_ipc_recv(void *dstva) =>請參閱source code (ipc.c & Blocking自己等待對方把資料傳送完 畢。可指定地址(小於UTOP) ,決定要跟 傳送者共享page的位置。
  21. 21. Client (fs.c) // Send an inter-environment request to the file server, and wait for // a reply. static int fsipc(unsigned type, void *dstva) { if (debug) { ……. } ipc_send(envs[1].env_id, type, &fsipcbuf, PTE_P | PTE_W | PTE_U); return ipc_recv(NULL, dstva, NULL); } 封裝IPC成一個fsipc給client使用。 IPC訊息格式定義於 fs.h。 =>請參閱source code (file.c &
  22. 22. FSServ (serv.c) ●初始OpenFile table。 //用於管理FSServ所開啟的檔案。 ●struct OpenFile opentab[MAXOPEN] = { { 0, 0, 1, 0 } }; ●struct OpenFile { uint32_t o_fileid; struct File *o_file; int o_mode; struct Fd *o_fd; }; =>請參閱source code (serv.
  23. 23. FSServ Client fd File fd FSREQ_OPEN ipc_send ipc_send ipc_recv ipc_recv 1 1 2 3 3 0xD0000000 0xD0000000 0x0ffff000 1.Client傳送一個IPC request 讓FSServ接收, request中夾帶路徑,並申請fd待,FSServ回 應fd page,用以共享page。 2.FSServ依路徑開啟File struct。 3.回應Client,並把開啟的fd address map到 clinet的fd上。 共享page =>請參閱source code (serv.
  24. 24. FSServ Client fd File FSREQ_WRITE ipc_send ipc_sendipc_recv ipc_recv 1 1 2 3 3 0xD0000000 0x0ffff000 1.Client傳送一個IPC request讓FSServ接收, request中夾帶file id及將寫入的資料buffer。 2.將資料buffer寫入檔案的block中。 3.回應Client,將file pointer位址返回。 =>請參閱source code (serv. fd
  25. 25. JOS file system RPC ●RPC base on IPC (ex : call file system server read a file) read( ) lib/fd.c devfile_read ( ) lib/file.c fsipc ( ) lib/file.c ipc_send ( ) file_read ( ) fs/fs.c serve_read ( ) fs/serv.c serve ( ) fs/serv.c ipc_recv( ) RPC mechanism JOS IPC Regular env FSServ env
  26. 26. Q&A
  27. 27. Reference ●[mit 6.828 lab 5] http://pdos.csail.mit.edu/6. 828/2011/labs/lab5/
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×