mogpres

154 views
143 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
154
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

mogpres

  1. 1. MogileFS Summit 2006-09-19
  2. 2. Format • Who’s Who? • Overview of MogileFS • What’s new in 2.0 • Wishlist / Future • Mounting Mogile • Plugins • ….. • (whatever)
  3. 3. Who’s Who? • ?
  4. 4. Overview: History • History – Storage company sales people intolerable • Milking little guys for all we’re worth • (asking us our yearly revenues before quote) – How hard can it be? • First version working after a weekend • Not hard. – Needed it for photo storage site
  5. 5. Overview: Parts • Clients • Trackers – mogilefsd – All queries / maintenance • Storage Nodes – mogstored (and/or lighttpd/apache) – Just HTTP/DAV server w/ disks. • Database – hopefully HA (more than 1, acts as 1)
  6. 6. New in 2.0 • all backwards compatible • code cleanup – OO, file per class, ... – well-defined protocol between all processes – Start of test suite • faster. no arbitrary sleeps. – Slowly adds sleep, otherwise stays busy
  7. 7. New in 2.0 • much more robust to error conditions during copying – detects difference between replicating failing reading from src vs dst. • watch dog for processes • configurable replication policies. removed from the mechanics. – Geographic, – Even/odd, – Whatever you want. • can MKCOL now, so works w/ any DAV server (apache and lighttpd tested)
  8. 8. New in 2.0 • Fsck job (partially done) – Double-check all replication decisions / locations – 3-4 levels of checking • Db location only • Stat (HEAD request) • Get contents, compare • Replication now uses new data structure (fids_to_replicate) – No longer queries files’s devcount – Means replication plugins can say: • “Put it here for now, but I’m not happy about it (not ideal)… revisit layout later.”
  9. 9. Future • Coming • Wishlist • ….?
  10. 10. Future • Mogstored --http=lighttpd, --http=apache, etc – Just do iostat and df interface (usage.txt files) – Smartd interface? • Mogstored improvements (aio isolation) • UUIDs, automounting in core (LVM integration?) • Move LJ’s out-of-band device reweighting using iostat into the core, so everybody gets it – Use it in more places – Add rebalance job, to move files around?
  11. 11. DB improvements • Partitioned DB – Necessary? So small. • Docs on MySQL HA • Or, mogilesd supporting master-master – Add DLM (ddlockd?) to mogilesd • Use for lock on namespace changess
  12. 12. Parallel meta files • 0/00/000/00000001.{fid,meta} • for fsck if lose db • Add checksumming info in there • Large file support (auto chunking?)
  13. 13. HTTP in mogilefsd • Add DAV server to mogilefsd – PUT / DELETE / GET (w/ redir or not) – Use any HTTP client / command line – Makes for easier mounting • Google “wdfs fuse”

×