Slideshare.net (beta)

 
Post to TwitterPost to Twitter
Post: 
Myspace Hi5 Friendster Xanga LiveJournal Facebook Blogger Tagged Typepad Freewebs BlackPlanet gigya icons

All comments

Add a comment on Slide 1

If you have a SlideShare account, login to comment; else you can comment as a guest


Showing 1-50 of 0 (more)

Compete File System

From dcancel, 6 months ago

File System Virtualization using Python.

175 views  |  0 comments  |  0 favorites  |  5 downloads  |  2 embeds (Stats)
 

Categories

Add Category
 
 

Groups / Events

 

 
Embed
options

More Info

This slideshow is Public
Total Views: 175
on Slideshare: 130
from embeds: 45

Slideshow transcript

Slide 1:CFS: The Compete File System File System Virtualization using Python Christopher Gillett Chief Software Architect Compete, Inc. Copyright © 2000-2005 Compete™ All Rights Reserved. 1

Slide 2:Background Compete, Inc. analyzes large amounts of data (gigabytes per day), and accrues terabytes of data every year supporting its predictive analysis business Technology platform is Unix clump/cluster/grid of 60+ machines • Job level parallelism managed using Portable Batch System • Storage is NFS with dedicated NFS servers • Platform has been evolving for a few years • Lots of storage devices with differing sizes • Software developed in-house in a variety of languages: • C++, C, Java, Python, etc. Problem: How to effectively manage lots of data on lots of different devices • Sub Issue: Sm