Gsummit apis-2012

1. GlusterFS APIs
2013-03-08
Bangalore

2. Libgfapi Basics
 Do some things manually that glusterfs[d] does in
main()
• Create a context
• Load a volfile into that context
• Set up logging, etc.
 Issue individual calls using glfs_xxx
• e.g. glfs_open, glfs_write

3. Libgfapi CodeExample
glfs_t *fs = NULL;
int ret = 0;
fs = glfs_new ("iops");
if (!fs) {
return 1;
}
ret = glfs_set_volfile_server (fs, "tcp", "localhost",
24007);
ret = glfs_set_logging (fs, "/dev/stderr", 7);
ret = glfs_init (fs);
fd = glfs_creat (fs, filename, O_RDWR, 0644);

4. Libgfapi inPython
vol = Volume("localhost",volid)
vol.set_logging("/dev/null",7)
vol.mount()
def test_create (vol, mypath, data):
fd = vol.creat(mypath,os.O_WRONLY|os.O_EXCL,0644)
if not fd:
print "creat error"
return False
rc = fd.write(data)
if rc != len(data):
print "wrote %d/%d bytes" % (rc, len(data))
return False
return True

5. Translator Basics
 Addfunctionalityfromsimplestorage“bricks” touser (at treeroot)
FUSE
DHT
AFR-1
Client-3Client-1
AFR-0
Client-2Client-0

6. Translator Environment
 All writtenin“PlainOldC” - thankyou, Dennis
 STACK_WINDtocall “down” (towardstorage) at thebeginningof a
request
 STACK_UNWINDtocall “up” (towarduser) at theendof arequest
 General filesystem-relatedfunctions
• get/set context oninodesandfds, dictionaries, …
 Utilityfunctions
• memoryallocation, stringhandling, logging, …
• moreabout thislater

7. HereBeDragons
 All callsareasynchronous
• ContinuationPassingStyle, node.js, etc.
 Youlosecontrol whenyoucall upor down
 Youregaincontrol viacallback
1267 STACK_WIND (frame, dht_unlink_cbk,
1268 cached_subvol, cached_subvol->fops->unlink,
1269 &local->loc);
1270
1271 return 0;
callback pointer
world might have changed

8. PrivateContext
 Accessedviaframe->local
 Availabletobothoriginal dispatchfunctionandcallback, soit'svery
handy
 Canbeanystructureyouwant (void*)
 Youareresponsiblefor allocatingit, but freedautomatically(if non-
null) alongwiththeframe
 Trickyobject-lifecyclerulesarearecurringtheme(e.g. sameissues
withdictionaries)

9. Request Types
 Mostlywhat you'dexpect fromPOSIXor LinuxVFS
• open, close, readv, writev, stat, …
• …but watchout e.g. for separatelookupfunction
 Most operateonfiledescriptorsor inodes
• sometimesboth, e.g. fstat vs. stat
 All haverequest-specificarguments
• paths, modes, flags, data(of course)
• hint: usedefault_xxxasatemplatefor eachxxxyouimplement

10. Dictionaries
 String-valuedkeyplusarbitraryvalue
 Usedfor translator options, andalsoasargumentsfor some
requests(e.g. getxattr)
 Lotsof functionstoset different kindsof values
• ints, strings(static/dynamic), binaryblobs
 Enumerationviadict_foreach
 Bothdictionaries(dict_t) anddataitems(data_t) haverefcounts

11. Persistent object context
 Inode(inode_t) andfiledescriptor (fd_t)
 Treat translator askey, arbitraryvalue
• settingvaluealsotriggersforget/releasecallbackwhentheobject itself is
destroyed
inode_ctx_put (inode, xlator, value)
inode_ctx_get (inode, xlator, &value)
inode_ctx_del (inode, xlator, &value)
fd_ctx_set (fd, xlator, value)
fd_ctx_get (fd, xlator, &value)
fd_ctx_del (fd, xlator, &value)

12. Translator CodeExample
int32_t
negative_lookup (call_frame_t *frame, xlator_t *this, loc_t *loc,
dict_t *xdata)
{
...
gf_log(this->name,GF_LOG_DEBUG,"%s/%s => MISS",
uuid_utoa(loc->gfid), loc->name);
gp = GF_CALLOC(1,sizeof(ghost_t),gf_negative_mt_ghost);
if (gp) {
uuid_copy(gp->gfid,loc->pargfid);
gp->name = gf_strdup(loc->name);
}
STACK_WIND_COOKIE (frame, negative_lookup_cbk, gp,
FIRST_CHILD(this),
FIRST_CHILD(this)->fops->lookup,
loc, xdata);
return 0;
}

13. Glupy
Normal xlator
Glupy “shim”
User's code
gluster.py
Normal xlator

14. GlupyCodeExample
def create_fop (self, frame, this, loc, flags, mode,
umask, fd, xdata):
pargfid = uuid2str(loc.contents.pargfid)
print "create FOP: %s:%s" % (pargfid,
loc.contents.name)
key = dl.get_id(frame)
self.requests[key] = (pargfid, loc.contents.name[:])
dl.wind_create(frame,POINTER(xlator_t)(),
loc,flags,mode,umask,fd,xdata)
return 0