We develop a consistent mutable replication extension for NFSv4 tuned to meet the rigorous demands of large-scale data sharing in global collaborations. The system uses a hierarchical replication control protocol that dynamically elects a primary server at various granularities. Experimental evaluation indicates a substantial performance advantage over a single server system. With the introduction of the hierarchical replication control, the overhead of replication is negligible even when applications mostly write and replication servers are widely distributed.

1. NFSv4 Replication for Grid Computing Peter Honeyman Center for Information Technology Integration University of Michigan, Ann Arbor

11. Replication control client When a client opens a file for writing, the selected server temporarily becomes the primary for that file Other replication servers are instructed to forward client requests for that file to the primary if concurrent writes occur SKIP SKIP SKIP wopen

12. Replication control client The primary server asynchronously distributes updates to other servers during file modification SKIP SKIP SKIP write

13. Replication control client When the file is closed and all replication servers are synchronized, the primary server notifies the other replication servers that it is no longer the primary server for the file SKIP SKIP SKIP close

22. Shallow & deep control /usr bin local /usr bin local A server with a shallow control on a file or directory is the primary server for that single object A server with a deep control on a directory is the primary server for everything in the subtree rooted at that directory

24. Ancestry table /root a b c f2 d2 controlled by S1 controlled by S0 controlled by S0 controlled by S2 …… Ancestry Table The data structure of entries in the ancestry table d1 f1 Ancestry Entry an ancestry entry has the following attributes id = unique identifier of the directory array of counters = set of counters recording which servers controls the directory’s descendants counter array S0 S1 S2 Id 2 0 0 c 0 0 1 b 2 1 0 a 2 1 1 root

28. Single remote NFS N.B.: log scale

29. Deep vs. shallow Shallow controls vs. deep + shallow controls

30. Deep control timer

34. Run-time model

40. PlanetLab failure CDF

41. Same-site correlated failures sites nodes 11 21 65 259 0.488 5 0.488 0.378 4 0.538 0.440 0.546 3 0.561 0.552 0.593 0.526 2

42. Different-site correlated failures

44. Simulated utilization F = one hour One backup server Four backup servers

45. Simulation results F = one day One backup server Four backup servers

46. Simulation results F = ten days One backup server Four backup servers

48. Checkpoint interval F = one day One backup server 20% checkpoint overhead F = ten days, 2% checkpoint overhead One backup server Four backup servers

52. Thank you for your attention! www.citi.umich.edu Questions?