The document summarizes updates to CephFS in the Pacific release, including improvements to usability, performance, ecosystem integration, multi-site capabilities, and quality. Key updates include MultiFS now being stable, MDS autoscaling, cephfs-top for performance monitoring, scheduled snapshots, NFS gateway support, feature bits for compatibility checking, and improved testing coverage. Performance improvements include ephemeral pinning, capability management optimizations, and asynchronous operations. Multi-site replication between clusters is now possible with snapshot-based mirroring.

1. 1
Ceph Month: CephFS Update
2021.06.21
Patrick Donnelly

2. 2
What’s new in Pacific!
https://ceph.io/community/new-in-pacific-cephfs-updates/

3. 3
Usability
Performance
Ecosystem
Multi-site
Quality
FIVE THEMES

4. 4
● MultiFS is marked stable!
○ Automated file system creation: use ‘ceph fs volume create NAME’
○ MDS automatically deployed with cephadm/rook
● MDS autoscaler (start/stop MDS based on file system max_mds, standby count)
● cephfs-top (preview)
○ See client sessions and performance of the file system
● Continued improvements to cephfs-shell
● Scheduled snapshots via new snap_schedule mgr module.
● First class NFS gateway support
○ active/active configurations + HA (haproxy + cephadm || rook)
○ automatically deployed via the Ceph orchestrator (Rook and cephadm)
● MDS-side encrypted file support (kernel-side development on-going): some patches still TBB
CEPHFS USABILITY

5. 5
Usability
Performance
Ecosystem
Multi-site
Quality
FIVE THEMES

6. 6
● Feature bit support for turning on/off required file system features
○ Clients not supporting features will be rejected
● Multiple MDS FS scrub (online integrity check)
● Kernel client (and mount.ceph) support for msgr2[.1]
○ Kernel mount option -o ms_mode=crc|secure|prefer-crc|prefer-secure
● Support for recovering mounts from blocklisting
○ Kernel reconnects with -o recover_session=clean
○ ceph-fuse reconnects with --client_reconnect_stale=1; page cache should be disabled
● Improved test coverage (doubled test matrix, 2500 -> 5000)
CEPHFS ROBUSTNESS

7. 7
Usability
Performance
Ecosystem
Multi-site
Quality
FIVE THEMES

8. 8
● Ephemeral pinning (policy based subtree pinning)
○ Distributed pins automatically shard sub-directories (think: /home)
○ Random pins shard descendent directories probabilistically
● Improved capability/cache management by MDS for large clusters
○ Cap recall defaults improved based on larger production clusters (CERN)
○ Capability acquisition throttling for some client workloads
● Asynchronous unlink/create (partial MDS support since Octopus)
○ Miscellaneous fixes and added testing
○ Kernel v5.7 and downstream in RHEL 8 / CentOS 8 (Stream)
○ libcephfs/ceph-fuse support in-progress: https://github.com/ceph/ceph/pull/38964
CEPHFS PERFORMANCE

9. 9
Usability
Performance
Ecosystem
Multi-site
Quality
FIVE THEMES

10. 10
● Replication targets (remote clusters) configured on any directory
● New cephfs-mirror daemon to migrate data
○ Managed by Rook or cephadm
● Snapshot-based
○ When snapshot is created on source cluster, it is replicated to remote cluster
● Initial implementation in Pacific
○ Single daemon
○ Inefficient incremental updates
○ Improvements will be backported
CEPHFS: SNAPSHOT-BASED MIRRORING

11. 11
Usability
Performance
Ecosystem
Multi-site
Quality
FIVE THEMES

12. 12
CSI / OPENSTACK MANILA
● RWX/ROX -- CephFS via mgr/volumes
○ New PV snapshot stabilization
■ Limits in place for snapshots on subvolumes.
○ New authorization API support (Manila)
○ New ephemeral pinning for volumes

13. 13
Windows client: cephfs-dokan
● ceph-dokan.exe -c c:ceph.conf -l x
● Developer preview
● QA integration TBD

14. 14
What’s coming in Quincy!

15. 15
● Async metadata operations
○ Async rmdir/mkdir + link/rename (same dir)
○ Support in libcephfs TBM: https://github.com/ceph/ceph/pull/38964
● Ceph-fuse performance
○ Take advantage of recent libfuse changes
● libcephfs (ceph-fuse/nfs-ganesha)
○ Break up big client lock
● fscache support in kernel client
● recursive unlink RPC (.trash directory)
Performance

16. 16
● MDS Rolling Upgrades (Planned)
○ per-MDS compat to simplify upgrades: https://github.com/ceph/ceph/pull/40511
● libcephfssqlite
○ See libcephsqlite: https://ceph.io/community/new-in-pacific-sql-on-ceph/
● mds_memory_target
Usability

17. 17
● HA / active-active (likely backported to Pacific)
● Automatic periodic snapshot + sync to remote cluster
○ Arbitrary source tree, destination in remote cluster
● Discussing more sophisticated models
○ Bidirectional, loosely/eventually consistent sync
○ Simple conflict resolution behavior?
CephFS multi-site replication

18. 18
Questions?