The slides for Jordan Hubbard's presentation, “FreeNAS 10: Challenges of Building a Modern Storage Appliance based on FreeBSD”, given at MeetBSD California 2016 in Berkeley, CA. A recording of the talk can be viewed at: http://bit.ly/2hH7nua.

1. FreeNAS 10
Challenges of Building a Modern Storage Appliance
based on FreeBSD

2. But first…

3. Welcome Back to Berkeley!
Last time Here: FreeBSDCon ’99!

4. FreeNAS 10
Key Requirements
Key Requirements
• Make a highly complex system as simple as
possible for end-users but still powerful for admins
• Provide maximum “situational awareness” to all
users
• Do as much as possible asynchronously
• Provide for flexible application hosting as well as
storage (“Hyperconverged storage”)

5. A Short Demo

6. Challenge: Building
• FreeBSD builds are not bit-for-bit reproducible
(build times and paths sneak into binaries; delta
packages are wheeeeeeeee!).
• Packaging: Appliances require cryptographically
signed update mechanism with config data
segregated
• Debugging symbols: No concept of the .DSYM file;
we need to keep two copies of all binaries (install
ISO is 670M - debug info is 908M!)

7. Challenge: Building
• FreeBSD is at least three different things:
• Kernel + Modules
• “Base”
• Ports (Poudriere)
• We had to create our own build system + DSL
(profile description language) to unify these
concepts for a software appliance

8. FreeBSD Today
Kernel
Base
Packages (ports - infinite size)
All Your Base
Self-serve
Cafe

9. FreeBSD Someday?
Kernel
Base
Packages (ports - moar infinite size)
Minimalist
Approach
Kernel Optional Modules
Appliance
Profile
Schema

10. Appliance Software
Lifecycle Management
• Install (for specific mission profile) needs to be
as automated as possible. ZFS boot
environments have been a big plus.
• Need to be able to upgrade entire application
profile (kern + base + pkgs) as one unified
concept.
• All config data needs to survive upgrades with
no user intervention (sorry, mergemaster).

11. Misc FreeBSD Challenges
• Future of SMB 3.1.1 Performance is iffy - FreeBSD is not a
poster-child for Samba and SMB is important.
• Some perf issues identified by Samba core team, now
need work from us.
• Unix Domain Sockets really suck for serious IPC (we’ve
found long-standing data corruption bugs, performance is
definitely not great, namespace is filesystem)
• /etc/rc.d is really not service management (highly porous,
doesn’t provide consistent pid tracking, etc etc etc)

12. Misc BSD Challenges
• More full-stack developers would reveal and fix
blatant userland + OS mismatches earlier
(python fork()! OMG!). More appliance folks
would be great
• We’re missing out on mainstream clustered
filesystems, some of the latest OpenZFS
encryption work (ZoL is pulling ahead), and
much other Enterprise goodness from Linux
space

13. Bhyve Improvements Made
• Added 9pfs client / server support to share host
data with VMs - still some challenges (ACLs! )�
• Created early VM guest tools (user kind of wants
to know what IP address a VM is on, among
many other things)
• Better virtual device support (E1000 / NE2000)
• Virtio Console driver for C&C, fast VNC console

14. Bhyve Improvements
Needed
• VGA Emulation still very green
• USB Device Mapping (like VMWare)
• Suspend / Resume (for snapshots / migration)
• CD image mount/eject after boot
• High-level management APIs / better ZFS
integration / other crazy dreams

15. Containers
• Jails are sort of cool and all but fragile (esp with
VIMAGE), implementation has long beard, and we
never standardized lifecycle management
(mechanism not policy again??)
• Docker / LXD combine privilege domains with
packaging / lifecycle and better networking
management out of the box
• Fight for mainstream DevOps hearts-and-minds has
been won by Docker / LXD / (TBD) - Now What?

16. Container Futures
• BHyve + VM guest tools allow us to simply wrap
containers and many other “service providers”
into VMs and proxy them as “bigger containers”
• Wrapping is a lot easier than making Linux
Emulation 100%, but if we go that route we also
need a roadmap and a clear idea of what
“victory” looks like

17. *BSD is still awesome
• … But every year it gets a little harder to justify
building Enterprise solutions like FreeNAS on top of
it
• It’s about making both tools and solutions
(chicken-and-egg problem). Software appliances
and HW/SW combined solutions are a good start
• We can move faster (less bikeshed, more
ambition)

18. Q&A