Ata Over Ethernet


Published on

A brief presentation on the ATA-over-Ethernet protocol and how it can be used to create an inexpensive enterprise-ready SAN.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Ata Over Ethernet

  1. 1. ATA Over Ethernet 7 March 2008 KULUA Meeting Lawrence Public Library, Lawrence, KS
  2. 2. What is it? • Encapsulation of ATA commands inside Ethernet frames • Non-routeable: transmitted directly via Ethernet, rather than (e.g.) TCP/IP • Hosts can use AoE-attached disks as if they were directly (e.g. via SATA) attached. • Cheaper lower-level alternative to iSCSI
  3. 3. How to get it? • Purchase EtherDrive appliance from Coraid. This is a commodity x86(64) server, generally SuperMicro, with a number of hot-swap SATA trays and Coraid OS. • Purchase a device based on LayerWalker MiniSAN chip • Roll your own
  4. 4. Roll your own? • Build commodity Linux server. • Put physical disks for export (or RAID arrays) into an LVM VG. • Create as many separate LVM LV’s as you want to export. • Export LV’s using vblade(d), kvblade, or ggaoed • Physical volumes need not be (S)ATA!
  5. 5. Wasn’t this ATA over Ethernet? • LVM appears to take care of all of it. • I have successfully created and exported LV’s via AoE on top of SATA, SCSI, and Adaptec SCSI RAID PV’s. • This means you can throw a bunch of old SCSI disks into a Linux box, export them via AoE, and use them for stuff.
  6. 6. Example configuration Gigabit Ethernet switch OS on SSD LAN AOE shelf AOE shelf AOE shelf AOE shelf
  7. 7. Example configuration explained • Linux server has OS mounted on SSD • Connected to AOE “shelves” via dedicated gigabit network • AOE shelves configured with appropriate RAID • Each shelf exports a single logical device • Logical devices configured on server as PV’s in single VG • LV’s created on server in VG
  8. 8. Advantages • In the example configuration, it is trivially easy to add storage to the VG on the server: • Attach another AOE shelf (e.g. #5) • Extend VG with device exported by shelf (e.g. 0) • vgextend vg00 /dev/etherd/e5.0 • Extend appropriate LV(s), filesystems
  9. 9. Caveats • AoE is non-routeable. SAN must be on same logical Ethernet LAN as server • Set up AoE devices on server as PV’s and put them into an LVM VG. Otherwise not so easy to extend later. • Set up syslog on related AoE shelves to point to single host for ease of management. • 1500 byte MTU only allows for two 512b sectors. Increase MTU to 9000 for better performance. • AoE volumes potentially accessible via multiple machines. If mounting the same volume from two machines, use an FS that supports this, like ClusterFS.
  10. 10. Required Software for Linux • AoE modules have been in the kernel since 2.6.11 • aoetools – tools to detect and display available AoE volumes, as well as tools to export LV’s/partitions via AoE –
  11. 11. More information • • • (my personal experiments with AoE) • (AoE bootable CD, can both mount and export volumes)
  12. 12. I can help! • Available to consult on AoE installations/deployments • • (843) 819-9065