Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Army of arm - NYC downtown tech meetup


Published on

see how igneous system shrunk the fault domain of a scalable object store down to almost nothing!

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Army of arm - NYC downtown tech meetup

  1. 1. Igneous Systems, Inc. RatioPerfect™ Architecture
  2. 2. • Team DNA Igneous Systems - Introduction • Started October 2013 • Based in Seattle • 12 patents to date • In production today 2
  3. 3. Who wants to be (or rely on) this guy? SETUP, PROVISION & SCALE MANAGE & MONITOR FAILURES TROUBLESHOOT & UPGRADE Client/App 3
  4. 4. What did we want to do • Provide IAAS on prem • “I” means : stuff people need, but hate dealing with • “AAS” means : customer doesn’t deal with it • Means: we don’t want to touch it if we can help it
  5. 5. Single Server Problems - I/O bottlenecks - Zero Server level protection True Cloud for Local Data SATA Traditional Concepts Features - Drive failure? No big deal - Simple for small scale /mnt/fileserver Tech - Direct Attached Storage - RAID/Parity
  6. 6. Dual Head SAS True Cloud for Local DataTraditional Concepts Problems - I/O bottlenecks - Scales only so far… - Mountpoint sprawl Features - Drive failure? No big deal - Single server protection - Simple to scale capacity /mnt/fileserver /mnt/fileserver2 /mnt/fileserver3 /mnt/fileserver4 /mnt/fileserver5 /mnt/fileserver7/mnt/fileserver6 /mnt/fileserver..100 Tech - SAS & FC - RAID/Parity/NVRAM
  7. 7. Traditional Concepts Scale out/Clustered Infiniband Infiniband Problems - One node part fail = node fail - Node rebuild is painful (weeks) Features - Distributed protection & perf - Drive failure? Who cares - Protection against node failure - Scale big on perf & capacity - One mountpoint /mnt/fileserver Tech - Node Based (CPU/RAM/DISK/NIC/Crtlr) - Infiniband Interconnect - Erasure Coding
  8. 8. What makes a server/node? CPU RAM NIC Controller OS Not redundant No Hot Swap.. Disks Hot Swap Highly redundant
  9. 9. Physical Concept Micron DDR3-256MB Marvell 88F6707 ARMv7 800MHZ Spansion bootFlash-128MB SAS-connector Top of Board Back side SATA-connector Plugs into HDD 4-inches We can run Ethernet here! 1-inch
  10. 10. A new platform is born First boot on July 11, 2014
  11. 11. True Cloud for Local Data11
  12. 12. Hot Swap Power supplies Hot Swap Ethernet Switches (IOMs) Chassis == Rack
  13. 13. Igneous HW Architecture –Each 4U Chassis contains ▪ 60 NanoServers, each attached to a 6TB SATA drive ▪ 212TB of Usable Capacity per Chassis –Start with one, scale out from there ▪ Add Chassis non-disruptively Basic Architecture
  14. 14. Top & Bottom half • 1:1:1 – CPU : Disk : I/O • No Impact Fail • Erasure Encoding with Distributed Repair • Small Fault Domain • Rack in a Box Redundancy 1 Stateful services X86 compute ARM Nano-servers Stateless Services
  15. 15. Control Plane 1 S3 API Customer Application Telemetry Provisioning Capacity Planning Customer does Monitoring Configuration Mitigation Software updates Igneous does
  16. 16. Data Path Architecture Object Layer Disk Layer Resilient Store Access control Namespaces Metadata Layout decisions Erasure coding Repair/rebalance Data placement Device health
  17. 17. Distributed Erasure Encoding Prioritized Repair - 65% space efficiency (20+8) - Prioritized repair D D D D L D D D D L D D D D L D D D D L D D D D L G G G Data block Local parity Global parity D L G
  18. 18. Protocol Services Journal Indexing Services Resilient Stores Disk Layer Ingest Services Extensible Data Path Zero-Touch Infrastructure Workflow Hooks Deep Inspection Indexing Extensible Data Engine Primary Data Path
  19. 19. Container ServicesKubernetes /Docker Index StoreSearch API Azure /Swift 2 Content Store S3 API Replication Encryption Compression Transcoding / Image Classifiers Customer Specific Microservice Search Microservice Third Party / ISV Microservices July 2016 Event Driven Computing Service What we’re building next…
  20. 20. 22 That’s all folks.. But btw… We are hiring @ @andypern @IgneousIO