This document provides an overview of Minio, an open source object storage server that is S3 compatible. It discusses Minio's background, architecture including the Minio server, client, and SDK stacks. It describes the different deployment flavors including filesystem, erasure coded, and distributed setups. The document also covers topics like erasure coding, accelerated blake2b hashing using SIMD, the Minio gateway for accessing multiple cloud storage backends through a single API, and other Minio projects.

1. DEEP DIVE

2. • Object Storage Server
• S3 Compatible
• Open Source (github.com/minio/minio)
• Founder: Anand Babu Periasamy
◦ Formerly GlusterFS
▪ Distributed File System
▪ Now part of Red Hat
MINIO
BACKGROUND

3. MINIO
STACK
MINIO SERVER MINIO CLIENT MINIO SDK

4. MINIO
FLAVORS
1 BINARY / 3 FLAVORS
FS backend
$ minio server /dir
XL backend with Erasure code & bitrot protection
$ minio server /disk1 /disk2 /disk3 /disk4 ... /disk16
Distributed up to 16 servers
$ minio server host1:/disk host2:/disk host3:/disk ... host16:/disk

5. DISTRIBUTED
SETUP
JBOD JBOD JBOD JBOD
MINIO 1 MINIO 2 MINIO 3 MINIO 16
ERASURE CODED BACKEND
Erasure Code (2x2, 4x4, 6x6, 8x8) - Maximum 16 Disks / Setup

6. ERASURE CODING

8. BLAKE2

9. PLAN 9/
GOLANG ASSEMBLY
• ‘Pseudo’ assembly language
◦ Generalized instructions like MOV, ADD, CMP
◦ Actual instructions are generated platform specifically (obviously)
◦ Mostly this translation is self-explanatory/logical, sometimes it is not
◦ Some architecture aspects shine through
• Data flows from left to right
◦ Eg: MOV R1, R2 // R2 = R1
• Some ‘pseudo’ registers
◦ FP, SP, PC, SB
• Not all instructions available
◦ However BYTE/WORD/LONG/QUAD constructs for inserting opcodes directly

10. PLAN 9/
GOLANG ASSEMBLY
• Benefits
◦ Fast compilation
◦ Enhanced execution speed
◦ No need for `cgo` dependency
◦ Avoid runtime call stack saving overhead etc.
• Allows to take advantage of SIMD (Intel) or NEON (ARM)
capabilities
◦ Single Instruction Multiple Data
• You are (or will be) a bit on your own…
◦ Learn / Take a look at existing code (grep for *.s in go/src/)
◦ Or see minio repos
• Full details, eg for ARM:
src/cmd/internal/obj/arm/a.out.go

11. ASM2PLAN9S: GENERATE BYTE
SEQUENCES FOR GO ASSEMBLY
• Developed as part of blake2b-simd
• Uses YASM behind the scenes
• Support for use within defines
• Works nicely together with asmfmt
• See github.com/minio/asm2plan9s
• Planning ARM support
// VPADDQ XMM0,XMM1,XMM8
$ asm2plan9s example.s
LONG $0xd471c1c4; BYTE $0xc0 // VPADDQ XMM0,XMM1,XMM8

12. ACCELERATING BLAKE2B
USING SIMD
• BLAKE2b hash
◦ SHA3 competition contender
◦ Characteristics: speed, simplicity, and security
◦ Optimized for 64-bit platforms (BLAKE2b)
• github.com/minio/blake2b-simd
◦ SIMD accelerated pure Go implementation using Go Assembly
◦ Three flavors: AVX2, AVX and SSE
◦ Speed up over pure go (non-assembly): 3.94x
• For bitrot detection (minio XL version)

13. TECH DETAILS
BLAKE 2B - SIMD
GOLANG
v0 += v4
v1 += v5
v2 += v6
v3 += v7
AVX USE 128-BIT WIDE XMM INSTRUCTIONS
VPADDQ XMM0,XMM0,XMM2 /* v0 += v4,v1 += v5 */
VPADDQ XMM1,XMM1,XMM3 /* v2 += v6,v3 += v7 */
AVX2 USE 256-BIT WIDE YMM INSTRUCTIONS
VPADDQ YMM0,YMM0,YMM1 /* v0 += v4,v1 += v5,v2 += v6,v3 += v7 */
SSE USE 128-BIT WIDE XMM
INSTRUCTIONS
(NON ‘VEX’-INSTRUCTIONS)

14. BLAKE 2B ROUNDS
12 Rounds of Processing

15. BLAKE 2B MACROS
Example G1 macro

16. GATEWAY

17. MINIO
GATEWAY
• Recent
addition
Google Cloud Storage Amazon S3 Azure Blob Storage

18. MINIO GATEWAY
FEATURES
• Serve as S3 compatible proxy for different back-ends
◦ Azure / GCS / S3
◦ Others to follow (plugin model?)
• Support full S3 API
◦ Multi-part uploads
◦ Any SDK (Minio, AWS, etc.)
◦ Any language
• Transparent for client application
• Minio browser for gateway

19. MULTI-CLOUD
STRATEGY
• Single API to access any object
• Universal access via mc (minio client)
• Deployment
◦ Multiple gateways on single server
◦ Multiple servers each with own gateway
• Caching
◦ Cache objects on local disk
◦ Faster delivery / create your own CDN
◦ Allow savings on eg bandwidth costs

20. OTHER RESOURCES

21. • Blog
◦ Golang Internals Part 1: Autogenerated functions
◦ Golang Internals Part 2: Named return values
• dsync
◦ distributed sync package
◦ github.com/minio/dsync
• c2goasm
◦ embed C/C++ generated assembly into Go
◦ github.com/minio/c2goasm
• asm2plan9
◦ generate BYTE sequences for Go assembly
◦ github.com/minio/asm2plan9
OTHER
RESOURCES

22. QUESTIONS?
slack.minio.io