SlideShare a Scribd company logo
1 of 22
Download to read offline
DEEP DIVE
• Object Storage Server
• S3 Compatible
• Open Source (github.com/minio/minio)
• Founder: Anand Babu Periasamy
◦ Formerly GlusterFS
▪ Distributed File System
▪ Now part of Red Hat
MINIO
BACKGROUND
MINIO
STACK
MINIO SERVER MINIO CLIENT MINIO SDK
MINIO
FLAVORS
1 BINARY / 3 FLAVORS
FS backend
$ minio server /dir
XL backend with Erasure code & bitrot protection
$ minio server /disk1 /disk2 /disk3 /disk4 ... /disk16
Distributed up to 16 servers
$ minio server host1:/disk host2:/disk host3:/disk ... host16:/disk
DISTRIBUTED
SETUP
JBOD JBOD JBOD JBOD
MINIO 1 MINIO 2 MINIO 3 MINIO 16
ERASURE CODED BACKEND
Erasure Code (2x2, 4x4, 6x6, 8x8) - Maximum 16 Disks / Setup
ERASURE CODING
BLAKE2
PLAN 9/
GOLANG ASSEMBLY
• ‘Pseudo’ assembly language
◦ Generalized instructions like MOV, ADD, CMP
◦ Actual instructions are generated platform specifically (obviously)
◦ Mostly this translation is self-explanatory/logical, sometimes it is not
◦ Some architecture aspects shine through
• Data flows from left to right
◦ Eg: MOV R1, R2 // R2 = R1
• Some ‘pseudo’ registers
◦ FP, SP, PC, SB
• Not all instructions available
◦ However BYTE/WORD/LONG/QUAD constructs for inserting opcodes directly
PLAN 9/
GOLANG ASSEMBLY
• Benefits
◦ Fast compilation
◦ Enhanced execution speed
◦ No need for `cgo` dependency
◦ Avoid runtime call stack saving overhead etc.
• Allows to take advantage of SIMD (Intel) or NEON (ARM)
capabilities
◦ Single Instruction Multiple Data
• You are (or will be) a bit on your own…
◦ Learn / Take a look at existing code (grep for *.s in go/src/)
◦ Or see minio repos
• Full details, eg for ARM:
src/cmd/internal/obj/arm/a.out.go
ASM2PLAN9S: GENERATE BYTE
SEQUENCES FOR GO ASSEMBLY
• Developed as part of blake2b-simd
• Uses YASM behind the scenes
• Support for use within defines
• Works nicely together with asmfmt
• See github.com/minio/asm2plan9s
• Planning ARM support
// VPADDQ XMM0,XMM1,XMM8
$ asm2plan9s example.s
LONG $0xd471c1c4; BYTE $0xc0 // VPADDQ XMM0,XMM1,XMM8
ACCELERATING BLAKE2B
USING SIMD
• BLAKE2b hash
◦ SHA3 competition contender
◦ Characteristics: speed, simplicity, and security
◦ Optimized for 64-bit platforms (BLAKE2b)
• github.com/minio/blake2b-simd
◦ SIMD accelerated pure Go implementation using Go Assembly
◦ Three flavors: AVX2, AVX and SSE
◦ Speed up over pure go (non-assembly): 3.94x
• For bitrot detection (minio XL version)
TECH DETAILS
BLAKE 2B - SIMD
GOLANG
v0 += v4
v1 += v5
v2 += v6
v3 += v7
AVX USE 128-BIT WIDE XMM INSTRUCTIONS
VPADDQ XMM0,XMM0,XMM2 /* v0 += v4,v1 += v5 */
VPADDQ XMM1,XMM1,XMM3 /* v2 += v6,v3 += v7 */
AVX2 USE 256-BIT WIDE YMM INSTRUCTIONS
VPADDQ YMM0,YMM0,YMM1 /* v0 += v4,v1 += v5,v2 += v6,v3 += v7 */
SSE USE 128-BIT WIDE XMM
INSTRUCTIONS
(NON ‘VEX’-INSTRUCTIONS)
BLAKE 2B ROUNDS
12 Rounds of Processing
BLAKE 2B MACROS
Example G1 macro
GATEWAY
MINIO
GATEWAY
• Recent
addition
Google Cloud Storage Amazon S3 Azure Blob Storage
MINIO GATEWAY
FEATURES
• Serve as S3 compatible proxy for different back-ends
◦ Azure / GCS / S3
◦ Others to follow (plugin model?)
• Support full S3 API
◦ Multi-part uploads
◦ Any SDK (Minio, AWS, etc.)
◦ Any language
• Transparent for client application
• Minio browser for gateway
MULTI-CLOUD
STRATEGY
• Single API to access any object
• Universal access via mc (minio client)
• Deployment
◦ Multiple gateways on single server
◦ Multiple servers each with own gateway
• Caching
◦ Cache objects on local disk
◦ Faster delivery / create your own CDN
◦ Allow savings on eg bandwidth costs
OTHER RESOURCES
• Blog
◦ Golang Internals Part 1: Autogenerated functions
◦ Golang Internals Part 2: Named return values
• dsync
◦ distributed sync package
◦ github.com/minio/dsync
• c2goasm
◦ embed C/C++ generated assembly into Go
◦ github.com/minio/c2goasm
• asm2plan9
◦ generate BYTE sequences for Go assembly
◦ github.com/minio/asm2plan9
OTHER
RESOURCES
QUESTIONS?
slack.minio.io

More Related Content

Similar to jedgskbhtigzmcf5w5hb-signature-c5e966851270c187b6839d7cc1c16887ce7dd17b275e0d984dc3a8e326aaf784-poli-170725011104.pdf

my Windows 7 info
my Windows 7 infomy Windows 7 info
my Windows 7 infoisky guard
 
Lecture 6 Kernel Debugging + Ports Development
Lecture 6 Kernel Debugging + Ports DevelopmentLecture 6 Kernel Debugging + Ports Development
Lecture 6 Kernel Debugging + Ports DevelopmentMohammed Farrag
 
Porting Android ABS 2011
Porting Android ABS 2011Porting Android ABS 2011
Porting Android ABS 2011Opersys inc.
 
qemu + gdb: The efficient way to understand/debug Linux kernel code/data stru...
qemu + gdb: The efficient way to understand/debug Linux kernel code/data stru...qemu + gdb: The efficient way to understand/debug Linux kernel code/data stru...
qemu + gdb: The efficient way to understand/debug Linux kernel code/data stru...Adrian Huang
 
PV-Drivers for SeaBIOS using Upstream Qemu
PV-Drivers for SeaBIOS using Upstream QemuPV-Drivers for SeaBIOS using Upstream Qemu
PV-Drivers for SeaBIOS using Upstream QemuThe Linux Foundation
 
Gentoo on a 486
Gentoo on a 486Gentoo on a 486
Gentoo on a 486yeokm1
 
Rete di casa e raspberry pi - Home network and Raspberry Pi
Rete di casa e raspberry pi - Home network and Raspberry Pi Rete di casa e raspberry pi - Home network and Raspberry Pi
Rete di casa e raspberry pi - Home network and Raspberry Pi Daniele Albrizio
 
Product Roadmap iEi 2017
Product Roadmap iEi 2017Product Roadmap iEi 2017
Product Roadmap iEi 2017Andrei Teleanu
 
[ENG] IPv6 shipworm + My little Windows domain pwnie
[ENG] IPv6 shipworm + My little Windows domain pwnie[ENG] IPv6 shipworm + My little Windows domain pwnie
[ENG] IPv6 shipworm + My little Windows domain pwnieZoltan Balazs
 
PGCon 2014 - What Do You Mean my Database Server Core Dumped? - How to Inspec...
PGCon 2014 - What Do You Mean my Database Server Core Dumped? - How to Inspec...PGCon 2014 - What Do You Mean my Database Server Core Dumped? - How to Inspec...
PGCon 2014 - What Do You Mean my Database Server Core Dumped? - How to Inspec...Faisal Akber
 
Developing Applications for Beagle Bone Black, Raspberry Pi and SoC Single Bo...
Developing Applications for Beagle Bone Black, Raspberry Pi and SoC Single Bo...Developing Applications for Beagle Bone Black, Raspberry Pi and SoC Single Bo...
Developing Applications for Beagle Bone Black, Raspberry Pi and SoC Single Bo...ryancox
 
Swift Install Workshop - OpenStack Conference Spring 2012
Swift Install Workshop - OpenStack Conference Spring 2012Swift Install Workshop - OpenStack Conference Spring 2012
Swift Install Workshop - OpenStack Conference Spring 2012Joe Arnold
 
Build Your Own Android Tablet
Build Your Own Android TabletBuild Your Own Android Tablet
Build Your Own Android TabletSGAndroidDevs
 
DEF CON 27 - HUBER AND ROSKOSCH - im on your phone listening attacking voip c...
DEF CON 27 - HUBER AND ROSKOSCH - im on your phone listening attacking voip c...DEF CON 27 - HUBER AND ROSKOSCH - im on your phone listening attacking voip c...
DEF CON 27 - HUBER AND ROSKOSCH - im on your phone listening attacking voip c...Felipe Prado
 
Scaling IO-bound microservices
Scaling IO-bound microservicesScaling IO-bound microservices
Scaling IO-bound microservicesSalo Shp
 
Bsdtw17: lightning talks/wip sessions
Bsdtw17: lightning talks/wip sessionsBsdtw17: lightning talks/wip sessions
Bsdtw17: lightning talks/wip sessionsScott Tsai
 
Virtualization with Vagrant (ua.pycon 2011)
Virtualization with Vagrant (ua.pycon 2011)Virtualization with Vagrant (ua.pycon 2011)
Virtualization with Vagrant (ua.pycon 2011)Dmitry Guyvoronsky
 
Zebra 2.0 in Hybrid Cloud Era
Zebra 2.0 in Hybrid Cloud EraZebra 2.0 in Hybrid Cloud Era
Zebra 2.0 in Hybrid Cloud EraKentaro Ebisawa
 

Similar to jedgskbhtigzmcf5w5hb-signature-c5e966851270c187b6839d7cc1c16887ce7dd17b275e0d984dc3a8e326aaf784-poli-170725011104.pdf (20)

my Windows 7 info
my Windows 7 infomy Windows 7 info
my Windows 7 info
 
Lecture 6 Kernel Debugging + Ports Development
Lecture 6 Kernel Debugging + Ports DevelopmentLecture 6 Kernel Debugging + Ports Development
Lecture 6 Kernel Debugging + Ports Development
 
Porting Android
Porting AndroidPorting Android
Porting Android
 
Porting Android ABS 2011
Porting Android ABS 2011Porting Android ABS 2011
Porting Android ABS 2011
 
Gcn performance ftw by stephan hodes
Gcn performance ftw by stephan hodesGcn performance ftw by stephan hodes
Gcn performance ftw by stephan hodes
 
qemu + gdb: The efficient way to understand/debug Linux kernel code/data stru...
qemu + gdb: The efficient way to understand/debug Linux kernel code/data stru...qemu + gdb: The efficient way to understand/debug Linux kernel code/data stru...
qemu + gdb: The efficient way to understand/debug Linux kernel code/data stru...
 
PV-Drivers for SeaBIOS using Upstream Qemu
PV-Drivers for SeaBIOS using Upstream QemuPV-Drivers for SeaBIOS using Upstream Qemu
PV-Drivers for SeaBIOS using Upstream Qemu
 
Gentoo on a 486
Gentoo on a 486Gentoo on a 486
Gentoo on a 486
 
Rete di casa e raspberry pi - Home network and Raspberry Pi
Rete di casa e raspberry pi - Home network and Raspberry Pi Rete di casa e raspberry pi - Home network and Raspberry Pi
Rete di casa e raspberry pi - Home network and Raspberry Pi
 
Product Roadmap iEi 2017
Product Roadmap iEi 2017Product Roadmap iEi 2017
Product Roadmap iEi 2017
 
[ENG] IPv6 shipworm + My little Windows domain pwnie
[ENG] IPv6 shipworm + My little Windows domain pwnie[ENG] IPv6 shipworm + My little Windows domain pwnie
[ENG] IPv6 shipworm + My little Windows domain pwnie
 
PGCon 2014 - What Do You Mean my Database Server Core Dumped? - How to Inspec...
PGCon 2014 - What Do You Mean my Database Server Core Dumped? - How to Inspec...PGCon 2014 - What Do You Mean my Database Server Core Dumped? - How to Inspec...
PGCon 2014 - What Do You Mean my Database Server Core Dumped? - How to Inspec...
 
Developing Applications for Beagle Bone Black, Raspberry Pi and SoC Single Bo...
Developing Applications for Beagle Bone Black, Raspberry Pi and SoC Single Bo...Developing Applications for Beagle Bone Black, Raspberry Pi and SoC Single Bo...
Developing Applications for Beagle Bone Black, Raspberry Pi and SoC Single Bo...
 
Swift Install Workshop - OpenStack Conference Spring 2012
Swift Install Workshop - OpenStack Conference Spring 2012Swift Install Workshop - OpenStack Conference Spring 2012
Swift Install Workshop - OpenStack Conference Spring 2012
 
Build Your Own Android Tablet
Build Your Own Android TabletBuild Your Own Android Tablet
Build Your Own Android Tablet
 
DEF CON 27 - HUBER AND ROSKOSCH - im on your phone listening attacking voip c...
DEF CON 27 - HUBER AND ROSKOSCH - im on your phone listening attacking voip c...DEF CON 27 - HUBER AND ROSKOSCH - im on your phone listening attacking voip c...
DEF CON 27 - HUBER AND ROSKOSCH - im on your phone listening attacking voip c...
 
Scaling IO-bound microservices
Scaling IO-bound microservicesScaling IO-bound microservices
Scaling IO-bound microservices
 
Bsdtw17: lightning talks/wip sessions
Bsdtw17: lightning talks/wip sessionsBsdtw17: lightning talks/wip sessions
Bsdtw17: lightning talks/wip sessions
 
Virtualization with Vagrant (ua.pycon 2011)
Virtualization with Vagrant (ua.pycon 2011)Virtualization with Vagrant (ua.pycon 2011)
Virtualization with Vagrant (ua.pycon 2011)
 
Zebra 2.0 in Hybrid Cloud Era
Zebra 2.0 in Hybrid Cloud EraZebra 2.0 in Hybrid Cloud Era
Zebra 2.0 in Hybrid Cloud Era
 

Recently uploaded

ROOT CAUSE ANALYSIS PowerPoint Presentation
ROOT CAUSE ANALYSIS PowerPoint PresentationROOT CAUSE ANALYSIS PowerPoint Presentation
ROOT CAUSE ANALYSIS PowerPoint PresentationAadityaSharma884161
 
What is Model Inheritance in Odoo 17 ERP
What is Model Inheritance in Odoo 17 ERPWhat is Model Inheritance in Odoo 17 ERP
What is Model Inheritance in Odoo 17 ERPCeline George
 
How to Configure Email Server in Odoo 17
How to Configure Email Server in Odoo 17How to Configure Email Server in Odoo 17
How to Configure Email Server in Odoo 17Celine George
 
Earth Day Presentation wow hello nice great
Earth Day Presentation wow hello nice greatEarth Day Presentation wow hello nice great
Earth Day Presentation wow hello nice greatYousafMalik24
 
ACC 2024 Chronicles. Cardiology. Exam.pdf
ACC 2024 Chronicles. Cardiology. Exam.pdfACC 2024 Chronicles. Cardiology. Exam.pdf
ACC 2024 Chronicles. Cardiology. Exam.pdfSpandanaRallapalli
 
AMERICAN LANGUAGE HUB_Level2_Student'sBook_Answerkey.pdf
AMERICAN LANGUAGE HUB_Level2_Student'sBook_Answerkey.pdfAMERICAN LANGUAGE HUB_Level2_Student'sBook_Answerkey.pdf
AMERICAN LANGUAGE HUB_Level2_Student'sBook_Answerkey.pdfphamnguyenenglishnb
 
Roles & Responsibilities in Pharmacovigilance
Roles & Responsibilities in PharmacovigilanceRoles & Responsibilities in Pharmacovigilance
Roles & Responsibilities in PharmacovigilanceSamikshaHamane
 
EPANDING THE CONTENT OF AN OUTLINE using notes.pptx
EPANDING THE CONTENT OF AN OUTLINE using notes.pptxEPANDING THE CONTENT OF AN OUTLINE using notes.pptx
EPANDING THE CONTENT OF AN OUTLINE using notes.pptxRaymartEstabillo3
 
Crayon Activity Handout For the Crayon A
Crayon Activity Handout For the Crayon ACrayon Activity Handout For the Crayon A
Crayon Activity Handout For the Crayon AUnboundStockton
 
Quarter 4 Peace-education.pptx Catch Up Friday
Quarter 4 Peace-education.pptx Catch Up FridayQuarter 4 Peace-education.pptx Catch Up Friday
Quarter 4 Peace-education.pptx Catch Up FridayMakMakNepo
 
How to do quick user assign in kanban in Odoo 17 ERP
How to do quick user assign in kanban in Odoo 17 ERPHow to do quick user assign in kanban in Odoo 17 ERP
How to do quick user assign in kanban in Odoo 17 ERPCeline George
 
Grade 9 Q4-MELC1-Active and Passive Voice.pptx
Grade 9 Q4-MELC1-Active and Passive Voice.pptxGrade 9 Q4-MELC1-Active and Passive Voice.pptx
Grade 9 Q4-MELC1-Active and Passive Voice.pptxChelloAnnAsuncion2
 
Judging the Relevance and worth of ideas part 2.pptx
Judging the Relevance  and worth of ideas part 2.pptxJudging the Relevance  and worth of ideas part 2.pptx
Judging the Relevance and worth of ideas part 2.pptxSherlyMaeNeri
 
Procuring digital preservation CAN be quick and painless with our new dynamic...
Procuring digital preservation CAN be quick and painless with our new dynamic...Procuring digital preservation CAN be quick and painless with our new dynamic...
Procuring digital preservation CAN be quick and painless with our new dynamic...Jisc
 
Full Stack Web Development Course for Beginners
Full Stack Web Development Course  for BeginnersFull Stack Web Development Course  for Beginners
Full Stack Web Development Course for BeginnersSabitha Banu
 
Romantic Opera MUSIC FOR GRADE NINE pptx
Romantic Opera MUSIC FOR GRADE NINE pptxRomantic Opera MUSIC FOR GRADE NINE pptx
Romantic Opera MUSIC FOR GRADE NINE pptxsqpmdrvczh
 
Planning a health career 4th Quarter.pptx
Planning a health career 4th Quarter.pptxPlanning a health career 4th Quarter.pptx
Planning a health career 4th Quarter.pptxLigayaBacuel1
 
Atmosphere science 7 quarter 4 .........
Atmosphere science 7 quarter 4 .........Atmosphere science 7 quarter 4 .........
Atmosphere science 7 quarter 4 .........LeaCamillePacle
 
Framing an Appropriate Research Question 6b9b26d93da94caf993c038d9efcdedb.pdf
Framing an Appropriate Research Question 6b9b26d93da94caf993c038d9efcdedb.pdfFraming an Appropriate Research Question 6b9b26d93da94caf993c038d9efcdedb.pdf
Framing an Appropriate Research Question 6b9b26d93da94caf993c038d9efcdedb.pdfUjwalaBharambe
 

Recently uploaded (20)

ROOT CAUSE ANALYSIS PowerPoint Presentation
ROOT CAUSE ANALYSIS PowerPoint PresentationROOT CAUSE ANALYSIS PowerPoint Presentation
ROOT CAUSE ANALYSIS PowerPoint Presentation
 
What is Model Inheritance in Odoo 17 ERP
What is Model Inheritance in Odoo 17 ERPWhat is Model Inheritance in Odoo 17 ERP
What is Model Inheritance in Odoo 17 ERP
 
How to Configure Email Server in Odoo 17
How to Configure Email Server in Odoo 17How to Configure Email Server in Odoo 17
How to Configure Email Server in Odoo 17
 
Earth Day Presentation wow hello nice great
Earth Day Presentation wow hello nice greatEarth Day Presentation wow hello nice great
Earth Day Presentation wow hello nice great
 
Raw materials used in Herbal Cosmetics.pptx
Raw materials used in Herbal Cosmetics.pptxRaw materials used in Herbal Cosmetics.pptx
Raw materials used in Herbal Cosmetics.pptx
 
ACC 2024 Chronicles. Cardiology. Exam.pdf
ACC 2024 Chronicles. Cardiology. Exam.pdfACC 2024 Chronicles. Cardiology. Exam.pdf
ACC 2024 Chronicles. Cardiology. Exam.pdf
 
AMERICAN LANGUAGE HUB_Level2_Student'sBook_Answerkey.pdf
AMERICAN LANGUAGE HUB_Level2_Student'sBook_Answerkey.pdfAMERICAN LANGUAGE HUB_Level2_Student'sBook_Answerkey.pdf
AMERICAN LANGUAGE HUB_Level2_Student'sBook_Answerkey.pdf
 
Roles & Responsibilities in Pharmacovigilance
Roles & Responsibilities in PharmacovigilanceRoles & Responsibilities in Pharmacovigilance
Roles & Responsibilities in Pharmacovigilance
 
EPANDING THE CONTENT OF AN OUTLINE using notes.pptx
EPANDING THE CONTENT OF AN OUTLINE using notes.pptxEPANDING THE CONTENT OF AN OUTLINE using notes.pptx
EPANDING THE CONTENT OF AN OUTLINE using notes.pptx
 
Crayon Activity Handout For the Crayon A
Crayon Activity Handout For the Crayon ACrayon Activity Handout For the Crayon A
Crayon Activity Handout For the Crayon A
 
Quarter 4 Peace-education.pptx Catch Up Friday
Quarter 4 Peace-education.pptx Catch Up FridayQuarter 4 Peace-education.pptx Catch Up Friday
Quarter 4 Peace-education.pptx Catch Up Friday
 
How to do quick user assign in kanban in Odoo 17 ERP
How to do quick user assign in kanban in Odoo 17 ERPHow to do quick user assign in kanban in Odoo 17 ERP
How to do quick user assign in kanban in Odoo 17 ERP
 
Grade 9 Q4-MELC1-Active and Passive Voice.pptx
Grade 9 Q4-MELC1-Active and Passive Voice.pptxGrade 9 Q4-MELC1-Active and Passive Voice.pptx
Grade 9 Q4-MELC1-Active and Passive Voice.pptx
 
Judging the Relevance and worth of ideas part 2.pptx
Judging the Relevance  and worth of ideas part 2.pptxJudging the Relevance  and worth of ideas part 2.pptx
Judging the Relevance and worth of ideas part 2.pptx
 
Procuring digital preservation CAN be quick and painless with our new dynamic...
Procuring digital preservation CAN be quick and painless with our new dynamic...Procuring digital preservation CAN be quick and painless with our new dynamic...
Procuring digital preservation CAN be quick and painless with our new dynamic...
 
Full Stack Web Development Course for Beginners
Full Stack Web Development Course  for BeginnersFull Stack Web Development Course  for Beginners
Full Stack Web Development Course for Beginners
 
Romantic Opera MUSIC FOR GRADE NINE pptx
Romantic Opera MUSIC FOR GRADE NINE pptxRomantic Opera MUSIC FOR GRADE NINE pptx
Romantic Opera MUSIC FOR GRADE NINE pptx
 
Planning a health career 4th Quarter.pptx
Planning a health career 4th Quarter.pptxPlanning a health career 4th Quarter.pptx
Planning a health career 4th Quarter.pptx
 
Atmosphere science 7 quarter 4 .........
Atmosphere science 7 quarter 4 .........Atmosphere science 7 quarter 4 .........
Atmosphere science 7 quarter 4 .........
 
Framing an Appropriate Research Question 6b9b26d93da94caf993c038d9efcdedb.pdf
Framing an Appropriate Research Question 6b9b26d93da94caf993c038d9efcdedb.pdfFraming an Appropriate Research Question 6b9b26d93da94caf993c038d9efcdedb.pdf
Framing an Appropriate Research Question 6b9b26d93da94caf993c038d9efcdedb.pdf
 

jedgskbhtigzmcf5w5hb-signature-c5e966851270c187b6839d7cc1c16887ce7dd17b275e0d984dc3a8e326aaf784-poli-170725011104.pdf

  • 2. • Object Storage Server • S3 Compatible • Open Source (github.com/minio/minio) • Founder: Anand Babu Periasamy ◦ Formerly GlusterFS ▪ Distributed File System ▪ Now part of Red Hat MINIO BACKGROUND
  • 4. MINIO FLAVORS 1 BINARY / 3 FLAVORS FS backend $ minio server /dir XL backend with Erasure code & bitrot protection $ minio server /disk1 /disk2 /disk3 /disk4 ... /disk16 Distributed up to 16 servers $ minio server host1:/disk host2:/disk host3:/disk ... host16:/disk
  • 5. DISTRIBUTED SETUP JBOD JBOD JBOD JBOD MINIO 1 MINIO 2 MINIO 3 MINIO 16 ERASURE CODED BACKEND Erasure Code (2x2, 4x4, 6x6, 8x8) - Maximum 16 Disks / Setup
  • 7.
  • 9. PLAN 9/ GOLANG ASSEMBLY • ‘Pseudo’ assembly language ◦ Generalized instructions like MOV, ADD, CMP ◦ Actual instructions are generated platform specifically (obviously) ◦ Mostly this translation is self-explanatory/logical, sometimes it is not ◦ Some architecture aspects shine through • Data flows from left to right ◦ Eg: MOV R1, R2 // R2 = R1 • Some ‘pseudo’ registers ◦ FP, SP, PC, SB • Not all instructions available ◦ However BYTE/WORD/LONG/QUAD constructs for inserting opcodes directly
  • 10. PLAN 9/ GOLANG ASSEMBLY • Benefits ◦ Fast compilation ◦ Enhanced execution speed ◦ No need for `cgo` dependency ◦ Avoid runtime call stack saving overhead etc. • Allows to take advantage of SIMD (Intel) or NEON (ARM) capabilities ◦ Single Instruction Multiple Data • You are (or will be) a bit on your own… ◦ Learn / Take a look at existing code (grep for *.s in go/src/) ◦ Or see minio repos • Full details, eg for ARM: src/cmd/internal/obj/arm/a.out.go
  • 11. ASM2PLAN9S: GENERATE BYTE SEQUENCES FOR GO ASSEMBLY • Developed as part of blake2b-simd • Uses YASM behind the scenes • Support for use within defines • Works nicely together with asmfmt • See github.com/minio/asm2plan9s • Planning ARM support // VPADDQ XMM0,XMM1,XMM8 $ asm2plan9s example.s LONG $0xd471c1c4; BYTE $0xc0 // VPADDQ XMM0,XMM1,XMM8
  • 12. ACCELERATING BLAKE2B USING SIMD • BLAKE2b hash ◦ SHA3 competition contender ◦ Characteristics: speed, simplicity, and security ◦ Optimized for 64-bit platforms (BLAKE2b) • github.com/minio/blake2b-simd ◦ SIMD accelerated pure Go implementation using Go Assembly ◦ Three flavors: AVX2, AVX and SSE ◦ Speed up over pure go (non-assembly): 3.94x • For bitrot detection (minio XL version)
  • 13. TECH DETAILS BLAKE 2B - SIMD GOLANG v0 += v4 v1 += v5 v2 += v6 v3 += v7 AVX USE 128-BIT WIDE XMM INSTRUCTIONS VPADDQ XMM0,XMM0,XMM2 /* v0 += v4,v1 += v5 */ VPADDQ XMM1,XMM1,XMM3 /* v2 += v6,v3 += v7 */ AVX2 USE 256-BIT WIDE YMM INSTRUCTIONS VPADDQ YMM0,YMM0,YMM1 /* v0 += v4,v1 += v5,v2 += v6,v3 += v7 */ SSE USE 128-BIT WIDE XMM INSTRUCTIONS (NON ‘VEX’-INSTRUCTIONS)
  • 14. BLAKE 2B ROUNDS 12 Rounds of Processing
  • 17. MINIO GATEWAY • Recent addition Google Cloud Storage Amazon S3 Azure Blob Storage
  • 18. MINIO GATEWAY FEATURES • Serve as S3 compatible proxy for different back-ends ◦ Azure / GCS / S3 ◦ Others to follow (plugin model?) • Support full S3 API ◦ Multi-part uploads ◦ Any SDK (Minio, AWS, etc.) ◦ Any language • Transparent for client application • Minio browser for gateway
  • 19. MULTI-CLOUD STRATEGY • Single API to access any object • Universal access via mc (minio client) • Deployment ◦ Multiple gateways on single server ◦ Multiple servers each with own gateway • Caching ◦ Cache objects on local disk ◦ Faster delivery / create your own CDN ◦ Allow savings on eg bandwidth costs
  • 21. • Blog ◦ Golang Internals Part 1: Autogenerated functions ◦ Golang Internals Part 2: Named return values • dsync ◦ distributed sync package ◦ github.com/minio/dsync • c2goasm ◦ embed C/C++ generated assembly into Go ◦ github.com/minio/c2goasm • asm2plan9 ◦ generate BYTE sequences for Go assembly ◦ github.com/minio/asm2plan9 OTHER RESOURCES