[GeeCON2024] How I learned to stop worrying and love the dark silicon apocalypse
•
0 likes•32 views
Computation is increasingly constrained by power. With each advancement in the manufacturing process, a decreasing percentage of the CPU can operate at full capacity, leading to the emergence of the term 'dark silicon'. This trend necessitates techniques that utilize chip area to optimize power efficiency through specialized accelerators. The presentation will outline key concepts that led to the dark silicon such as Moore’s law and breakdown of Dennard scaling, followed by an overview of current and upcoming CPU accelerators. The focus will then shift to vector units and the specifics of vector programming. Attendees will be introduced to registers, a range of vector operations, and methods to develop branchless algorithms such as sorting networks. The session will conclude with an overview of the new Java Vector API and how it was already picked up by projects to do AI inference (Llama 2) and vector search (AstraDB and Cassandra).
Recommended
More Related Content
Similar to [GeeCON2024] How I learned to stop worrying and love the dark silicon apocalypse
Similar to [GeeCON2024] How I learned to stop worrying and love the dark silicon apocalypse (20)
More from Tomasz Kowalczewski
More from Tomasz Kowalczewski (14)
Recently uploaded
Recently uploaded (20)
[GeeCON2024] How I learned to stop worrying and love the dark silicon apocalypse
- 1. HOW I LEARNED TO STOP WORRYING AND LOVE THE DARK SILICON APOCALYPSE HTTP://SLI.DO #GEECON ROOM 11
- 2. How I learned to stop worrying and love the dark silicon apocalypse
- 3. How I learned to stop worrying and love the dark silicon apocalypse
- 5. for(int i = 0; i<100; i++) { ... }
- 6. for(int i = 0; i<100; i++) { ... } .back: cmp rcx, 100 je .outside ... inc rcx jmp .back .outside:
- 7. M3 Max Graviton 4 AMD EPYC Genoa-X
- 8. Today: 1000 lights/m @ 100W 2024: 2000 lights/m @ 200W 2028: 4000 lights/m @ 400W
- 9. Dennard scaling As transistors get smaller, their power density stays constant, so that the power use stays in proportion with area; both voltage and current scale (downward) with length ROBERT H. DENNARD, IBM
- 10. S S
- 12. 75% DARK AFTER 2 GENERATIONS 93% DARK AFTER 4 GENERATIONS CPU x2 x2 x4 x4 CPU 4 EPOCHS
- 14. MICHAEL B. TAYLOR, IS DARK SILICON USEFUL?
- 15. MORE CORES, MORE BETTER MORE ACCELERATORS, MORE BETTER (C) PATRICK KENNEDY, SERVETHEHOME.COM
- 16. FJCVTZS Floating-point Javascript Convert to Signed fixed- point, rounding toward Zero.
- 17. PCMPESTRI Packed Compare Explicit Length Strings, Return Index
- 18. PICTURE CREDIT TO @FRITZCHENSFRITZ, ANNOTATED BY @GPUSAREMAGIC
- 19. PICTURE CREDIT TO @FRITZCHENSFRITZ, ANNOTATED BY @GPUSAREMAGIC
- 20. VECTOR REGISTER
- 21. VECTOR REGISTER
- 22. VECTOR REGISTER
- 24. VECTOR REGISTER
- 25. 1 2 3 4 5 6 7 8 VECTOR OPERATIONS + 10 20 30 40 50 60 70 80 = 11 22 33 44 55 66 77 88
- 27. Integer Integer Integer Integer Integer Integer Integer Integer VECTOR REGISTERS SHAPE + ELEMENT TYPE = SPECIES SHAPE 64 bit 128 bit 256 bit Long Long Long Long Double Double Double Double
- 28. Vector cosine similarity text-embedding-3-large is our new next generation larger embedding model and creates embeddings with up to 3072 dimensions. PICTURE CREDIT OPENAI
- 29. Vector cosine similarity text-embedding-3-large is our new next generation larger embedding model and creates embeddings with up to 3072 dimensions.
- 30. Origin Destination Flight No. Carrier Codeshare Cabin KRK DFW 1022 LH ... ... KRK BOS 933 LH ... ... KRK LHR 123 BA ... ... Row vs. Column layout in databases
- 31. Origin Destination Flight No. Carrier Codeshare Cabin KRK DFW 1022 LH ... ... KRK BOS 933 LH ... ... KRK LHR 123 BA ... ... Row vs. Column layout in databases 1022 933 123 ... IntVector
- 32. Origin Destination Flight No. Carrier Codeshare Cabin KRK DFW 1022 LH ... ... KRK BOS 933 LH ... ... KRK LHR 123 BA ... ... 1022 933 123 ... Row vs. Column layout in databases IntVector
- 33. Row vs. Column layout in databases KRK KRK KRK ... ... ... DFW BOS LHR ... ... ... 1022 933 123 ... ... ... Origin Destination Flight No.
- 34. Row vs. Column layout in databases KRK KRK KRK ... ... ... DFW BOS LHR ... ... ... 1022 933 123 ... ... ... Origin Destination Flight No. 1022 933 123 ... IntVector
- 35. Row vs. Column layout in databases 3xKRK 13xDFW 42xLHR ... ... ... 18xDFW 22xBOS 2xLHR ... ... ... 1022 -89 -810 ... ... ... Origin Destination Flight No. 1022 933 123 ... IntVector ?
- 36. SOURCE: HTTPS://EN.M.WIKIPEDIA.ORG/WIKI/FILE:TABLE_OF_X86_REGISTERS_SVG.SVG K0 K1 K2 K3 K4 K5 K6 K7
- 37. 1 2 3 4 1 1 1 1 0 0 0 0 1 2 3 4 0 0 0 0 MEMORY MASK Vector masks V1 LOAD
- 38. 1 2 3 4 5 6 7 8 1 1 1 1 0 0 0 0 1 2 3 4 0 0 0 0 V1 MASK Vector masks MEMORY STORE Page boundary, fault suppression
- 39. 1 2 3 4 5 6 7 8 7 6 5 4 3 2 1 0 8 7 6 5 4 3 2 1 V1 SHUFFLE Vector shuffles V2 REARRANGE
- 41. Sorting networks
- 46. 5 4 1 2 5 8 7 6 MAX(V1, V2) 2 6 3 7 2 5 1 9 5 6 3 7 5 8 7 9 V1 V2 VECTOR MIN/MAX MIN(V1, V2) 2 4 1 2 2 5 1 6
- 48. Sorting networks BERENGER BRAMAS, A NOVEL HYBRID QUICKSORT ALGORITHM VECTORIZED USING AVX-512 ON INTEL SKYLAKE
- 49. NEED FOR UPFRONT DESGIN OF DATA STRUCTURES AND MEMORY LAYOUTS COMPLICATED ALGORITHMS NEED TO EMPLOY ELABORATE CODE TACTICS TO NOT BREAK (RE)BOXING FAILURE TO OPTIMISE LEADS TO CATASTROPHIC DEGRADATION OF PERFORMANCE WILL INCUBATE UNTIL PROJECT VALHALLA BECOMES AVAILABLE NEED TO MEASURE, MEASURE, MEASURE ON TARGET HARDWARE
- 50. FAST AND ROBUST VECTORIZED IN-PLACE SORTING OF PRIMITIVE TYPES INTEL® INTRINSICS GUIDE JVECTOR SIMDOPS PERFORMANCE SPEED LIMITS DESIGNING IN 2023 - 10 PROBLEMS TO SOLVE (JIM KELLER) JLLAMA