Download to read offline
Uploaded byZvika Gutkin
Vertica on aws
The document discusses the implementation and performance of Vertica as a distributed database on AWS, highlighting significant performance enhancements and high compression rates. It details various technical aspects such as temp tables, partition streaming, and network placements. The content also reflects on geographic data comparisons and the scalability features offered by AWS for database optimization.
Vertica on aws
- 1. Vertica Distributed database inthe cloud wild west Zvika Gutkin Big data dbOps zvika.gutkin@convertro.com
- 2. Vertica on AWSConvertro Wehave 2 identical instances, Although one clearly peforms 30% better than the other.
- 3.
- 4.
- 5. 10x ~ 100x Performance Enhancements Highcompression rates Sorted data Object segmentation / replication Optimizer aware Vertica on AWS
- 6. Create Table ….. Verticaon AWSConvertro
- 7. USA Dallas 385 Sq. Miles Block BlockBlock Block Block Block 120000 Asia Israel Acres Tel Aviv 520000 Acres 450000 USA Dallas 385 Sq. Miles Asia Israel Acres Tel Aviv 520000 Acres 450000 N.America USA Dallas 385 Sq. Miles Block Block Block Block Block Block 120000 Asia Israel Acres Tel Aviv 520000 Acres 450000 USA Dallas 385 Sq. Miles Asia Israel Acres Tel Aviv 520000 Acres 450000 N.America Block Block Block Block Block Block Bad compression…
- 8. Continent Country Sizetype City size City Name Population Tel Aviv Jerusalem Haifa Dallas 450000 800000 268000 1200000 8200000 8800000 New York New Jersey 52000 78000 63000 385 468 8700 Acres Acres Acres Sq. Miles Sq. Miles Sq. Miles Israel Israel Israel USA USA USA Asia Asia Asia N. America N. America N. America Continent Country Size type City size City Name Population !@#$@a $%##! *&&^ !@#$ LZO Encoding 450000 268000 +532000 +932000 +7932000 +8532000 DeltaVal Encoding 52000 385 +83 +8315 +62615 +77615 DeltaVal Encoding Acres, 3 Sq. Miles, 3 RLE Encoding Israel, 3 USA, 3 RLE Encoding Asia, 3 N. America, 3 RLE Encoding
- 9.
- 10. Unified Temp Table Target Table/Partition Stream COPY Numberof parallel loads1 Number of parallel nodes2 Chunk size per loads3 TEMP TEMP TEMP MOVE PARTITIONS MOVE PARTITIONS Vertica The Convertro waConvertro
- 11. Real Time ETR selectA from B where C=‘D’ Business Logic Topology Sampling Lookup Aggregate Hydro Web Service Hydro
- 12.
- 13. Production Future Convertro Verticaon AWS Setup VPC Compatible instances Enhanced network Placement group EBS
- 14. Convertro Vertica onAWSThe cloud wild west
- 15. Convertro Vertica onAWSThe cloud wild west
- 16. Convertro Vertica onAWS Convertro VPC
- 17. Convertro Vertica onAWS Convertro VPC Placement group
- 18. Convertro Vertica onAWS Convertro VPC Throughput Network Disks EBS
- 19. Convertro Vertica onAWS Convertro VPC Vertica Cluster RAC
- 20. Production Future Nodes Fails AWSmagic Convertro Vertica on AWS Setup
- 21. Production Future Convertro Verticaon AWS Setup Up to date with AWS features Replay database Cluster elasticity
- 22.
Editor's Notes
- #3 Gotham The o.c Gladiator A beautiful mind
- #9 70% reduction in storage
- #11 150K rows per sec Regular days 10 – 15 billion rows per day 40 billion rows per day New vertica feature => Vhash
- #12 out of the box improvements => denormalize => data model changes are Game changer !!!.Vertica can handle big joins => merge joins MMM => measure measure measure => data collector tables .
- #14 Enhanced networking – EBS-optimized instance or an instance with 10 Gigabit network connectivity
- #15 Enhanced networking – EBS-optimized instance or an instance with 10 Gigabit network connectivity
- #16 Enhanced networking – EBS-optimized instance or an instance with 10 Gigabit network connectivity
- #17 Enhanced networking – EBS-optimized instance or an instance with 10 Gigabit network connectivity
- #18 Enhanced networking – EBS-optimized instance or an instance with 10 Gigabit network connectivity
- #19 Enhanced networking – EBS-optimized instance or an instance with 10 Gigabit network connectivity
- #20 Enhanced networking – EBS-optimized instance or an instance with 10 Gigabit network connectivity