Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Deep Dive on the Amazon Aurora MySQL-compatible Edition - DAT301 - re:Invent 2017

6,742 views

Published on

The Amazon Aurora MySQL-compatible Edition is a fully managed relational database engine that combines the speed and availability of high-end commercial databases with the simplicity and cost-effectiveness of open source databases. It is purpose-built for the cloud using a new architectural model and distributed systems techniques. It provides far higher performance, availability, and durability than previously possible using conventional monolithic database architectures. Amazon Aurora packs a lot of innovations in the engine and storage layers. In this session, we do a deep-dive into some key innovations behind Amazon Aurora MySQL-compatible edition. We explore new improvements to the service and discuss best practices and optimal configurations.

  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/yxufevpm } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/yxufevpm } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/yxufevpm } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/yxufevpm } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/yxufevpm } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/yxufevpm } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/yxufevpm } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/yxufevpm } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/yxufevpm } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/yxufevpm } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/yxufevpm } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/yxufevpm } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Real Ways To Make Money, Most online opportunities are nothing but total scams! ♥♥♥ http://scamcb.com/ezpayjobs/pdf
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Deep Dive on the Amazon Aurora MySQL-compatible Edition - DAT301 - re:Invent 2017

  1. 1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS re:INVENT DAT301: Deep-dive on Amazon Aurora A n u r a g G u p t a , V i c e P r e s i d e n t , A m a z o n W e b S e r v i c e s a w g u p t a @ a m a z o n . c o m
  2. 2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Agenda > > > Aurora fundamentals Recent improvements Coming soon
  3. 3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AURORA FUNDAMENTALS
  4. 4. Amazon Aurora: A re lational datab ase re imag ine d for the clou d  Speed and availability of high-end commercial databases  Simplicity and cost-effectiveness of open source databases  Drop-in compatibility with MySQL and PostgreSQL  Simple pay as you go pricing Delivered as a managed service
  5. 5. R e lational datab ase s we re not de sig ne d for the clou d Monolithic architecture Large failure blast radius SQL TRANSACTIONS CACHING LOGGING
  6. 6. Scale-out, distributed architecture Master Replica Replica Replica AVAILABILITY ZONE 1 SHARED STORAGE VOLUME AVAILABILITY ZONE 2 AVAILABILITY ZONE 3 STORAGE NODES WITH SSDS Logging pushed down to a purpose-built log-structured distributed storage system Storage volume is striped across hundreds of storage nodes distributed across 3 availability zones (AZ) Six copies of data, two copies in each AZ SQL TRANSACTIONS CACHING SQL TRANSACTIONS CACHING SQL TRANSACTIONS CACHING
  7. 7. Why are 6 copies necessary?  In a large fleet, always some failures  AZ failures have ”shared fate”  Need to tolerate AZ+1 failures and still be able to repair  For 3 AZs, requires 6 copies AZ 1 AZ 2 AZ 3 Quorum break on AZ failure 2/3 read 2/3 write AZ 1 AZ 2 AZ 3 Quorum survives AZ failure 3/6 read 4/6 write
  8. 8. Minimal time to repair  Small segments shorten repair  Aurora uses 10GB segments  Can use fault-tolerance to: • patch without impact • balance hot and cold nodes Segment size
  9. 9. Membership changes without stalls  Most systems use consensus for membership changes. Causes jitter and stalls.  Aurora uses quorum sets and epochs  No stalls, AZ+1 fault–tolerance, can aggressively detect failure A B C D E F A B C D E F A B C D E G A B C D E F A B C D E G Epoch 1: All node healthy Epoch 2: Node F is in suspect state; second quorum group is formed with node G; both quorums are active Epoch 3: Node F is confirmed unhealthy; new quorum group with node G is active.
  10. 10. Avoiding quorum reads  Reads are expensive in most quorum-based systems  Aurora knows which nodes are up to date, latency to each node  Read quorum only needed for repairs or crash recovery LSN N LSN N LSN N-1 LSN N LSN N-1 LSN N LSN N LSN N LSN N-1 LSN N LSN N-1 LSN N For each data block, at least 4 nodes in the quorum group will have the most recent data Read from any one of these four nodes will return most recent data
  11. 11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Aurora performance WRITE PERFORMANCE READ PERFORMANCE MySQL SysBench results; R4.16XL: 64cores / 488 GB RAM Aurora read/write throughput compared to MySQL 5.6 based on industry standard benchmarks. Aurora MySQL 5.6 0 100000 200000 300000 400000 500000 600000 700000 0 50000 100000 150000 200000 250000
  12. 12. Do fewer IOs Minimize network packets Cache prior results Offload the database engine DO LESS WORK Process asynchronously Reduce latency path Use lock-free data structures Batch operations together BE MORE EFFICIENT How did we achieve this? DATABASES ARE ALL ABOUT I/O NETWORK-ATTACHED STORAGE IS ALL ABOUT PACKETS/SECOND HIGH-THROUGHPUT PROCESSING IS ALL ABOUT CONTEXT SWITCHES
  13. 13. IO traffic in MySQL BINLOG DATA DOUBLE-WRITELOG FRM FILES T Y P E O F W R I T E MYSQL WITH REPLICA EBS MIRROREBS MIRROR AZ 1 AZ 2 AMAZON S3 EBS AMAZON ELASTIC BLOCK STORE (EBS) PRIMARY INSTANCE REPLICA INSTANCE 1 2 3 4 5 Issue write to EBS – EBS issues to mirror, ack when both done Stage write to standby instance Issue write to EBS on standby instance IO FLOW Steps 1, 3, 4 are sequential and synchronous This amplifies both latency and jitter Many types of writes for each user operation Have to write data blocks twice to avoid torn writes OBSERVATIONS 780K transactions 7,388K I/Os per million txns (excludes mirroring, standby) Average 7.4 I/Os per transaction PERFORMANCE 30 minute SysBench writeonly workload, 100GB dataset, RDS MultiAZ, 30K PIOPS
  14. 14. IO traffic in Aurora AZ 1 AZ 3 PRIMARY INSTANCE Amazon S3 AZ 2 REPLICA INSTANCE AMAZON AURORA ASYNC 4/6 QUORUM DISTRIBUTED WRITES BINLOG DATA DOUBLE-WRITELOG FRM FILES T Y P E O F W R I T E IO FLOW Only write redo log records; all steps asynchronous No data block writes (checkpoint, cache replacement) 6X more log writes, but 9X less network traffic Tolerant of network and storage outlier latency OBSERVATIONS 27,378K transactions 35X MORE 950K I/Os per 1M txns (6X amplification) 7.7X LESS PERFORMANCE Boxcar redo log records – fully ordered by LSN Shuffle to appropriate segments – partially ordered Boxcar to storage nodes and issue writesREPLICA INSTANCE
  15. 15. IO traffic in Aurora (Storage Node) LOG RECORDS Primary Instance INCOMING QUEUE STORAGE NODE S3 BACKUP 1 2 3 4 5 6 7 8 UPDATE QUEUE ACK HOT LOG DATA BLOCKS POINT IN TIME SNAPSHOT GC SCRUB COALESCE SORT GROUP PEER TO PEER GOSSIPPeer Storage Nodes All steps are asynchronous Only steps 1 and 2 are in foreground latency path Input queue is 46X less than MySQL (unamplified, per node) Favor latency-sensitive operations Use disk space to buffer against spikes in activity OBSERVATIONS IO FLOW ① Receive record and add to in-memory queue ② Persist record and ACK ③ Organize records and identify gaps in log ④ Gossip with peers to fill in holes ⑤ Coalesce log records into new data block versions ⑥ Periodically stage log and new block versions to S3 ⑦ Periodically garbage collect old versions ⑧ Periodically validate CRC codes on blocks
  16. 16. TRADITIONAL DATABASE Have to replay logs since the last checkpoint Typically 5 minutes between checkpoints Single-threaded in MySQL; requires a large number of disk accesses AMAZON AURORA Underlying storage replays redo records on demand as part of a disk read Parallel, distributed, asynchronous No replay for startup Checkpointed Data Redo Log Crash at T0 requires a re-application of the SQL in the redo log since last checkpoint T0 T0 Crash at T0 will result in redo logs being applied to each segment on demand, in parallel, asynchronously Instant crash recovery
  17. 17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Crash recovery in Aurora CRASH Log records Gaps Volume Complete LSN (VCL) AT CRASH IMMEDIATELY AFTER CRASH RECOVERY Consistency Point LSN (CPL) Consistency Point LSN (CPL)  Instance opens all storage nodes for the volume  Volume Complete LSN (VCL) is the highest point where all records have met quorum  Consistency Point LSN (CPL) is the highest commit record below VCL  Everything past CPL is deleted at crash recovery  Removes the need for 2PC at each commit spanning storage nodes  No redo or undo processing is required before the database is opened for processing
  18. 18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. RECENT IMPROVEMENTS
  19. 19. Fast database cloning *August 2017* Clone database without copying data  Creation of a clone is nearly instantaneous  Data copy happens only on write – when original and cloned volume data differ Example use cases  Clone a production DB to run tests  Reorganize a database  Save a point in time snapshot for analysis without impacting production system PRODUCTION DATABASE CLONE CLONE CLONE DEV/TEST APPLICATIONS BENCHMARKS PRODUCTION APPLICATIONS PRODUCTION APPLICATIONS
  20. 20. Backup and restore in Aurora Take periodic snapshot of each segment in parallel; stream the redo logs to Amazon S3 Backup happens continuously without performance or availability impact At restore, retrieve the appropriate segment snapshots and log streams to storage nodes Apply log streams to segment snapshots in parallel and asynchronously SEGMENT SNAPSHOT LOG RECORDS RECOVERY POINT SEGMENT 1 SEGMENT 2 SEGMENT 3 TIME
  21. 21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Backtrack provides near-instantaneous restores * C o m i n g s o o n * Backtrack quickly brings the database to a desired point in time. No restore from backup. No copying of data. Not destructive – can backtrack many times. Quickly recover from unintentional DML/DDL operations. T0 T1 T2 T0 T1 T2 T3 T4 T3 T4 REWIND TO T1 REWIND TO T3 INVISIBLE INVISIBLE
  22. 22. Read replica end-point auto-scaling *Nov. 2017* Up to 15 promotable read replicas across multiple availability zones Re-do log based replication leads to low replica lag – typically < 10ms Reader end-point with load balancing and auto-scaling * NEW * MASTER READ REPLICA READ REPLICA READ REPLICA SHARED DISTRIBUTED STORAGE VOLUME READER END-POINT
  23. 23. Online DDL: Aurora vs. MySQL  Full Table copy in the background  Rebuilds all indexes – can take hours or days  DDL operation impacts DML throughput  Table lock applied to apply DML changes INDEX LEAFLEAFLEAF LEAF INDEX ROOT table name operation column-name time-stamp Table 1 Table 2 Table 3 add-col add-col add-col column-abc column-qpr column-xyz t1 t2 t3  Use schema versioning to decode the block  Modify-on-write primitive to upgrade to latest schema  Currently support add NULLable column at end of table  Add column anywhere and with default coming soon MySQL Amazon Aurora
  24. 24. Online DDL performance On r3.large On r3.8xlarge Aurora MySQL 5.6 MySQL 5.7 10GB table 0.27 sec 3,960 sec 1,600 sec 50GB table 0.25 sec 23,400 sec 5,040 sec 100GB table 0.26 sec 53,460 sec 9,720 sec Aurora MySQL 5.6 MySQL 5.7 10GB table 0.06 sec 900 sec 1,080 sec 50GB table 0.08 sec 4,680 sec 5,040 sec 100GB table 0.15 sec 14,400 sec 9,720 sec
  25. 25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Asynchronous key prefetch *Oct. 2017* Latency improvement factor vs. Batched Key Access (BKA) join algorithm. Decision support benchmark, R3.8xlarge 14.57 - 2.00 4.00 6.00 8.00 10.00 12.00 14.00 16.00 Query-1 Query-2 Query-3 Query-4 Query-5 Query-6 Query-7 Query-8 Query-9 Query-10 Query-11 Query-12 Query-13 Query-14 Query-15 Query-16 Query-17 Query-18 Query-19 Query-20 Query-21 Query-22 Cold buffer Works for queries using Batched Key Access (BKA) join algorithm + Multi-Range Read (MRR) Optimization Performs a secondary to primary index lookup during JOIN evaluation Uses background thread to asynchronously load pages into memory AKP used in queries 2, 5, 8, 9, 11, 13, 17, 18, 19, 20, 21
  26. 26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Batched scans *coming soon* Standard MySQL pprocesses rows one-at- a-time resulting in high overhead due to:  Repeated function calls  Locking and latching  cursor store and restore  InnoDB to MySQL format conversion Amazon Aurora scan tuples from the InnoDB buffer pool in batches for  Table full scans  Index full scans  Index range scans Latency improvement factor vs. Batched Key Access (BKA) join algorithm Decision support benchmark, R3.8xlarge 1.78X - 0.20 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00 Query-1 Query-2 Query-3 Query-4 Query-5 Query-6 Query-7 Query-8 Query-9 Query-10 Query-11 Query-12 Query-13 Query-14 Query-15 Query-16 Query-17 Query-18 Query-19 Query-20 Query-21 Query-22
  27. 27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Hash Joins *coming soon* Latency improvement factor vs. Batched Key Access (BKA) join algorithm Decision support benchmark, R3.8xlarge, cold buffer cache (lower improvement if all data cached) 8.22 - 1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 9.00 Hash join used in queries 2, 3, 5, 7, 8, 9, 10, 11, 12, 15, 16, 17,18, 19, 21
  28. 28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AURORA SERVERLESS
  29. 29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Aurora Serverless  Starts up on demand, shuts down when not in use  Scales up/down automatically  No application impact when scaling  Pay per second, 1 minute minimum WARM POOL OF INSTANCES APPLICATION DATABASE STORAGE SCALABLE DB CAPACITY REQUEST ROUTER DATABASE END-POINT
  30. 30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Database end-point provisioning When you provision a database, Aurora Serverless:  Provisions VPC end-points for the application connectors  Initializes request routers to accept connections  Creates an Aurora storage volume A database instance is only provisioned when the first request arrives APPLICATION CUSTOMER VPC VPC END-POINTS VPC END-POINTS NETWORK LOAD BALANCER STORAGE VOLUME REQUEST ROUTERS
  31. 31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Instance provisioning and scaling  First request triggers instance provisioning. Usually 1-3 seconds  Instance auto-scales up and down as workload changes. Usually 1-3 seconds  Instances hibernate after user-defined period of inactivity  Scaling operations are transparent to the application – user sessions are not terminated  Database storage is persisted until explicitly deleted by user DATABASE STORAGE WARM POOL APPLICATION REQUEST ROUTER CURRENT INSTANCE NEW INSTANCE
  32. 32. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AURORA MULTI-MASTER
  33. 33. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Distributed Lock Manager GLOBAL RESOURCE MANAGER SQL TRANSACTIONS CACHING LOGGING SQL TRANSACTIONS CACHING LOGGING SHARED DISK CLUSTER STORAGE APPLICATION LOCKING PROTOCOL MESSAGES SHARED STORAGE M1 M2 M3 M1 M1 M1M2 M3 M2 Cons Heavyweight cache coherency traffic, on per-lock basis Networking can be expensive Negative scaling when hot blocks Pros All data available to all nodes Easy to build applications Similar cache coherency as in multi-processors
  34. 34. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Consensus with two phase or Paxos commit DATA RANGE #1 DATA RANGE #2 DATA RANGE #4 DATA RANGE #3 DATA RANGE #5 L L L L L SHARED NOTHING SQL TRANSACTIONS CACHING LOGGING SQL TRANSACTIONS CACHING LOGGING APPLICATION STORAGE STORAGE Cons Heavyweight commit and membership change protocols Range partitioning can result in hot partitions, not just hot blocks. Re-partitioning expensive. Cross partition operations expensive. Better at small requests Pros Query broken up and sent to data node Less coherence traffic – just for commits Can scale to many nodes
  35. 35. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Conflict resolution using distributed ledgers There are many “oases” of consistency in Aurora The database nodes know transaction orders from that node The storage nodes know transaction orders applied at that node Only have conflicts when data changed at both multiple database nodes AND multiple storage nodes Much less coordination required 2 3 4 5 61 BT1 [P1] BT2 [P1] BT3 [P1] BT4 [P1] BT1 BT2 BT3 BT4 Page1 Quorum OT1 OT2 OT3 OT4 Page 1 Page 2 2 3 4 5 61 OT1[P2] OT2[P2] OT3[P2] OT4[P2] PAGE1 PAGE2 MASTER MASTER Page 2
  36. 36. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Hierarchical conflict resolution Both masters are writing to two pages P1 and P2 BLUE master wins the quorum at page P1; ORANGE master wins quorum at P2 Both masters recognize the conflict and have two choices: (1) roll back the transactions or (2) escalate to regional resolver Regional arbitrator decides who wins the tie breaker 2 3 4 5 61 BT1 [P1] OT1 [P1] 2 3 4 5 61 OT1[P2] BT1[P2] PAGE1 PAGE2 MASTER MASTER BT1 OT1 Regional resolver Page 1 Page 2 Page 1 Page 2 Quorum X X
  37. 37. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Crash recovery in multi-master CRASH MULTI-MASTERSINGLE MASTER Log records Gaps Volume Complete LSN (VCL) AT CRASH IMMEDIATELY AFTER RECOVERY MASTER 1 CRASHES GAPS AT CRASH Consistency Point LSN(CPL) VCLVCL CPL CPL MASTER 1 MASTER 2 IMMEDIATELY AFTER RECOVERY Gap filled New LSNs and Gaps Master 1 Recovery Point Consistency Point LSN(CPL)
  38. 38. Multi-region Multi-Master Write accepted locally Optimistic concurrency control – no distributed lock manager, no chatty lock management protocol REGION 1 REGION 2 HEAD NODES HEAD NODES MULTI-AZ STORAGE VOLUME MULTI-AZ STORAGE VOLUME LOCAL PARTITION LOCAL PARTITIONREMOTE PARTITION REMOTE PARTITION Conflicts handled hierarchically – at head nodes, at storage nodes, at AZ and region level arbitrators Near-linear performance scaling when there is no or low levels of conflicts
  39. 39. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AURORA PARALLEL QUERY
  40. 40. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Parallel query processing Aurora storage has thousands of CPUs  Presents opportunity to push down and parallelize query processing using the storage fleet  Moving processing close to data reduces network traffic and latency However, there are significant challenges  Data stored in storage node is not range partitioned – require full scans  Data may be in-flight  Read views may not allow viewing most recent data  Not all functions can be pushed down to storage nodes DATABASE NODE STORAGE NODES PUSH DOWN PREDICATES AGGREGATE RESULTS
  41. 41. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Processing at head node STORAGE NODES OPTIMIZER EXECUTOR INNODB NETWORK STORAGE DRIVER AGGREGATOR APPLICATION PARTIAL RESULTS STREAM RESULTS IN-FLIGHT DATA PQ CONTEXT PQ PLAN Query Optimizer produces PQ Plan and creates PQ context based on leaf page discovery PQ request is sent to storage node along with PQ context Storage node produces  Partial results streams with processed stable rows  Raw stream of unprocessed rows with pending undos Head node aggregates these data streams to produce final results
  42. 42. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Processing at storage node Each storage node runs up to 16 PQ processes, each associated with a parallel query PQ process receives PQ context  List of pages to scan  Read view and projections  Expression evaluation byte code PQ process makes two passes through the page list  Pass 1: Filter evaluation on InnoDB formatted raw data  Pass 2: Expression evaluation on MySQL formatted data PQ PROCESS PQ PROCESS Up to 16 TO/FROM HEAD NODE STORAGE NODE PROCESS PAGE LISTS
  43. 43. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Example: Parallel hash join Head node sends hash join context to storage node consisting of:  List of pages – Page ID and LSNs  Join expression byte code  Read view – what rows to include  Projections – what columns to select PQ process performs the following steps:  Scans through the lists of pages  Builds partial bloom filters; collect stats  Sends two streams to head node  Rows with matching hashes  Rows with pending undo processing Head node performs the Join function after merging data streams from different storage nodes JOIN CONTEXT PAGE SCAN STABLE ROWS ROWS WITH PENDING UNDO BLOOM FILTER FILTERED AND PROJECTED ROW STREAM UNPROCESSED ROW STREAM
  44. 44. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Parallel query performance Latency (seconds) Decision support benchmark, R3.8xlarge, cold buffer cache Improvement factor with Parallel Query 24.6x 18.3x 5.0x 0 200 400 600 800 1000 1200 1400 1600 1800 2000 Aggregate + 2-table join Aggregate query Point query on non-indexed column With Parallel Query Without Parallel Query
  45. 45. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Thank you!

×