Memory Management Made Simple: Kevin Mcghee, Madelyn Olson

© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Memory Management Made Simple
Kevin McGehee (Software Development Engineer)
Madelyn Olson (Software Development Engineer)
Amazon ElastiCache

• Introduction to memory
• How Redis uses memory & how to tune it
1. Dataset
2. Input & output buffers
3. Cron jobs + background threads
4. Additional Redis metadata
5. Snapshotting
6. Allocator fragmentation + metadata
• Summary of best practices
Outline

Fast
<1ms latency for
most commands
Open source
Simple
Highly available
Replication
Atomic operations
Supports transactions
In-memory
key-value store
Powerful
~200 commands, Lua scripting,
Pub/Sub
Various data structures
Strings, lists, hashes, sets, sorted sets, streams,
geospatial, bitmaps, streams
and HyperLogLogs
Backup/Restore
Enables snapshotting

Introduction to memory

• Random Access Memory (RAM)
• Byte-addressable device that allows consistent, low-latency access to data
• Traditionally volatile
• Data is lost without power
• RAM is a finite resource
• Lacks the same scalability properties of disks
Memory: how computers remember

• Swap pretends to give you more RAM
• But it’s cheating
Memory: how computers remember
B
C
D
F
Swap In
A E
Swap Out

• All Redis data is in RAM all the time
• Important to understand how Redis uses memory & how to tune
• No hard limit on process memory usage
• When out of memory (OOM), exits
• Swap can prevent crashes but degrades performance
Redis & memory
static void zmalloc_default_oom(size_t size) {
fprintf(stderr, "zmalloc: Out of memory trying
to allocate %zu bytesn", size);
fflush(stderr);
abort();
}

How Redis uses memory

Redis memory info
# Memory
used_memory:327700504
used_memory_human:312.52M
used_memory_rss:338391040
used_memory_rss_human:322.71M
used_memory_peak:327778768
used_memory_peak_human:312.59M
used_memory_peak_perc:99.98%
used_memory_overhead:93743400
used_memory_startup:509680
used_memory_dataset:233957104
used_memory_dataset_perc:71.50%
allocator_allocated:327700616
allocator_active:328024064
allocator_resident:337584128
total_system_memory:1044770816
total_system_memory_human:996.37M
used_memory_lua:37888
used_memory_lua_human:37.00K
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
allocator_frag_ratio:1.00
allocator_frag_bytes:323448
allocator_rss_ratio:1.03
allocator_rss_bytes:9560064
rss_overhead_ratio:1.00
rss_overhead_bytes:806912
mem_fragmentation_ratio:1.03
mem_fragmentation_bytes:10773560
mem_allocator:jemalloc-4.0.3
active_defrag_running:0
lazyfree_pending_objects:0

• Keys and values
• Value size depends on data type used
• Values may require different amounts of memory on primary and replica
Item #1: Redis dataset
used_memory_dataset:233957104
used_memory_dataset_perc:71.50%

• used_memory value
• All the memory Redis has allocated within the process (not just dataset)
• maxmemory parameter
• Goal for maximum memory consumption
• When reached, will trigger eviction and possibly reject further writes
Redis dataset control

• maxmemory-policy parameter
• What to do when used_memory >= maxmemory
• Eviction policy: how Redis selects which items to remove when
maxmemory is reached
• volatile-{lru,lfu,random,ttl}
– volatile == TTL is set
• allkeys-{lru,lfu,random}
• noeviction
• Best effort check before commands are executed
Redis dataset control

Redis per item overhead
Bytes
1-2 (assuming 0.5-1.0 load
factor) pointers in the hash
table
8-16
dictEntry Structure: 3
Pointers
24
Key Sds structure: Header
size 3 byte assuming key
size > 32 bytes, 1 byte for
null terminator
4
Value Redis Object Structure 16
If expiry is set 32-40
Total 60, 100 (if expiry set)

• Different data structures have different tradeoffs
• Sds (Strings)
• Embedded string (Strings)
• Integer (Strings)
• HashMap (Main DB, Hash, Sorted Set, Set)
• Intset (Set)
• ZipList (Hash, Sorted Set)
• Skiplist (Sorted Set)
• QuickList (List)
• And more (Radix, zipmap, linked list)
Redis data structures are represented differently

• Some data structures scale better
• Memory locality
• Some data-structures have one way conversions
• Intset -> HashMap
• Embedded String -> String
• Integer -> String
• ZipList -> HashMap
• ZipList <-> Skiplist
• Reload the data to reset data structure
One way conversions can increase memory

• Simple Dynamic String
• Can scale dynamically
• Low overhead for string, used throughout Redis
• Different encodings have different overhead
• Embedded String
• Removes pointer to value, better memory utilization
• String is up to 44 bytes, total 64 bytes with redis-object
• Integer Encoded
• Converts a string to an integer if possible
Example – Strings

Example – Strings
“123456789” – 29 bytes
<8 bytes of overhead>
<8 bytes for pointer>
<3 bytes of header>
<10 bytes for string>
“123456789” - 21 bytes
<3 bytes of header>
“123456789” – 16 bytes

• Ziplist is a packed binary encoding of the data in the list
• 1-9 bytes per entry of overhead, 11 bytes for datastructure structure
overhead
• Fast when data structure is small from memory locality
• Generic Solution for key/value store
• 24 Bytes per entry + (8-16) pointer from main table
• 64 Bytes of overhead
• Scales well
• Supports dynamic resizing
Example – Hashes (Ziplist or HashMap)

• ”Hello”: “World” – 25 Bytes
• [0f 00 00 00] [0c 00 00 00] [02 00] - Overhead
• [00] [05] [48 65 6c 6c 6f] - ”Hello”
• [07] [06] [57 6f 72 6c 64] [ff] – “World” + end marker
Ziplist - example

HashMap - Example
• Total Size: 112 Bytes
• Dictionary – 64 Bytes
• Entry array - 8 bytes
• Entry – 24 bytes
• Data x2 – 16 bytes

• Datatype conversions
• hash-max-ziplist-entries (Default 512)
• hash-max-ziplist-value (Default 64)
• set-max-intset-entries (Default 512)
• zset-max-ziplist-entries (Default 128)
• zset-max-ziplist-value (Default 64)
• List compression
• list-compress-depth (Default 0)
• list-max-ziplist-size (Default -2)
Redis data structure configs

• Store blobs of data as strings when possible
• HyperLogLog (just a string) for object counting
• Use Bitmaps instead of individual keys
• Integers and intsets are compact
Better utilize data structures

• JSON -> MsgPack
• Supported in LUA already
• {"compact":true,"schema":0}
• 82 a7 63 6f 6d 70 61 63 74 c3 a6 73 63 68 65 6d 61 00
• Fixed structure -> protobuf
Use a better serializer for your data

• Trade off client side performance for server side memory
reduction
• LZO, Google Snappy – Fast compression, low ratio
• bzip2, Brotli – Slow compression, high ratio
• https://quixdb.github.io/squash-benchmark/
• Some Clients have built in support
• https://github.com/lettuce-io/lettuce-core
Compression

• https://rdbtools.com/
• Runs on a running cluster or
snapshot
• Profiles the amount of memory by
keys and patterns
• Shows which data structures are using
the most memory
Use RDBTools to analyze data
Image from https://rdbtools.com/

• Client commands are buffered internally
• Redis reads from socket incrementally
• Large commands are aggregated until the end-of-command
• Limits default to 1GB: client-query-buffer-limit
Item #2: Input & Output buffers
HSET user:john favorite_food
John input buffer
John
kale smoothies
Execute!
client_biggest_input_buf:0

• Each connected client has a 16KB output buffer
• Expands to a linked list for larger responses
• Output to commands are buffered to allow atomic processing
• Data is copied into per-client buffers (e.g. KEYS *)
• Redis writes to socket incrementally
• Limits are per client type: client-output-buffer-limit {normal, pubsub, replica}
HGETALL user:alice
Alice input buffer
Alice
Execute!
Alice output buffer
1) "favorite_food"
2) "lemonade"
3) "favorite_color"
4) "green"
client_longest_output_list:0

• Replication Output Buffers
• Used for replication to feed replicas with all incoming writes
• Accumulates quickly during high write throughput
• Grows without draining during replica bootstrap (snapshotting)
• If too large in steady-state, could point to other problems
Alice
Bob
Replica output buffer
INCR mycounter
INCR mycounter
Primary Replica

• Activerehashing
• Hash table expansion + rehashing
• Resizes the main dictionary when it reaches a specified load factor
• Provides better performance in hash lookups
• Two hash tables exist simultaneously
• TTL expiration
• Items are not guaranteed to reclaim memory when TTL fires
• Background job samples items with TTLs and deletes expired items
• Maintains carrying capacity of 25% expired items
• Can use SCAN to force expiration
Item #3: Cron jobs & background threads

• Script cache
• Parameterize input arguments instead of loading multiple scripts
• Modules
• Replication backlog buffer
• Fixed circular buffer to allow PSYNC (default 1MB)
• AOF rewrite buffer
• Expanding in-memory buffer for writes to flush to disk
Item #4: Additional Redis metadata

• fork() OS call is used to provide copy-on-write behavior
• Parent keeps processing requests, child iterates over existing keyspace
• Every time a page of RAM is written to by parent, it is copied
Item #5: Snapshotting (BGSAVE)

DEL session:cindy
Cindy
Replica output buffer
A
B
C
D
A’
DEL session:cindy
Primary Replica
BGSAVE
Process

• Reserve more memory (use lower maxmemory)
• 50% recommended
RDB: 100 MB of memory used by copy-on-write

• Buffer cache memory may go up while snapshotting / AOF
• This memory can be reclaimed by OS when needed
$ free -m
total used free shared buffers cached
Mem: 2007 1834 172 0 67 914
-/+ buffers/cache: 853 1153
Swap: 2007 0 2007

• Fragmentation occurs when there are unused holes in memory
• This memory is “wasted” and not available to application
Item #6: Fragmentation
mem_fragmentation_ratio:1.03
mem_fragmentation_bytes:10773560

• Use jemalloc
• Jemalloc minimizes fragmentation
• Splits up pages into size classes, limiting internal fragmentation
• Activedefrag
• Moves memory around to reduce fragmentation
• MEMORY PURGE
• Restart the process
• Reloads data from RDB/AOF
Item #6: Fragmentation

Key takeaways

• Set maxmemory and eviction policy
• Swap adds protection against crashes
• Data structures vary based on size, and can be tuned for usage
• Consider serialization and compression to reduce dataset size
Best Practices

• Replication and snapshotting require more memory, especially
with lots of writes
• Buffering can be a large source of memory usage, especially with
many clients
• Account for overhead in sizing your database
Best Practices

• REDIS INFO MEMORY
• Hopefully you can parse the output better!
• MEMORY USAGE <key> [SAMPLES <count>]
• Estimate the size of a {key, value} item + metadata in bytes
• MEMORY DOCTOR
• Analyze memory stats for possible problems + suggests fixes
Tools

Fully-managed
Extreme performance
In-memory data store and cache for
sub-millisecond response times
Platform tuning and optimization (e.g.
R5/M5)
Fully managed
AWS manages all hardware and software
setup, configuration, monitoring
Easily scalable
Read scaling with replicas
Write and memory scaling with sharding
(up to 250 nodes / 170TB of data)
Redis compatible
Redis 5 Support
Redis client compatible
Reliable
Multi-AZ
Deep monitoring
Automatic failover
Secure and compliant
Amazon VPC
HIPAA, PCI, FedRAMP
Encryption at-rest and in-transit
Authentication

AWS Redis page
aws.amazon.com/redis
ElastiCache for Redis page
aws.amazon.com/elasticache/redis
Contact us
aws.amazon.com/contact-us/
Get started

Thank you!
Kevin McGehee | mcgehee@amazon.com
Madelyn Olson | matolson@amazon.com

Memory Management Made Simple: Kevin Mcghee, Madelyn Olson

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Memory Management Made Simple: Kevin Mcghee, Madelyn Olson

Similar to Memory Management Made Simple: Kevin Mcghee, Madelyn Olson (20)

More from Redis Labs

More from Redis Labs (20)

Recently uploaded

Recently uploaded (20)

Memory Management Made Simple: Kevin Mcghee, Madelyn Olson