PostgreSQL Query Cache - "pqc"

Overview• By caching result responses from backend, a query cache daemon (`pqcd’) makes query response faster “extremely” (x10~x100). – Delegates queries in front of the backend, like a proxy. – Waits connections on the different port (9999 by default). – Intercepts and caches SELECT query results. (Query Cache) – Manages lifecycle of the query cache. Server side Direct execution PostgreSQL PostgreSQL client backend(s) Execution pqcd with the query cache Cache Memory Copyright 2010-2011 Uptime Technologies, LLC. All rights reserved.

Tech Specs• Supports simple query executions, and prepared statements. – Supports V3 protocol only. (V2 is not supported)• Supports “Active cache mode” – All SELECT query results would be stored in the query cache. – User specified query would not be cached.• Supports “Passive cache mode” – All SELECT query results would NOT be cached. – Only user specified SELECT queries would be cached. – And/or queries which took longer execution time than the user-specified duration, would be cached. (Not implemented yet)• Supports cache invalidation (expiration or refreshing) – By using the cache expiration timeout. – By using “hint” with SELECT statements. – By using “hint” to expire all query cache. (Not implemented yet) Copyright 2010-2011 Uptime Technologies, LLC. All rights reserved.

The Query Cache Flow Overview• After `pqcd’ receives a SELECT statement, if some result is available in the query cache, `pqcd’ returns the result from the query cache. If not available, it relays the query to the backend. Receive a query from frontend Is cache No available? Yes Not expired No Invalidate Forward the query yet? the cache to backend Yes Fetch a result Retreive a result from query cache from the backend Store a result Send to the frontend in the query cache Copyright 2010-2011 Uptime Technologies, LLC. All rights reserved.

Implementation• Using KVS (Memcached) to hold the query cache. – “libmemcached” as a memcached client library for C. – A SQL statement as a key, and response messages as a value.• `pqcd’, a query cache daemon, is implemented as a derivative of pgpool. – Un-used feature (code) removed, and several “hook” routines added. Server side PostgreSQL PostgreSQL client pqcd backend(s) Hook routines libmemcached Memcached Copyright 2010-2011 Uptime Technologies, LLC. All rights reserved.

Deployment• Required packages – libevent 1.4.14b (required by memcached) – memcached 1.4.5 – libmemcached 0.43• How to install – automake – ./configure --prefix=$PREFIX – make – sudo make install – cd $PREFIX/etc – cp pqcd_hba.conf.sample pqcd_hba.conf• Starting pqcd – $PREFIX/bin/pqcd• Stopping pqcd – $PREFIX/bin/pqcd stop Copyright 2010-2011 Uptime Technologies, LLC. All rights reserved.

Deployment (with RPMs)• Required RPM – uqc-libevent-1.4.14b-1.i386.rpm – uqc-memcached-1.4.5-1.i386.rpm – uqc-libmemcached-0.43-1.i386.rpm – uqc-querycache-20110223-1.i386.rpm• Configuration files – /opt/uptime/querycache/etc/pqcd.conf – /opt/uptime/querycache/etc/pqcd_hba.conf• Starting pqcd – /opt/uptime/querycache/bin/pqcd• Stopping pqcd – /opt/uptime/querycache/bin/pqcd stop Copyright 2010-2011 Uptime Technologies, LLC. All rights reserved.

Configurations• `pqcd.conf’ may be configured as you need. (in $PREFIX/etc)• memcached_bin – A path name to Memcached executable. (default is “/opt/uptime/querycache/bin/memcached”)• query_cache_mode – The query cache mode. “active” or “passive”. (default is “active”)• query_cache_expiration – The timeout value for query cache expiration (in seconds, default is 30) Copyright 2010-2011 Uptime Technologies, LLC. All rights reserved.

Cache hints• Add as a comment at the beginning of SELECT statements – /* cache:refresh */SELECT * FROM … – <slash> <asterrisk> <space> <hint> <space> <asterisk> <slash>• cache:on (default in Active cache mode) – Looks at the cache first. If not found, then executes on the backend, and puts the result into the cache.• cache:off (default in Passive cache mode) – Doesn’t look at the cache. Execute on the backend first, and doesn’t put the result into the cache.• cache:refresh – Doesn’t look at the cache first. Executes on the backend, and puts the result into the cache.• cache:expire – Invalidates query cache for specified statement. (Not implemented yet)• cache:expireall – Invalidates all query cache. (Not implemented yet) Copyright 2010-2011 Uptime Technologies, LLC. All rights reserved.

TODO• To Do – Handle a transaction state correctly on “Z (ready for query)” response. – Make expandable cache values for >8192bytes. – Make enable to handle cache key with >250 charactors. (limitation from Memcached) – Automatic query caching by watching duration time under the Passive cache mode. – Run benchmarks!• Done – PreparedStatement. – Cache hints. – Rename the binary to ‘pqcd’ in Makefile – Start/stop Memcached from pqcd. – Invalidation query cache by timeout. (query_cache_expiration option) – Completes a pgbench run. – Remove V2 protocol code. (prevent to be conneced with V2 at startup) – `memcached_bin’ option added in pqcd.conf. – Put a connected database name into the cache key. – Remove un-used code (replication, master-slave, dual server under connection pooling) – “Keep it simple, stupid!” – Make it public as an opensource software! Copyright 2010-2011 Uptime Technologies, LLC. All rights reserved.

http://pgsnaga.blogspot.com	1390
http://pgsnaga.blogspot.in	56
http://pgsnaga.blogspot.co.uk	51
http://pgsnaga.blogspot.com.br	50
http://pgsnaga.blogspot.ca	43
http://pgsnaga.blogspot.fr	42
http://pgsnaga.blogspot.it	38
http://pgsnaga.blogspot.com.es	36
http://pgsnaga.blogspot.de	35
http://pgsnaga.blogspot.com.au	25
http://pgsnaga.blogspot.pt	24
http://pgsnaga.blogspot.nl	16
http://pgsnaga.blogspot.hu	15
http://pgsnaga.blogspot.jp	15
http://pgsnaga.blogspot.ru	14
http://pgsnaga.blogspot.kr	12
http://pgsnaga.blogspot.ro	9
http://pgsnaga.blogspot.co.nz	9
http://pgsnaga.blogspot.tw	8
http://pgsnaga.blogspot.be	7
http://pgsnaga.blogspot.co.il	7
http://paper.li	7
http://pgsnaga.blogspot.dk	7
http://pgsnaga.blogspot.se	6
http://pgsnaga.blogspot.fi	6
http://pgsnaga.blogspot.ch	5
http://pgsnaga.blogspot.hk	5
http://pgsnaga.blogspot.com.ar	5
http://pgsnaga.blogspot.mx	5
http://pgsnaga.blogspot.cz	5
http://translate.googleusercontent.com	5
http://pgsnaga.blogspot.gr	4
http://pgsnaga.blogspot.no	4
http://pgsnaga.blogspot.sg	2
http://pgsnaga.blogspot.co.at	2
https://p.yammer.com	1
http://webcache.googleusercontent.com	1
http://pgsnaga.blogspot.sk	1

PostgreSQL Query Cache - "pqc"

by Uptime Technologies LLC

Accessibility

Categories

Upload Details

Usage Rights

38 Embeds 1,973

Statistics

PostgreSQL Query Cache - "pqc" Presentation Transcript