DrupalCamp Japan 2017 in Tokyo (http://2017camp.drupaljapan.org/ ) 「大規模サイトにおけるユーザーレベルのキャッシュ活用によるパフォーマンスチューニング」のスライドです。 HTMLバージョンは https://blauerberg.github.io/slides/drupalcamp_2017_tokyo/#/ です。 I talked about performance tuning for large scale system by user level cache management. HTML version is here: https://blauerberg.github.io/slides/drupalcamp_2017_tokyo/#/

1. ⼤規模サイトにおけるユーザーレ
ベルのキャッシュ活⽤による
パフォーマンスチューニング
Drupal Camp Japan 2017 in Tokyo
Yoshikazu Aoyama
ANNAI Inc

2. ⻘⼭ 義万
(あおやま よしかず)
About me
Facebook:
Twitter:
Drupal.org:
ANNAI Inc
Drupal Japan User Group
Drupalさっぽろ
https://www.facebook.com/yoshikazu.aoyama
@blauerberg
blauerberg

4. Experience
Drupal developer (4.x〜)
Drupal contributor (Core, Views, Console, etc)
Maintainer of Drupal Console (Japanese Document)
Internet of Things
Web application (Ruby on Rails, Java, Node.js, etc)
Cloud/Server side design (Infrastructure as Code, DevOps)
Embedded Linux
TCP/IP Network stack
Network stack for telecommunication network (ATM, xDSL, 3G,
3.5G)

5. ANNAI株式会社について
ANNAIは2007年からDrupal専⾨のWebシステム開発会社として、世界
規模で展開するグローバル企業や⼤学・⾃治体を中⼼に、数多くの
Webソリューションを提供してきました。
Drupal CoreやModuleのコントリビューターなど、Drupalエキスパー
トが多数在籍しており、Drupalの国内ユーザーコミュニティの代表と
して、定期的な勉強会の主催やセミナーの開催を通じて、オープンソー
ス技術の衆知や普及、さらに海外Drupalコミュニティとの緊密な連携
を図っています。
https://annai.co.jp

6. Drupal 8 スタートブック
2017/1/20 発売予定！
https://www.amazon.co.jp/dp/4295000582

8. Topics
About Drupal cache
Usecase of large scale system
For Drupal 8

9. About Drupal cache

10. Provided by core
11 cache tables (D7)
cache
cache_block
cache_boostrap
cache_field
cache_filter
cache_form
cache_image
cache_menu
cache_page
cache_path
cache_update

11. Provided by contribute
modules
cache_apachesolr
cache_libraries
cache_location
cache_panels
cache_rules
cache_tmgmt
cache_variable
cache_views
cache_views_data
etc..

12. Structure of cache table
MariaDB [drupal]> describe cache;
+------------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+------------+--------------+------+-----+---------+-------+
| cid | varchar(255) | NO | PRI | | |
| data | longblob | YES | | NULL | |
| expire | int(11) | NO | MUL | 0 | |
| created | int(11) | NO | | 0 | |
| serialized | smallint(6) | NO | | 0 | |
+------------+--------------+------+-----+---------+-------+
5 rows in set (0.00 sec)

13. Structure of cache table
MariaDB [drupal]> describe cache_page;
+------------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+------------+--------------+------+-----+---------+-------+
| cid | varchar(255) | NO | PRI | | |
| data | longblob | YES | | NULL | |
| expire | int(11) | NO | MUL | 0 | |
| created | int(11) | NO | | 0 | |
| serialized | smallint(6) | NO | | 0 | |
+------------+--------------+------+-----+---------+-------+
5 rows in set (0.00 sec)

14. Structure of cache table
MariaDB [drupal]> describe cache_views;
+------------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+------------+--------------+------+-----+---------+-------+
| cid | varchar(255) | NO | PRI | | |
| data | longblob | YES | | NULL | |
| expire | int(11) | NO | MUL | 0 | |
| created | int(11) | NO | | 0 | |
| serialized | smallint(6) | NO | | 0 | |
+------------+--------------+------+-----+---------+-------+
5 rows in set (0.00 sec)

15. All of caches has same
structure and can access by
same API.

16. cache_get, cache_set
cache_get($cid, $bin = 'cache')
cache_set($cid, $data, $bin = 'cache', $expire = CACHE_PERMANENT)
https://api.drupal.org/api/drupal/includes%21cache.inc/function/cache_get/7.x
https://api.drupal.org/api/drupal/includes%21cache.inc/function/cache_set/7.x

17. Create a custom cache table
/**
* Implements of hook_schema().
*/
function custom_module_scheme() {
$schema = [];
$schema['cache_custom'] = drupal_get_schema_unprocessed('system', 'cache');
return $scheme;
}
https://api.drupal.org/api/drupal/modules%22system%21system.api.php/function/hook_schema/7.x
https://api.drupal.org/api/drupal/includes!common.inc/function/drupal_get_schema_unprocessed/7.x

18. Structure of custom cache
table
MariaDB [drupal]> describe cache_custom;
+------------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+------------+--------------+------+-----+---------+-------+
| cid | varchar(255) | NO | PRI | | |
| data | longblob | YES | | NULL | |
| expire | int(11) | NO | MUL | 0 | |
| created | int(11) | NO | | 0 | |
| serialized | smallint(6) | NO | | 0 | |
+------------+--------------+------+-----+---------+-------+
5 rows in set (0.00 sec)

19. cidにユーザーのコンテキストが特定できる識別⼦を組み込むだけで、
ユーザー単位のキャッシュが管理できる
function respond_contents() {
global $user;
$cid = "{$user->uid}-SOME-CACHE-HASH-STRING";
$cache = cache_get($cid, 'cache_custom');
if (is_null($cache) || is_expire($cache)) {
$generated_contents = create_some_user_specific_contents();
cache_set($cid, serialize($generated_contents), 'cache_custom', time() + 600);
return $generated_contents;
}
return $cache;
}

20. Usecase of large scale system

21. Requirements of our system
Multilingual (25+ countries)
Fast response to access from all over the world
More than 1,000,000 PV/month
More than 100,000 authenticated users
Many user/role specific contents, almost user logged in to use the
system
Access log for every user
etc..

22. User specific contents?

26. ユーザー特有のコンテンツが多い
とHTML全体をキャッシュするこ
とは難しい

27. では、このようなユースケースで
はCDNなどのキャッシュ機構は使
えないのか？

28. Lazy Loading
Lazy loading is a design pattern commonly used in
computer programming to defer initialization of
an object until the point at which it is needed. It
can contribute to efficiency in the program's
operation if properly and appropriately used. The
opposite of lazy loading is eager loading.
https://en.wikipedia.org/wiki/Lazy_loading

30. Cache strategy
CDN deriver contents from the edge server near the user.
CDN won't not always return a completed page. User specific
contents will be loaded lazy by Drupal (parallelly & asynchronous),
and then derivery and update partially.
Drupal try to return user specific contents from internal cache that
is managed per user or role.
If internal cache doesn't exists, Drupal generate these contents
and set contents to the internal cache to response faster in next
time.

31. System structure

32. if CDN has cached content

33. if Varnish has cached content

34. if internal cache has cached content

35. no cache

36. Lazy loading user specific contents.

37. Implementation
Per user/role cache control
Custom drush command for cache management
Lazy loadable all elements (menu, sidebar, main content, etc..)

38. For Drupal 8

39. Can we use this architecture in
Drupal 8 ?
you can implement more easily using by
awesome Cache API.
https://www.drupal.org/docs/8/api/cache-api

40. Cache Contexts
複数のコンテキストがcoreで定義され、独⾃のコンテキストの定義も
可能
cookies
:name
headers
:name
ip
languages
:type
...
session
.exists
...
user
.is_super_user
.node_grants
:operation
.permissions
.roles
:role

41. core.services.yml
...
cache_context.user:
class: DrupalCoreCacheContextUserCacheContext
arguments: ['@current_user']
tags:
- { name: cache.context}
cache_context.user.permissions:
...

42. UserCacheContext.php
class UserCacheContext extends UserCacheContextBase implements CacheContextInterface {
/**
* {@inheritdoc}
*/
public static function getLabel() {
return t('User');
}
/**
* {@inheritdoc}
*/
public function getContext() {
return $this->user->id();
}
/**
* {@inheritdoc}
*/
public function getCacheableMetadata() {
return new CacheableMetadata();
}
}

43. DatabaseBackend.php
class DatabaseBackend implements CacheBackendInterface {
...
public function get($cid, $allow_invalid = FALSE) {
$cids = array($cid);
$cache = $this->getMultiple($cids, $allow_invalid);
return reset($cache);
}
public function getMultiple(&$cids, $allow_invalid = FALSE) {
$cid_mapping = array();
foreach ($cids as $cid) {
$cid_mapping[$this->normalizeCid($cid)] = $cid;
}
$result = array();
try {
$result = $this->connection->query('SELECT cid, data, created, expire, serialized, tags, checksum FROM {' . $this->connection->escapeTable($this->bin) . '} WHERE
}
...
return $cache;
}
...

44. Cache API
D7同様のバックエンドに依存しないシンプルで統⼀されたAPI
public CacheBackendInterface::get($cid, $allow_invalid = FALSE)
public CacheBackendInterface::set($cid, $data, $expire = Cache::PERMANENT, array $tags = array())

45. Cache tag
キャッシュタグにより簡単にコントロール可能に
ex. "user:1", "role:admin", "node:10"
Cache tags = data dependencies
Cache tags describe dependencies on data
managed by Drupal

46. ANNAIでは⼀緒に働く仲間を募集
中です！

47. リモートワーク可能です！

48. Contact us
https://annai.co.jp/recruit

49. Thank you!