The document discusses Scala technologies used in Dwango's account system. It focuses on three Scala programming techniques: 1) using continuation monads to build web applications in a modular way, 2) using the Minimal Cake Pattern for dependency injection in large codebases, and 3) introducing a new transaction monad called Fujitask that abstracts storage access and enables transaction composition through subtyping.

1. 1
Scala technology supporting
the Dwango account system
Seitaro Yuki / @pab_tech / DWANGO Co., Ltd.
ドワンゴアカウントシステムを支えるScala技術
結城清太郎/ @pab_tech / 株式会社ドワンゴ

2. 2
Overview
Dwango is often named as a leading company to adopt
Scala among the enterprises in Japan.
The account system is known to be largest Scala project
in our company, responsible for registering niconico users
and authenticating them.
In this session we will introduce a variety of Scala
programming techniques that are used in the account
system
ドワンゴのアカウントシステムで使われている
Scalaプログラミング技術について解説します

3. 3
Contents
Web applications by continuation monads
Dependency injection by Minimal Cake Pattern
New transaction monad using subtyping
継続モナド、Minimal Cake Pattern、トランザ
クションモナドについて解説します

4. 4
Web applications by continuation monads
継続モナドによるWebアプリケーション

5. 5
Technical problems:
How to construct web components
Web application needs to be decomposed to request
processing, response processing, filtering processing, etc.
Each component needs to be easy to understand and test
We want a simple and powerful component technology
課題: Webアプリケーションの部品化

6. 6
Simple and powerful component technology:
Monads
We will explain how to build a web application using
continuation monads
継続モナドを使ったWebアプリケーションの
構成方法を説明します

7. 7
Similarity of a typical web application and
continuation monads
まずは一般的なWebアプリケーションの構造
と継続モナドの類似性から見ていきましょう

8. 8
Typical web application structure
Typical web applications have nested structure like
onions
e.g. Java Servlet, Python WSGI, Ruby Rack
Middleware performs the following operations
It processes a request, and passes the result to the next
middleware
It modifies a response that came back, and returns the
result to the before middleware
Or, when an error occurs it returns the result without
calling the next middleware
典型的なWebアプリケーションの構造

9. 9
Typical web application structure
Web application
Web Browser Web Server Filter A Filter B Main Processing
request
request
request
request
response
response
response
response
典型的なWebアプリケーションの構造

10. 10
Authentication middleware
Let's consider authentication middleware
We assume that users logined already somewhere
Authentication middleware retrieves a session ID from
cookies, confirms using a storage such as Redis, passes
the result to the next middleware
Or, if there is no session ID or the session is invalid,
authentication middleware redirects to a login form
認証処理をおこなうミドルウェアについて考
えてみる

11. 11
Authentication middleware
Web Browser Web Server Authentication Filter Main Processing
request
request
alt ["Authentication is successful"]
request
request
response
response
response
["Authentication is failed"]
redirect
response
認証処理のシークエンス図

12. 12
Code this authentication
middleware using continuation
monad
では、この認証処理を継続モナドを使って書
いてみましょう

13. 13
What is continuation?
Continuation represents the rest of the
computation at a given point in the
process's execution Continuation from A
Continuation from B
B
R
Caller
A
Caller
プログラムの実行のある時点において評価さ
れていない残りのプログラムを継続と呼ぶ

14. 14
Continuation-passing style (CPS)
CPS is a style of programming in which control is passed a
continuation explicitly
Caller
Caller
A
A
B
B
R
R
pass a continuation after A and execute
pass a continuation after B and execute
pass a continuation after R and execute
execute the continuation
execute the continuation
execute the continuation
継続を明示的に受け渡しするプログラミング
スタイルを継続渡しスタイルと呼ぶ

15. 15
CPS and web applications are similar
Web application seems to be able to be written in CPS
Further if continuation is monad, it becomes a simple and
powerful component
CPSとWebアプリケーションは似ている
継続がモナドならさらによい

16. 16
Continuation monad
Ais a type of value at a given point, Ris a final result type
A=>Rrepresents a continuation
(A=>R)=>Rrepresents a function that take a
continuation
Cont[R,A]is a constructor that creates a monad from a
function that take a continuation
継続モナドは、継続を受け取る関数を受け取
って、モナドを作る
case class Cont[R, A](run: (A => R) => R) {
def map[B](f: A => B): Cont[R, B] =
Cont(k => run(a => k(f(a))))
def flatMap[B](f: A => Cont[R, B]): Cont[R, B] =
Cont(k => run(a => f(a).run(k)))
}

17. 17
Let's make web application using
continuation monads
継続モナドを使ってWebアプリケーションを
作ってみよう

18. 18
Authentication of continuation monad
継続モナドによる認証処理
def readSessionIdFromCookie(request: Request): Option[SessionId] = ???
def readSessionFromRedis(sessionId: SessionId): Option[Session] = ???
def redirectLoginForm: Response = ???
def auth(request: Request): Cont[Response, Session] =
Cont((k: Session => Response) =>
readSessionIdFromCookie(request)
.flatMap(readSessionFromRedis)
.map(k)
.getOrElse(redirectLoginForm)

19. 19
Web application using for-expression
for式を使ってWebアプリケーションを組み
立てる
def compress(request: Request): Cont[Response, Unit] = ???
def mainProcessing(request: Request, session: Session): Cont[Response,
def webApplication(request: Request): Cont[Response, Response] =
for {
session <- auth(request)
_ <- compress(request)
response <- mainProcessing(request, session)
} yield response
// execution
webApplication(request).run(identity) // => Response

20. 20
Let's look at Play Action composition
methods for comparison
比較のためにPlay標準のAction合成方法を見
てみよう

21. 21
Play Action composition methods
ActionFunction
def invokeBlock
ActionBuilder ActionRefiner
ActionFilter ActionTransformerAction
The methods of Play Action composision is performed by
functional composition
PlayのAction合成方法
ActionFunctionの合成によっておこなわれる

22. 22
Actually continuation also appears in Play
block:P[A]=>Future[Result]is continuation
But it was not a monad
実はPlayでも継続の概念が使われている
しかしモナドではなかった
trait ActionFunction[-R[_], +P[_]] {
def invokeBlock[A](request: R[A], block: P[A] => Future[Result]): Future
}

23. 23
Problems of Play Action composition
Functional composition of ActionFunctionis less flexible
than monadic composition
For example, AuthenticatedBuilderreturns
AuthenticatedRequest
Therefore, AuthenticatedBuilder can be composed to only
ActionFunctionreceives AuthenticatedRequest
PlayのAction合成の問題点
ActionFunctionの合成は柔軟性に欠ける

24. 24
Advantages of using a continuation monad
Continuation monad is simple
It can be represented by 6 lines
Monadic composition is more flexible rather than
functional composition
It is possible to combine with other kinds of monad by
using monad transformers
The account system uses ContT[Future,Result,A]
for error handling
継続モナドをWebアプリケーションの記述に
使う利点

25. 25
Dependency Injection by Minimal
Cake Pattern
Minimal Cake PatternによるDependency
Injection

26. 26
Technical problems:
Large-scale system development
The account system has 230,000 lines in only code of
Scala
For comparison, Play has 90,000 lines, Slick has 30,000
lines
How to modularize
How to test
課題: 大規模システム開発
23万行のScalaコードがある

27. 27
Let's solve these problems by the
dependency injection
Dependency Injectionでこれらの課題を解決
しよう

28. 28
DI technology that we have adopted
DI technology that we have adopted is defined as
"Minimal Cake Pattern"
We will explain the following contents
Comparison of various other DI technologies of Scala
What is difference from the existing Cake Pattern
Advantages for large-scale system development
ここでは我々が採用している"Minimal Cake
Pattern"というDI技術について解説をします

29. 29
Various methods of DI in Scala
DI Container such as Guice
Cake pattern
Reader monad
様々なScalaのDependency Injection方法

30. 30
DI Container such as Guice
Advantages
Experience in Java
Also adopted in Play
Dependencies in source code are eliminated
completely
Shortening of compile time
Disadvantages
DI container often causes a run-time error of object
creation
We should learn about the DI container
GuiceなどのDIコンテナを使う方法の利点と
欠点

31. 31
Cake pattern
Advantages
Checking at compile time
Frameworks such as DI container are not required
Disadvantages
Cake patterns would complicate a system as a
component technology
Increase boilerplates
Cakeパターンを使う方法の利点と欠点

32. 32
Reader monad
Advantages
Same as Cake Pattern
Disadvantages
It will be full of reader monads in everywhere
It is difficult to manage a large number of
dependencies
It is sensitive to change such as IO monads in Haskell
We conclude that reader monads should not be used
for DI
Readerモナドを使う方法の利点と欠点

33. 33
Which is good for large-scale system
development?
大規模システム開発ならどの方法がいいの
か？

34. 34
Rethink of Cake Pattern
Cake Pattern is a proposed component technique in
"Scalable Component Abstractions" by Odersky's et al.
But, there are some problems as a component technique
By using an abstract type, types are likely to become
too complicated
Cakes are likely to be tightly coupled to each other
We will consider Cake Pattern not as a component
technique
コンポーネント技術としてのCake Patternの
問題点

35. 35
Mimimal Cake Pattern
Do not use advanced features such as abstract types
Provide only one field variable
Do not nest class
Define the Cake Pattern with such features as "Mimimal
Cake Pattern"
Mimimal Cake Patternとはコンポーネント技
術ではないCake Patternである

36. 36
Example of not use DI
まずはDIを使わない例
// Interface
trait Repository
// Implementation
object RepositoryImpl extends Repository
// Client
object Service {
// ↓Here's the problem referring to the Implementation
val repository: Repository = RepositoryImpl
}

37. 37
Example of Mimimal Cake Pattern (Interface)
Add Usestrait to declare interface
Usestrait just provides one field variable
Do not use abstract types and nested classes
Interfaceの例
Usesトレイトを追加する
trait UsesRepository {
val repository: Repository
}
trait Repository

38. 38
Example of Mimimal Cake Pattern
(Implementation)
Add MixIntrait to provide the implementation
実装の例
MixInトレイトを追加する
trait MixInRepository extends UsesRepository {
val repository = repositoryImpl
}
object RepositoryImpl extends Repository

39. 39
Example of Mimimal Cake Pattern(Client)
Extend UsesRepositoryfor using Repository
Use MixInRepositoryin MixInService for providing
Service implementation
We can replace the implementation here
UsesとMixInを使ったServiceの例
ServiceもMixInトレイトを提供する
trait Service extends UsesRepository
trait MixInService extends UsesService {
val service = new Service with MixInRepository
}

40. 40
Development style using Minimal Cake
Pattern
Our development style is based on Minimal Cake Pattern
Interface, Usestrait, implementation, MixIntraits and
test form a development unit
We should write a comment for all methods in the
interface, including all error values
Tests should include all error cases in the interface
Only when there are all present, the code is ready to be
reviewed
Mimimal Cake Patternによる開発スタイル

41. 41
Advantage of Minimal Cake Pattern
The account system consists of 400 or more modules
We need to simplify cake pattern by discarding extra
features not related to DI
Static checking is desirable in large-scale system
If we use DI container, it should be verified by starting
the system each time
DI has established our development style
We want to re-evaluate Cake Pattern by limiting only to DI
Mimimal Cake Patternの利点の説明
DIだけのCake Patternを再評価したい

42. 42
New transaction monad using subtyping
サブタイピングを使った新たなトランザクシ
ョンモナド

43. 43
Technical problems:
Abstraction of storage access
We got the power to abstract components by DI
We want to do even abstraction of storage access, such as
MySQL and Redis
How to abstract transaction objects of the infrastructure
layer, such as DB session
How to compose transactions
We want to facilitate a programming of Master/Slave
structure
技術的な課題: ストレージアクセスの抽象化

44. 44
We will introduce a new transaction monad
to solve these problems
これらの問題を解決する新しいトランザクシ
ョンモナドを紹介します

45. 45
Transaction monad: Fujitask
Fujitask enables abstraction of storage access and
composition of transactions
Fujitask treats transaction objects in the same way as a
reader monad
Fujitask determines whether the composed monad will
query Master or Slave by using subtyping
トランザクションモナド: Fujitask

46. 46
Common problems of the Master / Slave
structure
In programming for Master/Slave structure, often we
should explicitly specify whether each code will query
Master or Slave
A run-time errors when send a query of update to Slave
It is hard to notice that we send a query that should be
sent to Slave to Master because no error occurs
Fujitask will solve the these problems
Master/Slave構成のよくある問題
Fujitaskはこのような問題を解決します

47. 47
Basic parts of Fujitask
Fujitaskの基本部分
trait Task[-R, +A] { lhs =>
// To be implemented
def execute(r: R)(implicit ec: ExecutionContext): Future[A]
def flatMap[S <: R, B](f: A => Task[S, B]): Task[S, B] = ???
def map[B](f: A => B): Task[R, B] = ???
def run[S <: R]()(implicit runner: TaskRunner[S]): Future[A] = ???
}
trait TaskRunner[R] {
// To be implemented
def run[A](task: Task[R, A]): Future[A]
}

48. 48
How to use Fujitask
We'll explain how to use the Fujitask through the
implementation of ScalikeJDBC
ScalikeJDBC版のFujitaskの実装を通じて
Fujitaskの使い方を説明します

49. 49
Define types of transaction objects
Define kinds of transaction
These are assigned to a left type variable of Task[R,A]
The important point is that ReadWriteTransaction inherits
ReadTransaction
トランザクションオブジェクトを定義する
ReadWriteがReadを継承するのがポイント
trait Transaction
trait ReadTransaction extends Transaction
trait ReadWriteTransaction extends ReadTransaction

50. 50
Implement transaction objects of
ScalikeJDBC
These classes inherit Transactionand wrap DBSession of
ScalikeJDBC
ScalikeJDBCのトランザクションオブジェク
トの型を定義する
abstract class ScalikeJDBCTransaction(val session: DBSession)
class ScalikeJDBCReadTransaction(session: DBSession)
extends ScalikeJDBCTransaction(session)
with ReadTransaction
class ScalikeJDBCReadWriteTransaction(session: DBSession)
extends ScalikeJDBCTransaction(session)
with ReadWriteTransaction

51. 51
Implement TaskRunner of
ScalikeJDBC (1)
Define askmethod to get the transaction object
Derived from the ask function of Reader monad
ScalikeJDBC版のTaskRunnerを実装する
askメソッドを定義する
package object scalikejdbc {
def ask: Task[Transaction, DBSession] =
new Task[Transaction, DBSession] {
def execute(transaction: Transaction)
(implicit ec: ExecutionContext): Future[DBSession] =
Future.successful(transaction.asInstanceOf[ScalikeJDBCTransaction
}
}

52. 52
Implement TaskRunner of
ScalikeJDBC (2)
Define TaskRunnerfor ReadTransaction
ReadTransaction用のTaskRunnerを定義する
package object scalikejdbc {
implicit def readRunner[R >: ReadTransaction] : TaskRunner[R] =
new TaskRunner[R] {
def run[A](task: Task[R, A]): Future[A] = {
val session = DB.readOnlySession()
val future = task.execute(new ScalikeJDBCReadTransaction(session))
future.onComplete(_ => session.close())
future
}
}
}

53. 53
Implement TaskRunner of
ScalikeJDBC (3)
Define TaskRunnerfor ReadWriteTransaction
ReadWriteTransaction用のTaskRunnerを定義
する
package object scalikejdbc {
implicit def readWriteRunner[R >: ReadWriteTransaction]: TaskRunner[
new TaskRunner[R] {
def run[A](task: Task[R, A]): Future[A] = {
DB.futureLocalTx(session =>
task.execute(new ScalikeJDBCReadWriteTransaction(session)))
}
}
}

54. 54
Now, we can use Fujitask of
ScalikeJDBC
TaskRunnercan be implemented for another DB library
and Redis client in the same way
Next, we create domain code using this Fujitask of
ScalikeJDBC
これでScalikeJDBCのFujitaskを使うことがで
きるようになりました

55. 55
Define an interface of Repository
Repositoryのインターフェースを定義する
trait UserRepository {
def create(name: String): Task[ReadWriteTransaction, User]
def read(id: Long): Task[ReadTransaction, Option[User]]
}

56. 56
Implement Repository using ScalikeJDBC
Fujitask
ScalikeJDBC版Fujitaskを使ったRepositoryの
実装
object UserRepositoryImpl extends UserRepository {
def create(name: String): Task[ReadWriteTransaction, User] =
ask.map { implicit session =>
val sql = sql"""insert into users (name) values ($name)"""
val id = sql.updateAndReturnGeneratedKey.apply()
User(id, name)
}
def read(id: Long): Task[ReadTransaction, Option[User]] =
ask.map { implicit session =>
val sql = sql"""select * from users where id = $id"""
sql.map(rs => User(rs.long("id"), rs.string("name"))).single.apply()
}
}

57. 57
Implement Service using Repository
Fujitaskで作ったRepositoryを使ってみる
trait MessageService
extends UsesUserRepository
with UsesMessageRepository {
def createByUserId(message: String, userId: Long): Future[Message] = {
val task =
for {
userOpt <- userRepository.read(userId)
user = userOpt.getOrElse(
throw new IllegalArgumentException("User Not Found"))
message <- messageRepository.create(message, user.name)
} yield message
task.run()
}
}

58. 58
How Fujitask works
Transactions are combined by for-expression
userRepository.readhas ReadTransactionand
messageRepository.createhas ReadWritetransaction,
then the combined monad has ReadWritetransactionby
subtyping
And, when task.runis invoked, an appropriate
TaskRunneris selected by implicit parameter mechanism
Fujitaskはどのように動作するか

59. 59
The advantage of Fujitask
Storage access can be abstracted by Fujitask
We could compose transactions by Fujitask
Master/Slave structure Programming is made easier
Fujitaskの利点

60. 60
Sharding by Fujitask
Use ShardedTaskhaving a shard key instead of the Task if
we want sharding
Type of shard key is checked statically by ShardedTask
Also, the value of the shard key is checked at run time
whether the same in flatMap
Fujitaskによるシャーディング
abstract class ShardedTask[Key, -R, +A](val shardKey: Key)

61. 61
Conclusion
総括

62. 62
Let's consider the commonalities
among these techniques
最後に今回説明した技術の共通点を考えたい

63. 63
Simple
Continuation monad can be represented by 6 lines
Minimal Cake Pattern is the minimized Cake Pattern for DI
Even though Fujitask has a variety of storage access
functions, it can be written in about 30 lines
今回我々が紹介した技術はどれもシンプルで
ある

64. 64
Use new features introduced in Scala
Continuation monad can be briefly composed by Scala
monads syntax
Also, It can be combined with other monads in monad
transformers by higher kinded types
Minimal Cake Pattern uses a mixin of traits
Fujitask uses variances and ad hoc polymorphism by
implicit parameter
どれもScalaで新しく導入された機能を使っ
ている

65. 65
Without frameworks
Web components are implemented by web framework in
other languages
We have proposed a method to use continuation
monad
Dependency Injection has been implemented by DI
container in Java
We can use cake pattern instead of DI container
We would have to use AOP If we try to implement the
functions of the Fujitask in other languages
我々の手法ではフレームワークは必要ではな
い

66. 66
The Dwango account system is supported by
the advanced features of Scala
Thanks to the advanced features of Scala, our system
became more simple, more understandable and more
convenient
ドワンゴアカウントシステムはScalaの高度
な言語機能に支えられている

67. 67
So, we want to conclude this session that ...
よって、このセッションをこう締め括りたい

68. 68
We will continue to use the advanced
languages like Scala
我々はScalaのような進歩した言語を使って
いきたい

69. 69
And, we hope that programming languages
continue to make progress
そして、これからもプログラミング言語は進
歩しつづけてほしい

70. 70
Thank you for listening
ご清聴ありがとうございました