SlideShare a Scribd company logo
1 of 74
Download to read offline
7 ways
to make your Scala
RedHot high velocity
x1(Yuri Inoue)
For ScalaMatsuri 2016
あなたのScalaを爆速にする7つの方法
Yuri Inoue
Cyberagent,Inc.
AdTech Studio, AMoAd.
twitter: @iyunoriue
GitHub: x1-
HP: Batsuichi and Inken’s engineer blog
http://x1.inkenkun.com/
profile
井上ゆりと申します。
• Parallel execution is not taken into account.
• This session is for beginners of Scala.
• The environment of benchmark is shown at next page.
• @Keen advised some performances. Special thanks!
acknowledgement
謝辞。並列実行に関しては考慮していません。
environment of benchmark
instance Amazon EC2 m3.2xlarge
vCPU 8
memory 30GB
disk generalized SSD
OS CentOS6.7-Final
jdk jdk1.8.0u65(oracle, server vm)
scala 2.11.7
build tool sbt
benchmark tool sbt-jmh 0.2.5
libraries
com.bionicspirit.shade:1.6.0
net.debasishg.redisclient:3.0
org.scalatest.scalatest:2.2.4
ベンチマーク環境
No.1 Random Read
- File vs KVS -
Q1
Search for some string
in 2GB data.
Which of the following
would be faster?
2GBデータから文字列を探します。どちらが速いでしょう?
from 2GB file
on SSD.
A
from total 2GB
strings
on memcache.
(divided into chunk)
B
prerequisite prerequisite
* SSD is generalized
SSD on EC2.
* not using RamDisk.
* memcache version
is 1.4.0.
* memcache stand up
on local
A:SSD上の2GBファイル B:memcache上の2GBデータ
Answer
A. from 2GB file on SSD.
答え A.SSD上に置いた2GBのファイルから探す です。
benchmark
The figure shows the average for the times it took
to search for the string ”SUIKA” from the following
2GB data.
• Searching from a file completed in less than 8sec.
• Searching from memcache completed in near by 19sec.
• Searching from Redis(also KVS) completed in 14sec.
'''<code>&amp;h</code>''' を用い、<code>&amp;h0F</code> (十進で15)のように表現する。
nn[[Standard Generalized Markup Language|SGML]]、[[Extensible Markup Language|XML]]、
[[HyperText Markup Language|HTML]]では、アンパサンドをSUIKA使って[[SGML実体]]を参照する。
SUIKAという文字列を探したときの平均タイムです。
Why is file faster than memcache?
• This is Memory Mapped File(MMF) power.
• Memory Mapped File is used to map a file on disk to a
region of address space on memory.
• It avoids unnecessary file I/O operations and file
buffering.
• MappedByteBuffer included java.nio, that directly
access Memory Mapped File.
physical
file
mapping address on memory
fileが高速なのはMemoryMappedFileを使ったからです。
• memcache(<1.4.2) has limitation below,
The maximum size of a value you can store in memcached is 1MB.
• Although memcache is on the same host, accessing it
many times to retrieve data takes long time.
• Redis can save 1GB per 1 key. Using 2GB data divided
into 4 key(each 500MB), got below benchmark.
• It completed in about 8sec, but cannot win file.
Why is memcache slower than file?
memcache(<1.4.2)は1key<=1MBのサイズ制限があります。
• Memory Mapped File can only map up to 2GB file on JVM.
http://stackoverflow.com/questions/8076472/why-does-
filechannel-map-take-up-to-integer-max-value-of-data
• Apache Spark MLlib development supervisor, Reynold Xin
wrote the following Gist.
He measured the performance over various approaches to
read ByteBuffer in Scala.
https://gist.github.com/rxin/5087680
more information
jvmではMemoryMappedFileは2GBまでしか扱えません。
RedHot high velocity No.1
Using Memory Mapped File,
you can operate on files
at high speed.
爆速その1. MMFで高速ファイル操作が可能に!
No.2 for comprehension
vs
flatMap & map
Q2
for comprehension behaves
same as flatMap & map.
Which one is faster than?
for内包表記とflatMap&map、どちらが速いでしょう?
A B
code
same. flatMap & map
for {

a <- data

b <- a

} yield {

b

}
for comprehension
data.flatMap( a =>
a.map( b => b )
)
flatMap & map
Answer
A. same.
答え A.同じ です。
benchmark
All of Throughput、Average、Sample don’t show
significant difference between for comprehension and
flatMap & map.
10,000times
for内包表記とflatMap&mapで優位差が見られません。
• for comprehension and flatMap & map are logically same.
• Here is a comparison after decompiling them. They are
same!
Why are they the same speed?
for内包表記とflatMap&mapは論理的に同じです。
public Option<String> forComprehension()

{

data().flatMap(new AbstractFunction1()

{

public static final long serialVersionUID = 0L;



public final Option<String> apply(Some<String> a)

{

a.map(new AbstractFunction1()

{

public static final long serialVersionUID = 0L;



public final String apply(String b)

{

return b;

}

});

}

});

}
public Option<String> flatMapAndMap()

{

data().flatMap(new AbstractFunction1()

{

public static final long serialVersionUID = 0L;



public final Option<String> apply(Some<String> a)

{

a.map(new AbstractFunction1()

{

public static final long serialVersionUID = 0L;



public final String apply(String b)

{

return b;

}

});

}

});

}
for comprehension flatMap & map
RedHot high velocity No.2
for comprehension and flatMap & map
are same, level of byte code.
爆速その2. for内包表記とflatMap&mapは同じです。
No.3 append & insert
- collection -
Collections Performance Characteristics at Scala
cite: http://docs.scala-lang.org/overviews/collections/performance-characteristics.html
Scalaにおけるコレクションの性能特性
Q3-1
Mutable variable
“var xs: Vector”
and
Immutable variable
“val xs: ArrayBuffer”
Which one is faster, when appending
to the tail of a collection?
可変なVectorと不変なArrayBuffer、末尾追加が速いのは?
var Vector
A
val ArrayBuffer
B
code
var xs = Vector.empty[Int]
xs = xs :+ a
var xs: Vector val xs: ArrayBuffer
val xs = ArrayBuffer.empty[Int]
xs += a
Answer
B. val ArrayBuffer
答え B.val ArrayBufferです。
benchmark
ArrayBuffer is faster than Vector.
ArrayBufferの方がVectorよりも速いです。
Throughput of appending n times
This benchmark shows throughputs of appending N elements.
For example, type:Vector and times:10k indicates appending 10,000
elements in an empty Vector.
benchmark
The benchmarks of the other immutable objects
are below.
VectorはListやStreamよりは速いのですが。
Throughput of appending n times
Vector is faster than List and Stream as same as
immutable.
Why is ArrayBuffer faster than Vector?
When appending new element ...
Vectorは新インスタンスを新たにつくるので遅くなります。
add new element,
after coping elements to
new instance.
update tail position, after
resizing instance.
var Vector val ArrayBuffer
val b = bf(repr)

b ++= thisCollection

b += elem

b.result()
ensureSize(size0 + 1)

array(size0) =
elem.asInstanceOf[AnyRef]

size0 += 1

this
Thease processes are absolutely different.
Q3-2
Mutable variable
“var xs: List”
and
Immutable variable
“val xs: ListBuffer”
Which one is faster, when inserting
into the head of a collection?
可変なListと不変なListBuffer、先頭挿入が速いのは?
var List
A
val ListBuffer
B
code
var xs = List.empty[Int]
xs = a :: xs
var xs: List val xs: ListBuffer
val xs = ListBuffer.empty[Int]
a +=: xs
Answer
A. var List
答え A.var Listです。
benchmark
List is faster than ListBuffer.
Listの方がListBufferよりも速いです。
Throughput of inserting n times
This benchmark shows throughputs of inserting N elements.
For example, type:List and times:1k indicates inserting 1,000 elements
in an empty List.
benchmark
The benchmarks of the other objects are below.
Listだけ飛び抜けて速いです。
List is enormously faster than the others.
Throughput of inserting n times
Why is List faster than the others?
When inserting new element into the begining position...
Listではほとんど計算せずに先頭挿入が行えます。
Because of having head and
tail, List can create new
instance immediately.
ListBuffer is almost same
as List, but inner variable
is reassigned to.
var List val ListBuffer
new
scala.collection.immutable
.::(x, this)
if (exported) copy()

val newElem = new :: (x, start)

if (isEmpty) last0 = newElem

start = newElem

len += 1

this
List calculate a little, in case of inserting.
RedHot high velocity No.3
When appending elements, using ArrayBuffer
or ListBuffer is a better way.
But inserting the begining position, using
List works best performance.
爆速その3.末尾追加は**Bufferを!先頭挿入はListを!
No.4 remove
- collection -
Q4
Mutable variable
“var xs: List”
and
Immutable variable
“val xs: ListBuffer”
Which one is faster, when removing
an item from a collection?
可変なListと不変なListBuffer、削除が速いのは?
var List
A
val ListBuffer
B
code
var xs: List val xs: ListBuffer
var xs =
List( 1 to n: _* )
// head
xs = xs.tail
// tail
xs = xs.dropRight(0)
var xs =
ListBuffer( 1 to n: _* )
// head
xs.remove(0)
// tail
xs.remove(xs.size - 1)
Answer
B. val ListBuffer
答え B.val ListBuffer です。
benchmark
ListBuffer is much faster than List.
ListBufferの方がListよりも相当速いです。
Throughput of removing n times
This benchmark shows throughputs of removing N elements from N length array.
For example, Benchmark:removeTail, type:List and times:1k indicates removing
1,000 elements from the last of 1,000 length List.
benchmark
The benchmarks of the other objects are below.
Buffer系以外はどれも超絶遅いです。
They are all very slow, except for the Buffer family.
Throughput of removing n times
Why is ListBuffer faster than List?
When remove element from collection...
ListのdropRightはO(n)の時間がかかります。
The operation - dropRight
of List takes time O(n).
The operation - remove of
ListBuffer takes constant
time.
var List val ListBuffer
def dropRight(n: Int): Repr = {

val b = newBuilder

var these = this

var lead = this drop n

while (!lead.isEmpty) {

b += these.head

these = these.tail

lead = lead.tail

}

b.result()

}
def remove(n: Int): A = {
:
var cursor = start

var i = 1

while (i < n) {

cursor = cursor.tail

i += 1

}

old = cursor.tail.head

if (last0 eq cursor.tail) last0 =
cursor.asInstanceOf[::[A]]

cursor.asInstanceOf[::[A]].tl
= cursor.tail.tail
benchmark of List’s dropRight
Because the benchmark has increased by 10 times the size,
we have to display the throughput logarithmic graph of.
dropRightのスループットは線形に下降しているだけです。
Throughput is just only lowered to linear.
It is the horror of linear increase.
log(Throughput) of List’s dropRight
reference benchmark
参考のためtakeと比較しました。takeは少し速いです。
Since both of dropRight(1) and take(n-1) are same
function, so compared with dropRight and take.
take is slightly faster than dropRight.
Throughput of List’ dropRight and take
RedHot high velocity No.4
When dropping elements,
using ListBuffer or ArrayBuffer
is a better way.
爆速その4. 要素削除を行うならListBuffer or ArrayBuffer。
No.5 random read
- collection -
Q5
Vector vs ListBuffer
Which one is faster,
when reading randomly?
VectorとListBuffer, ランダム・リードが速いのは?
Vector
A
ListBuffer
B
code
val data =
Vector( 1 to n: _* )



( 1 to data.size ) map { _ =>
data( Random.nextInt( n ) )
}
val data =
ListBuffer( 1 to n: _* )



( 1 to data.size ) map { _ =>
data( Random.nextInt( n ) )
}
Answer
A. Vector
答え A.Vectorが速いです。
benchmark
Vector is faster than ListBuffer.
Vectorの方がListBufferより速いです。
Throughput of reading n times
This benchmark shows throughputs of getting an item randomly from N
length array.
For example, type:ListBuffer and times:1k indicates getting an item
1,000 times from 1,000 length List.
benchmark
The benchmarks of the other objects are below.
速いのはArray, ArrayBuffer, Vectorです。
Array, ArrayBuffer, Vector are fast.
Throughput of reading n times
Why Array, ArrayBuffer, Vector are fast?
Vectorなどは内部的に定数時間のArrayを使っています。
• Array - random read takes constant time.
• ArrayBuffer and Vector have Array internal.
protected var array: Array[AnyRef]
= new Array[AnyRef](math.max(initialSize, 1))
:
def apply(idx: Int) = {

if (idx >= size0) throw new IndexOutOfBoundsException(idx.toString)

array(idx).asInstanceOf[A]

}
e.g ArrayBuffer
RedHot high velocity No.5
It is a cool way,
using Array family when
reading randomly.
爆速その5. Arrayの仲間はランダム・リードが速いです。
No.6 fibonacci sequence
Q6
Considering the above,
Stream vs Array
Which one is faster,
producing
a fibonacci sequence?
フィボナッチ数列の生成が速いのはStream?Array?
Stream
A
Array
B
code
def fibonacci(
h: Int = 1,
n: Int = 1 ): Stream[Int] =

h #:: fibonacci( n, h + n )
val fibo = fibonacci().take( n )
def fibonacci( n: Int = 1 ): Array[Int] =
if ( n == 0 ) {

Array.empty[Int]

} else {

val b = new Array[Int](n)

b(0) = 1

for ( i <- 0 until n - 1 ) {

val n1 = if ( i == 0 ) 0 else b( i - 1 )

val n2 = b( i )

b( i + 1 ) = n1 + n2

}

b

}
val fibo = fibonacci( n )
* calculate recursively * operation takes O(n)
Answer
A. Stream.
答え A.Stream です。
benchmark
Stream is overwhelmingly faster than Array.
Streamが圧倒的に速いです。
Throughput of creating n length fibonacci sequence
fibonacci sequence: ( 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, .. )
Why is Stream so much faster?
Streamは遅延評価なので必要なときに計算します。
• Stream implements lazy lists where elements are only
evaluated when they are needed.
• Actually, these have not yet been calculated.
• This is the power of lazy evaluation.
• But, if the sequence is materialized, it takes
calculating time.
→ see next page.
Why is Stream so much faster?
ただしStreamは具現化するときにコストがかかります。
This is the benchmark when called toList on Stream.
Array is fastest, if it calls toList on Stream.
It shows calculation method using Array is better than
recursive.
Throughput of creating n length fibonacci sequence
RedHot high velocity No.6
Lazy evaluation is very useful.
But, it takes cost at materialized.
爆速その6. 遅延評価は便利ですが具現化すると普通です。
No.7 regular expression
Showing the benchmark of w+w
Regular Expression consumes CPU resource a lot.
In particular, back tracking will occur using the
expression like “ww..w”, so takes time O(n^2).
正規表現はCPUリソースをたくさん消費します。
The following is the performance of Regular Expression.
Throughput of each regular expression execution(1,000 times)
Last. Q7
When looking for behind the specific
expression(w+/),
“findAllIn and look behind(?<=)”
and
“findPrefixOf and quantifier(+)”
which one is faster than?
from this string→ abcdef..0123../abc.. (1024byte)
ある文字列の後方を探したいです。どちらが速いでしょう?
same.
A
findPrefixOf
B
code
val re = """(?<=w+)/.+""" r
val ms = re findAllIn data

if ( ms.isEmpty ) None

else Some( ms.next )
val re = “""w+/""" r

re findPrefixMatchOf data map(
_.after.toString
)
findAllIn and look behind findPrefixOf and quantifier
Answer
B. findPrefixOf
& quantifier
答え B.findPrefixOfと量指定子(+)の方が速いです。
benchmark
findPrefixOfと量指定子の組み合わせの方が速いです。
findPrefixOf and quantifier is faster than findAllIn.
Throughput of 1,000 times execution
This benchmark shows throughputs of running n times findAllIn with
“(?<=w+)/.+” and findPrefixOf(findPrefixMatchOf and after) with “w/”.
Why are findPrefixOf and quantifier faster?
findAllInは部分マッチ、findPrefixOfは先頭マッチ。
• The above expression causes back tracking.
• Look behind assertion is not a problem.
• In addition, findPrefixOf is faster than findAllIn in this
case.
• findAllIn returns all non-overlapping matches of the
regular expression in the given character sequence.
• findPrefixOf returns a match of the regular expression at
the beginning of the given character sequence.
(?<=w+)/.+
matching same character sequence!
benchmark of various regular expressions
同じ正規表現を実行してもfindPrefixOfの方が速いです。
Even if they are given same regular expression, findPrefixOf is
fastest.
Further, the combination of findPrefixOf and look behind is so
Throughput of 1,000 times execution
findPrefixOf usage in famous library
Because routing trees are constructed by consuming
the beginning of uri path, findPrefixOf is sufficient
rather than findAllIn.
spray-routingでもfindPrefixOfを使っています。
implicit def regex2PathMatcher(regex: Regex): PathMatcher1[String] =
regex.groupCount match {

case 0 ⇒ new PathMatcher1[String] {

def apply(path: Path) = path match {

case Path.Segment(segment, tail) ⇒ regex findPrefixOf segment match {

case Some(m) ⇒ Matched(segment.substring(m.length) :: tail, m :: HNil)

case None ⇒ Unmatched

}

case _ ⇒ Unmatched

}

}
:
https://github.com/spray/spray/blob/master/spray-routing/src/main/scala/spray/routing/PathMatcher.scala
PathMatcher.scala line:211
The following code is a part of spray-routing.
RedHot high velocity No.7
Considering the effective utilization of
findPrefixOf.
When using regular expressions, be
mindful of the computational complexity.
爆速その7. 正規表現を使うときは計算量を考えましょう。
No. 1 Using Memory Mapped File, you can operate on files at
high speed.
No. 2 for comprehension and flatMap & map are same, level of
byte code.
No. 3 When updating a collection, Buffers are better choice.
No. 4 When inserting, List works best performance. So, you
feel happy using sorted List after inserting elements.
No. 5 It is cool way, using Array family when reading
randomly.
No. 6 Lazy evaluation is very useful.
No. 7 Be mindful of the computational complexity when using
regular expressions.
https://github.com/x1-/scala-benchmark
Source code is here !
Thank you for listening!

More Related Content

What's hot

Effective testing for spark programs Strata NY 2015
Effective testing for spark programs   Strata NY 2015Effective testing for spark programs   Strata NY 2015
Effective testing for spark programs Strata NY 2015
Holden Karau
 
Building a High-Performance Distributed Task Queue on MongoDB
Building a High-Performance Distributed Task Queue on MongoDBBuilding a High-Performance Distributed Task Queue on MongoDB
Building a High-Performance Distributed Task Queue on MongoDB
MongoDB
 

What's hot (20)

Kirk Shoop, Reactive programming in C++
Kirk Shoop, Reactive programming in C++Kirk Shoop, Reactive programming in C++
Kirk Shoop, Reactive programming in C++
 
Joblib for cloud computing
Joblib for cloud computingJoblib for cloud computing
Joblib for cloud computing
 
Writing Hadoop Jobs in Scala using Scalding
Writing Hadoop Jobs in Scala using ScaldingWriting Hadoop Jobs in Scala using Scalding
Writing Hadoop Jobs in Scala using Scalding
 
Weaving Dataflows with Silk - ScalaMatsuri 2014, Tokyo
Weaving Dataflows with Silk - ScalaMatsuri 2014, TokyoWeaving Dataflows with Silk - ScalaMatsuri 2014, Tokyo
Weaving Dataflows with Silk - ScalaMatsuri 2014, Tokyo
 
Internship final report@Treasure Data Inc.
Internship final report@Treasure Data Inc.Internship final report@Treasure Data Inc.
Internship final report@Treasure Data Inc.
 
Parallel-Ready Java Code: Managing Mutation in an Imperative Language
Parallel-Ready Java Code: Managing Mutation in an Imperative LanguageParallel-Ready Java Code: Managing Mutation in an Imperative Language
Parallel-Ready Java Code: Managing Mutation in an Imperative Language
 
Clojure made-simple - John Stevenson
Clojure made-simple - John StevensonClojure made-simple - John Stevenson
Clojure made-simple - John Stevenson
 
Introduction of failsafe
Introduction of failsafeIntroduction of failsafe
Introduction of failsafe
 
JRuby 9000 - Optimizing Above the JVM
JRuby 9000 - Optimizing Above the JVMJRuby 9000 - Optimizing Above the JVM
JRuby 9000 - Optimizing Above the JVM
 
A Brief Intro to Scala
A Brief Intro to ScalaA Brief Intro to Scala
A Brief Intro to Scala
 
Clojure, Plain and Simple
Clojure, Plain and SimpleClojure, Plain and Simple
Clojure, Plain and Simple
 
Storm Anatomy
Storm AnatomyStorm Anatomy
Storm Anatomy
 
whats new in java 8
whats new in java 8 whats new in java 8
whats new in java 8
 
Start Wrap Episode 11: A New Rope
Start Wrap Episode 11: A New RopeStart Wrap Episode 11: A New Rope
Start Wrap Episode 11: A New Rope
 
Apache Storm Tutorial
Apache Storm TutorialApache Storm Tutorial
Apache Storm Tutorial
 
Effective testing for spark programs Strata NY 2015
Effective testing for spark programs   Strata NY 2015Effective testing for spark programs   Strata NY 2015
Effective testing for spark programs Strata NY 2015
 
Building a High-Performance Distributed Task Queue on MongoDB
Building a High-Performance Distributed Task Queue on MongoDBBuilding a High-Performance Distributed Task Queue on MongoDB
Building a High-Performance Distributed Task Queue on MongoDB
 
05 pig user defined functions (udfs)
05 pig user defined functions (udfs)05 pig user defined functions (udfs)
05 pig user defined functions (udfs)
 
RubyKaigi2015 making robots-with-mruby
RubyKaigi2015 making robots-with-mrubyRubyKaigi2015 making robots-with-mruby
RubyKaigi2015 making robots-with-mruby
 
Shooting the Rapids
Shooting the RapidsShooting the Rapids
Shooting the Rapids
 

Viewers also liked

Viewers also liked (12)

Ironicを運用して半年が経過しました - OpenStack最新情報セミナー(2016年7月)
Ironicを運用して半年が経過しました  - OpenStack最新情報セミナー(2016年7月)Ironicを運用して半年が経過しました  - OpenStack最新情報セミナー(2016年7月)
Ironicを運用して半年が経過しました - OpenStack最新情報セミナー(2016年7月)
 
ご注文はRxですか? -RxSwiftを実際に導入してみた件-
ご注文はRxですか? -RxSwiftを実際に導入してみた件-ご注文はRxですか? -RxSwiftを実際に導入してみた件-
ご注文はRxですか? -RxSwiftを実際に導入してみた件-
 
Apache Drill で見る Twitter の世界
Apache Drill で見る Twitter の世界Apache Drill で見る Twitter の世界
Apache Drill で見る Twitter の世界
 
F.O.Xを支える技術
F.O.Xを支える技術F.O.Xを支える技術
F.O.Xを支える技術
 
サイバーエージェント様 発表「OpenStackのNWと物理の話」
サイバーエージェント様 発表「OpenStackのNWと物理の話」サイバーエージェント様 発表「OpenStackのNWと物理の話」
サイバーエージェント様 発表「OpenStackのNWと物理の話」
 
Atomic Design powered by React @ AbemaTV
Atomic Design powered by React @ AbemaTVAtomic Design powered by React @ AbemaTV
Atomic Design powered by React @ AbemaTV
 
AWSとGCPを使用したインフラ環境
AWSとGCPを使用したインフラ環境AWSとGCPを使用したインフラ環境
AWSとGCPを使用したインフラ環境
 
Wowzaを用いた配信基盤 Takusuta tech conf01
Wowzaを用いた配信基盤 Takusuta tech conf01Wowzaを用いた配信基盤 Takusuta tech conf01
Wowzaを用いた配信基盤 Takusuta tech conf01
 
GKEで半年運用してみた
GKEで半年運用してみたGKEで半年運用してみた
GKEで半年運用してみた
 
ポコロンダンジョンズを彩るアニメーションノウハウ
ポコロンダンジョンズを彩るアニメーションノウハウポコロンダンジョンズを彩るアニメーションノウハウ
ポコロンダンジョンズを彩るアニメーションノウハウ
 
レスポンシブWebデザインでうまくやるための考え方
レスポンシブWebデザインでうまくやるための考え方レスポンシブWebデザインでうまくやるための考え方
レスポンシブWebデザインでうまくやるための考え方
 
[社内勉強会]ELBとALBと数万スパイク負荷テスト
[社内勉強会]ELBとALBと数万スパイク負荷テスト[社内勉強会]ELBとALBと数万スパイク負荷テスト
[社内勉強会]ELBとALBと数万スパイク負荷テスト
 

Similar to あなたのScalaを爆速にする7つの方法

r,rstats,r language,r packages
r,rstats,r language,r packagesr,rstats,r language,r packages
r,rstats,r language,r packages
Ajay Ohri
 
My first experience with lambda expressions in java
My first experience with lambda expressions in javaMy first experience with lambda expressions in java
My first experience with lambda expressions in java
Scheidt & Bachmann
 

Similar to あなたのScalaを爆速にする7つの方法 (20)

Java 8
Java 8Java 8
Java 8
 
Real Time Big Data Management
Real Time Big Data ManagementReal Time Big Data Management
Real Time Big Data Management
 
Exciting JavaScript - Part II
Exciting JavaScript - Part IIExciting JavaScript - Part II
Exciting JavaScript - Part II
 
Java 7 Whats New(), Whats Next() from Oredev
Java 7 Whats New(), Whats Next() from OredevJava 7 Whats New(), Whats Next() from Oredev
Java 7 Whats New(), Whats Next() from Oredev
 
Quantifying Container Runtime Performance: OSCON 2017 Open Container Day
Quantifying Container Runtime Performance: OSCON 2017 Open Container DayQuantifying Container Runtime Performance: OSCON 2017 Open Container Day
Quantifying Container Runtime Performance: OSCON 2017 Open Container Day
 
Exploitation Crash Course
Exploitation Crash CourseExploitation Crash Course
Exploitation Crash Course
 
Scaling an ELK stack at bol.com
Scaling an ELK stack at bol.comScaling an ELK stack at bol.com
Scaling an ELK stack at bol.com
 
Scala Talk at FOSDEM 2009
Scala Talk at FOSDEM 2009Scala Talk at FOSDEM 2009
Scala Talk at FOSDEM 2009
 
Scala final ppt vinay
Scala final ppt vinayScala final ppt vinay
Scala final ppt vinay
 
Spark - The Ultimate Scala Collections by Martin Odersky
Spark - The Ultimate Scala Collections by Martin OderskySpark - The Ultimate Scala Collections by Martin Odersky
Spark - The Ultimate Scala Collections by Martin Odersky
 
SparkSQL: A Compiler from Queries to RDDs
SparkSQL: A Compiler from Queries to RDDsSparkSQL: A Compiler from Queries to RDDs
SparkSQL: A Compiler from Queries to RDDs
 
Exploring Clojurescript
Exploring ClojurescriptExploring Clojurescript
Exploring Clojurescript
 
Charles Sharp: Java 8 Streams
Charles Sharp: Java 8 StreamsCharles Sharp: Java 8 Streams
Charles Sharp: Java 8 Streams
 
New Features in JDK 8
New Features in JDK 8New Features in JDK 8
New Features in JDK 8
 
r,rstats,r language,r packages
r,rstats,r language,r packagesr,rstats,r language,r packages
r,rstats,r language,r packages
 
Porting a Streaming Pipeline from Scala to Rust
Porting a Streaming Pipeline from Scala to RustPorting a Streaming Pipeline from Scala to Rust
Porting a Streaming Pipeline from Scala to Rust
 
10 Reasons to Start Your Analytics Project with PostgreSQL
10 Reasons to Start Your Analytics Project with PostgreSQL10 Reasons to Start Your Analytics Project with PostgreSQL
10 Reasons to Start Your Analytics Project with PostgreSQL
 
My first experience with lambda expressions in java
My first experience with lambda expressions in javaMy first experience with lambda expressions in java
My first experience with lambda expressions in java
 
Functional programming with Scala
Functional programming with ScalaFunctional programming with Scala
Functional programming with Scala
 
Flink internals web
Flink internals web Flink internals web
Flink internals web
 

More from x1 ichi

More from x1 ichi (10)

リアルタイムにデータ分析してWebサービスの面白さを伝えたい
リアルタイムにデータ分析してWebサービスの面白さを伝えたいリアルタイムにデータ分析してWebサービスの面白さを伝えたい
リアルタイムにデータ分析してWebサービスの面白さを伝えたい
 
逆引き!Scala x ビッグデータ
逆引き!Scala x ビッグデータ逆引き!Scala x ビッグデータ
逆引き!Scala x ビッグデータ
 
競馬の格言を地方競馬で検証してみた
競馬の格言を地方競馬で検証してみた競馬の格言を地方競馬で検証してみた
競馬の格言を地方競馬で検証してみた
 
あなたのScalaを爆速にする7つの方法(日本語版)
あなたのScalaを爆速にする7つの方法(日本語版)あなたのScalaを爆速にする7つの方法(日本語版)
あなたのScalaを爆速にする7つの方法(日本語版)
 
本当にあったApache Spark障害の話
本当にあったApache Spark障害の話本当にあったApache Spark障害の話
本当にあったApache Spark障害の話
 
MillWheel Fault-Tolerant Stream Processing at Internet Scaleの意訳
MillWheel Fault-Tolerant Stream Processing at Internet Scaleの意訳MillWheel Fault-Tolerant Stream Processing at Internet Scaleの意訳
MillWheel Fault-Tolerant Stream Processing at Internet Scaleの意訳
 
広告配信現場で使うSpark機械学習
広告配信現場で使うSpark機械学習広告配信現場で使うSpark機械学習
広告配信現場で使うSpark機械学習
 
女性エンジニアの1週間
女性エンジニアの1週間女性エンジニアの1週間
女性エンジニアの1週間
 
Sparkを用いたビッグデータ解析 〜 前編 〜
Sparkを用いたビッグデータ解析 〜 前編 〜Sparkを用いたビッグデータ解析 〜 前編 〜
Sparkを用いたビッグデータ解析 〜 前編 〜
 
解説: a semantic approach to recommending text advertisements for images
解説: a semantic approach to recommending text advertisements for images解説: a semantic approach to recommending text advertisements for images
解説: a semantic approach to recommending text advertisements for images
 

Recently uploaded

Microkernel in Operating System | Operating System
Microkernel in Operating System | Operating SystemMicrokernel in Operating System | Operating System
Microkernel in Operating System | Operating System
Sampad Kar
 
Online crime reporting system project.pdf
Online crime reporting system project.pdfOnline crime reporting system project.pdf
Online crime reporting system project.pdf
Kamal Acharya
 
Maher Othman Interior Design Portfolio..
Maher Othman Interior Design Portfolio..Maher Othman Interior Design Portfolio..
Maher Othman Interior Design Portfolio..
MaherOthman7
 
Activity Planning: Objectives, Project Schedule, Network Planning Model. Time...
Activity Planning: Objectives, Project Schedule, Network Planning Model. Time...Activity Planning: Objectives, Project Schedule, Network Planning Model. Time...
Activity Planning: Objectives, Project Schedule, Network Planning Model. Time...
Lovely Professional University
 
Tembisa Central Terminating Pills +27838792658 PHOMOLONG Top Abortion Pills F...
Tembisa Central Terminating Pills +27838792658 PHOMOLONG Top Abortion Pills F...Tembisa Central Terminating Pills +27838792658 PHOMOLONG Top Abortion Pills F...
Tembisa Central Terminating Pills +27838792658 PHOMOLONG Top Abortion Pills F...
drjose256
 

Recently uploaded (20)

Seismic Hazard Assessment Software in Python by Prof. Dr. Costas Sachpazis
Seismic Hazard Assessment Software in Python by Prof. Dr. Costas SachpazisSeismic Hazard Assessment Software in Python by Prof. Dr. Costas Sachpazis
Seismic Hazard Assessment Software in Python by Prof. Dr. Costas Sachpazis
 
analog-vs-digital-communication (concept of analog and digital).pptx
analog-vs-digital-communication (concept of analog and digital).pptxanalog-vs-digital-communication (concept of analog and digital).pptx
analog-vs-digital-communication (concept of analog and digital).pptx
 
Research Methodolgy & Intellectual Property Rights Series 1
Research Methodolgy & Intellectual Property Rights Series 1Research Methodolgy & Intellectual Property Rights Series 1
Research Methodolgy & Intellectual Property Rights Series 1
 
Microkernel in Operating System | Operating System
Microkernel in Operating System | Operating SystemMicrokernel in Operating System | Operating System
Microkernel in Operating System | Operating System
 
5G and 6G refer to generations of mobile network technology, each representin...
5G and 6G refer to generations of mobile network technology, each representin...5G and 6G refer to generations of mobile network technology, each representin...
5G and 6G refer to generations of mobile network technology, each representin...
 
litvinenko_Henry_Intrusion_Hong-Kong_2024.pdf
litvinenko_Henry_Intrusion_Hong-Kong_2024.pdflitvinenko_Henry_Intrusion_Hong-Kong_2024.pdf
litvinenko_Henry_Intrusion_Hong-Kong_2024.pdf
 
AI in Healthcare Innovative use cases and applications.pdf
AI in Healthcare Innovative use cases and applications.pdfAI in Healthcare Innovative use cases and applications.pdf
AI in Healthcare Innovative use cases and applications.pdf
 
Lab Manual Arduino UNO Microcontrollar.docx
Lab Manual Arduino UNO Microcontrollar.docxLab Manual Arduino UNO Microcontrollar.docx
Lab Manual Arduino UNO Microcontrollar.docx
 
Involute of a circle,Square, pentagon,HexagonInvolute_Engineering Drawing.pdf
Involute of a circle,Square, pentagon,HexagonInvolute_Engineering Drawing.pdfInvolute of a circle,Square, pentagon,HexagonInvolute_Engineering Drawing.pdf
Involute of a circle,Square, pentagon,HexagonInvolute_Engineering Drawing.pdf
 
Filters for Electromagnetic Compatibility Applications
Filters for Electromagnetic Compatibility ApplicationsFilters for Electromagnetic Compatibility Applications
Filters for Electromagnetic Compatibility Applications
 
Quiz application system project report..pdf
Quiz application system project report..pdfQuiz application system project report..pdf
Quiz application system project report..pdf
 
The battle for RAG, explore the pros and cons of using KnowledgeGraphs and Ve...
The battle for RAG, explore the pros and cons of using KnowledgeGraphs and Ve...The battle for RAG, explore the pros and cons of using KnowledgeGraphs and Ve...
The battle for RAG, explore the pros and cons of using KnowledgeGraphs and Ve...
 
Online crime reporting system project.pdf
Online crime reporting system project.pdfOnline crime reporting system project.pdf
Online crime reporting system project.pdf
 
Electrical shop management system project report.pdf
Electrical shop management system project report.pdfElectrical shop management system project report.pdf
Electrical shop management system project report.pdf
 
Maher Othman Interior Design Portfolio..
Maher Othman Interior Design Portfolio..Maher Othman Interior Design Portfolio..
Maher Othman Interior Design Portfolio..
 
Lesson no16 application of Induction Generator in Wind.ppsx
Lesson no16 application of Induction Generator in Wind.ppsxLesson no16 application of Induction Generator in Wind.ppsx
Lesson no16 application of Induction Generator in Wind.ppsx
 
Activity Planning: Objectives, Project Schedule, Network Planning Model. Time...
Activity Planning: Objectives, Project Schedule, Network Planning Model. Time...Activity Planning: Objectives, Project Schedule, Network Planning Model. Time...
Activity Planning: Objectives, Project Schedule, Network Planning Model. Time...
 
Tembisa Central Terminating Pills +27838792658 PHOMOLONG Top Abortion Pills F...
Tembisa Central Terminating Pills +27838792658 PHOMOLONG Top Abortion Pills F...Tembisa Central Terminating Pills +27838792658 PHOMOLONG Top Abortion Pills F...
Tembisa Central Terminating Pills +27838792658 PHOMOLONG Top Abortion Pills F...
 
Introduction to Heat Exchangers: Principle, Types and Applications
Introduction to Heat Exchangers: Principle, Types and ApplicationsIntroduction to Heat Exchangers: Principle, Types and Applications
Introduction to Heat Exchangers: Principle, Types and Applications
 
Vip ℂall Girls Karkardooma Phone No 9999965857 High Profile ℂall Girl Delhi N...
Vip ℂall Girls Karkardooma Phone No 9999965857 High Profile ℂall Girl Delhi N...Vip ℂall Girls Karkardooma Phone No 9999965857 High Profile ℂall Girl Delhi N...
Vip ℂall Girls Karkardooma Phone No 9999965857 High Profile ℂall Girl Delhi N...
 

あなたのScalaを爆速にする7つの方法

  • 1. 7 ways to make your Scala RedHot high velocity x1(Yuri Inoue) For ScalaMatsuri 2016 あなたのScalaを爆速にする7つの方法
  • 2. Yuri Inoue Cyberagent,Inc. AdTech Studio, AMoAd. twitter: @iyunoriue GitHub: x1- HP: Batsuichi and Inken’s engineer blog http://x1.inkenkun.com/ profile 井上ゆりと申します。
  • 3. • Parallel execution is not taken into account. • This session is for beginners of Scala. • The environment of benchmark is shown at next page. • @Keen advised some performances. Special thanks! acknowledgement 謝辞。並列実行に関しては考慮していません。
  • 4. environment of benchmark instance Amazon EC2 m3.2xlarge vCPU 8 memory 30GB disk generalized SSD OS CentOS6.7-Final jdk jdk1.8.0u65(oracle, server vm) scala 2.11.7 build tool sbt benchmark tool sbt-jmh 0.2.5 libraries com.bionicspirit.shade:1.6.0 net.debasishg.redisclient:3.0 org.scalatest.scalatest:2.2.4 ベンチマーク環境
  • 5. No.1 Random Read - File vs KVS -
  • 6. Q1 Search for some string in 2GB data. Which of the following would be faster? 2GBデータから文字列を探します。どちらが速いでしょう?
  • 7. from 2GB file on SSD. A from total 2GB strings on memcache. (divided into chunk) B prerequisite prerequisite * SSD is generalized SSD on EC2. * not using RamDisk. * memcache version is 1.4.0. * memcache stand up on local A:SSD上の2GBファイル B:memcache上の2GBデータ
  • 8. Answer A. from 2GB file on SSD. 答え A.SSD上に置いた2GBのファイルから探す です。
  • 9. benchmark The figure shows the average for the times it took to search for the string ”SUIKA” from the following 2GB data. • Searching from a file completed in less than 8sec. • Searching from memcache completed in near by 19sec. • Searching from Redis(also KVS) completed in 14sec. '''<code>&amp;h</code>''' を用い、<code>&amp;h0F</code> (十進で15)のように表現する。 nn[[Standard Generalized Markup Language|SGML]]、[[Extensible Markup Language|XML]]、 [[HyperText Markup Language|HTML]]では、アンパサンドをSUIKA使って[[SGML実体]]を参照する。 SUIKAという文字列を探したときの平均タイムです。
  • 10. Why is file faster than memcache? • This is Memory Mapped File(MMF) power. • Memory Mapped File is used to map a file on disk to a region of address space on memory. • It avoids unnecessary file I/O operations and file buffering. • MappedByteBuffer included java.nio, that directly access Memory Mapped File. physical file mapping address on memory fileが高速なのはMemoryMappedFileを使ったからです。
  • 11. • memcache(<1.4.2) has limitation below, The maximum size of a value you can store in memcached is 1MB. • Although memcache is on the same host, accessing it many times to retrieve data takes long time. • Redis can save 1GB per 1 key. Using 2GB data divided into 4 key(each 500MB), got below benchmark. • It completed in about 8sec, but cannot win file. Why is memcache slower than file? memcache(<1.4.2)は1key<=1MBのサイズ制限があります。
  • 12. • Memory Mapped File can only map up to 2GB file on JVM. http://stackoverflow.com/questions/8076472/why-does- filechannel-map-take-up-to-integer-max-value-of-data • Apache Spark MLlib development supervisor, Reynold Xin wrote the following Gist. He measured the performance over various approaches to read ByteBuffer in Scala. https://gist.github.com/rxin/5087680 more information jvmではMemoryMappedFileは2GBまでしか扱えません。
  • 13. RedHot high velocity No.1 Using Memory Mapped File, you can operate on files at high speed. 爆速その1. MMFで高速ファイル操作が可能に!
  • 15. Q2 for comprehension behaves same as flatMap & map. Which one is faster than? for内包表記とflatMap&map、どちらが速いでしょう?
  • 16. A B code same. flatMap & map for {
 a <- data
 b <- a
 } yield {
 b
 } for comprehension data.flatMap( a => a.map( b => b ) ) flatMap & map
  • 18. benchmark All of Throughput、Average、Sample don’t show significant difference between for comprehension and flatMap & map. 10,000times for内包表記とflatMap&mapで優位差が見られません。
  • 19. • for comprehension and flatMap & map are logically same. • Here is a comparison after decompiling them. They are same! Why are they the same speed? for内包表記とflatMap&mapは論理的に同じです。 public Option<String> forComprehension()
 {
 data().flatMap(new AbstractFunction1()
 {
 public static final long serialVersionUID = 0L;
 
 public final Option<String> apply(Some<String> a)
 {
 a.map(new AbstractFunction1()
 {
 public static final long serialVersionUID = 0L;
 
 public final String apply(String b)
 {
 return b;
 }
 });
 }
 });
 } public Option<String> flatMapAndMap()
 {
 data().flatMap(new AbstractFunction1()
 {
 public static final long serialVersionUID = 0L;
 
 public final Option<String> apply(Some<String> a)
 {
 a.map(new AbstractFunction1()
 {
 public static final long serialVersionUID = 0L;
 
 public final String apply(String b)
 {
 return b;
 }
 });
 }
 });
 } for comprehension flatMap & map
  • 20. RedHot high velocity No.2 for comprehension and flatMap & map are same, level of byte code. 爆速その2. for内包表記とflatMap&mapは同じです。
  • 21. No.3 append & insert - collection -
  • 22. Collections Performance Characteristics at Scala cite: http://docs.scala-lang.org/overviews/collections/performance-characteristics.html Scalaにおけるコレクションの性能特性
  • 23. Q3-1 Mutable variable “var xs: Vector” and Immutable variable “val xs: ArrayBuffer” Which one is faster, when appending to the tail of a collection? 可変なVectorと不変なArrayBuffer、末尾追加が速いのは?
  • 24. var Vector A val ArrayBuffer B code var xs = Vector.empty[Int] xs = xs :+ a var xs: Vector val xs: ArrayBuffer val xs = ArrayBuffer.empty[Int] xs += a
  • 25. Answer B. val ArrayBuffer 答え B.val ArrayBufferです。
  • 26. benchmark ArrayBuffer is faster than Vector. ArrayBufferの方がVectorよりも速いです。 Throughput of appending n times This benchmark shows throughputs of appending N elements. For example, type:Vector and times:10k indicates appending 10,000 elements in an empty Vector.
  • 27. benchmark The benchmarks of the other immutable objects are below. VectorはListやStreamよりは速いのですが。 Throughput of appending n times Vector is faster than List and Stream as same as immutable.
  • 28. Why is ArrayBuffer faster than Vector? When appending new element ... Vectorは新インスタンスを新たにつくるので遅くなります。 add new element, after coping elements to new instance. update tail position, after resizing instance. var Vector val ArrayBuffer val b = bf(repr)
 b ++= thisCollection
 b += elem
 b.result() ensureSize(size0 + 1)
 array(size0) = elem.asInstanceOf[AnyRef]
 size0 += 1
 this Thease processes are absolutely different.
  • 29. Q3-2 Mutable variable “var xs: List” and Immutable variable “val xs: ListBuffer” Which one is faster, when inserting into the head of a collection? 可変なListと不変なListBuffer、先頭挿入が速いのは?
  • 30. var List A val ListBuffer B code var xs = List.empty[Int] xs = a :: xs var xs: List val xs: ListBuffer val xs = ListBuffer.empty[Int] a +=: xs
  • 31. Answer A. var List 答え A.var Listです。
  • 32. benchmark List is faster than ListBuffer. Listの方がListBufferよりも速いです。 Throughput of inserting n times This benchmark shows throughputs of inserting N elements. For example, type:List and times:1k indicates inserting 1,000 elements in an empty List.
  • 33. benchmark The benchmarks of the other objects are below. Listだけ飛び抜けて速いです。 List is enormously faster than the others. Throughput of inserting n times
  • 34. Why is List faster than the others? When inserting new element into the begining position... Listではほとんど計算せずに先頭挿入が行えます。 Because of having head and tail, List can create new instance immediately. ListBuffer is almost same as List, but inner variable is reassigned to. var List val ListBuffer new scala.collection.immutable .::(x, this) if (exported) copy()
 val newElem = new :: (x, start)
 if (isEmpty) last0 = newElem
 start = newElem
 len += 1
 this List calculate a little, in case of inserting.
  • 35. RedHot high velocity No.3 When appending elements, using ArrayBuffer or ListBuffer is a better way. But inserting the begining position, using List works best performance. 爆速その3.末尾追加は**Bufferを!先頭挿入はListを!
  • 37. Q4 Mutable variable “var xs: List” and Immutable variable “val xs: ListBuffer” Which one is faster, when removing an item from a collection? 可変なListと不変なListBuffer、削除が速いのは?
  • 38. var List A val ListBuffer B code var xs: List val xs: ListBuffer var xs = List( 1 to n: _* ) // head xs = xs.tail // tail xs = xs.dropRight(0) var xs = ListBuffer( 1 to n: _* ) // head xs.remove(0) // tail xs.remove(xs.size - 1)
  • 39. Answer B. val ListBuffer 答え B.val ListBuffer です。
  • 40. benchmark ListBuffer is much faster than List. ListBufferの方がListよりも相当速いです。 Throughput of removing n times This benchmark shows throughputs of removing N elements from N length array. For example, Benchmark:removeTail, type:List and times:1k indicates removing 1,000 elements from the last of 1,000 length List.
  • 41. benchmark The benchmarks of the other objects are below. Buffer系以外はどれも超絶遅いです。 They are all very slow, except for the Buffer family. Throughput of removing n times
  • 42. Why is ListBuffer faster than List? When remove element from collection... ListのdropRightはO(n)の時間がかかります。 The operation - dropRight of List takes time O(n). The operation - remove of ListBuffer takes constant time. var List val ListBuffer def dropRight(n: Int): Repr = {
 val b = newBuilder
 var these = this
 var lead = this drop n
 while (!lead.isEmpty) {
 b += these.head
 these = these.tail
 lead = lead.tail
 }
 b.result()
 } def remove(n: Int): A = { : var cursor = start
 var i = 1
 while (i < n) {
 cursor = cursor.tail
 i += 1
 }
 old = cursor.tail.head
 if (last0 eq cursor.tail) last0 = cursor.asInstanceOf[::[A]]
 cursor.asInstanceOf[::[A]].tl = cursor.tail.tail
  • 43. benchmark of List’s dropRight Because the benchmark has increased by 10 times the size, we have to display the throughput logarithmic graph of. dropRightのスループットは線形に下降しているだけです。 Throughput is just only lowered to linear. It is the horror of linear increase. log(Throughput) of List’s dropRight
  • 44. reference benchmark 参考のためtakeと比較しました。takeは少し速いです。 Since both of dropRight(1) and take(n-1) are same function, so compared with dropRight and take. take is slightly faster than dropRight. Throughput of List’ dropRight and take
  • 45. RedHot high velocity No.4 When dropping elements, using ListBuffer or ArrayBuffer is a better way. 爆速その4. 要素削除を行うならListBuffer or ArrayBuffer。
  • 46. No.5 random read - collection -
  • 47. Q5 Vector vs ListBuffer Which one is faster, when reading randomly? VectorとListBuffer, ランダム・リードが速いのは?
  • 48. Vector A ListBuffer B code val data = Vector( 1 to n: _* )
 
 ( 1 to data.size ) map { _ => data( Random.nextInt( n ) ) } val data = ListBuffer( 1 to n: _* )
 
 ( 1 to data.size ) map { _ => data( Random.nextInt( n ) ) }
  • 50. benchmark Vector is faster than ListBuffer. Vectorの方がListBufferより速いです。 Throughput of reading n times This benchmark shows throughputs of getting an item randomly from N length array. For example, type:ListBuffer and times:1k indicates getting an item 1,000 times from 1,000 length List.
  • 51. benchmark The benchmarks of the other objects are below. 速いのはArray, ArrayBuffer, Vectorです。 Array, ArrayBuffer, Vector are fast. Throughput of reading n times
  • 52. Why Array, ArrayBuffer, Vector are fast? Vectorなどは内部的に定数時間のArrayを使っています。 • Array - random read takes constant time. • ArrayBuffer and Vector have Array internal. protected var array: Array[AnyRef] = new Array[AnyRef](math.max(initialSize, 1)) : def apply(idx: Int) = {
 if (idx >= size0) throw new IndexOutOfBoundsException(idx.toString)
 array(idx).asInstanceOf[A]
 } e.g ArrayBuffer
  • 53. RedHot high velocity No.5 It is a cool way, using Array family when reading randomly. 爆速その5. Arrayの仲間はランダム・リードが速いです。
  • 55. Q6 Considering the above, Stream vs Array Which one is faster, producing a fibonacci sequence? フィボナッチ数列の生成が速いのはStream?Array?
  • 56. Stream A Array B code def fibonacci( h: Int = 1, n: Int = 1 ): Stream[Int] =
 h #:: fibonacci( n, h + n ) val fibo = fibonacci().take( n ) def fibonacci( n: Int = 1 ): Array[Int] = if ( n == 0 ) {
 Array.empty[Int]
 } else {
 val b = new Array[Int](n)
 b(0) = 1
 for ( i <- 0 until n - 1 ) {
 val n1 = if ( i == 0 ) 0 else b( i - 1 )
 val n2 = b( i )
 b( i + 1 ) = n1 + n2
 }
 b
 } val fibo = fibonacci( n ) * calculate recursively * operation takes O(n)
  • 58. benchmark Stream is overwhelmingly faster than Array. Streamが圧倒的に速いです。 Throughput of creating n length fibonacci sequence fibonacci sequence: ( 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, .. )
  • 59. Why is Stream so much faster? Streamは遅延評価なので必要なときに計算します。 • Stream implements lazy lists where elements are only evaluated when they are needed. • Actually, these have not yet been calculated. • This is the power of lazy evaluation. • But, if the sequence is materialized, it takes calculating time. → see next page.
  • 60. Why is Stream so much faster? ただしStreamは具現化するときにコストがかかります。 This is the benchmark when called toList on Stream. Array is fastest, if it calls toList on Stream. It shows calculation method using Array is better than recursive. Throughput of creating n length fibonacci sequence
  • 61. RedHot high velocity No.6 Lazy evaluation is very useful. But, it takes cost at materialized. 爆速その6. 遅延評価は便利ですが具現化すると普通です。
  • 63. Showing the benchmark of w+w Regular Expression consumes CPU resource a lot. In particular, back tracking will occur using the expression like “ww..w”, so takes time O(n^2). 正規表現はCPUリソースをたくさん消費します。 The following is the performance of Regular Expression. Throughput of each regular expression execution(1,000 times)
  • 64. Last. Q7 When looking for behind the specific expression(w+/), “findAllIn and look behind(?<=)” and “findPrefixOf and quantifier(+)” which one is faster than? from this string→ abcdef..0123../abc.. (1024byte) ある文字列の後方を探したいです。どちらが速いでしょう?
  • 65. same. A findPrefixOf B code val re = """(?<=w+)/.+""" r val ms = re findAllIn data
 if ( ms.isEmpty ) None
 else Some( ms.next ) val re = “""w+/""" r
 re findPrefixMatchOf data map( _.after.toString ) findAllIn and look behind findPrefixOf and quantifier
  • 66. Answer B. findPrefixOf & quantifier 答え B.findPrefixOfと量指定子(+)の方が速いです。
  • 67. benchmark findPrefixOfと量指定子の組み合わせの方が速いです。 findPrefixOf and quantifier is faster than findAllIn. Throughput of 1,000 times execution This benchmark shows throughputs of running n times findAllIn with “(?<=w+)/.+” and findPrefixOf(findPrefixMatchOf and after) with “w/”.
  • 68. Why are findPrefixOf and quantifier faster? findAllInは部分マッチ、findPrefixOfは先頭マッチ。 • The above expression causes back tracking. • Look behind assertion is not a problem. • In addition, findPrefixOf is faster than findAllIn in this case. • findAllIn returns all non-overlapping matches of the regular expression in the given character sequence. • findPrefixOf returns a match of the regular expression at the beginning of the given character sequence. (?<=w+)/.+ matching same character sequence!
  • 69. benchmark of various regular expressions 同じ正規表現を実行してもfindPrefixOfの方が速いです。 Even if they are given same regular expression, findPrefixOf is fastest. Further, the combination of findPrefixOf and look behind is so Throughput of 1,000 times execution
  • 70. findPrefixOf usage in famous library Because routing trees are constructed by consuming the beginning of uri path, findPrefixOf is sufficient rather than findAllIn. spray-routingでもfindPrefixOfを使っています。 implicit def regex2PathMatcher(regex: Regex): PathMatcher1[String] = regex.groupCount match {
 case 0 ⇒ new PathMatcher1[String] {
 def apply(path: Path) = path match {
 case Path.Segment(segment, tail) ⇒ regex findPrefixOf segment match {
 case Some(m) ⇒ Matched(segment.substring(m.length) :: tail, m :: HNil)
 case None ⇒ Unmatched
 }
 case _ ⇒ Unmatched
 }
 } : https://github.com/spray/spray/blob/master/spray-routing/src/main/scala/spray/routing/PathMatcher.scala PathMatcher.scala line:211 The following code is a part of spray-routing.
  • 71. RedHot high velocity No.7 Considering the effective utilization of findPrefixOf. When using regular expressions, be mindful of the computational complexity. 爆速その7. 正規表現を使うときは計算量を考えましょう。
  • 72. No. 1 Using Memory Mapped File, you can operate on files at high speed. No. 2 for comprehension and flatMap & map are same, level of byte code. No. 3 When updating a collection, Buffers are better choice. No. 4 When inserting, List works best performance. So, you feel happy using sorted List after inserting elements. No. 5 It is cool way, using Array family when reading randomly. No. 6 Lazy evaluation is very useful. No. 7 Be mindful of the computational complexity when using regular expressions.
  • 74. Thank you for listening!