* Scala 基本介紹 * 以 Spark sample 介紹 Scala 語法

1. Scala 初探
亦思科技 鄭紹志
vito@is-land.com.tw

2. WHO AM I
• 鄭紹志
• IS-LAND Technical Manager
• HareDB
• Hadoop + HBase
• Scala + Spark

3. WHY SCALA
WHY NOT SCALA
• Scala - The good, the bad and the very ugly
• http://www.slideshare.net/Bozho/scala-the-good-the-bad-and-the-very-ugly
•Why Scala
• http://www.slideshare.net/al3x/why-scala-presentation
•Today Gist: http://tinyurl.com/z8xx6ab

4. SCALABLE LANGUAGE
• Runs on JVM
• Concise, 簡潔
• Statically typed, 靜態型別
• Object Oriented Programming, 物件導向開發
• Functional Programming, 函數式開發

5. Sample Code - SparkPi

7. SCALA INSTALL
• Scala 2.10.x - Spark 1.0 - 1.6 都使⽤ 2.10.x
• Download Scala 2.10.x
• http://www.scala-lang.org/download/all.html
• 解壓縮 & 設定路徑
• http://www.scala-lang.org/download/install.html

8. SCALA INTERPRETER
• scala: Read-Evaluate-Print-Loop(REPL)
[~]$ scala
Welcome to Scala version 2.11.7 (Java HotSpot(TM) 64-Bit
Server VM, Java 1.7.0_80).
Type in expressions to have them evaluated.
Type :help for more information.
scala> val name = "Vito"
name: String = Vito
scala> println( s"Hello, $name" )
Hello, Vito
scala> println( name.getClass )
class java.lang.String
scala>

9. SCALA INTERPRETER
• One line execute
• Run script
• Run main class
• Shell script
• Load file: :load
• Paste mode: :paste
$ scala -e 'val name="Vito"; println(s"Hello,$name")'
$ scala hello.scala
$ scala HelloWorld.scala
$ ./script.sh

10. Value & Variable
• 定義常數與變數
• 優先考慮使⽤ val
var price1 = 100
price1 = 200
val price2 = 100
price2 = 200 // error: reassignment to val

11. Type Inference
• 推斷使⽤的型別
val name = "Vito"
val n1 = 100
val n2 = 100L
val n3: Int = 100
val n4 = 20.6F
val list = Seq("a", "b", "c")
import java.util._
val list1 = new ArrayList[String]()

12. STRING
//
val s1 = "Hello"
//
val s2 = """object HelloWorld extends App {
println("Hello, World!")
}
"""
// string interpolation
val s3 = s"[$s1] source code:n$s2"
// no escape
val s4 = raw"Hello n world."
val s5 = "Hello"(1)
val s6 = "Hello".charAt(1)

13. TUPLE
• Tuple2 - Tuple22
val t2a = ("a", 2)
val t2b = Tuple2("a", 2)
val t2c = new Tuple2("a", 2)
val t2d = "a" -> 2
val t5 = ("a", 100, "c", 20.5, (1,2,3))
val v1 = t2a._1 == t2a._1 // true
val v2 = t5._5 == (1,2,3) // true

14. OBJECT ORIENTED
//
// . ( )
"Hello World" split " "
"Hello World".split(" ")
// operator
"a" + "b" + "c"
"a".+("b").+("c")

15. IMPORT CLASS
• 預設 import
• java.lang._
• scala._
• Java *, Scala _
• scala.Predef object
import java.util._
import java.util.{List, ArrayList}
import java.util.{List=>JList, HashMap=>JHashMap}
import java.util.{Date=>_, _}

16. IMPORT CLASS
• Import anywhere
import org.apache.spark.{SparkContext, SparkConf, Logging}
class MyClass {
import java.util._
def printDate() = {
val date = new Date()
println(date)
}
def printFilePath() = {
import java.io.File
val file = new File("data.txt")
println(file.getAbsolutePath)
}
}

17. OBJECT
• Scala 沒有 Java 的 static 宣告, 可⽤ object 代替
• 想⽤ Singletion pattern 就⽤ object
• Companion Objects: 同名的 class 與 object 同時
存在
object Student {
def sayHi = println("Hi~")
}
Student.sayHi

18. OBJECT
class Student(val name: String) {
override def toString = s"[Student]: $name"
}
object Student {
def apply(names: String*) = {
names.map { n => new Student(n) }
}
def count(ss: Seq[Student]) = ss.size
}
val ss = Student("Vito", "John")
Student.count(ss)

19. RETURN VALUE
• method 傳回值不須加 return
• Every statement(expression) yields a value
val a = 5
val b = if (a>2) 0 else 1
val data = Seq(1,2,3,4,5)
val data1 = for (
num <- data
if num % 2 == 0
) yield {
num + 10
} // data1: List(12, 14)

20. RETURN VALUE
• Every statement(expression) yields a value
def calc(a: Int, b: Int) = try {
a / b
} catch {
case _: Throwable => 0
}
val a = calc(100, 0)
calc: (a: Int, b: Int)Int
a: Int = 0

21. Anonymous Functions
val data = Seq.range(1, 5)
val data2 = data.map( (s: Int) => s*2 )
val data3 = data.map( (s) => s*2 )
val data4 = data.map( _*2 )
data: Seq[Int] = List(1, 2, 3, 4)
data2: Seq[Int] = List(2, 4, 6, 8)
data3: Seq[Int] = List(2, 4, 6, 8)
data4: Seq[Int] = List(2, 4, 6, 8)

22. Anonymous Functions
val data = Seq.range(1, 5)
val multi = (s: Int) => s * 2
val multi2 = new Function1[Int, Int]() {
override def apply(v1: Int): Int = v1 * 2
}
val data2 = data.map(multi)
val data3 = data map multi
val data4 = data map multi2
multi: Int => Int = <function1>
multi2: Int => Int = <function1>
data2: Seq[Int] = List(2, 4, 6, 8)
data3: Seq[Int] = List(2, 4, 6, 8)
data4: Seq[Int] = List(2, 4, 6, 8)

23. Function ? Method ?
• ⽬前不必追究
• (想追究的😆)可從這裡先開始, 再 Google
• http://stackoverflow.com/q/2529184

24. map

25. map
val data = Seq.range(1, 5)
val multi = (s: Int) => s * 2
val data2 = data.map( _*2 )
val data3 = data map multi
data: Seq[Int] = List(1, 2, 3, 4)
multi: Int => Int = <function1>
data2: Seq[Int] = List(2, 4, 6, 8)
data3: Seq[Int] = List(2, 4, 6, 8)

26. Reduce

27. Reduce
val data = Seq.range(1, 5)
val data2 = data.reduce(_+_)
val data3 = data.reduce { (v1, v2) =>
println( (v1, v2) )
v1 + v2
}
data: Seq[Int] = List(1, 2, 3, 4)
data2: Int = 10
(1,2)
(3,3)
(6,4)
data3: Int = 10

28. PRACTICE
• 看懂第⼀個 Spark scala sample code 使⽤的語法
• SparkPi.scala

30. CLASS
// Scala
// Both getter and setter
class Student1(var name: String)
// Only getter
class Student2(val name: String)
// private
class Student3(val name: String, private val
id: Int)

31. CASE CLASS
• 同時建⽴ class 與 object
• 為 class 建⽴ getter & setter
• 為 object 建⽴ apply & unapply method
case class Student(val id: Int, val name:
String)
val s = Student(100, "Vito")
Student.unapply(s)

32. Pattern Matching
• 第⼀眼看起來很像 switch...case...
val b = true
b match {
case true => println("true")
case false => println("false")
}

33. Pattern Matching
• 實際與 switch 差很⼤, example: Source

34. class extends & with
• Scala 沒有 Java 的 interface
• Trait - 結合 interface 與 class
class Person {}
trait PlayGame {}
trait ReadBook {}
class Student extends Person
with PlayGame with ReadBook {
}
class LittleBoy extends Person with PlayGame {
}

35. Sample Code - RDDRelation

36. SBT
• The interactive build tool, run task from shell
• Installing sbt
• Spark
• Maven is recommented for packaging Spark
• SBT is supported for day-to-day development

37. SBT
• Commands
• run, run-main
• test, testOnly
• compile
• reload
• 威⼒強⼤的 ~
• Demo

38. SCALA 實務
• Scala 還蠻難的
• 直接使⽤ Java SDK & 第三⽅ Libraries
• SBT project 可混合 Java & Scala
• 好的 IDE 帶你快速上路: Intellij IDEA
• Spark 是分散式的 Scala

39. SCALA 實務
• Build tool: recommand SBT
• Testing: Scalatest, Specs2, JUnit
• SBT PermGen space resolve
• http://www.scala-sbt.org/0.13/docs/Manual-Installation.html
• Book

40. Thank you
and
Question ?