1.
Scala at Netflix
Scala Bay Meetup
Netflix, Sept 9th 2013
Manish Pandit
mpandit@netflix.com
@lobster1234

2.
Agenda
Background
Netflix OSS Components
Development
Deployment/Delivery
Open Floor

3.
Partner Product Engineering
Smart devices
+
Certification
=
Lots of Device Metadata!

4.
Model
Firmware
Screen Resolution
Subtitle Support
3D
DRM
Remote Control
Netflix SDK
…

5.
Architecture
Cassandra
HTTP Layer, and Manager Layer EVCache
Crowd/SSO
RDS
Astyanax
Netflix OSS Cloud Components

6.
Netflix OSS Components
https://github.com/netflix
http://techblog.netflix.com

7.
Simian army

8.
Asgard

9.
Eureka

10.
Karyon

11.
Astyanax
https://github.com/Netflix/astyanax
Astyanax is a Java Client for Cassandra

12.
EVCache

13.
Development
We to code.

14.
Test first
We write a ton of tests.

15.
Test coverage is a measure of confidence.

16.
ScalaTest
simple
intuitive
english-like
rich
promotes BDD

17.
Test first
Manager Layer
No HTTP requests, simple method invocation

18.
/**
Lets try to build a login endpoint. It should support a method called
login(user:String,pass:String) that returns an Option[String].
*/
class LoginManagerSpec extends FlatSpec with ShouldMatchers {
}

19.
/**
Lets try to build a login endpoint. It should support a method called
login(user:String,pass:String) that returns an Option[String].
*/
class LoginManagerSpec extends FlatSpec with ShouldMatchers {
it should " Be able to login a valid user and get a token " in {
fail()
}
}

20.
/**
Lets try to build a login endpoint. It should support a method called
login(user:String,pass:String) that returns an Option[String].
*/
class LoginManagerSpec extends FlatSpec with ShouldMatchers {
it should " Be able to login a valid user and get a token " in {
val token = LoginManager.login("someuser", "somepassword")
token should not be None
}
}

21.
/**
Lets try to build a login endpoint. It should support a method called
login(user:String,pass:String) that returns an Option[String].
*/
class LoginManagerSpec extends FlatSpec with ShouldMatchers {
it should " Be able to login a valid user and get a token " in {
val token = LoginManager.login("someuser", "somepassword")
token should not be None
}
it should " Fail to login an invalid user " in {
fail
}
}

22.
/**
Lets try to build a login endpoint. It should support a method called
login(user:String,pass:String) that returns an Option[String].
*/
class LoginManagerSpec extends FlatSpec with ShouldMatchers {
it should " Be able to login a valid user and get a token " in {
val token = LoginManager.login("someuser", "somepassword")
token should not be None
}
it should " Fail to login an invalid user " in {
val token = LoginManager.login("fail", "fail")
token should be (None)
}
}

23.
HTTP, or the Scalatra Layer

24.
Scalatra
very simple, Sinatra-inspired framework
routes defined with code
json4s
Swagger
ScalatraSpec

25.
A Simple API in Scalatra
class LoginService extends ScalatraServlet

26.
A Simple API in Scalatra
class LoginService extends ScalatraServlet {
post("/") {
}
}

27.
A Simple API in Scalatra
class LoginService extends ScalatraServlet {
before() {
contentType = formats("json")
}
post("/") {
val token = LoginManager.login(params("user"), params("password"))
token match{
case None => Unauthorized(Map("message"->"Login failed"))
case Some(x) => Ok(Map("token"->x))
}
}
}

28.
Putting it all together..
class ScalatraBootstrap extends LifeCycle {
override def init(context: ServletContext) {
context.mount(new LoginService, "/login/*")
}
}
<listener>
<listener-class>org.scalatra.servlet.ScalatraListener</listener-class>
</listener>

29.
$ curl http://localhost:8080/login --form "user=foo&password=bar"
{"token":"02055794-1928-11e3-9a3b-f23c91aec05e"}

30.
ScalatraSpec
traits to take ScalaTest to the next level
support for all HTTP methods
helpers for JSON parsing
plenty of wrappers (body, headers..)

31.
class LoginServiceSpec extends ScalatraFlatSpec {
}

32.
class LoginServiceSpec extends ScalatraFlatSpec {
addServlet(classOf[LoginService], "/*")
}

33.
class LoginServiceSpec extends ScalatraFlatSpec {
addServlet(classOf[LoginService], "/*")
it should "log in valid users" in {
post("/", body = """user=gooduser&password=goodpassword""") {
status should equal(200)
body should include "token"
}
}
}

34.
class LoginServiceSpec extends ScalatraFlatSpec {
addServlet(classOf[LoginService], "/*")
it should "log in valid users" in {
post("/", body = """user=gooduser&password=goodpassword""") {
status should equal(200)
body should include "token"
}
}
it should "not allow invalid users to log in" in {
post("/", body = """user=baduser&password=badpassword""") {
status should equal(401)
body should include "message"
}
}
}

35.
APIs Best Practices
Use Proper HTTP Response Codes
Set Proper HTTP Headers
Break up your data into groups

36.
Pop Quiz!
Lets do some HTTP response codes..

37.
Pop Quiz!
What is the response code for an async operation?

38.
Pop Quiz!
…forbidden?

39.
Pop Quiz!
…a delete?

40.
Git Workflow
work on the dev branch
write tests
leave the rest to Jenkins

41.
Git Workflow
$ git status
# On branch dev
# Changes not staged for commit:
# (use "git add <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# modified: src/main/scala/com/netflix/nrdportal/http/DpiService.scala
# modified: src/test/scala/com/netflix/nrdportal/http/DpiServiceSpec.scala

42.
Automated Code Pushes
Push to dev
Jenkins runs
dev build,
tests, merges
to master
Jenkins runs
master build,
makes an
RPM
Aminator
bakes an AMI
from the RPM
asgard
deploys the
AMI in staging
cloud

43.
Scala Best Practices
Using Options
Using Try[A] vs. Exceptions
Wrappers
Control Abstractions

44.
The dreaded null
public String willReturnNullForOdds(int x){
if(x%2==0) return "Even";
else return null;
}

45.
Using Options
def willReturnNullForOdds(x: Int): Option[String] = {
if (x % 2 == 0) Some("Even") else None
}

46.
Using Options as Wrappers
…where you have to call code that can return null
def returnNull(x:Int) = if(x%2 == 0) "Even" else null
scala> Option(returnNull(3))
res01: Option[String] = None
scala> Option(returnNull(2))
res02: Option[String] = Some(Even)

47.
Exceptions?
public String willThrowExceptionForOdds(int x){
if(x%2==0) return "Even";
else throw new IllegalArgumentException("Odd Number!");
}

48.
Using Try[A]
def someFunction(x: Int): Try[String] = {
if (x % 2 == 0) Success("Even")
else Failure(new IllegalArgumentException("Odd number!"))
}

49.
Control Abstractions
def withAuthenticatedUser(f: (String) => ActionResult) = {
getUserNameFromCookie match {
case Some(userName) => f(userName)
case None => Unauthorized("You are not logged in!")
}
}
def printCurrentUserName = {
withAuthenticatedUser {
userName => Ok(s"Your username is ${userName}")
}
}

50.
Finally…
Avoid writing complex code at all costs – there are better
ways to prove your awesomeness!

51.
Manish Pandit
@lobster1234
mpandit@netflix.com
linkedin.com/in/mpandit
slideshare.net/lobster1234