Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Scala at Netflix
Scala Bay Meetup
Netflix, Sept 9th 2013
Manish Pandit
mpandit@netflix.com
@lobster1234
Agenda
Background
Netflix OSS Components
Development
Deployment/Delivery
Open Floor
Partner Product Engineering
Smart devices
+
Certification
=
Lots of Device Metadata!
Model
Firmware
Screen Resolution
Subtitle Support
3D
DRM
Remote Control
Netflix SDK
…
Architecture
Cassandra
HTTP Layer, and Manager Layer EVCache
Crowd/SSO
RDS
Astyanax
Netflix OSS Cloud Components
Netflix OSS Components
https://github.com/netflix
http://techblog.netflix.com
Simian army
Asgard
Eureka
Karyon
Astyanax
https://github.com/Netflix/astyanax
Astyanax is a Java Client for Cassandra
EVCache
Development
We to code.
Test first
We write a ton of tests.
Test coverage is a measure of confidence.
ScalaTest
simple
intuitive
english-like
rich
promotes BDD
Test first
Manager Layer
No HTTP requests, simple method invocation
/**
Lets try to build a login endpoint. It should support a method called
login(user:String,pass:String) that returns an O...
/**
Lets try to build a login endpoint. It should support a method called
login(user:String,pass:String) that returns an O...
/**
Lets try to build a login endpoint. It should support a method called
login(user:String,pass:String) that returns an O...
/**
Lets try to build a login endpoint. It should support a method called
login(user:String,pass:String) that returns an O...
/**
Lets try to build a login endpoint. It should support a method called
login(user:String,pass:String) that returns an O...
HTTP, or the Scalatra Layer
Scalatra
very simple, Sinatra-inspired framework
routes defined with code
json4s
Swagger
ScalatraSpec
A Simple API in Scalatra
class LoginService extends ScalatraServlet
A Simple API in Scalatra
class LoginService extends ScalatraServlet {
post("/") {
}
}
A Simple API in Scalatra
class LoginService extends ScalatraServlet {
before() {
contentType = formats("json")
}
post("/")...
Putting it all together..
class ScalatraBootstrap extends LifeCycle {
override def init(context: ServletContext) {
context...
$ curl http://localhost:8080/login --form "user=foo&password=bar"
{"token":"02055794-1928-11e3-9a3b-f23c91aec05e"}
ScalatraSpec
traits to take ScalaTest to the next level
support for all HTTP methods
helpers for JSON parsing
plenty of wr...
class LoginServiceSpec extends ScalatraFlatSpec {
}
class LoginServiceSpec extends ScalatraFlatSpec {
addServlet(classOf[LoginService], "/*")
}
class LoginServiceSpec extends ScalatraFlatSpec {
addServlet(classOf[LoginService], "/*")
it should "log in valid users" i...
class LoginServiceSpec extends ScalatraFlatSpec {
addServlet(classOf[LoginService], "/*")
it should "log in valid users" i...
APIs Best Practices
Use Proper HTTP Response Codes
Set Proper HTTP Headers
Break up your data into groups
Pop Quiz!
Lets do some HTTP response codes..
Pop Quiz!
What is the response code for an async operation?
Pop Quiz!
…forbidden?
Pop Quiz!
…a delete?
Git Workflow
work on the dev branch
write tests
leave the rest to Jenkins
Git Workflow
$ git status
# On branch dev
# Changes not staged for commit:
# (use "git add <file>..." to update what will ...
Automated Code Pushes
Push to dev
Jenkins runs
dev build,
tests, merges
to master
Jenkins runs
master build,
makes an
RPM
...
Scala Best Practices
Using Options
Using Try[A] vs. Exceptions
Wrappers
Control Abstractions
The dreaded null
public String willReturnNullForOdds(int x){
if(x%2==0) return "Even";
else return null;
}
Using Options
def willReturnNullForOdds(x: Int): Option[String] = {
if (x % 2 == 0) Some("Even") else None
}
Using Options as Wrappers
…where you have to call code that can return null
def returnNull(x:Int) = if(x%2 == 0) "Even" el...
Exceptions?
public String willThrowExceptionForOdds(int x){
if(x%2==0) return "Even";
else throw new IllegalArgumentExcept...
Using Try[A]
def someFunction(x: Int): Try[String] = {
if (x % 2 == 0) Success("Even")
else Failure(new IllegalArgumentExc...
Control Abstractions
def withAuthenticatedUser(f: (String) => ActionResult) = {
getUserNameFromCookie match {
case Some(us...
Finally…
Avoid writing complex code at all costs – there are better
ways to prove your awesomeness!
Manish Pandit
@lobster1234
mpandit@netflix.com
linkedin.com/in/mpandit
slideshare.net/lobster1234
Scala at Netflix
Upcoming SlideShare
Loading in …5
×

Scala at Netflix

My talk at Scala Bay Meetup at Netflix about Powering the Partner APIs with Scalatra and Netflix OSS. This talk was delivered on September 9th 2013, at 8 PM at Netflix, Los Gatos.

Scala at Netflix

  1. 1. Scala at Netflix Scala Bay Meetup Netflix, Sept 9th 2013 Manish Pandit mpandit@netflix.com @lobster1234
  2. 2. Agenda Background Netflix OSS Components Development Deployment/Delivery Open Floor
  3. 3. Partner Product Engineering Smart devices + Certification = Lots of Device Metadata!
  4. 4. Model Firmware Screen Resolution Subtitle Support 3D DRM Remote Control Netflix SDK …
  5. 5. Architecture Cassandra HTTP Layer, and Manager Layer EVCache Crowd/SSO RDS Astyanax Netflix OSS Cloud Components
  6. 6. Netflix OSS Components https://github.com/netflix http://techblog.netflix.com
  7. 7. Simian army
  8. 8. Asgard
  9. 9. Eureka
  10. 10. Karyon
  11. 11. Astyanax https://github.com/Netflix/astyanax Astyanax is a Java Client for Cassandra
  12. 12. EVCache
  13. 13. Development We to code.
  14. 14. Test first We write a ton of tests.
  15. 15. Test coverage is a measure of confidence.
  16. 16. ScalaTest simple intuitive english-like rich promotes BDD
  17. 17. Test first Manager Layer No HTTP requests, simple method invocation
  18. 18. /** Lets try to build a login endpoint. It should support a method called login(user:String,pass:String) that returns an Option[String]. */ class LoginManagerSpec extends FlatSpec with ShouldMatchers { }
  19. 19. /** Lets try to build a login endpoint. It should support a method called login(user:String,pass:String) that returns an Option[String]. */ class LoginManagerSpec extends FlatSpec with ShouldMatchers { it should " Be able to login a valid user and get a token " in { fail() } }
  20. 20. /** Lets try to build a login endpoint. It should support a method called login(user:String,pass:String) that returns an Option[String]. */ class LoginManagerSpec extends FlatSpec with ShouldMatchers { it should " Be able to login a valid user and get a token " in { val token = LoginManager.login("someuser", "somepassword") token should not be None } }
  21. 21. /** Lets try to build a login endpoint. It should support a method called login(user:String,pass:String) that returns an Option[String]. */ class LoginManagerSpec extends FlatSpec with ShouldMatchers { it should " Be able to login a valid user and get a token " in { val token = LoginManager.login("someuser", "somepassword") token should not be None } it should " Fail to login an invalid user " in { fail } }
  22. 22. /** Lets try to build a login endpoint. It should support a method called login(user:String,pass:String) that returns an Option[String]. */ class LoginManagerSpec extends FlatSpec with ShouldMatchers { it should " Be able to login a valid user and get a token " in { val token = LoginManager.login("someuser", "somepassword") token should not be None } it should " Fail to login an invalid user " in { val token = LoginManager.login("fail", "fail") token should be (None) } }
  23. 23. HTTP, or the Scalatra Layer
  24. 24. Scalatra very simple, Sinatra-inspired framework routes defined with code json4s Swagger ScalatraSpec
  25. 25. A Simple API in Scalatra class LoginService extends ScalatraServlet
  26. 26. A Simple API in Scalatra class LoginService extends ScalatraServlet { post("/") { } }
  27. 27. A Simple API in Scalatra class LoginService extends ScalatraServlet { before() { contentType = formats("json") } post("/") { val token = LoginManager.login(params("user"), params("password")) token match{ case None => Unauthorized(Map("message"->"Login failed")) case Some(x) => Ok(Map("token"->x)) } } }
  28. 28. Putting it all together.. class ScalatraBootstrap extends LifeCycle { override def init(context: ServletContext) { context.mount(new LoginService, "/login/*") } } <listener> <listener-class>org.scalatra.servlet.ScalatraListener</listener-class> </listener>
  29. 29. $ curl http://localhost:8080/login --form "user=foo&password=bar" {"token":"02055794-1928-11e3-9a3b-f23c91aec05e"}
  30. 30. ScalatraSpec traits to take ScalaTest to the next level support for all HTTP methods helpers for JSON parsing plenty of wrappers (body, headers..)
  31. 31. class LoginServiceSpec extends ScalatraFlatSpec { }
  32. 32. class LoginServiceSpec extends ScalatraFlatSpec { addServlet(classOf[LoginService], "/*") }
  33. 33. class LoginServiceSpec extends ScalatraFlatSpec { addServlet(classOf[LoginService], "/*") it should "log in valid users" in { post("/", body = """user=gooduser&password=goodpassword""") { status should equal(200) body should include "token" } } }
  34. 34. class LoginServiceSpec extends ScalatraFlatSpec { addServlet(classOf[LoginService], "/*") it should "log in valid users" in { post("/", body = """user=gooduser&password=goodpassword""") { status should equal(200) body should include "token" } } it should "not allow invalid users to log in" in { post("/", body = """user=baduser&password=badpassword""") { status should equal(401) body should include "message" } } }
  35. 35. APIs Best Practices Use Proper HTTP Response Codes Set Proper HTTP Headers Break up your data into groups
  36. 36. Pop Quiz! Lets do some HTTP response codes..
  37. 37. Pop Quiz! What is the response code for an async operation?
  38. 38. Pop Quiz! …forbidden?
  39. 39. Pop Quiz! …a delete?
  40. 40. Git Workflow work on the dev branch write tests leave the rest to Jenkins
  41. 41. Git Workflow $ git status # On branch dev # Changes not staged for commit: # (use "git add <file>..." to update what will be committed) # (use "git checkout -- <file>..." to discard changes in working directory) # # modified: src/main/scala/com/netflix/nrdportal/http/DpiService.scala # modified: src/test/scala/com/netflix/nrdportal/http/DpiServiceSpec.scala
  42. 42. Automated Code Pushes Push to dev Jenkins runs dev build, tests, merges to master Jenkins runs master build, makes an RPM Aminator bakes an AMI from the RPM asgard deploys the AMI in staging cloud
  43. 43. Scala Best Practices Using Options Using Try[A] vs. Exceptions Wrappers Control Abstractions
  44. 44. The dreaded null public String willReturnNullForOdds(int x){ if(x%2==0) return "Even"; else return null; }
  45. 45. Using Options def willReturnNullForOdds(x: Int): Option[String] = { if (x % 2 == 0) Some("Even") else None }
  46. 46. Using Options as Wrappers …where you have to call code that can return null def returnNull(x:Int) = if(x%2 == 0) "Even" else null scala> Option(returnNull(3)) res01: Option[String] = None scala> Option(returnNull(2)) res02: Option[String] = Some(Even)
  47. 47. Exceptions? public String willThrowExceptionForOdds(int x){ if(x%2==0) return "Even"; else throw new IllegalArgumentException("Odd Number!"); }
  48. 48. Using Try[A] def someFunction(x: Int): Try[String] = { if (x % 2 == 0) Success("Even") else Failure(new IllegalArgumentException("Odd number!")) }
  49. 49. Control Abstractions def withAuthenticatedUser(f: (String) => ActionResult) = { getUserNameFromCookie match { case Some(userName) => f(userName) case None => Unauthorized("You are not logged in!") } } def printCurrentUserName = { withAuthenticatedUser { userName => Ok(s"Your username is ${userName}") } }
  50. 50. Finally… Avoid writing complex code at all costs – there are better ways to prove your awesomeness!
  51. 51. Manish Pandit @lobster1234 mpandit@netflix.com linkedin.com/in/mpandit slideshare.net/lobster1234

×