Introducing Scala to your     Java/Ruby Shop      My experiences at IGN                           Manish Pandit           ...
About me             Manish Pandit      Director of Engineering, IGN             @lobster1234       linkedin.com/in/mpandit
About IGNWe are a leading online media and servicescompanyobsessed with gaming and entertainment.56MM Monthly Unique Views...
My ObjectiveIs:To share my story of introducing Scala to IGN, and(hopefully) show you the path to enlightenment.Isnt:To in...
The IGN Tech Stack
Best tool for the job• Front End : JS, CSS3, HTML5, backbone,  Coffeescript, jQuery• Middle Tier : ZF2 (PHP), Rails• APIs ...
APIs at IGN : Numbers• ~5 Billion requests a month• Average Response time of under  20ms on cache misses• 17 APIs between ...
The Kitchen Sink• CMS – JSPs talking to Oracle DB and some Java  services• V1 API : Java Services using Oracle DB with  Me...
The API Evolution : V3 [2012]• A fresh look at the APIs and IGN’s direction• First attempt to  – Integrate all content typ...
Roadblocks• Resistance to change  – Culture  – Willingness and commitment• Too many choices – making the right pick  – Inv...
The hunt for…•   A runtime as fast, if not faster than Java•   Concise, yet expressive•   Less boilerplate•   Smooth learn...
Why Scala : Strategic Reasons• Evolve the talent brand along side evolving  the platform• Establish an API platform by rep...
Why Scala : Tactical Reasons• Performance tied to cores than processor  speed, i.e. concurrency• Ideal for API development...
Why Scala : Tactical Reasons•   Richer Collections API•   Traits enabling Mix-Ins and behavior re-use•   Statically Typed ...
Yes, Virginia, Scala is hard : DPPSo, how can you figure out if Scala will be "easy" or "hard" for yourorganization:•Your ...
Picking up Scala : Week 1• Scala for Java Developers fast-tracked the  learning• Syntax Familiarity• Java without semicolo...
Picking up Scala : Week 4• Reduce the lines of code, embrace  immutability, use the functional paradigms  built into the l...
Picking up Scala : Today• Focus on immutability• Think functional – with the knowledge gained  from using the functional a...
Less Boilerplate  public class HelloWorld{     public static void main(String... args){           System.out.println("Hell...
Expressive, yet concise  val x = if(n%2==0) "Even" else "Odd"  val x = for(i <- 1 to 10) yield i*2  val x = for(i <- 1 to ...
Less pain points  No more dreaded null or throwing unwanted exceptions :  Use Option[T]  def thisMayReturnNull[T](x:T):Opt...
Try-catch messtry{       //Construct a URL which can throw a MalformedURLException       //do something with a URL which c...
Type Inferencescala> val x = "Sunday"x: java.lang.String = Sundayscala> def x(n:Int) = n*3x: (n: Int)Intscala> 1 to 5res2:...
Dynamic Mix-insclass Home(bedrooms:Int=3,bathrooms:Double=2.5)trait Garage{  def park(n:Int) = println("Parking " + n + "c...
Concurrency• Actors make concurrency so much easier  – Messaging vs. Blocking  – Lightweight, 300-500 bytes per instance  ...
Scala API Components•   Scalatra•   Lift-MongoRecord for MongoDB•   Casbah for MongoDB•   PlayFramework 2•   Actors for ta...
Other API Components•   Varnish Cache•   Elasticsearch•   Yammer Metrics (ping, healthcheck)•   Swagger (self-documenting ...
Challenges with Scala                             *• Steepness of the Learning curve depends  entirely on your org culture...
Challenges with Scala• Slow compilation based on the source• No (binary) compatibility of dependencies for  the major rele...
Lessons Learned : Culture• Culture of Meetups, Hack  Nights and Code Jams• Being able to connect with  the contributors an...
Lessons Learned : Team•   Small team (~5 engineers)•   Java background helped a lot•   10% time set aside for skill buildi...
Lessons Learned : Material• Books that have exercises, so you  can quantify your learning.• Online tutorials• StackOverflo...
Resources• Books   – Programming in Scala by Odersky   – Scala for the Impatient by Horstmann• Interwebz   – http://debasi...
scala> case class Me(name:String="ManishPandit",twitter:String="@lobster1234")defined class Mescala> val me = new Meme: Me...
Introducing Scala to your Ruby/Java Shop : My experiences at IGN
Upcoming SlideShare
Loading in …5
×

Introducing Scala to your Ruby/Java Shop : My experiences at IGN

1,725 views
1,560 views

Published on

With the move to SOA, IGN's API engineering became a traditional Java/Ruby shop. We introduced Scala to the team in August '11. As of April '12 we have our major services running on Scala, and we could not be happier. I'll talk about the strategic & tactical rationale, gradual introduction to the new language, cultivating experts and mentoring junior developers. We will also cover the common roadblocks the leaders may run into, addressing the concerns of engineers and stakeholders, and building an engineering culture that facilitates innovation and talent growth. Please note that this talk is not a Ruby/Java/Scala religious debate - we believe in using the best tool for the job; and what works for us may not work for every org (which is where the culture comes in). My team's journey from "Holy crap! This looks academic and scary!" to "Let me whip out a service by EOD today" in a matter of months is something I am very proud of, and would love to share it with fellow developer community.

Published in: Technology
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,725
On SlideShare
0
From Embeds
0
Number of Embeds
22
Actions
Shares
0
Downloads
15
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide
  • ----- Meeting Notes (10/4/12 16:32) ----- Scala in production? Playing around with Scala?
  • Introducing Scala to your Ruby/Java Shop : My experiences at IGN

    1. 1. Introducing Scala to your Java/Ruby Shop My experiences at IGN Manish Pandit Silicon Valley Code Camp ‘12 Oct 7th, 2012
    2. 2. About me Manish Pandit Director of Engineering, IGN @lobster1234 linkedin.com/in/mpandit
    3. 3. About IGNWe are a leading online media and servicescompanyobsessed with gaming and entertainment.56MM Monthly Unique Views737MM Monthly Page Views20MM Monthly Video Views
    4. 4. My ObjectiveIs:To share my story of introducing Scala to IGN, and(hopefully) show you the path to enlightenment.Isnt:To incite a functional vs. imperative, or Scala vs. ____debate.
    5. 5. The IGN Tech Stack
    6. 6. Best tool for the job• Front End : JS, CSS3, HTML5, backbone, Coffeescript, jQuery• Middle Tier : ZF2 (PHP), Rails• APIs : Scala, some Java• Persistence : MongoDB, MySQL, Redis• Search : ElasticSearch• Caching : Memcached, Varnish
    7. 7. APIs at IGN : Numbers• ~5 Billion requests a month• Average Response time of under 20ms on cache misses• 17 APIs between Social, OAuth, and Content• Hardest hit APIs doing about 25K RPM at peak
    8. 8. The Kitchen Sink• CMS – JSPs talking to Oracle DB and some Java services• V1 API : Java Services using Oracle DB with Memcached• V2 API : Rails based API layer using MongoDB and a Java/Memcached front cache• Lots of duct tape like messaging and cron jobs for simple tasks• Too diverse of a stack in the API infrastructure
    9. 9. The API Evolution : V3 [2012]• A fresh look at the APIs and IGN’s direction• First attempt to – Integrate all content types – Open up the APIs to external consumers – Evolve the CMS, traditionally a monolithic system – ElasticSearch – Varnish Response Caching• A learning opportunity for traditional Java stack engineers
    10. 10. Roadblocks• Resistance to change – Culture – Willingness and commitment• Too many choices – making the right pick – Involve every one – Do not decide by committee, decide by learning – Time box and measure everything – Be accountable, as you’re looked upon as the expert
    11. 11. The hunt for…• A runtime as fast, if not faster than Java• Concise, yet expressive• Less boilerplate• Smooth learning curve and ramp up• Re-use ecosystem of libraries• Growing adoption and developer velocity• Built in support for concurrency• Fast and fun development• Next level of programming – disrupt!
    12. 12. Why Scala : Strategic Reasons• Evolve the talent brand along side evolving the platform• Establish an API platform by replacing “rewrite” with learning instead of porting• Start thinking functional• Align with the leaders in the APIs• Get involved with a technology as it is evolving (vs. established). Helps with influence.
    13. 13. Why Scala : Tactical Reasons• Performance tied to cores than processor speed, i.e. concurrency• Ideal for API development – (relatively) simple to handle concurrency and immutability• JVM based = Performance• Re-use Java toolkits and libraries• Concise, yet expressive code• Actor model makes the difficult parts easy
    14. 14. Why Scala : Tactical Reasons• Richer Collections API• Traits enabling Mix-Ins and behavior re-use• Statically Typed w/Type inference• Functional, but not alien (it can be!)• REPL
    15. 15. Yes, Virginia, Scala is hard : DPPSo, how can you figure out if Scala will be "easy" or "hard" for yourorganization:•Your company has speakers at JavaOne, OSCON, Strangle Loop, QCon: Scalawill be easy•Lunch-time discussions involve the criteria for moving from a developer to asenior developer: Scala will be hard•Your developers can write code in NotePad if they have to: Easy•Your developers stare blankly or say 3 "Hail Marys" when they hear thename "Zed Shaw": Scala == Hard•Developers all follow Dean Wampler on Twitter: Scala Easy•Your developers come in at 9:15 and leave before 6 and dont check workemail at night: Hard http://blog.goodstuff.im/yes-virginia-scala-is-hard
    16. 16. Picking up Scala : Week 1• Scala for Java Developers fast-tracked the learning• Syntax Familiarity• Java without semicolons? def total(nums: List[Int]) :Int = { var sum:Int=0 for(num<-nums) { sum+=num } sum }
    17. 17. Picking up Scala : Week 4• Reduce the lines of code, embrace immutability, use the functional paradigms built into the language• Unmystrify the implicits magic nums.foldLeft(0)((n,c) => n+c)
    18. 18. Picking up Scala : Today• Focus on immutability• Think functional – with the knowledge gained from using the functional aspects of the language, build control abstractions• Explore and implement ScalaZ• Keep learning, and applying• Move all the Actor processing to Akka
    19. 19. Less Boilerplate public class HelloWorld{ public static void main(String... args){ System.out.println("Hello, World!"); } } object HelloWorld extends App{ println("Hello, World!") } scala> println("Hello, World!")
    20. 20. Expressive, yet concise val x = if(n%2==0) "Even" else "Odd" val x = for(i <- 1 to 10) yield i*2 val x = for(i <- 1 to 10; if(i%2==0)) yield i*2 val y = 20 match{ case p if(p%2==0) => "Even" case _ => "Odd" } val largest = List(1,5,2,6).foldLeft(0)((a,b)=> if(a>b) a else b)
    21. 21. Less pain points No more dreaded null or throwing unwanted exceptions : Use Option[T] def thisMayReturnNull[T](x:T):Option[T]={ if(true) Some(x) else None } No more creating classes only to return more than 1 value : Use Tuples def doublePair(x:Int)= (x,x*2)
    22. 22. Try-catch messtry{ //Construct a URL which can throw a MalformedURLException //do something with a URL which can throw IOException //do something with string encoding which can throw // UnsupportedEncodingException } catch{ case mal:MalformedURLException => //something case ioe: IOException => //something case e: Exception => //something }
    23. 23. Type Inferencescala> val x = "Sunday"x: java.lang.String = Sundayscala> def x(n:Int) = n*3x: (n: Int)Intscala> 1 to 5res2: scala.collection.immutable.Range.Inclusive =Range(1, 2, 3, 4, 5)scala> def x = if(true) Left("Yes!") else Right(newException("Didn’t work out!"))x: Product with Serializable withEither[java.lang.String,java.lang.Exception]
    24. 24. Dynamic Mix-insclass Home(bedrooms:Int=3,bathrooms:Double=2.5)trait Garage{ def park(n:Int) = println("Parking " + n + "cars")}val someHome = new Homeval someHomeWithGarage = new Home with GaragesomeHomeWithGarage.park(2)
    25. 25. Concurrency• Actors make concurrency so much easier – Messaging vs. Blocking – Lightweight, 300-500 bytes per instance – Isolated
    26. 26. Scala API Components• Scalatra• Lift-MongoRecord for MongoDB• Casbah for MongoDB• PlayFramework 2• Actors for tasks like syndication• sbt and Maven for Builds• MongoDB for persistence
    27. 27. Other API Components• Varnish Cache• Elasticsearch• Yammer Metrics (ping, healthcheck)• Swagger (self-documenting RESTful APIs)• 3Scale for Partner APIs• IntelliJ IDEA and Eclipse with ScalaIDE• PlayFramework 2.0 (New!)• ScalaTest for Testing
    28. 28. Challenges with Scala *• Steepness of the Learning curve depends entirely on your org culture• Scala is what you make of it • http://scalaz.github.com/scalaz/scalaz-2.9.1- 6.0.4/doc.sxr/scalaz/BKTree.scala.html • Middle Ground between Simplicity and, well..elegance* None of these challenges outweigh the benefits. They’re just something tobe aware of.
    29. 29. Challenges with Scala• Slow compilation based on the source• No (binary) compatibility of dependencies for the major releases (2.8, 2.9, 2.10(?))• Tooling not as rich as Java – Changing (rapidly) with Typesafe investing in ScalaIDE – ScalaIDE comes with a Scala Worksheet, which is like REPL but richer
    30. 30. Lessons Learned : Culture• Culture of Meetups, Hack Nights and Code Jams• Being able to connect with the contributors and language creators• Willingness to invest in learning, evolving vs. maintaining
    31. 31. Lessons Learned : Team• Small team (~5 engineers)• Java background helped a lot• 10% time set aside for skill building• Learn and share• It is a commitment• Communication!
    32. 32. Lessons Learned : Material• Books that have exercises, so you can quantify your learning.• Online tutorials• StackOverflow• Coursera : A course on Functional Programming by Martin Odersky
    33. 33. Resources• Books – Programming in Scala by Odersky – Scala for the Impatient by Horstmann• Interwebz – http://debasishg.blogspot.com – http://www.reddit.com/r/scala – http://stackoverflow.com/tags/scala/info – http://twitter.github.com/scala_school – https://www.coursera.org/course/progfun
    34. 34. scala> case class Me(name:String="ManishPandit",twitter:String="@lobster1234")defined class Mescala> val me = new Meme: Me = Me(Manish Pandit,@lobster1234)scala> println(“Questions!”)Questions!

    ×