Scala at foursquare

3,996 views

Published on

Presentation at the New York Scala Meetup - 10/20/2010

Published in: Technology, Education
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,996
On SlideShare
0
From Embeds
0
Number of Embeds
93
Actions
Shares
0
Downloads
41
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Scala at foursquare

  1. 1. Scala @foursquare Jorge Ortiz @jorgeortiz85
  2. 2. foursquare runs on... • Scala 2.8.0 • Lift 2.1 • MongoDB • Twitter open source projects • ~46k lines of Scala
  3. 3. foursquare server team • 9 out of 11 had no major prior Scala experience • Previously: Java, Python, C# • Most exposed to: ML, Lisp
  4. 4. foursquare loves Scala • Pimped: Option, List, Map • Type-safe equality: =? • BackgroundOperation { ... } • Throttle(“secret feature!”) { ... } • Any Java library
  5. 5. 2.7.7 => 2.8.0 (painless migration) • Compile: 2.7.7 with -deprecation • Branch • Compile 2.8.0 without -deprecation • Compile 2.8.0 with -Xmigration • Compile 2.8.0 with -Xwarninit • Switch
  6. 6. foursquare on Lift • REST API • Out-of-the-box: • Rich Ajax/Comet support • ORMs for PostgresQL, MongoDB • Random: Email, Facebook, JSON, utils...
  7. 7. Lift Deployment • nginx • HAProxy • API machines - stateless • WWW machines - stateful
  8. 8. GC issues • Stateless API machines • Tens of millions of requests per day • Only a dozen or so servers • 12 GB heap => 15sec GCs *ouch*
  9. 9. GC issues • GC tuning? • “There's no way to get around the fact that we're allocating faster than we can collect.” • Allocating ~200 MB/s • Opportunity to reduce wasteful allocations • Better load-balancing strategy • Cap # of concurrent connections
  10. 10. GC issues • Stateful WWW machines • Stateful Lift + Scala closures + Java GC • Lots of variables getting caught in Session • val venues: RequestCache[List[Venue]] = RequestCache(Venue.findAll(...))
  11. 11. MongoDB in Scala • Some support built-in to Lift • No expressive query language, so we wrote one: Venue where (_.mayor eqs userid) modify (_.tags push “I’m mayor!”) updateMulti
  12. 12. Twitter open source • Kestrel • Ostrich • GrabbyHands • (probably more soon...)
  13. 13. We’re hiring http://foursquare.com/jobs
  14. 14. Questions?

×