This document summarizes Foursquare's use of Scala, Lift, MongoDB, and other technologies. Foursquare runs on Scala 2.8.0, Lift 2.1, MongoDB, and Twitter open source projects with around 46k lines of Scala code. Their server team of 11 people had little prior Scala experience but took to it well. They leverage many Scala features and had a painless migration from 2.7.7 to 2.8.0. Lift provides out-of-the-box support for their REST API, ORMs, and utilities. Their deployment uses nginx, HAProxy, stateless API machines, and stateful WWW machines, but

1. Scala @foursquare
Jorge Ortiz
@jorgeortiz85

2. foursquare runs on...
• Scala 2.8.0
• Lift 2.1
• MongoDB
• Twitter open source projects
• ~46k lines of Scala

3. foursquare server team
• 9 out of 11 had no major prior Scala experience
• Previously: Java, Python, C#
• Most exposed to: ML, Lisp

4. foursquare loves Scala
• Pimped: Option, List, Map
• Type-safe equality: =?
• BackgroundOperation { ... }
• Throttle(“secret feature!”) { ... }
• Any Java library

5. 2.7.7 => 2.8.0
(painless migration)
• Compile: 2.7.7 with -deprecation
• Branch
• Compile 2.8.0 without -deprecation
• Compile 2.8.0 with -Xmigration
• Compile 2.8.0 with -Xwarninit
• Switch

7. foursquare on Lift
• REST API
• Out-of-the-box:
• Rich Ajax/Comet support
• ORMs for PostgresQL, MongoDB
• Random: Email, Facebook, JSON, utils...

8. Lift Deployment
• nginx
• HAProxy
• API machines - stateless
• WWW machines - stateful

9. GC issues
• Stateless API machines
• Tens of millions of requests per day
• Only a dozen or so servers
• 12 GB heap => 15sec GCs *ouch*

10. GC issues
• GC tuning?
• “There's no way to get around the fact that
we're allocating faster than we can collect.”
• Allocating ~200 MB/s
• Opportunity to reduce wasteful allocations
• Better load-balancing strategy
• Cap # of concurrent connections

11. GC issues
• Stateful WWW machines
• Stateful Lift + Scala closures + Java GC
• Lots of variables getting caught in Session
• val venues: RequestCache[List[Venue]] =
RequestCache(Venue.findAll(...))

12. MongoDB in Scala
• Some support built-in to Lift
• No expressive query language, so we wrote one:
Venue where (_.mayor eqs userid)
modify (_.tags push “I’m mayor!”)
updateMulti

13. Twitter open source
• Kestrel
• Ostrich
• GrabbyHands
• (probably more soon...)

14. We’re hiring
http://foursquare.com/jobs

15. Questions?