This document discusses the development of a real-time ad server on Heroku that serves 8.5 million requests per week. It describes how the application evolved over time to address issues like redundancy, scale, disaster recovery, load balancing, and monitoring through the use of Heroku add-ons like Redis, Postgres, New Relic, SendGrid, and Papertrail. Key aspects of the architecture discussed include using Redis for caching, Postgres for the database, S3 for file storage, and services like Cloudfront, RabbitMQ, and Memcache.

1. Developing a Real Time Ad-Server on Heroku
The evolution of an application that today serves 8.5
million requests a week to airline passengers.
Abhinav Keswani & Bry Ashman, Trineo Ltd
@trineoltd

2. Safe Harbor
Safe harbor statement under the Private Securities Litigation Reform Act of 1995:
This presentation may contain forward-looking statements that involve risks, uncertainties, and assumptions. If any such uncertainties materialize or if
any of the assumptions proves incorrect, the results of salesforce.com, inc. could differ materially from the results expressed or implied by the forward-
looking statements we make. All statements other than statements of historical fact could be deemed forward-looking, including any projections of
product or service availability, subscriber growth, earnings, revenues, or other financial items and any statements regarding strategies or plans of
management for future operations, statements of belief, any statements concerning new, planned, or upgraded services or technology developments
and customer contracts or use of our services.
The risks and uncertainties referred to above include – but are not limited to – risks associated with developing and delivering new functionality for our
service, new products and services, our new business model, our past operating losses, possible fluctuations in our operating results and rate of growth,
interruptions or delays in our Web hosting, breach of our security measures, the outcome of intellectual property and other litigation, risks associated
with possible mergers and acquisitions, the immature market in which we operate, our relatively limited operating history, our ability to expand, retain,
and motivate our employees and manage our growth, new releases of our service and successful customer deployment, our limited history reselling
non-salesforce.com products, and utilization and selling to larger enterprise customers. Further information on potential factors that could affect the
financial results of salesforce.com, inc. is included in our annual report on Form 10-Q for the most recent fiscal quarter ended July 31, 2012. This
documents and others containing important disclosures are available on the SEC Filings section of the Investor Information section of our Web site.
Any unreleased services or features referenced in this or other presentations, press releases or public statements are not currently available and may
not be delivered on time or at all. Customers who purchase our services should make the purchase decisions based upon features that are currently
available. Salesforce.com, inc. assumes no obligation and does not intend to update these forward-looking statements.

3. Abhinav Keswani
Director // Principal, Bespoke Solutions
@wasabhi

4. Bry Ashman
Lead Developer, Bespoke Solutions
@yrb

6. SpotU

8. SpotU Architecture

9. Why Heroku?

10. Diversifying
Redundancy
Automation Disaster Recovery
Scale
Replication
OS
Tuning DIY?
Load Balancing Database
Monitoring Networking

11. BACKUPS?!!?

12. Heroku Add-Ons // Modularity
New Relic, Sendgrid, RedisToGo,
Postgres, BlitzIO, Scheduler,
Memcache, Papertrail.

13. Campaign
Redirector Redis Postgres
Manager

14. Backing Services
(12factor.net)

15. Data Stores
Redis // Postgres

16. Redis
in memory key:value store
high speed atomic ops
ultimate ‘scratchpad’

17. Heroku Redis Add-On
RedisToGo
Fast to provision
Options for growth

18. Redis
Used by resque
Used for campaign map
Ad cache

19. Postgres
Relational Database of choice

20. Heroku Postgres
Provisioning
Backups
Follow/Fork
Data Clips

21. Content
S3
Cloudfront

22. S3
The only option for
storing files in the cloud?

23. Cloudfront

24. Nifty Extras: Blitz

25. Blitz
Lay siege to your app.

26. Operational Management
New Relic
Papertrail

27. New Relic
Know what’s going on when your
app is under siege

28. Papertrail
Logs: streaming, searchable and
actionable events.

29. Production Traits
Architectural Evolution

30. Redis

31. Messaging // AMQP

32. Response Cache // Memcache

33. Data Architecture

34. Memcache
Redirector Redis Campaign Postgres
Manager
Analytics
Logging
RabbitMQ
...

35. Latency // Multi Region

36. Perspective
1 million requests
100ms latency/request
~ 28 hours in transit

37. Memcache
Redirector Redis Campaign Postgres
Manager
Analytics
Logging
RabbitMQ
...

38. Abhinav Keswani Bry Ashman
Chief Context Switcher, Chief Worrier,
@wasabhi @yrb