OK… let’s get started. Welcome to Build your API with Force.com and Heroku. I’m Jeff Douglas and as the title aptly describes I’m going to give you some pointers on building an API on top of force.com and then we’ll build a quick API using Node.js. We only have 30 minutes so let’s get going.
OK… so here’s the ever popular safe harbor statement. I’m sure you are quite familiar with it and have read it thoroughly numerous times.
Again I’m Jeff Douglas, a senior technical consultant at appirio and the platform architect at CloudSpokes. Now they told me to toot my own horn, so I’m one of the original 10 Force.com MVPs, I won the Force.com Developer Hero award at Dreamforce 2010, I have a number of salesforce certifications, I write a popular salesforce blog at blog.jeffdouglas.com with tons of code and example applications and I’m also the co-author of the Salesfoce Handbook.
I’m going to be talking about our experience building an API on top of Force.com for CloudSpokes. Is anyone familiar with CloudSpokes? CloudSpokes a crowdsourcding development community where roughly 600K members compete in design, coding and algorithm challenges for cash, prizes, badges and bragging rights. The entire community is built on Force.com so we manage members, challenges, payments, settings, virtually everything in force.com. We expose this functionality to our website using an API that we originally wrote in ruby a couple of years ago but recently ported to node. I’m going to talk about some lessons we’ve learned over the past two years, some best practices to keep in mind and some gotchas to watch out for.
So the big question is WHY build an API when you can just use the existing SOAP or REST APIs that salesforce offers. I think there are a number of valuable use cases for building your own API that wraps Force.com. 1. Perhaps your company has a large development shop and doesn’t want to give everyone direct access to the salesforce org. Let’s say the SAP team wants some sales data and perhaps the web team needs some type of real-time integration with pricing. In this situation, you can build an API that exposes only the data and functionality that these teams need without having to manage their access to your org. They are limited to the functionality that your api provides. 2. Maybe you are an ISV with an existing Force.com application and want to make it easier for existing customer to consume your platform. Spin up an api that exposes this functionality and make it super easy for developers who don’t necessary know how to work with Force.com. 3. Or you want to build an entire business on top of Force.com. This is exactly what we did at CloudSpokes. We started off with the paradigm that Force.com would hold our data and business logic and our API would expose this functionality to anyone that wanted to consume it. They don’t know or care that we are using Force.com for our backend and that’s the way it should be.
Why chose Force.com when there are so many other alternatives? This is my standard “why is force.com so awesome” slide. There are too many reasons for this slide but some of the obvious ones are that it’s quick and relatively easy to build your business logic in Apex. It’s a fairly mature and feature rich language and enforces good development practices. Plus the declarative nature of the platform allows non-coders to contribute to the development process with configuration, workflows, formulas and email notifications. Security is probably the biggest advantage to using Force.com. I’m just telling you right now, don’t build your own user authentication, profile management and access layer. What you get out of the box with force.com is rock solid. So at CloudSpokes we have a private sharing model for our Challenge object but by default each challenge is shared with an “all members” public group so everyone has access to it.So if we want to make a challenge private to only CloudSpokes administrators and a x number of members, we remove the “all members” share, and add shares for the individual members and the “cloudspokes admin” public group. So now when the API makes a SOQL query, if the member has access to the challenge, it will be returned normally. If they don’t have access, the API simply returns a 404. From the API side I don’t have to do anything special at all. Force.com handles all of the security for me. Super simple. How much time and development effort do you think companies spend building and maintaining an admin UI for their application? My guess is probably 20%-30%. With Force.com you can spend this time building a better product as the as you get most of this admin functionality free with Force.com. Add a new field to a object and the CRUD UI is immediately updated, it’s available in reporting, for web services and much much more. Plus three times a year you get new features without even doing anything. Another advantage is the complete development lifecycle and tools available. If you’re running a Java stack on Windows with SQL Server how long does it take your IT department to get a new server provisioned? My guess is significantly longer than the amount of time it takes me to spin up a new sandbox or sign up for a DE org to test newly released features. I have an number of friends whose companies spend a good chuck of change at Amazon just on development AMIs. We get that with force.com as part of the platform. Lastly I’m just going to say two words: reports and dashboards. I don’t want to build them by hand. Let the business users take care of that stuff.
We use heroku for everything at cloudspokes and absolutely love it. In my opinion heroku is the rainbow colored unicorn of application development. You can do almost anything with heroku. It’s a polyglot platform that supports node.js, java, python, ruby, scala and almost any other language with buildpacks. You can easily scale application up and down with a single command to handle spikes in traffic and deployment is a breeze with git. Heroku has baked in a lot of features that make development stupid simple. Let’s say you need a copy of your application for QA testing or training or something, you can create an exact replica along with all of its addons simply by forking the application. It’s a one line command and the entire process takes about 15 seconds. If you haven’t used postgres you should try it. We love postgres and heroku adds some awesomesauce by allowing you to fork a database to make an exact copy, create a read-only follower for things like reporting and analytics and you can even share queries externally with dataclips. They also have a growing number of addons for a bunch of services like mail, caching, queueing, elasticsearch all for ridiclously low prices. Why should I build a logger service when I can easily add Papertrail for $7 month. They wrap this all up with a free usage tier to get started plus excellent support. So the answer is yes….they are a rainbow colored unicorn of application development.
So lets talk about designing your api. There’s a lot that goes into designing a greate API but I’ll touch on a few high level ideas. Keep it simple. As a developer I want to look at an API and be able to grok it quickly. I want great documentation when I get stuck that includes code samples and test cases.
When building your api don’t go crazy and try to reinvent the wheel. Use existing standards like REST, JSON and Oauth. Sure, you could use SOAP and XML but honestly no one likes that stuff anymore.
When building your API use proper REST principles but don’t be dogmatic. There will be times where you need to wander off the beaten path. Just remember: Nouns are good. Verbs are bad. The number one principle in pragmatic RESTful design is: keep simple things simple. The base URL is the most important part of your API design. A simple and intuitive base URL makes using your API easy to understand. Verbs shouldn’t be in the base URL. Everything is centered around collections and resources and you use HTTP verbs to operate on them. You can think of these as CRUD methods.
Some other important factors when designing your API. Always return proper HTTP responses but don’t go crazy. Start off with a 200 for success, a 400 for bad request, 401 for not authorized, 404 for not found and a 500 for a internal server error. Don’t get more complex unless unless you absolutely have to. You’ll want to think about your endpoint versioning strategy ahead of time. Typically you can either pass the version number a part of the URL or in the header. We prefer the the URL route so you can easily test endpoints in the browser, the version number is obvious and it’s fewer keystrokes for cURL commands. For simple security, use API Keys to restrict access to your api and endpoints. I’ll show you an example of that in the demo but basically your API should be able to produce a API key that is passed from the client with each request. If the API key is found in the local datastore then they are granted access. If not simply return a 401 response. And finally, hide the complexity of your API behind the “?”. It provides for a more expressive API with far fewer endpoints. And finally, there’s a great presentation called “Teach a Dog to REST” from Dreamforce a couple of years ago that I would recommend you listen to.
OK… so now we know which platforms to use and have a good idea of how to design our API. Now the big choice. Which language to write it in. We actually really don’t care. Some of the more popular languages for APIs are ruby, node and java. Fortunately there are a number of language specific wrappers around the Force.com APIs that you can use along with some other frameworks and tools to make api development much more enjoyable. I would recommend the restforce gem if you are using ruby but my favorite is the nforce package for node.js. Both of these provide authentication, SOQL and DML statements, calling Apex rest services, streaming api and much more. I’ve also written a “forcifier” utility package for ruby and node to make it easier to work with funky salesforce field names. Since languages like ruby and node are case sensitive, the “forcifier” will take the mixed case object field names from force.com and make them lower case with the “__c” removed plus some other helper functions as well. This way the client code never has to know about or deal with the funky salesforce field names. There are a number of language specific frameworks for building RESTful web services. For ruby there’s rocket pants and grape. Grape is more mature and feature-rich but we chose rocket pants since it was a little more light weight and honestly I just really liked saying “rocket pants” all the time. For node you can certainly use Express but you’ll find a lot of applications using Restify. We chose the new Actionhero package and are really happy with it. We’ll use that in our demo in second. If you are using Java, the Play! Framework and Jersey are really popular.
One last thing before we get into some code. One of the questions that come up regularly is the use of the REST API vs. Apex REST services. We highly suggest you use Apex REST Services whenever possible as they are super fast and easy to build! Unlike the REST API where you essentially operate on a single record or collection of records, you can wrap as much functionality as you’d like into a single request with Apex REST Services. For instance, the leaderboard page at CloudSpokes originally made 3 REST calls: one to return the leaderboard for current month, for this year and all time. We dramatically reduced the load time by replacing these three calls with a single Apex REST service that returned all 3 time periods with a single request. You should build your Apex REST Services to run as fast as possible to prevent hitting long running request limits. You can only have 10 concurrent request longer than 5 seconds before the platform starts spitting out errors. Whenever possible, hand off long running requests to asynchronous processes. So when we create a new CloudSpokes member, we quickly return the new member and then hand off the setup and email notification to an asynchronous process to complete when the platform has resource available.
OK… let’s write some code. So this has become my stock “getting down to business” picture. Does everyone know Pat Patterson? He’s a developer evangelist at SFDC? I took this picture of him in Portland Maine last month at the Monktoberfest conference. I love that picture. So the api we are going to build is rather basic since we only have a few minutes. If you need some sample code or reference you can go to the cloudspokes github repo and find the ruby or node version of our api. The code is all open source so feel free to poke around. Ok… so let’s build an API using actionhero, node.js and force.com that uses both the REST API, Apex REST services and handles security with some simple key management.
One last thing to talk about is authentication. You can either one a single “share” user for each call to Force.com or individual named users. You can certainly go the shared user route but we suggest using named users instead for a number of reasons including granular record security, chatter integration, audit trail and record ownership to name a few.
Prototype using the REST API then once you’re happy with the the process, refactor to use Apex REST Services. Cache results from Force.com whenever possible. With our new node API we’ve started using Cloudconnect for some of the site’s functionality. Cloudconnect syncs your data between force.com and either mysql or postgres. So now your developers are happy. They can work with a local database like they are used to instead of dealing with an API. Instead of polling for changes in force.com, use the streaming API to “listen” for any record changes and then process them in your API. Lastly, you’ll need to think carefully about how you version things like Apex classes, triggers and workflows. We try to keep our endpoints in sync with an internal version number inside Force.com but it can become challenging.
The workbench is fantastic tool for development. Can you test the REST API, Apex Rest Services, the Streaming API and much more. If you are not using it, you should. We also use Runscope all the time for debugging, testing and sharing API calls. You saw a minute ago I was using Postman to make REST calls. I can’t live without Postman. Then we use IODocs for documenting and testing calls. You can also look at Swagger which does the same job. We’ve recently started using Apiary.io for API development. You can declaratively build your interface with RABL and it provides a mock interface that you can gradually replace with your actual implementation. It’s pretty sweet.
Lastly a couple of salesforce gotchas: maintenance downtime and sandbox refreshes and upgrades. Since your API is dependent upon Force.com then it’s offline during this period. This is one of the reasons we implemented Cloudconnect. One of the things that’s a real pain is writing rspecs or mocha tests for your API. Typically when you write your test for ruby or node it’s against some local testing database that is rolled back after each test run. However, since your tests are hitting Force.com these are not rolled back because it looks like a normal call for Force.com and not an actual test. This can be a real pain so think ahead about how to handle this with some common test data or perhaps setup and tear down methods in your test framework.
OK.. So that’s it. Feel free to ask any questions now or find me after the session.
Build your API with Force.com and Heroku
Build your API with Force.com &
Jeff Douglas, Appirio, CloudSpokes Platform Architect
Safe harbor statement under the Private Securities Litigation Reform Act of 1995:
This presentation may contain forward-looking statements that involve risks, uncertainties, and assumptions. If any such uncertainties
materialize or if any of the assumptions proves incorrect, the results of salesforce.com, inc. could differ materially from the results
expressed or implied by the forward-looking statements we make. All statements other than statements of historical fact could be
deemed forward-looking, including any projections of product or service availability, subscriber growth, earnings, revenues, or other
financial items and any statements regarding strategies or plans of management for future operations, statements of belief, any
statements concerning new, planned, or upgraded services or technology developments and customer contracts or use of our services.
The risks and uncertainties referred to above include – but are not limited to – risks associated with developing and delivering new
functionality for our service, new products and services, our new business model, our past operating losses, possible fluctuations in our
operating results and rate of growth, interruptions or delays in our Web hosting, breach of our security measures, the outcome of any
litigation, risks associated with completed and any possible mergers and acquisitions, the immature market in which we operate, our
relatively limited operating history, our ability to expand, retain, and motivate our employees and manage our growth, new releases of
our service and successful customer deployment, our limited history reselling non-salesforce.com products, and utilization and selling to
larger enterprise customers. Further information on potential factors that could affect the financial results of salesforce.com, inc. is
included in our annual report on Form 10-K for the most recent fiscal year and in our quarterly report on Form 10-Q for the most recent
fiscal quarter. These documents and others containing important disclosures are available on the SEC Filings section of the Investor
Information section of our Web site.
Any unreleased services or features referenced in this or other presentations, press releases or public statements are not currently
available and may not be delivered on time or at all. Customers who purchase our services should make the purchase decisions
based upon features that are currently available. Salesforce.com, inc. assumes no obligation and does not intend to update these
CloudSpokes Platform Architect
All about CloudSpokes
CloudSpokes is a crowdsourcing development community
where members compete in design, coding and algorithm
challenges for cash, prizes, badges and bragging rights.
~600,000 member community
40-60 development challenges in-flight at any one time
Force.com platform manages members, challenges, payments, etc.
API calls Force.com via REST and JSON
Why build an API?
IT departments can provide limited access
to internal development teams without
exposing the implementation details of
ISVs can better serve existing customers
by extending their platform
Startups and entrepreneurs can build their
business upon Force.com
Why choose Force.com?
Build your business logic in Apex
Security – FTW!
No admin UI to develop and maintain
New features three times a year
Sandbox environments, tools, deployment processes, etc.
Reports & dashboards
Why choose Heroku?
Scale up and down rapidly
Distributed development with Git
Fork applications for testing, QA, etc.
Postgres database (fork, follow, share)
Free to get started
Designing your API
Developers want the following:
Simple, intuitive and easy to learn
Building your API
Adopt web standards. Don’t reinvent the wheel.
Use proper REST principles
Nouns are good. Verbs are bad.
Other API design considerations
HTTP response and error codes
• 200, 400, 401, 404 & 500
Versioning (header vs. URL)
Key management & security
Hide complexity behind “?”
Poor design: /challenges/open/5
Better design: /challenges?status=open&limit=5
Teach a Dog to REST
Which development language is best?
Language specific REST wrappers for Force.com as well as tools
to make it easier to build well-designed APIs.
• restforce gem, forcifier gem
• Rocket_pants, Grape
• nforce package, forcifier-node package
• Actionhero, Restify, Express
• Play!, Jersey
Apex REST Services vs. REST API
The REST API is great for interacting with records while Apex
REST Services can encapsulate numerous operations into a
Design to run as quickly as possible
Handoff long running requests to asynchronous processes
Authenticate your API with Force.com
Single “shared” user
• Audit trail & history
• Record ownership
Prototype with REST API then refactor to Apex REST Services
Caching is cheap and fast with Memcache and redis
Sync to local datastore with Cloudconnect.com
“Listen” for record changes using the Streaming API
Think about how to version Apex classes, triggers and workflows
Runscope for debugging, testing and sharing API calls
Chrome extensions: Postman, REST Console
Documenting and testing calls with IODocs, Swagger and Apiary.io
Salesforce.com maintenance downtime
Sandbox refreshes and upgrades
Hard to write API unit tests against sandbox environments