Build your API with and Heroku


Published on

Dreamforce 13 Presentation

Published in: Technology, Design
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • OK… let’s get started. Welcome to Build your API with and Heroku. I’m Jeff Douglas and as the title aptly describes I’m going to give you some pointers on building an API on top of and then we’ll build a quick API using Node.js. We only have 30 minutes so let’s get going.
  • OK… so here’s the ever popular safe harbor statement. I’m sure you are quite familiar with it and have read it thoroughly numerous times.
  • Again I’m Jeff Douglas, a senior technical consultant at appirio and the platform architect at CloudSpokes. Now they told me to toot my own horn, so I’m one of the original 10 MVPs, I won the Developer Hero award at Dreamforce 2010, I have a number of salesforce certifications, I write a popular salesforce blog at with tons of code and example applications and I’m also the co-author of the Salesfoce Handbook.
  • I’m going to be talking about our experience building an API on top of for CloudSpokes. Is anyone familiar with CloudSpokes? CloudSpokes a crowdsourcding development community where roughly 600K members compete in design, coding and algorithm challenges for cash, prizes, badges and bragging rights. The entire community is built on so we manage members, challenges, payments, settings, virtually everything in We expose this functionality to our website using an API that we originally wrote in ruby a couple of years ago but recently ported to node. I’m going to talk about some lessons we’ve learned over the past two years, some best practices to keep in mind and some gotchas to watch out for.
  • So the big question is WHY build an API when you can just use the existing SOAP or REST APIs that salesforce offers. I think there are a number of valuable use cases for building your own API that wraps
    1. Perhaps your company has a large development shop and doesn’t want to give everyone direct access to the salesforce org. Let’s say the SAP team wants some sales data and perhaps the web team needs some type of real-time integration with pricing. In this situation, you can build an API that exposes only the data and functionality that these teams need without having to manage their access to your org. They are limited to the functionality that your api provides.
    2. Maybe you are an ISV with an existing application and want to make it easier for existing customer to consume your platform. Spin up an api that exposes this functionality and make it super easy for developers who don’t necessary know how to work with
    3. Or you want to build an entire business on top of This is exactly what we did at CloudSpokes. We started off with the paradigm that would hold our data and business logic and our API would expose this functionality to anyone that wanted to consume it. They don’t know or care that we are using for our backend and that’s the way it should be.
  • Why chose when there are so many other alternatives? This is my standard “why is so awesome” slide. There are too many reasons for this slide but some of the obvious ones are that it’s quick and relatively easy to build your business logic in Apex. It’s a fairly mature and feature rich language and enforces good development practices. Plus the declarative nature of the platform allows non-coders to contribute to the development process with configuration, workflows, formulas and email notifications.
    Security is probably the biggest advantage to using I’m just telling you right now, don’t build your own user authentication, profile management and access layer. What you get out of the box with is rock solid. So at CloudSpokes we have a private sharing model for our Challenge object but by default each challenge is shared with an “all members” public group so everyone has access to it.So if we want to make a challenge private to only CloudSpokes administrators and a x number of members, we remove the “all members” share, and add shares for the individual members and the “cloudspokes admin” public group. So now when the API makes a SOQL query, if the member has access to the challenge, it will be returned normally. If they don’t have access, the API simply returns a 404. From the API side I don’t have to do anything special at all. handles all of the security for me. Super simple.
    How much time and development effort do you think companies spend building and maintaining an admin UI for their application? My guess is probably 20%-30%. With you can spend this time building a better product as the as you get most of this admin functionality free with Add a new field to a object and the CRUD UI is immediately updated, it’s available in reporting, for web services and much much more. Plus three times a year you get new features without even doing anything.
    Another advantage is the complete development lifecycle and tools available. If you’re running a Java stack on Windows with SQL Server how long does it take your IT department to get a new server provisioned? My guess is significantly longer than the amount of time it takes me to spin up a new sandbox or sign up for a DE org to test newly released features. I have an number of friends whose companies spend a good chuck of change at Amazon just on development AMIs. We get that with as part of the platform.
    Lastly I’m just going to say two words: reports and dashboards. I don’t want to build them by hand. Let the business users take care of that stuff.
  • We use heroku for everything at cloudspokes and absolutely love it. In my opinion heroku is the rainbow colored unicorn of application development. You can do almost anything with heroku. It’s a polyglot platform that supports node.js, java, python, ruby, scala and almost any other language with buildpacks. You can easily scale application up and down with a single command to handle spikes in traffic and deployment is a breeze with git. Heroku has baked in a lot of features that make development stupid simple. Let’s say you need a copy of your application for QA testing or training or something, you can create an exact replica along with all of its addons simply by forking the application. It’s a one line command and the entire process takes about 15 seconds.
    If you haven’t used postgres you should try it. We love postgres and heroku adds some awesomesauce by allowing you to fork a database to make an exact copy, create a read-only follower for things like reporting and analytics and you can even share queries externally with dataclips. They also have a growing number of addons for a bunch of services like mail, caching, queueing, elasticsearch all for ridiclously low prices. Why should I build a logger service when I can easily add Papertrail for $7 month. They wrap this all up with a free usage tier to get started plus excellent support. So the answer is yes….they are a rainbow colored unicorn of application development.
  • So lets talk about designing your api. There’s a lot that goes into designing a greate API but I’ll touch on a few high level ideas. Keep it simple. As a developer I want to look at an API and be able to grok it quickly. I want great documentation when I get stuck that includes code samples and test cases.
  • When building your api don’t go crazy and try to reinvent the wheel. Use existing standards like REST, JSON and Oauth. Sure, you could use SOAP and XML but honestly no one likes that stuff anymore.
  • When building your API use proper REST principles but don’t be dogmatic. There will be times where you need to wander off the beaten path. Just remember: Nouns are good. Verbs are bad. The number one principle in pragmatic RESTful design is: keep simple things simple. The base URL is the most important part of your API design. A simple and intuitive base URL makes using your API easy to understand. Verbs shouldn’t be in the base URL. Everything is centered around collections and resources and you use HTTP verbs to operate on them. You can think of these as CRUD methods.
  • Some other important factors when designing your API. Always return proper HTTP responses but don’t go crazy. Start off with a 200 for success, a 400 for bad request, 401 for not authorized, 404 for not found and a 500 for a internal server error. Don’t get more complex unless unless you absolutely have to. You’ll want to think about your endpoint versioning strategy ahead of time. Typically you can either pass the version number a part of the URL or in the header. We prefer the the URL route so you can easily test endpoints in the browser, the version number is obvious and it’s fewer keystrokes for cURL commands.
    For simple security, use API Keys to restrict access to your api and endpoints. I’ll show you an example of that in the demo but basically your API should be able to produce a API key that is passed from the client with each request. If the API key is found in the local datastore then they are granted access. If not simply return a 401 response.
    And finally, hide the complexity of your API behind the “?”. It provides for a more expressive API with far fewer endpoints. And finally, there’s a great presentation called “Teach a Dog to REST” from Dreamforce a couple of years ago that I would recommend you listen to.
  • OK… so now we know which platforms to use and have a good idea of how to design our API. Now the big choice. Which language to write it in. We actually really don’t care. Some of the more popular languages for APIs are ruby, node and java. Fortunately there are a number of language specific wrappers around the APIs that you can use along with some other frameworks and tools to make api development much more enjoyable. I would recommend the restforce gem if you are using ruby but my favorite is the nforce package for node.js. Both of these provide authentication, SOQL and DML statements, calling Apex rest services, streaming api and much more. I’ve also written a “forcifier” utility package for ruby and node to make it easier to work with funky salesforce field names. Since languages like ruby and node are case sensitive, the “forcifier” will take the mixed case object field names from and make them lower case with the “__c” removed plus some other helper functions as well. This way the client code never has to know about or deal with the funky salesforce field names.
    There are a number of language specific frameworks for building RESTful web services. For ruby there’s rocket pants and grape. Grape is more mature and feature-rich but we chose rocket pants since it was a little more light weight and honestly I just really liked saying “rocket pants” all the time. For node you can certainly use Express but you’ll find a lot of applications using Restify. We chose the new Actionhero package and are really happy with it. We’ll use that in our demo in second. If you are using Java, the Play! Framework and Jersey are really popular.
  • One last thing before we get into some code. One of the questions that come up regularly is the use of the REST API vs. Apex REST services. We highly suggest you use Apex REST Services whenever possible as they are super fast and easy to build! Unlike the REST API where you essentially operate on a single record or collection of records, you can wrap as much functionality as you’d like into a single request with Apex REST Services. For instance, the leaderboard page at CloudSpokes originally made 3 REST calls: one to return the leaderboard for current month, for this year and all time. We dramatically reduced the load time by replacing these three calls with a single Apex REST service that returned all 3 time periods with a single request.
    You should build your Apex REST Services to run as fast as possible to prevent hitting long running request limits. You can only have 10 concurrent request longer than 5 seconds before the platform starts spitting out errors. Whenever possible, hand off long running requests to asynchronous processes. So when we create a new CloudSpokes member, we quickly return the new member and then hand off the setup and email notification to an asynchronous process to complete when the platform has resource available.
  • OK… let’s write some code. So this has become my stock “getting down to business” picture. Does everyone know Pat Patterson? He’s a developer evangelist at SFDC? I took this picture of him in Portland Maine last month at the Monktoberfest conference. I love that picture.
    So the api we are going to build is rather basic since we only have a few minutes. If you need some sample code or reference you can go to the cloudspokes github repo and find the ruby or node version of our api. The code is all open source so feel free to poke around. Ok… so let’s build an API using actionhero, node.js and that uses both the REST API, Apex REST services and handles security with some simple key management.
  • One last thing to talk about is authentication. You can either one a single “share” user for each call to or individual named users. You can certainly go the shared user route but we suggest using named users instead for a number of reasons including granular record security, chatter integration, audit trail and record ownership to name a few.
  • Prototype using the REST API then once you’re happy with the the process, refactor to use Apex REST Services. Cache results from whenever possible. With our new node API we’ve started using Cloudconnect for some of the site’s functionality. Cloudconnect syncs your data between and either mysql or postgres. So now your developers are happy. They can work with a local database like they are used to instead of dealing with an API. Instead of polling for changes in, use the streaming API to “listen” for any record changes and then process them in your API. Lastly, you’ll need to think carefully about how you version things like Apex classes, triggers and workflows. We try to keep our endpoints in sync with an internal version number inside but it can become challenging.
  • The workbench is fantastic tool for development. Can you test the REST API, Apex Rest Services, the Streaming API and much more. If you are not using it, you should. We also use Runscope all the time for debugging, testing and sharing API calls. You saw a minute ago I was using Postman to make REST calls. I can’t live without Postman. Then we use IODocs for documenting and testing calls. You can also look at Swagger which does the same job. We’ve recently started using for API development. You can declaratively build your interface with RABL and it provides a mock interface that you can gradually replace with your actual implementation. It’s pretty sweet.
  • Lastly a couple of salesforce gotchas: maintenance downtime and sandbox refreshes and upgrades. Since your API is dependent upon then it’s offline during this period. This is one of the reasons we implemented Cloudconnect.
    One of the things that’s a real pain is writing rspecs or mocha tests for your API. Typically when you write your test for ruby or node it’s against some local testing database that is rolled back after each test run. However, since your tests are hitting these are not rolled back because it looks like a normal call for and not an actual test. This can be a real pain so think ahead about how to handle this with some common test data or perhaps setup and tear down methods in your test framework.
  • OK.. So that’s it. Feel free to ask any questions now or find me after the session.
  • Build your API with and Heroku

    1. 1. Build your API with & Heroku Jeff Douglas, Appirio, CloudSpokes Platform Architect @jeffdonthemic
    2. 2. Safe harbor Safe harbor statement under the Private Securities Litigation Reform Act of 1995: This presentation may contain forward-looking statements that involve risks, uncertainties, and assumptions. If any such uncertainties materialize or if any of the assumptions proves incorrect, the results of, inc. could differ materially from the results expressed or implied by the forward-looking statements we make. All statements other than statements of historical fact could be deemed forward-looking, including any projections of product or service availability, subscriber growth, earnings, revenues, or other financial items and any statements regarding strategies or plans of management for future operations, statements of belief, any statements concerning new, planned, or upgraded services or technology developments and customer contracts or use of our services. The risks and uncertainties referred to above include – but are not limited to – risks associated with developing and delivering new functionality for our service, new products and services, our new business model, our past operating losses, possible fluctuations in our operating results and rate of growth, interruptions or delays in our Web hosting, breach of our security measures, the outcome of any litigation, risks associated with completed and any possible mergers and acquisitions, the immature market in which we operate, our relatively limited operating history, our ability to expand, retain, and motivate our employees and manage our growth, new releases of our service and successful customer deployment, our limited history reselling products, and utilization and selling to larger enterprise customers. Further information on potential factors that could affect the financial results of, inc. is included in our annual report on Form 10-K for the most recent fiscal year and in our quarterly report on Form 10-Q for the most recent fiscal quarter. These documents and others containing important disclosures are available on the SEC Filings section of the Investor Information section of our Web site. Any unreleased services or features referenced in this or other presentations, press releases or public statements are not currently available and may not be delivered on time or at all. Customers who purchase our services should make the purchase decisions based upon features that are currently available., inc. assumes no obligation and does not intend to update these forward-looking statements.
    3. 3. Jeff Douglas CloudSpokes Platform Architect @jeffdonthemic
    4. 4. All about CloudSpokes CloudSpokes is a crowdsourcing development community where members compete in design, coding and algorithm challenges for cash, prizes, badges and bragging rights.  ~600,000 member community  40-60 development challenges in-flight at any one time  platform manages members, challenges, payments, etc.  API calls via REST and JSON
    5. 5. Why build an API?  IT departments can provide limited access to internal development teams without exposing the implementation details of  ISVs can better serve existing customers by extending their platform  Startups and entrepreneurs can build their business upon
    6. 6. Why choose  Build your business logic in Apex  Declarative development  Security – FTW!  No admin UI to develop and maintain  New features three times a year  Sandbox environments, tools, deployment processes, etc.  Reports & dashboards
    7. 7. Why choose Heroku?  Polyglot platform  Scale up and down rapidly  Distributed development with Git  Fork applications for testing, QA, etc.  Postgres database (fork, follow, share)  Heroku add-ons  Free to get started  Excellent support
    8. 8. Designing your API Developers want the following:  Simple, intuitive and easy to learn  Great documentation  Code samples  Test cases
    9. 9. Building your API Adopt web standards. Don’t reinvent the wheel.  REST  JSON  OAuth2
    10. 10. Use proper REST principles Nouns are good. Verbs are bad.
    11. 11. Other API design considerations  HTTP response and error codes • 200, 400, 401, 404 & 500  Versioning (header vs. URL)  Key management & security  Hide complexity behind “?” Poor design: /challenges/open/5 Better design: /challenges?status=open&limit=5 Teach a Dog to REST
    12. 12. Which development language is best? Language specific REST wrappers for as well as tools to make it easier to build well-designed APIs.  Ruby • restforce gem, forcifier gem • Rocket_pants, Grape  Node.js • nforce package, forcifier-node package • Actionhero, Restify, Express  Java • Play!, Jersey
    13. 13. Apex REST Services vs. REST API The REST API is great for interacting with records while Apex REST Services can encapsulate numerous operations into a single request.  Design to run as quickly as possible  Handoff long running requests to asynchronous processes
    14. 14. Let’s build an API!
    15. 15. User authentication Authenticate your API with  Single “shared” user  Named user • Security • Chatter • Audit trail & history • Record ownership
    16. 16. Best practices  Prototype with REST API then refactor to Apex REST Services  Caching is cheap and fast with Memcache and redis  Sync to local datastore with  “Listen” for record changes using the Streaming API  Think about how to version Apex classes, triggers and workflows
    17. 17. Development tools  Workbench  Runscope for debugging, testing and sharing API calls  Chrome extensions: Postman, REST Console  Documenting and testing calls with IODocs, Swagger and
    18. 18. Gotchas!  maintenance downtime  Sandbox refreshes and upgrades  Hard to write API unit tests against sandbox environments
    19. 19. Jeff Douglas CloudSpokes Platform Architect, @jeffdonthemic