API Days 2012 - 1 billion SMS through an API !

  • 452 views
Uploaded on

I gave this presentation at the API Days conference on Dec 3rd 2012 …

I gave this presentation at the API Days conference on Dec 3rd 2012

Once upon a time, people wanted to send a billion SMS per month through an API ... This session will tell you the story of how this can be achieved.
Taking example on the design behind the apiGrove opensource project, you will hear about the under the hood technology details relating to API policy distribution across large scale clusters of API gateways. In the cloud. With throughput above tens of thousands transactions per second. With high availability and high accuracy in rate limit enforcement.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
452
On Slideshare
0
From Embeds
0
Number of Embeds
2

Actions

Shares
Downloads
0
Comments
0
Likes
2

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. American  Idol  :  The  tale  of  1  Billion  SMS  /  month   through  an  API   Guilhem  Ensuque   @apigrove   #apidays  
  • 2. How  it  started   •  10s  of  millions  of  viewers   •  Worldwide  franchise   •  Vote  by  SMS   •  2-­‐hour  window  to  vote     •  Through  an  API  
  • 3. The  Big  Picture  
  • 4. Mission  :  Impossible  •  Replace  legacy  system  in  <  1  year  •  99.999%  reliability  (5  mins  downSme  /  year)  •  American  Idol-­‐scale   –  Peak  throughput  =  20,000  TPS   –  Average  volume  =  1  billion  messages  /  month  •  Secure  •  In  the  cloud  
  • 5. Key  decisions  •  Code  fast,  show  results  early  •  Open  Source  2-­‐ways  •  K.I.S.S.  :  meter  /  protect  /  scale  •  Integrate  easily  in  bigger  system  through  …                                APIs  
  • 6. OpSons  vs  Challenges   Web  Proxy  (ligh2pd)     ESB  (Fuse)   +     +   Custom  C/C++   Custom  Java            Performance                 Dev.  Time      
  • 7. Technology    •  Choosing  a  technology  is  important  …    •  …  unSl  you  find  a  be^er  one          DON’T  BE  RELIGIOUS  •  “Business”  code  is  middleware-­‐agnosSc      
  • 8. Policy  Model   AUTH API - authId-apiId - status-endpoint - type-type - key-validationType - username-authenticationTypes - password-nbRequestCertificate - policyContext id-notificationFlag - apiContext id-contexts - id - status auths0 : ..* - targetHosts - sbRequestsType - sbRequestsCredentials - maxRateLimits apis 0 : ..* POLICY - policyId - contexts - id - status - quotas - rateLimits
  • 9. E3Route Camel  Pipeline   HTTP Request Jetty%Camel Validation Auth0Manager Rates0Quatas Custom <jetty> <validate> <auth> <rates0quotas> <other> Client TargetHostApplication OK Exception Service Error Gateway%Error% Processor Status% Error%TDR Billing%TDR TDR Tdr0Lib <TDR%Queue> Emit%tdr File% System !
  • 10. Clustering  
  • 11. Hard  Stuff:  Rate  limits  Sync  •  How  to  sync  rate  limits  data  ?  •  Main  difficulty:  large  cluster  (40  instances)  •  SoluSon  :  Speaker  Manager  +  maths  
  • 12. Wrap  Up  /  Q&As   guilhem.ensuque@alcatel-­‐lucent.com     Twi^er:  @gensuque_alu         h^p://apigrove.net     h^p://github.com/apigrove/apigrove