AdWords API Workshops – All rights reserved
API SERVER

Rate Limits
and API Best Practices

<SPEAKER>, Google, Inc.

AdWords API Workshops – All rights reserved
Agenda
● API Best Practices
● What are Rate Limits?
● How to handle Rate Limits

AdWords API Workshops – All rights reserv...
Best Practices
Some simple things to speed up your applications

AdWords API Workshops – All rights reserved
Batch Operations Together

AdWords API Workshops – All rights reserved

● Requests to the API have certain costs
● Network...
Group Operations by Target
● Multiple operations on the same AdGroup / Campaign
are faster
● Subsequent edits to same AdGr...
Only Update What you Need to
● Updating an object?
● Only send the values that will change!
● Sending in all the other val...
A Couple of Other Ideas …
● Compress your requests / response
with gzip
● Make sure User-Agent: has “gzip”
● Accept-Encodi...
Defining Rate Limits

AdWords API Workshops – All rights reserved
Rate Limits
● Are not fixed
● Vary on server load
● Vary per feature area
● Will change over time
● Different per AdWords ...
Rate Limit Errors
● RATE_EXCEEDED
○ at least wait retryAfterSeconds seconds

● CONCURRENT_MODIFICATIONS
○ Exponential back...
How to Handle Rate Limits
Careful coding...

AdWords API Workshops – All rights reserved
Basic Example
Java

ApiError[] errorArray = apiException.getErrors();
for (ApiError apiError : errorArray) {
if (apiError ...
Basic Example - Explained
● Any request can generate a RateExceededError
● Handling these errors is very important
● Wait ...
Important Points About the Basic Solution
● Synchronous solution to the problem
● Blocks the thread until it succeeds, or ...
A More Advanced Solution
● Message Queues
● The perfect solution to distribute load and do
throttling
● Very good out of t...
Message Queues (cont’d)
● More control over the rates and limits
● Better break down into specialized modules
● Let’s take...
1. Single Queue
Producer

Producers will create tasks to be
executed using the API, and add
them to the queue

Producer

C...
First Approach - Pros & Cons
● Pros:
● Very easy to implement
● Only one control point to change rate speed
● Easy to hand...
2. Single Queue with Selectors
Producers
Producers

Producers will create specific tasks
to be executed, and add them to
t...
Second Approach - Pros & Cons
● Pros:
● Group operations by type
● Some control over throttling depending on type
● More e...
3. Multiple Queues
Executors do the request

Producers
Consumer

Producers
Producers

Producers
Consumer

Throttling

Prod...
Third Approach - Pros & Cons
● Pros:
●
●
●
●
●
●

Total control over rate limits
Very robust - very easy to add new parts
...
Message Queues - Takeaways
● Complexity of your platform dictates the best solution
● You can combine parts of other solut...
Resources
API Best Practices - https://developers.google.
com/adwords/api/docs/guides/bestpractices
ActiveMQ - http://acti...
Questions?
Thank you!

AdWords API Workshops – All rights reserved
AdWords API Workshops – All rights reserved
Upcoming SlideShare
Loading in …5
×

Rate limits and performance Talk

1,509 views

Published on

How to implement rate limiting and performance tips and tricks for the AdWords API.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,509
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
33
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Rate limits and performance Talk

  1. 1. AdWords API Workshops – All rights reserved
  2. 2. API SERVER Rate Limits and API Best Practices <SPEAKER>, Google, Inc. AdWords API Workshops – All rights reserved
  3. 3. Agenda ● API Best Practices ● What are Rate Limits? ● How to handle Rate Limits AdWords API Workshops – All rights reserved
  4. 4. Best Practices Some simple things to speed up your applications AdWords API Workshops – All rights reserved
  5. 5. Batch Operations Together AdWords API Workshops – All rights reserved ● Requests to the API have certain costs ● Network transfer, serialisation, auth, etc. ● Batching operations into a single request reduces these ● mutate methods accept arrays of operations ● MutateJobService for batching AdWords API Workshops – All rights reserved
  6. 6. Group Operations by Target ● Multiple operations on the same AdGroup / Campaign are faster ● Subsequent edits to same AdGroup or Campaign can cause CONCURRENT_MODIFICATION errors ● Backend systems can take time to do their work ● Try to do all edits on AdGroups/Campaigns at once AdWords API Workshops – All rights reserved
  7. 7. Only Update What you Need to ● Updating an object? ● Only send the values that will change! ● Sending in all the other values wastes time ● The system still validates them, stores to DB, etc. AdWords API Workshops – All rights reserved
  8. 8. A Couple of Other Ideas … ● Compress your requests / response with gzip ● Make sure User-Agent: has “gzip” ● Accept-Encoding: lists gzip ● Use partial failure feature ● Tells API to serialise those ops that succeeded ● Returns list of those that failed AdWords API Workshops – All rights reserved
  9. 9. Defining Rate Limits AdWords API Workshops – All rights reserved
  10. 10. Rate Limits ● Are not fixed ● Vary on server load ● Vary per feature area ● Will change over time ● Different per AdWords service AdWords API Workshops – All rights reserved Defining Rate Limits
  11. 11. Rate Limit Errors ● RATE_EXCEEDED ○ at least wait retryAfterSeconds seconds ● CONCURRENT_MODIFICATIONS ○ Exponential backoff, few retries only ● UNEXPECTED_INTERNAL_API_ERROR ○ Exponential back off, few retries only ○ Report the issue to your CSR or forum AdWords API Workshops – All rights reserved Defining Rate Limits
  12. 12. How to Handle Rate Limits Careful coding... AdWords API Workshops – All rights reserved
  13. 13. Basic Example Java ApiError[] errorArray = apiException.getErrors(); for (ApiError apiError : errorArray) { if (apiError instanceof RateExceededError) { int seconds = ((RateExceededError) apiError) .getRetryAfterSeconds(); // wait the amount of seconds the server asked Thread.sleep(seconds * 1000); } } AdWords API Workshops – All rights reserved How to handle Rate Limits
  14. 14. Basic Example - Explained ● Any request can generate a RateExceededError ● Handling these errors is very important ● Wait and retry is the best strategy in this case ● Even more important when doing multiple requests AdWords API Workshops – All rights reserved
  15. 15. Important Points About the Basic Solution ● Synchronous solution to the problem ● Blocks the thread until it succeeds, or fails completely ● No control over request rate speed ● Very difficult to group operations AdWords API Workshops – All rights reserved
  16. 16. A More Advanced Solution ● Message Queues ● The perfect solution to distribute load and do throttling ● Very good out of the box solutions available ● ActiveMQ, RabbitMQ, … etc. ● A lot of tools / client libraries to handle comms layer AdWords API Workshops – All rights reserved
  17. 17. Message Queues (cont’d) ● More control over the rates and limits ● Better break down into specialized modules ● Let’s take a look to three possible approaches to the problem... AdWords API Workshops – All rights reserved
  18. 18. 1. Single Queue Producer Producers will create tasks to be executed using the API, and add them to the queue Producer Consumers Consumers Consumers X Queue Throttling Producer Logging Error Consumers will retrieve the messages/tasks from the queue with a controlled rate limit AdWords API Workshops – All rights reserved How to handle Rate Limits
  19. 19. First Approach - Pros & Cons ● Pros: ● Very easy to implement ● Only one control point to change rate speed ● Easy to handle errors ● Cons: ● ● ● ● Only one control point to change rate speed Hard to group operations Tasks can take too long to be executed Not all MQs deal with message priority AdWords API Workshops – All rights reserved
  20. 20. 2. Single Queue with Selectors Producers Producers Producers will create specific tasks to be executed, and add them to the queue with a specific header Producers Producers Each specific consumer can group the operations, and execute within your own rate limit using selectors X X X X Producers Producers Queue Throttling Producers Producers Producers Consumer AdWords API Workshops – All rights reserved Producers Consumer Producers Consumer Producers Consumer Error How to handle Rate Limits
  21. 21. Second Approach - Pros & Cons ● Pros: ● Group operations by type ● Some control over throttling depending on type ● More efficient - better parallel access to the API ● Cons: ● ● ● ● Only one queue - difficult to administer No main control on how fast the API is called Managing all the pieces can be difficult Operation logging might be complicated AdWords API Workshops – All rights reserved
  22. 22. 3. Multiple Queues Executors do the request Producers Consumer Producers Producers Producers Consumer Throttling Producers Producers Producers Producers Producers Producers Producers Consumer Producers Executors Executors Executors Producers Consumer Queues Consumers just group the tasks Error Logging X X X X Producers just spawn a lot of tasks Throttling AdWords API Workshops – All rights reserved How to handle Rate Limits
  23. 23. Third Approach - Pros & Cons ● Pros: ● ● ● ● ● ● Total control over rate limits Very robust - very easy to add new parts Complete picture on what is happening with the queues Easy to scale based on demand If done properly, easy to do maintenance! Super efficient - Parallel grouping; Parallel access to the API ● Cons: ● Hard to implement properly - experience required AdWords API Workshops – All rights reserved
  24. 24. Message Queues - Takeaways ● Complexity of your platform dictates the best solution ● You can combine parts of other solutions! ● It is a complex subject, but will save you in the long run ● Invest a lot in monitoring. A lot! ● There is no silver bullet... AdWords API Workshops – All rights reserved
  25. 25. Resources API Best Practices - https://developers.google. com/adwords/api/docs/guides/bestpractices ActiveMQ - http://activemq.apache.org/ RabbitMQ - http://www.rabbitmq.com/ AdWords API Workshops – All rights reserved
  26. 26. Questions? Thank you! AdWords API Workshops – All rights reserved
  27. 27. AdWords API Workshops – All rights reserved

×