This document discusses rate limits and performance when making API requests. It recommends using message queues to distribute the load and implement throttling. Three approaches using message queues are described: 1) a single queue, 2) a single queue with selectors to group operations, and 3) multiple queues to give more control over rate limits and make scaling and maintenance easier. Monitoring is emphasized as important for any solution.

1. Rate Limits &
Performance
Handling Rate limits, boosting
performance
Gustavo Moreira
<Presenter>

2. Agenda
● API best practices
● What are rate limits?
● How to handle rate limits?

3. Best Practices
Some simple things to speed up your applications

4. Batch operations together
● Requests to the API have certain costs
○ Network transfer, serialisation, auth, etc.
● Batching operations into a single request
● mutate methods accept operation arrays

5. Group operations by target
● Multiple operations on the same
AdGroup/Campaign are faster
● Subsequent edits can cause
CONCURRENT_MODIFICATION errors
○ Backend can take time to do its work
○ Try all edits to AdGroups/Campaigns at once

6. Only update what you need
● Updating an object?
○ Only send the values that will change!
○ Sending all other values wastes time
■ System validation
■ DB storing
■ etc.

7. A couple of other things...
● Compress your traffic with
gzip
○ User-Agent: has "gzip"
○ Accept-Encoding: "gzip"
● Use partial failure
○ Tells API to serialise those ops
that succeeded
○ Returns list of those that failed

8. Defining Rate Limits

9. Rate Limits
● Not fixed
● Vary on server load
● Vary per feature area
● Will change over time
● Differs per AdWords Service

10. Limit based on Access Level
● This limit doesn't fluctuate
● The access level can be Basic or Standard
Standard: More than 10K
Basic: 10K operations per day
operations per day allowed

11. Rate Limit Errors
● RATE_EXCEEDED
○ wait retryAfterSeconds seconds
● CONCURRENT_MODIFICATIONS
○ Exponential backoff, few retries only
● UNEXPECTED_INTERNAL_API_
ERROR
○ Exponential back off, few retries only
○ Report the issue to your CSR or forum

12. How to Handle
Rate Limits

13. Basic Example
ApiError[] errorArray = apiException.getErrors();
for (ApiError apiError : errorArray) {
if (apiError instanceof RateExceededError) {
int seconds = ((RateExceededError) apiError)
.getRetryAfterSeconds();
// wait the amount of seconds the server asked
Thread.sleep(seconds * 1000);
}
}

14. Basic Example - Explained
● Any request can generate a
RateExceededError
● Handle these errors!
● Best strategy: wait and retry
● Even more important when
doing multiple requests

15. Important points - Basic Solution
● Synchronous solution to the problem
● Blocks the thread until it succeeds, or fails
completely
● No control over request rate speed
● Very difficult to group operations

16. A More Advanced Solution
● Message Queues
○ The perfect solution to distribute load and do
throttling
○ Very good out of the box solutions available
■ ActiveMQ, RabbitMQ, … etc.
○ A lot of tools / client libraries to handle the
communication layer

17. Message Queues - Cont.
● More control over the rates and limits
● Better break down into specialized modules
● Let’s take a look to three possible
approaches to the problem...

18. 1. Single Queue
Consume
rs
Queue
Producer
Producer
Producer
Consume
rs
Throttling
Producers will create tasks to
be executed using the API, and
add them to the queue
Consumers will retrieve the
messages/tasks from the queue
with a controlled rate limit
X
Error
Consumers
Logging

19. Pros & Cons
● Pros:
○ Very easy to implement
○ Only one control point to change rate speed
○ Easy to handle errors
● Cons:
○ Only one control point to change rate speed
○ Hard to group operations
○ Tasks can take too long to be executed
○ Not all MQs deal with message priority

20. 2. Single Queue with Selectors
Queue
Producers
Producers
Producers
Producers
Consumer
Consumer
Consumer
Consumer
Throttling
Producers will create specific tasks
to be executed, and add them to
the queue with a specific header
Each specific consumer can group
the operations, and execute within
your own rate limit using selectors
X
X
X
X
Error

21. Pros & Cons
● Pros:
○ Group operations by type
○ Some control over throttling depending on type
○ More efficient - better parallel access to the API
● Cons:
○ Only one queue - difficult to administer
○ No main control on how fast the API is called
○ Managing all the pieces can be difficult
○ Operation logging might be complicated

22. 3. Multiple Queues
Queues
Producers
Producers
Producers
Producers
Consumer
Consumer
Consumer
ConsumerThrottling
X
X
X
X
Error
Executors
Throttling
Logging
Producers just
spawn a lot of tasks
Consumers just group the tasks
Executors do the request

23. Pros & Cons
● Pros:
○ Total control over rate limits
○ Very robust - very easy to add new parts
○ Complete picture on what is happening with the
queues
○ Easy to scale based on demand
○ If done properly, easy to do maintenance!
○ Super efficient - Parallel grouping; Parallel access to
the API
● Cons:
○ Hard to implement properly - experience required

24. Message Queues - Takeaway
● Complexity of your platform dictates the
best solution
● You can combine parts of other solutions!
● It is a complex subject, but will
save you in the long run
● Invest a lot in monitoring. A lot!
● There is no silver bullet...

25. Resources
● API Best Practices - https://developers.
google.
com/adwords/api/docs/guides/bestpractic
es
● ActiveMQ - http://activemq.apache.org/
● RabbitMQ - http://www.rabbitmq.com/