This document discusses three methods to optimize an API: 1) Build indexes for fields in WHERE clauses to improve search time; 2) Sort data in the database rather than in Python code for better performance; 3) Only retrieve needed fields rather than all fields to reduce data transfer size and indexing issues. The document demonstrates how these methods improved a query's execution time from 2.5 minutes to 2.5 seconds. It also covers caveats like indexes slowing down other database operations and provides tips on using EXPLAIN clauses and debugging tools.

1. How to Optimize Our API
Performance
–David Jr.

2. 1. Title
2. Why dost it matters?
3. Three methods to optimize our API
4. Demo
5. Caveat
6. Conclusion
Table of Contents

3. 2. Why does it matters?

4. 2. Why does it matters?

5. 3. Three methods to optimize our API
• Build index for the field specified in WHERE Clause
• Sort in DB instead of sorting in Python
• Only retrieve the fields you need

6. 3.1 Build index for the field specified in WHERE Clause
BTree: The worst case search time is O(log(n))

7. 3.2 Sort in DB instead of sorting in Python
Just traverse all the leaves and you’ll get what you want

8. 3.2 MySQL use B+Tree
• It stores data in the leaf nodes
• B+Tree is faster for sorting
• BTree is faster when you insert values in the middle

9. 3.3 Only retrieve the fields you need
• Inefficiency in moving data to the consumer
• Indexing issue
• Ref: https://stackoverflow.com/questions/29377548/is-it-faster-to-
only-query-specific-columns

10. 3.3 Only retrieve the fields you need
• The origin Query use objects.all() => SELECT *
• Use objects.all().values(col…) instead => SELECT col…

11. 4. Demo

12. 4. Demo
• 2.5 minutes -> 2.5 seconds
• 60X faster than original method

13. 5. Caveat!

14. 5.1 Index is not always the silver bullet
• It requires more disk I/O
• It would slow down INSERT, UPDATE, DELETE
• It might mislead the query planner and slow down SELECT

15. 5.1 Use EXPLAIN Clause
Check the query plan to locate the bottle neck

16. 5.2 Pysnooper
Powerful Debugging Tool

17. 5.2 Pysnooper
Powerful Debugging Tool

18. 6. Conclusion
• Use Index if you can
• But sometimes index would hurt performance

19. –David Jr.
Q & A