Talk for the socal piggies meetup 2013-01-17 at Telesign's office.

1. Reliable and Efficient
Facebook data
processing
Andres Buritica
Socal Piggies
2013-01-17

2. http://thelinuxkid.com
Python developer
Facebook Graph API experience
Ubernear
FounderDating

3. Meta
Draws on lessons from Ubernear
Based on traversal of public nodes in Graph

4. Use case
Pages or users (owners) with public events
Event discovery

5. Flow
Update owners
Owner IDs Update owner events
Expire events
Event details Partial events
Complete events

6. Why?
Separation of concerns
Parallel processing with separate
user/apps/servers
Less data load on batch requests
Want to store all data

7. Update owners
Check for migrated owners
○ (#21) Page ID <old_id> was migrated to page ID
<new_id>. Please update your API calls to the new
ID
Move migrated owners to separate table
Add new owners

8. Update owner events
Events for owners not checked since datetime
Stop at last event previously collected

9. Expire events
end_time has passed move to another table
False
○ Data might exist
Should have returned False?
○ [100] Unsupported get request
○ Data might exist

10. Expire events
Alias not found
○ (#803) Some of the aliases you requested do not
exist...

11. Event details
Skip completed (no refreshing)
Transient errors (retry)
○ None
○ OAuthException...Error validating application
○ (#1) "Unknown error occured"
○ "(#2) Service temporarily unavailable"
○ "(#4) User request limit reached" (throttle)
○ "(#4) Application request limit reached" (throttle)
○ "(#17) User request limit reached" (throttle)

12. Datetimes
All in ISO-8601
Events
○ date_format modifier has no effect
○ timezones after "Events Timezone Migration"
○ is_date_only
○ legacy without timezone

13. Batch requests
POST
50 requests in one
Large or complex can time out
Nested calls count towards rate limiting
One top level access token, many nested

14. Batch request example
User's profile and 50 friends
batch=[{
"method":"GET",
"relative_url":"me"
},
{
"method":"GET",
"relative_url":"me/friends?limit=50"
}]

15. Batch dependencies
Reference results of a previous operation
JSONPath
Child operation executed after parent
Parent returns None unless forced

16. Batch dependencies example
Get details of 5 friends
batch=[{
"method":"GET",
"name":"get-friends",
"relative_url":"me/friends?limit=5",
},
{
"method":"GET",
"relative_url":"?ids={result=get-friends:$.data.*.id}"
}]

17. Field expansion
GET
"join" multiple graph queries into a single call
Replacement for FQL
fields, connections, modifiers and identifiers
No JSONPath

18. Field expansion example
User's name and birthday plus id and picture of
the last 10 photos
/me?
fields=
name,
birthday,
photos.limit(10).fields(id,picture)

19. Batch request with field expansions
User's profile and picture link of 10 photos
batch=[{
"method":"GET",
"relative_url":"me"
},
{
"method":"GET",
"relative_url":"me?fields=photos.limit(10).fields(picture)"
}]

20. Process
Facebook is constantly improving
Testing crucial
Beta tier

21. Not covered
Refreshing data
Pagination
Throttling
Insights

22. Thanks
Sample code at http://ubernear.com
Questions
http://thelinuxkid.com