PyCon APAC 2014 - Social Network Analysis Using Python (David Chiu)

6,449 views
6,178 views

Published on

PyCon APAC 2014 - Social Network Analysis Using Python (David Chiu)

Published in: Social Media, Technology
0 Comments
19 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
6,449
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
115
Comments
0
Likes
19
Embeds 0
No embeds

No notes for slide

PyCon APAC 2014 - Social Network Analysis Using Python (David Chiu)

  1. 1. Social Network Analysis With Python @ PyCon APAC 2014David Chiu
  2. 2. About Me Co-founder of Ex-Trend Micro Engineer NumerInfo ywchiu.com
  3. 3. Social Network http://libeltyseo.com/wp-content/uploads/2013/03/social-networking.png
  4. 4. Human Nature http://cdn.macado.com/assets/2010/03/peeping-tom.gif
  5. 5. What do we want to know? Who knows whom, and which people are common to their social networks? How frequently are particular people communicating with one another? Which social network connections generate the most value for a particular niche? How does geography affect your social connections in an online world? Who are the most influential/popular people in a social network? What are people chatting about (and is it valuable)? What are people interested in based upon the human language that they use in a digital world?
  6. 6. Explore Facebook
  7. 7. OAuth2 Flow Open standard for authorization. OAuth provides a method for clients to access server resources on behalf of a resource owner
  8. 8. Connect to Facebook https://developers.facebook.com/
  9. 9. Get Access Token https://developers.facebook.com/tools/explorer/
  10. 10. User Permission
  11. 11. Permission List User Data Permissions: user_hometown user_location user_interests user_likes user_relationships Friends Data Permissions: friends_hometown friends_location friends_interests friends_likes friends_relationships Extended Permissions: read_friendlists
  12. 12. Copy Token
  13. 13. Social Network Analysis With Python Let's Hack
  14. 14. Get Information From Facebook
  15. 15. Test On API Explorer
  16. 16. Required Packages requests Sending HTTP Request to Retrieve Data From Facebook json For Parsing JSON Format
  17. 17. Facebook Connect import requests import json access_token="<access_token>" url = "https://graph.facebook.com/me?access_token=%s" response = requests.get(url%(access_token)) fb_data = json.loads(response.text) print fb_data
  18. 18. Question: Who Likes My Post The Most?
  19. 19. Get Likes Count of Posts access_token = '<access_token>' url="https://graph.facebook.com/me/posts?access_token=%s" response = requests.get(url%(access_token)) fb_data = json.loads(response.text) count_dic = {} for post in fb_data['data']: if 'likes' in post: for rec in post['likes']['data']: if rec['name'] in count_dic: count_dic[rec['name']] += 1 else: count_dic[rec['name']] = 1
  20. 20. Simple Ha! Ask Harder Question!
  21. 21. Question: What's People Talking About
  22. 22. Take Cross-Strait Agreement As Example keyword_dic = {} posts_url = 'https://graph.facebook.com/%s/posts?access_token=%s' post_response = rs.get(posts_url%(userid, access_token)) post_json = json.loads(post_response.text) for post in post_json['data']: if 'message' in post: m = re.search('服貿', post['message'].encode('utf-8')) if m: if userid not in keyword_dic: keyword_dic[userid] = 1 else: keyword_dic[userid] += 1
  23. 23. Text Mining NLTK!
  24. 24. Sorry! My Facebook Friends Speak In Mandarin Jieba!
  25. 25. Using Jieba For Word Tokenization import jieba data = post_json['data'] dic = {} for rec in post_json['data']: if 'message' in rec: seg_list = jieba.cut(rec['message']) for seg in seg_list: if seg in dic: dic[seg] = dic[seg] + 1 else: dic[seg] = 1
  26. 26. Question: How to Identify Social Groups?
  27. 27. Required Packages networkx Analyze Social Network community Community Detection Using Louvain Method
  28. 28. Social Network Man As , Connection AsNode Edge
  29. 29. Build Friendship Matrix import networkx as nx mutual_friends = {} for friend in friends_obj['data']: mutual_url = "https://graph.facebook.com/%s/mutualfriends?access_token=%s" res = requests.get( mutual_url % (friend['id'], access_token) ) response_data = json.loads(res.text)['data'] mutual_friends[friend['name']] = [ data['name'] for data in response_data ] nxg = nx.Graph() [ nxg.add_edge('me', mf) for mf in mutual_friends ] [ nxg.add_edge(f1, f2) for f1 in mutual_friends for f2 in mutual_friends[f1] ]
  30. 30. Draw Network Plot nx.draw(nxg)
  31. 31. Calculate Network Property betweenness_centrality(nxg) degree_centrality(nxg) closeness_centrality(nxg)
  32. 32. Community Detection import community def find_partition(graph): g = graph partition = community.best_partition(g) return partition new_G = find_partition(nxg)
  33. 33. Draw Social Network Communities import matplotlib.pyplot as plt size = float(len(set(new_G.values()))) pos = nx.spring_layout(nxg) count = 0. for com in set(new_G.values()) : count = count + 1. list_nodes = [nodes for nodes in new_G.keys() if new_G[nodes] == com] nx.draw_networkx_nodes(nxg, pos, list_nodes, node_size = 20, node_color = str(count / size)) nx.draw_networkx_edges(nxg,pos, alpha=0.5) plt.show()
  34. 34. Community Partitioned Plot
  35. 35. Gephi Gephi, an open source graph visualization and manipulation software
  36. 36. One More Thing
  37. 37. To build your own data service
  38. 38. jsnetworkx A JavaScript port of the NetworkX graph library.
  39. 39. juimee.com
  40. 40. THANK YOU

×