The document discusses peer-to-peer (P2P) content distribution systems like BitTorrent and Spotify. It describes how BitTorrent works by breaking files into pieces that are distributed among peers using a tracker. Peers select rare pieces first and cooperate through a tit-for-tat approach. Spotify's P2P system supplements streaming from servers by also getting pieces from other users' devices to improve performance and scalability.

1. P2P Content Distribution
BitTorrent and Spotify
Amir H. Payberah
amir@sics.se
Amirkabir University of Technology
(Tehran Polytechnic)
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 1 / 62

2. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 2 / 62

3. Possible Solutions for
Content Distribution
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 3 / 62

4. Client-Server Model
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 4 / 62

5. Client-Server Model
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 5 / 62

6. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 6 / 62

7. The Client-Server Model Problems
Scalability?
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 7 / 62

8. The Client-Server Model Problems
Scalability?
Single Point of failure?
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 7 / 62

9. The Client-Server Model Problems
Scalability?
Single Point of failure?
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 7 / 62

10. Client-Server Systems
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 8 / 62

11. The Client-Server Model Problem
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 9 / 62

12. Scalable and Fault-Tolerant Client-Server Model
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 10 / 62

13. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 11 / 62

14. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 12 / 62

15. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 13 / 62

16. Peer-to-Peer Model
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 14 / 62

17. Peer-to-Peer (P2P) Model
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 15 / 62

18. P2P Challenges
Churn in the system
Free-riding problem
Bottleneck in the overlay network
Connectivity problem, e.g., NAT
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 16 / 62

19. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 17 / 62

20. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 18 / 62

21. How To Discover Data?
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 19 / 62

22. Possible Solutions - First Generation
Central directory
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 20 / 62

23. Possible Solutions - Second Generation
Flooding
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 21 / 62

24. Possible Solutions - Third Generation
Distributed Hash Table (DHT)
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 22 / 62

25. P2P Content Distribution Applications - File Sharing
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 23 / 62

26. P2P Content Distribution Applications - Media Streaming
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 24 / 62

27. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 25 / 62

28. BitTorrent
BitTorrent is a system for efficient and scalable replication of large
amounts of static data.
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 26 / 62

29. BitTorrent Players
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 27 / 62

30. Files
Files are broken into pieces of size between 64KB and 1MB.
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 28 / 62

31. .torrent Files
Metadata
Contains:
• URL of tracker
• Information about the file,
e.g., filename, length, ...
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 29 / 62

32. The Core Idea
A peer obtains .torrent file.
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 30 / 62

33. The Core Idea
A peer obtains .torrent file.
It, then, connects to the tracker.
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 30 / 62

34. The Core Idea
A peer obtains .torrent file.
It, then, connects to the tracker.
The tracker tells the peers from
which other peers to download the
pieces of the file.
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 30 / 62

35. The Core Idea
A peer obtains .torrent file.
It, then, connects to the tracker.
The tracker tells the peers from
which other peers to download the
pieces of the file.
Peers use this information to communicate with each other.
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 30 / 62

36. The Core Idea
A peer obtains .torrent file.
It, then, connects to the tracker.
The tracker tells the peers from
which other peers to download the
pieces of the file.
Peers use this information to communicate with each other.
The peers send information about the file and themselves to tracker.
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 30 / 62

37. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 31 / 62

38. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 32 / 62

39. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 33 / 62

40. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 34 / 62

41. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 35 / 62

42. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 36 / 62

43. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 37 / 62

44. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 38 / 62

45. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 39 / 62

46. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 40 / 62

47. What About Free Riders?
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 41 / 62

48. Tit-For-Tat
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 42 / 62

49. Question
From which peers download the pieces?
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 43 / 62

50. Peer Selection
Use choking algorithm to choose peer to download pieces.
Decision to choke/unchoke based on tit-for-tat.
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 44 / 62

51. Discover More Cooperating Peers
Optimistic unchoking
Allocate an upload slot to a randomly chosen uncooperative peer
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 45 / 62

52. Snubbed Peers
If all its peers choke it.
Increase the number of optimistic unchokes.
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 46 / 62

53. Question
Which piece?
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 47 / 62

54. Piece Selection
Rarest first: common parts left for later
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 48 / 62

55. Piece Selection
Rarest first: common parts left for later
Random first piece: start-up need to get a complete piece
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 48 / 62

56. Piece Selection
Rarest first: common parts left for later
Random first piece: start-up need to get a complete piece
Endgame mode: broadcast for all remaining blocks
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 48 / 62

57. BitTorrent Extension
Distributed tracker
Peer-exchange
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 49 / 62

58. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 50 / 62

59. Spotify
Active users: over 50 million
Number of songs: over 20 million
Number of songs added per day: over 20000
Number of playlists: over 1.5 billion created so far
Available in 58 countries
Legal
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 51 / 62

60. The Core Idea
Request first piece from Spotify servers.
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 52 / 62

61. The Core Idea
Request first piece from Spotify servers.
Meanwhile, search P2P network for remainder.
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 52 / 62

62. The Core Idea
Request first piece from Spotify servers.
Meanwhile, search P2P network for remainder.
Switch back and forth between Spotify servers and peers as needed.
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 52 / 62

63. The Core Idea
Request first piece from Spotify servers.
Meanwhile, search P2P network for remainder.
Switch back and forth between Spotify servers and peers as needed.
Towards end of a track, start prefetching the next one.
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 52 / 62

64. Main Problem in Using Spotify P2P Network
Peer Discovery
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 53 / 62

65. Peer Discovery
Sever-side tracker (BitTorrent style)
• Only remembers 20 peers per track.
• Returns 10 (online) peers to client on query.
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 54 / 62

66. Peer Discovery
Sever-side tracker (BitTorrent style)
• Only remembers 20 peers per track.
• Returns 10 (online) peers to client on query.
Broadcast query in small (2 hops) neighborhood in overlay (Gnutella
style)
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 54 / 62

67. Peer Discovery
Sever-side tracker (BitTorrent style)
• Only remembers 20 peers per track.
• Returns 10 (online) peers to client on query.
Broadcast query in small (2 hops) neighborhood in overlay (Gnutella
style)
LAN peer discovery
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 54 / 62

68. Downloading in P2P
Ask for most urgent pieces first.
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 55 / 62

69. Downloading in P2P
Ask for most urgent pieces first.
If a peer is slow, re-request from new peers.
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 55 / 62

70. Downloading in P2P
Ask for most urgent pieces first.
If a peer is slow, re-request from new peers.
When buffers are low, download from central server as well.
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 55 / 62

71. Downloading in P2P
Ask for most urgent pieces first.
If a peer is slow, re-request from new peers.
When buffers are low, download from central server as well.
If buffers are very low, stop uploading.
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 55 / 62

72. Spotify vs. BitTorrent
One (well, three) P2P overlay for all tracks (not per-torrent).
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 56 / 62

73. Spotify vs. BitTorrent
One (well, three) P2P overlay for all tracks (not per-torrent).
Does not inform peers about downloaded blocks.
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 56 / 62

74. Spotify vs. BitTorrent
One (well, three) P2P overlay for all tracks (not per-torrent).
Does not inform peers about downloaded blocks.
Downloads blocks in order.
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 56 / 62

75. Spotify vs. BitTorrent
One (well, three) P2P overlay for all tracks (not per-torrent).
Does not inform peers about downloaded blocks.
Downloads blocks in order.
Does not enforce fairness (such as tit-for-tat).
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 56 / 62

76. Spotify vs. BitTorrent
One (well, three) P2P overlay for all tracks (not per-torrent).
Does not inform peers about downloaded blocks.
Downloads blocks in order.
Does not enforce fairness (such as tit-for-tat).
Informs peers about urgency of request.
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 56 / 62

77. Caching
Player caches tracks it has played.
Use 10% of free space (capped at 10GB)
Least Recently Used policy for cache eviction.
Over 50% of data comes from local cache.
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 57 / 62

78. Spotify Data Usage
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 58 / 62

79. Spotify Says Goodbye to P2P
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 59 / 62

80. Summary
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 60 / 62

81. Summary
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 61 / 62

82. Questions?
Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 62 / 62