The document discusses the components and challenges of building a social platform, including technology, user profiles, relationships, and activity streams. It emphasizes the importance of flexibility in design, infrastructure, and processes, along with performance optimization strategies such as caching and scaling. Additionally, it notes the need for effective integration, automated testing, and the use of various technologies like RabbitMQ and NoSQL databases for managing social interactions efficiently.

2. Components of Social PlatformChallengesTechnologyAPIEngineering Process

3. User ProfileRelationshipsActivity Streams

4. Basic user informationExtended information based on the contextPlayer CardsBragging rights Points, levelsAchievements, badgesActivities

5. FriendingUnidirectional (the Twitter model)Bidirectional (the Facebok model) AssociationCommentsRatings, ThumbsBookmarking/favoritingRecommendations

6. Who is doing what and whenAll about Actors, Actions, Objects and TargetsActivitystrea.ms standard vs. OpenSocialCommentable

7. AuthenticationPerformanceActivityStremsIntegrationFlexibilityTesting

8. People are tired of creating accounts on every siteNeed to support existing login method if the platform caters to an existing audienceExisting auth may not work well with Open API initiativesOpen API and Oauth2 legged: Service to Service3 legged: User to App to Service

9. Identify the bottlenecks Measure everythingUse CDNs for all static contentFront end optimization via async loadingDatabase optimization via indexes, shardingCaching Scaling the sortsScaling up vs. Scaling outCAP theoremRelational vs. NOSQL storageRead vs. Write heaviness

10. Query vs. PropagationQueries are read heavyPropagation is write heavyDeletion is a pain with propagationActivity AggregationAggregation on actor vs. objectNormalized vs. Denormalized storageCommentsDecorating the activities on each request

11. Integration with legacy touchpointsOpening up the APIMore channels like MobileMore independent applicationsRate limiting and access controlDon’t forget existing dataData outlives code

12. Flexibility in the code to adapt changing requirements quickly and seamlesslyGood designDRY SOCsFlexibility in the infrastructure to adapt changing traffic and behaviorVirtualizationHeavy replicationFlexibility in the team to respond to changesProcess

13. Automated Testing wherever possibleDeveloper Focus on test coverage (80+%)Continuous Integration and DeploymentCucumber + HudsonCross browser testing (yes, including IE)

15. Java servicesTomcat with Shindig 1.1, 4 nodesREST/JSONRuby Rails Admin App for moderation and points/levelsMigration ScriptsTwitter bot for routing #myign tweets to the platformMisc. scripts to invalidate memache keys and test service endpoints

16. MemcachedExtremely trivial to set up and maintainAlmost never diesMassive scale outCareful withCache hotspotsConcurrent writesOn the fly scale-outKey/Value size limits

17. MySQLProven, cheap to develop and operateMaslow’s hammerEasy scale outHard to store (and retrieve) network graphsWrite scaling with single masterNot the best choice for activitystreamsSchema changes lock the table(s)

18. Awesome write scalingGreat for activity propagation modelIn place updatesUsing $push and $setExcellent for storing social relationships as documentsVery easy to clusterWe are running replica pairs, plan to move to replica setsSchema-lessNo need to run alter scripts on 18M-row table

19. QueryableRich Query language ($in, $size, $exists, $slice)MapReduce for heavy data crunchingSupports IndexingYou can even index collections inside a documentStorage ~4x storage compared to relational dataEmerging technologyIndex defragmentation $or and indexing (to be supported in 1.7)Load balancing support in the driver (coming soon)

20. RabbitMQ for messagingEase of clusteringWritten in Erlang for high performance and availabilityUsed forPropagation of activitiesSending out email alertsIndexing data in Solr

21. PersonGET @self, @friends, @followers, @all, PUT/POST @self, @friendsActivitiesGET @global, @self, @friends, POST @self MediaItemsGET @self, @all and POST @selfAppDataFor applications to store/retrieve data as key-value pairs GET/POST @selfStatus GET @friends, @self, @followers , POST @self

22. Must have for any Java/Ruby webappMonitoring and troubleshootingSave a ton of $ and time by efficient root cause analysis toolsAgents for Ruby and JavaIGN Engineers helped write PHP and Memcached agents

23. Social Applications and communityCheck the pulse of the communityUserVoice (http://ign.uservoice.com)Less is moreDistinguish yourself and focus on your nicheBe Agile - Release early, release oftenDo not shock your audienceAnnounce the changes/features on a blogEat your own dog foodhttp://people.ign.com/ign-labs

24. Released July 2010 as betaDaily API requests ~25MDaily page views ~30KDaily Uniques ~12K6ms response timesExpected traffic 8-10x with more integration and mobile platform

25. Manish PanditEngineering Manager, Social Platform at IGNEmail: pandit.manish-at-gmail.comTwitter: @lobster1234LinkedIn: http://www.linkedin.com/in/mpanditBlog: http://contrarianwisdom.blogspot.comMyIGN: http://people.ign.com/mpanditign

26. http://corp.ign.comhttp://labs.ign.comhttp://my.ign.comhttp://people.ign.com/ign-labs