Predicting the “Stars of Tomorrow” on Social Media
•
1 like•2,440 views
People are interested in predicting the future. For example, which films will bomb or who will win the upcoming Grammy awards? Making predictions about the future is not only fun matters but can bring real value to those who correctly predict the course of world events, such as which stocks are the best purchases for short-term gains. Predictive analytics is thus a field that has attracted major attention in both academia and the industry. As social media has become an inseparable part of modern life, there has been increasing interest in research of leveraging and exploiting social media as an information source for inferring rich social facts and knowledge. In this talk, we will address an interesting and challenging problem in social media research, i.e., predicting social media popularity. We aim to discover which image posts on social media are the “stars of tomorrow”, those will be the most engaging for social media audiences, e.g., receiving the most likes. Sociological finding and our novel solutions to effectively develop a structured modeling for popularity dynamics will be presented.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Predicting the “Stars of Tomorrow” on Social Media
Similar to Predicting the “Stars of Tomorrow” on Social Media (20)
More from Förderverein Technische Fakultät
More from Förderverein Technische Fakultät (20)
Recently uploaded
Recently uploaded (20)
Predicting the “Stars of Tomorrow” on Social Media
- 1. Predicting the “Stars of Tomorrow” on Social Media Wen-Huang Cheng (鄭文皇) Multimedia Computing Lab (MCLab) Research Center for Information Technology Innovation (CITI), Academia Sinica, Taipei, Taiwan whcheng@citi.sinica.edu.tw Presented at on 10 May 2017
- 2. 2
- 3. Academia Sinica (中央研究院) • The highest national research institute in Taiwan – with about 1,000 professors (60 in EE/CS) 3 located in Nangang, Taipei
- 4. Multimedia Computing Lab (MCLab) 4 http://mclab.citi.sinica.edu.tw
- 5. We are social… 5 Real World Digital World
- 13. Leading Social Networks 13 [Ref] http://www.smartinsights.com/social-media-marketing/social-media-strategy/new-global-social-media-research/
- 14. 14
- 15. Sociology and Human Interaction • With the huge number of people who are involved nowadays with social networks, it is very interesting to note how they are influenced by each other in many different ways. – e.g., identity in the age of social media 15 [Ref] http://edition.cnn.com/2015/10/05/health/being-13-teens-social-media-study/index.html
- 16. 100 years after 100 years ago
- 17. Social Popularity Prediction • General Popularity Prediction: Predicting the popularity score of a new social media post by combining post content (photo, text or video) and user cues 17 Score: 4.9 ? Model A new post Predicted Popularity Training Images 5.6 2.3 5.1 2.8 7.8 3.1 History data
- 18. Why is it important? • wide applications and high business value – e.g., predicting the “Stars of Tomorrow” (top popular models) within the fashion Industry using social media 18 [Ref] “Style in the Age of Instagram: Predicting Success within the Fashion Industry using Social Media,” CSCW 2016. Fashion Model Directory (FMD) profile page Can you tell who will be the “top”?
- 19. People are desired for knowing the future… 19 [Ref] https://www.oreilly.com/ideas/inside-the-washington-posts-popularity-prediction-experiment
- 21. 21
- 23. 競賽數據 23
- 24. Our Related Publications • “Sequential Prediction of Social Media Popularity with Deep Temporal Context Networks,” IJCAI 2017. • “Time Matters: Multi-scale Temporalization of Social Media Popularity,” ACM Multimedia 2016 (full paper). • “Unfolding Temporal Dynamics: Predicting Social Media Popularity Using Multi-scale Temporal Decomposition,” AAAI 2016. • “SocialCRC: Enabling Socially-Consensual Rendezvous Coordination by Mobile Phones,” Pervasive and Mobile Computing, 2016. 24
- 25. What Makes A Post Popular? 25 [Ref]“What Makes an Image Popular?” WWW, 2014.
- 26. What Makes A Post Popular? • Features for prediction – Post content • e.g., visual sentiment features (color and texture) 26 [Ref]“Analyzing and predicting sentiment of images on the social web,” ACM Multimedia 2010.
- 27. What Makes A Post Popular? • Features for prediction – User cues • e.g., followers (a user’s follower count), friends (how many users a user follows), statuses (a user’s current total post count), user time (a user’s account creation time), etc. 27 A friend graph: 6 1
- 28. What Makes A Post Popular? • Features for prediction – User cues (topological features) • e.g., closeness centrality, the average length of the shortest path between the node and all other nodes in the graph 28 1 2 3 4 5 6 7
- 30. Latent Factor Models • The popularity prediction task is formulated as a matrix completion problem of filling in the missing entries of a partially observed matrix. 30 known popularity to be estimated
- 31. Our Observations: Time Matters 31 [Ref] http://www.adweek.com/socialtimes/best-time-to-post-social-media/504222
- 32. Temporal Modeling for Popularity • To incorporate the temporal evolving structures in popularity prediction 32
- 33. • The popularity evolving at multi-granularities with different patterns 33 Challenge 1: Temporal Evolving Multi-granularities Characteristics of Popularity Dynamics
- 34. Challenge 2: Data Noise • Popularity patterns are covered in very noisy behavior data or information Popularity distribution on time series
- 35. Our Solution#1 [AAAI’16]: Incorporating Multi-Scale Temporal Decomposition 35 popularity matrix time scales Solver: Multiple Update Rule (D.Lee and Sebastian.Seung 1999)
- 36. Datasets • Data Sets – Over 1.8M photos – Over 70K users – Views, User profile, Photo stream – Metadata, Images, Annotations • Settings 36 User-specific Dataset (UsD) Users 400 Images 600K Photo-mix Dataset (PmD) Users 70K Images 1200K
- 37. Experiments • Metric: Spearman Correlation • Time scales: – period, week, month, season • period: “morning (8:00am-12:00am)”, “lunch time (12:00am-14:00pm)”, “afternoon (14:00am-17:00pm)”, “dinner time (17:00am- 20:00pm)”, “evening (20:00am-24:00pm)” and “sleeping (0:00am-8:00am)” 37
- 38. Our Solution#2 [MM’16]: A Multi-scale Temporalization (MT) Framework 38
- 39. Algorithm: Multi-scale Temporalization (MT) 39 Optimization Updating Steps Optimization Updating Steps
- 40. Experiments 40
- 41. Our Solution#3 [IJCAI’17]: Deep Temporal Context Networks (DTCN) • We address the problem as a sequential prediction task, where the input is a user-photo sequence (with time order) while the output is the popularity of a “future” photo (a photo before its publication on social media) 41
- 42. Experiments • Prediction performances on TPIC17-100K, 200K, and 400K datasets – Metric: Spearman Ranking Correlation 42
- 43. More Influential Factors: Cultures • A voting survey of the 2014 TripAdvisor's Top 10 Attractions in Japan by visitors from different countries shows how much the favorites for attractions can vary among people from different regions, i.e., different cultures. 43 [Ref] 2014 TripAdvisor’s Top 20 Attractions in Japan: http://www.tripadvisor.com/pages/- HotSpotsJapan.html.
- 44. Foursquare Dataset https://sites.google.com/site/yangdingqi/home/foursquare-dataset • Individual “check-ins” data of the more than 10 million users on Foursquare 44
- 45. A Pilot Study: Understanding Foursquare Venue Popularity in Taiwan 45 Performed by Mr. Mrinal Kanti Baowaly in 2016
- 46. Taiwan vs. USA – Venue Distribution of Top 10 Categories 46 Taiwan USA
- 47. More Influential Factors: Personalization • What Your Facial Features Say About Your Personality (MM13) 47 personality report facial image
- 48. Learning Relevance by Neighbor Voting 48 [Ref] X. Li, C.G.M. Snoek, M. Worring, “Learning tag relevance by neighbor voting for social image retrieval,” Proc. ACM Intl. Conf. Multimedia Information Retrieval (MIR), 2008.
- 49. More Influential Factors: Personal Fashion Flavor 49 [Ref] “Fashion Analysis: Current Techniques and Future Directions,” IEEE Multimedia, 2014.
- 50. Urban Tribes: Analyzing Group Photos from a Social Perspective [CVPR’12] 50 Urban tribe: the term to describe subcultures of people who share common interests and tend to have similar styles of dress, to behave similarly, and to congregate together. (coined by French sociologist Michel Maffesoli in 1985) Which groups of people would more likely choose to interact socially? (a) and (b) or (a) and (c)?
- 51. Clothing Fashion Analysis 51 "i-Stylist: Finding the Right Dress Through Your Social Networks," MMM 2017. "A Framework of Enlarging Face Datasets Used for Makeup Face Analysis," BigMM 2016. "What are the Fashion Trends in New York?" MM 2014. (Grand Challenge Prize) "Clothing Genre Classification by Exploiting the Style Elements," MM 2012.
- 52. Clothing fashion is a reflection of the society of a period • The global fashion apparel market today has surpassed 1 trillion US dollars since 2013, and accounts for nearly 2 percent of the world's Gross Domestic Product (GDP) 52
- 53. Trend Analysis for the Clothing Fashion Our work received “Multimedia Grand Challenge Award” in 2014 ACM Multimedia Conference.
- 54. Applications: “Fashion is becoming mobile first with apps that help track down must-have clothes, accessories and shoes” - theguardian.com LIKEtoKNOW.it The Netbook Snap Fashion The Hunt
- 55. http://www.fashiontv.com/videos/fashion-weeks Construct a fashion show dataset Source: New York Fashion Weeks
- 56. Color Cut Pattern Head decoration major elements for fashion style investigation key factors for discovering fashion trends: coherence (frequently occur within a fashion week) uniqueness (occur much more often in a fashion week than in other fashion weeks)
- 57. http://www.fashiontv.com/videos/fashion-weeks Detect the presence of catwalk models over all video frames . . . . . . . . . . . . . . . . . . http://www.fashiontv.com/videos/fashion-weeks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Identify distinct catwalk models
- 58. . . . . . . . . . . . . . . . . . . Identify distinct catwalk modelsExtract model location and the full-body image
- 59. Collect full-body image of catwalk models Catwalk Models e.g. NYFW Autumn/Winter 2014 Positive set Negative set e.g. all catwalk models at NYFW except for Autumn/Winter 2014 Divide the collection of full-body images into two sets Distributional clustering technique W.‐H. Cheng et al., "Learning and Recognition of On‐Premise Signs (OPSs) from Weakly Labeled Street View Images," IEEE Tran. on Image Processing (TIP), 2014.
- 60. Query Image Query Image Color Analysis Texture Analysis Color + Texture Analysis Query ImageQuery Image Color Analysis Texture Analysis Color + Texture Analysis Query Image Query Image Query Image Color Analysis Texture Analysis Color + Texture Analysis Query Image Query Image Query Image Color Analysis Texture Analysis Color + Texture Analysis Query Image Query Image Query Image Spring/Summer 2011 Spring/Summer 2013 Spring/Summer 2013 Spring/Summer 2011 Spring/Summer 2013 Spring/Summer 2013
- 61. Predicting Occupation via Human Clothing and Contexts [ICCV’11] • Diving into the recognition of high-level semantic categories of human such as occupations 61
- 62. Recognizing City Identity via Attribute Analysis of Geo-tagged Images [ECCV’14] • A set of 7 high-level attributes is used to describe the spatial form of a city (amount of vertical buildings, type of architecture, water coverage, and green space coverage) and its social functionality (transportation network, athletic activity, and social activity). 62
- 63. From Scene Attributes to City Attributes • 102 scene attributes are defined. • Each of the city attribute classifier is modeled as an ensemble of SVMs. 63
- 64. Spatial Analysis of City Attributes • The city perception map visualizes the spatial distribution of the 7 city attributes in different colors and exhibits the visitors’ and inhabitants’ own experience and perception of the cities, while it reflects the spatial popularity of places in the city across attributes. 64
- 65. Attribute-Based City Identity Recognition 65
- 66. 66 Sociological understanding of humans and human interactions is fun but still a long way to go!
- 67. 67 ACM Multimedia 2017 http://www.acmmm.org/2017/ Grand Challenge Social Media Prediction (SMP): Predicting the “Stars of Tomorrow” on Social Media https://social-media-prediction.github.io/MM17PredictionChallenge/ Organizers Wen-Huang Cheng Academia Sinica Bo Wu Chinese Academy of Sciences Yongdong Zhang Chinese Academy of Sciences Tao Mei Microsoft Research Asis
- 68. 68
- 70. YFCC100M • This dataset contains 100 million media objects and explain the rationale behind its creation. This list is compiled from data available on Yahoo! Flickr. 70 Two photos of real world scenes from photographers in the YFCC100M dataset.
- 71. YFCC100M • Global coverage of a sample of one million photos from the YFCC100M dataset. 71
- 75. ICWSM-16 Dataset International Conference on Web and Social Media • http://www.icwsm.org/2016/datasets/datasets/ 75
- 76. 76 General Chairs Program Chairs Wan-Chi Siu Hong Kong Polytechnic University Chia-Wen Lin National Tsinghua University Wen-Huang Cheng Academia Sinica Gene Cheung National Institute of Informatics vcip2018.org
- 77. Let’s exchange ideas! 77 whcheng@citi.sinica.edu.tw Wen-Huang Cheng wenhuangcheng