Data Mining of Informational Stream in Social Networks

4,097 views
4,304 views

Published on

Data Mining of Informational Stream in Social Networks Forecasting of Social, Market and Financial Trends

Published in: Business, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
4,097
On SlideShare
0
From Embeds
0
Number of Embeds
2,541
Actions
Shares
0
Downloads
44
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • {}
  • Data Mining of Informational Stream in Social Networks

    1. 1. Data Mining of Informational Stream in Social Networks Forecasting of Social, Market and Financial Trends Bohdan Pavlyshenko e-mail: b.pavlyshenko@gmail.com blog: bpavlyshenko.blogspot.com
    2. 2. Used technologies: R, Python, Java, Hadoop/MapReduce/Pig/Hive The prototypes of data mining systems are based on the theory of formal concept analysis and on the theory of frequent itemsets. Using a model of a semantic concept lattice makes it possible to analyze semantically related sets of words and to construct association rules. The use of quantitative characteristics of informational streams for marketing trend forecasting and for the analysis of users’ attitude towards different goods and services (Opinion Mining) Detection of predictive potential of association rules in informational streams and the use of these rules in autoregressive models (ARIMA, VAR) for predicting, in particular, the financial trends on stock markets. Such a model takes into account both the past behavior of financial time row of a company and the time dynamics of quantitative characteristics of association rules.
    3. 3. The analysis of communities and their leaders who form analyzed trends in social networks. The analysis of the presence of manipulative formation of users’ attitude towards this or that commodity or economic trend. The causality analysis on the basis of Granger tests for singling out the principal and subordinate time rows, particularly for informational streams, economic indicators, etc. The creation of a subsystem of recommendations for users. For example, in an online store, this system analyzes users’ behavior, their purchases, their feedback towards goods or services. Based on the user’s activity, one can create his/her semantic profile and then make various offers to this user, taking into account his/her activity and the decisions of users with similar profiles. Such an approach may shorten significantly the time the user spent while searching goods and services, and give him/her unknown but necessary offers, revealed on the basis of other similar users’ activities.
    4. 4. The analysis of financial tweets The package “Tweet Miner for Stock Market”
    5. 5. The analysis of financial tweets The formation of keyword frequent sets with the biggest support value
    6. 6. The analysis of financial tweets The analysis of causal relationship between the frequent sets in tweets and Apple stock prices. The results obtained show that it is possible to predict stock prices on the basis of data mining of informational streams in social networks.
    7. 7. The analysis of financial tweets Forecasting based on ARIMA model Granger causality test between quantitative characteristics of tweets and Apple stock prices. test 1 Granger causality test Model 1: V3 ~ Lags(V3, 1:1) + Lags(V2, 1:1) Model 2: V3 ~ Lags(V3, 1:1) Res.Df Df F Pr(>F) 1 87 2 88 -1 10.05 0.002103 ** --Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 test 2 Granger causality test Model 1: V2 ~ Lags(V2, 1:1) + Lags(V3, 1:1) Model 2: V2 ~ Lags(V2, 1:1) Res.Df Df F Pr(>F) 1 87 2 88 -1 0.3261 0.5694 Forecasting based on VAR model
    8. 8. The examples of the studies of semantic concepts in Twitter messages
    9. 9. The examples of the studies of semantic concepts in Twitter messages The Final Olympic Tennis Tournament (2012)
    10. 10. The examples of test studies of semantic concepts in Twitter messages The prediction of Eurovision 2013 favorites Before the Eurovision 2013 final we published our forecasting of a winner and the favorites in our blog. Later on it proved to be correct.
    11. 11. The examples of test studies of semantic concepts in Twitter messages Travel trends The analysis of travel trends
    12. 12. The examples of test studies of semantic concepts in Twitter messages Travel trends The analysis of travel trends
    13. 13. The examples of test studies of semantic concepts in Twitter messages Market analysis of iPhone concept
    14. 14. The examples of test studies of semantic concepts in Twitter messages Market analysis of iPhone concept
    15. 15. The examples of test studies of semantic concepts in Twitter messages The prediction of Royal baby’s name In this work, we analyze the existence of possible correlation between public opinion of twitter users and the decision-making of persons who are influential in the society. We carry out this analysis on the example of the discussion of probable name of the British crown baby, born in July, 2013. In our study, we use the methods of quantitative processing of natural language, the theory of frequent sets, the algorithms of visual displaying of users' communities. We also analyzed the time dynamics of keyword frequencies. The analysis showed that the main predictable name was dominating in the spectrum of names before the official announcement. Using the theories of frequent sets, we showed that the full name consisting of three component names was the part of top 5 by the value of support. It was revealed that the structure of dynamically formed users' communities participating in the discussion is determined by only a few leaders who influence significantly the viewpoints of other users.
    16. 16. The examples of test studies of semantic concepts in Twitter messages Royal baby’s name forecasting The name George was dominating in the spectrum of names before the official announcement.
    17. 17. The examples of test studies of semantic concepts in Twitter messages Royal baby’s name forecasting 10 first frequent sets were created by five names, the three of which are the components of Prince’s full name George Alexander Louis.
    18. 18. The examples of test studies of semantic concepts in Twitter messages The Royal baby’s name forecasting Users’ societies, which formed the discussion trends.
    19. 19. More test examples and studies are in my blog http://bpavlyshenko.blogspot.com Thank you for your attention! Bohdan Pavlyshenko, Ph.D., e-mail: b.pavlyshenko@gmail.com

    ×