Recomendação de Conteúdo para Redes Sociais Educativas
Upcoming SlideShare
Loading in...5
×
 

Recomendação de Conteúdo para Redes Sociais Educativas

on

  • 3,786 views

Palestra ministrada no X Seminário de Engenharia da Computação (SEC) Na Escola Politécnica de Pernambuco, em Recife Pernambuco em 21.10.2011

Palestra ministrada no X Seminário de Engenharia da Computação (SEC) Na Escola Politécnica de Pernambuco, em Recife Pernambuco em 21.10.2011

Statistics

Views

Total Views
3,786
Views on SlideShare
1,741
Embed Views
2,045

Actions

Likes
1
Downloads
14
Comments
0

17 Embeds 2,045

http://www.mobideia.com 1796
http://mobideia.blogspot.com.br 180
http://mobideia.blogspot.com 22
http://flavors.me 19
http://mobideia.blogspot.pt 9
http://www.linkedin.com 4
http://marcelcaraciolo.flavors.me 3
http://mobideia.blogspot.gr 2
http://mobideia.blogspot.tw 2
http://mobideia.blogspot.fr 1
http://mobideia.blogspot.in 1
http://mobideia.blogspot.sg 1
http://translate.googleusercontent.com 1
http://mobideia.blogspot.co.uk 1
http://pt.flavors.me 1
http://36095344_aacee614e82de80bb5f0ee0f1f7e06666148f883.blogspot.com 1
http://www.yatedo.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Recomendação de Conteúdo para Redes Sociais Educativas Recomendação de Conteúdo para Redes Sociais Educativas Presentation Transcript

  • Content  Recommenda,on  Based   on  Data  Mining  in  Adap,ve   Social  Networks Marcel  Pinheiro  Caraciolo mpc@cin.ufpe.brOrientador:  Germano  Crispim gcv@cin.ufpe.br
  • Who is Marcel ?Marcel Pinheiro Caraciolo - @marcelcaraciolo Sergipano, porém Recifense. Cientista Chefe da Orygens (http://orygens.com) Mestrando em Ciência da Computação no CIN/UFPE na área de mineração de dados Membro e Moderador da Celúla de Usuários Python de Pernambuco (PUG-PE) Minhas áreas de interesse: Computação móvel e Computação inteligente Meus blogs: http://www.mobideia.com (sobre Mobilidade desde 2006) http://aimotion.blogspot.com (sobre I.A. desde 2009) Jovem Aprendiz ainda nas artes pythonicas.... (desde 2007) 2
  • The  Problem Provide features that can enhance online social learning environment One social software in special are recommender systems Several approaches have been applied to web-based educational systems [1] [2] But only a few recommender systems use data mining and explanations in the recommendations
  • Objec,ves Design a methodology for social recommender systems that incorporate different knowledge sources It can be structured such as messages, personal profile, timeline, questions , answers It can be non-structured study plans, disciplines, classes watched, relationships educational resourcesents in a social envi- 01 g a recommender en- (%)*++%&$%, 01 -&./&% 01different recommenda- ation called Crab [15]. !"#$%&" (%)*++%&$23*&4 al domains, including 56786- suggest online learn- 599786-eir preferences, knowl-terests. The system is 8&;*,+23*&ational social network 9/<"%,/&. =%)>&/?#%4dents, helping them to !"#$%&":47*. The innovative aspect ing on social networks ions at the given rec- components of Architecture proposal for the recom- It uses several Figure 1: the social networkudent more confidence mender system arning understanding. s use data mining and [5]. 3. CONTENT RECOMMENDATION IN ED-st we describe the re- UCATIONAL SOCIAL NETWORKS
  • Objec,vesDesign an improved explanation of the recommendation to the user Generally in the currently available systems, the recommendation only comes with an overall score Increase the recommendation acceptance rate in order to enhance the student experience in social networks You answered questions about X,Y, Z. You answered questions about X,Y, Z. You have A,B,C friends in common Your friend asked this question. Your friends P,X and Y follows him. You both live at the same state Pernambuco. You have X,Y, Z followers in common You are both interested in portuguese, maths and english.You have X,Y, Z concursos in common You have P,Q, R study groups in common. Because you have difficulty in portuguese, maths and english. Because it is a most searched course
  • What recommender systems really do? 1. Predict how much you would like a certain product or service 2. Suggest a list of N items ranked based on your interests3. Suggest a list of N users ranked for a product/ service4. Explain to you why those recommendations were given5. Adjust the prediction and the recommendation based on your feedback and from others.
  • Mee,ng  Recommenda,on  Systems Content  Based  Filtering Similar Text Portuguese Maths Biology Items Interpretation recommend likes Marcel Users
  • Mee,ng  Recommenda,on  Systems Collabora,ve  Based  Filtering Portuguese Literature Maths Physics Items like like recommends Marcel Rafael Amanda Users Similar
  • Mee,ng  Recommenda,on  Systems Our  Approach  -­‐  Hybrid  Recommender  System Meta recommender system architectureCustomized control over the generation ofa recommendation list !"#$%*+,-)% !"#$%&$()#%It adapts in accordance to the structure of the data ./0#$-+1/% Recommendation of Friends ? More weight to Collaborative Filtering % % % % !"#$%#$&()%*&+,-$%.,#/& !"--(0".(12%&()%*&+,-$%.,#/& % Recommendation of Courses ? 2$,#/3"%456575689% % !"#"$%&&%()*&+,-(.&/,-0&+,-(.& :+-1/;%%<#+,=#% %12%&303#2,&(",&2,"&34& More weight to Content Filtering *+>)-$">,?;%%@$-3A-0#3% % % Cold Start: Mitigate using Popular Recommendations B#0-%<#+CC#/3#$% %% %&-$-C#0#$"%% accepted from another users <#+CC#/3-1/"% Feedback and temporal slicing: Learn from users and select limited results by time Figure 2: Meta Recommender Components Interac- tion be highly beneficial given that students do not meet phys- ically. It may result in their becoming more socially con-
  • Methodology  and  Current  Results !"#$%*+,-)% ./0#$-+1/% this engine with the popular brazilian social network AtéPassar Integrated More than 70.000 students registered studying for the public examinations Recommend StudyGroups, Friends,Video Classes, Questions and Concursos More than 70.000 items available for recommend % % !"--(0".(12%&()%*&+,-$%.,#/& % Written in Python using a open-source framework Crab !"#"$%&&%()*&+,-(.&/,-0&+,-(.& %12%&303#2,&(",&2,"&34& % Framework available for building recommender systems (My contribution) It is running since January 2011 In March B#0-%<#+CC#/3#$% was performed. 2011 , questionnaire %% %&-$-C#0#$"%% Liked Not Liked -1/"% 23%mender Components Interac- 77% Figure 3: AtePassar Recommender Syste face
  • Expected  Results Improve the Learning Process Analyze how the recommenders can increase the learning process in online educational social networks. Mining all types of source in social networks in forms of recommendations Explore the Hidden Knowledge There are several knowledge sources in a educational social network. All those sources as basis for discovery novel content and learning resources Better Recommendation Understanding The explanations can be quite helpful to better understanding of the given resource Better learning interaction and simplification of the human/computer interaction
  • ConclusionsProposal of a personalized recommender systemthat incorporate different types of source of knowledgeIt can be applied in adaptive social networksBring machine learning to web educational systemsHelp the students to find an optimal path to learning resourcesHelp the researchers and designers how to designthe social network to achieve those tasks efficiently
  • Contribu)ons
  • Crab A Python Framework for Building Recommendation EnginesMarcel Caraciolo Ricardo Caspirro Bruno Melo @marcelcaraciolo @ricardocaspirro @brunomelo
  • What is Crab ? A python framework for building recommendation enginesA Scikit module for collaborative, content and hybrid filtering Mahout Alternative for Python Developers :D Open-Source under the BSD license https://github.com/muricoca/crab
  • The current CrabCollaborative Filtering algorithms User-Based, Item-Based and Slope OneEvaluation of the Recommender Algorithms Precision, Recall, F1-Score, RMSE Precision-Recall Charts
  • The current Crab
  • The current Crab>>>#load the dataset
  • The current Crab>>>#load the dataset>>> from crab.datasets import load_sample_movies
  • The current Crab>>>#load the dataset>>> from crab.datasets import load_sample_movies>>> data = load_sample_movies()
  • The current Crab>>>#load the dataset>>> from crab.datasets import load_sample_movies>>> data = load_sample_movies()>>> data
  • The current Crab>>>#load the dataset>>> from crab.datasets import load_sample_movies>>> data = load_sample_movies()>>> data{DESCR: sample_movies data set was collected by the book called nProgramming the Collective Intelligence by Toby Segaran nnNotesn----- nThis data set consists ofnt* n ratings with (1-5) from n users to n movies., data: {1: {1: 3.0, 2: 4.0, 3: 3.5, 4: 5.0, 5: 3.0},  2: {1: 3.0, 2: 4.0, 3: 2.0, 4: 3.0, 5: 3.0, 6: 2.0},  3: {2: 3.5, 3: 2.5, 4: 4.0, 5: 4.5, 6: 3.0},  4: {1: 2.5, 2: 3.5, 3: 2.5, 4: 3.5, 5: 3.0, 6: 3.0},  5: {2: 4.5, 3: 1.0, 4: 4.0},  6: {1: 3.0, 2: 3.5, 3: 3.5, 4: 5.0, 5: 3.0, 6: 1.5},  7: {1: 2.5, 2: 3.0, 4: 3.5, 5: 4.0}}, item_ids: {1: Lady in the Water,  2: Snakes on a Planet,  3: You, Me and Dupree,  4: Superman Returns,  5: The Night Listener,  6: Just My Luck}, user_ids: {1: Jack Matthews,  2: Mick LaSalle,  3: Claudia Puig,  4: Lisa Rose,  5: Toby,  6: Gene Seymour,  7: Michael Phillips}}
  • The current Crab
  • The current Crab>>> from crab.models import MatrixPreferenceDataModel
  • The current Crab>>> from crab.models import MatrixPreferenceDataModel>>> m = MatrixPreferenceDataModel(data.data)
  • The current Crab>>> from crab.models import MatrixPreferenceDataModel>>> m = MatrixPreferenceDataModel(data.data)>>> print mMatrixPreferenceDataModel (7 by 6)         1 2 3 4 5 ...1 3.000000 4.000000 3.500000 5.000000 3.0000002 3.000000 4.000000 2.000000 3.000000 3.0000003 --- 3.500000 2.500000 4.000000 4.5000004 2.500000 3.500000 2.500000 3.500000 3.0000005 --- 4.500000 1.000000 4.000000 ---6 3.000000 3.500000 3.500000 5.000000 3.0000007 2.500000 3.000000 --- 3.500000 4.000000
  • The current Crab
  • The current Crab>>> #import pairwise distance
  • The current Crab>>> #import pairwise distance>>> from crab.metrics.pairwise import euclidean_distances
  • The current Crab>>> #import pairwise distance>>> from crab.metrics.pairwise import euclidean_distances>>> #import similarity
  • The current Crab>>> #import pairwise distance>>> from crab.metrics.pairwise import euclidean_distances>>> #import similarity>>> from crab.similarities import UserSimilarity
  • The current Crab>>> #import pairwise distance>>> from crab.metrics.pairwise import euclidean_distances>>> #import similarity>>> from crab.similarities import UserSimilarity>>> similarity = UserSimilarity(m, euclidean_distances)
  • The current Crab>>> #import pairwise distance>>> from crab.metrics.pairwise import euclidean_distances>>> #import similarity>>> from crab.similarities import UserSimilarity>>> similarity = UserSimilarity(m, euclidean_distances)>>> similarity[1]
  • The current Crab>>> #import pairwise distance>>> from crab.metrics.pairwise import euclidean_distances >>> #import similarity >>> from crab.similarities import UserSimilarity >>> similarity = UserSimilarity(m, euclidean_distances) >>> similarity[1] [(1, 1.0),(6, 0.66666666666666663),(4, 0.34054242658316669),(3, 0.32037724101704074),(7, 0.32037724101704074),(2, 0.2857142857142857),(5, 0.2674788903885893)]
  • The current Crab>>> #import pairwise distance>>> from crab.metrics.pairwise import euclidean_distances >>> #import similarity >>> from crab.similarities import UserSimilarity >>> similarity = UserSimilarity(m, euclidean_distances) >>> similarity[1] [(1, 1.0),(6, 0.66666666666666663), MatrixPreferenceDataModel (7 by 6)          1 2 3 4 5(4, 0.34054242658316669), 1 3.000000 4.000000 3.500000 5.000000 3.000000(3, 0.32037724101704074), 2 3.000000 4.000000 2.000000 3.000000 3.000000 3 --- 3.500000 2.500000 4.000000 4.500000(7, 0.32037724101704074), 4 2.500000 3.500000 2.500000 3.500000 3.000000 5 --- 4.500000 1.000000 4.000000 ---(2, 0.2857142857142857), 6 3.000000 3.500000 3.500000 5.000000 3.000000(5, 0.2674788903885893)] 7 2.500000 3.000000 --- 3.500000 4.000000
  • The current Crab
  • The current Crab>>> from crab.recommenders.knn import UserBasedRecommender
  • The current Crab>>> from crab.recommenders.knn import UserBasedRecommender>>> recsys = UserBasedRecommender(model=m,similarity=similarity, capper=True,with_preference=True)
  • The current Crab>>> from crab.recommenders.knn import UserBasedRecommender>>> recsys = UserBasedRecommender(model=m,similarity=similarity, capper=True,with_preference=True)>>> recsys.recommend(5)array([[ 5. , 3.45712869],       [ 1. , 2.78857832],       [ 6. , 2.38193068]])
  • The current Crab>>> from crab.recommenders.knn import UserBasedRecommender>>> recsys = UserBasedRecommender(model=m,similarity=similarity, capper=True,with_preference=True)>>> recsys.recommend(5)array([[ 5. , 3.45712869],       [ 1. , 2.78857832],       [ 6. , 2.38193068]])>>> recsys.recommended_because(user_id=5,item_id=1)array([[ 2. , 3. ],       [ 1. , 3. ],       [ 6. , 3. ],       [ 7. , 2.5],       [ 4. , 2.5]])
  • The current Crab>>> from crab.recommenders.knn import UserBasedRecommender>>> recsys = UserBasedRecommender(model=m,similarity=similarity, capper=True,with_preference=True)>>> recsys.recommend(5)array([[ 5. , 3.45712869],       [ 1. , 2.78857832],       [ 6. , 2.38193068]])>>> recsys.recommended_because(user_id=5,item_id=1)array([[ 2. , 3. ],       [ 1. , 3. ], MatrixPreferenceDataModel (7 by 6)          1 2 3 4 5 ...       [ 6. , 3. ], 1 3.000000 4.000000 3.500000 5.000000 3.000000 2 3.000000 4.000000 2.000000 3.000000 3.000000       [ 7. , 2.5], 3 --- 3.500000 2.500000 4.000000 4.500000       [ 4. , 2.5]]) 4 2.500000 3.500000 2.500000 3.500000 3.000000 5 --- 4.500000 1.000000 4.000000 --- 6 3.000000 3.500000 3.500000 5.000000 3.000000 7 2.500000 3.000000 --- 3.500000 4.000000
  • Distributed Computing with mrJob https://github.com/Yelp/mrjob
  • Distributed Computing with mrJob https://github.com/Yelp/mrjobIt supports Amazon’s Elastic MapReduce(EMR) service, your own Hadoop cluster or local (for testing)
  • Distributed Computing with mrJob https://github.com/Yelp/mrjobIt supports Amazon’s Elastic MapReduce(EMR) service, your own Hadoop cluster or local (for testing)
  • Distributed Computing with mrJob https://github.com/Yelp/mrjob """The classic MapReduce job: count the frequency of words. """ from mrjob.job import MRJob import re WORD_RE = re.compile(r"[w]+") class MRWordFreqCount(MRJob):     def mapper(self, _, line):         for word in WORD_RE.findall(line):             yield (word.lower(), 1)     def reducer(self, word, counts):         yield (word, sum(counts)) if __name__ == __main__:     MRWordFreqCount.run()It supports Amazon’s Elastic MapReduce(EMR) service, your own Hadoop cluster or local (for testing)
  • Studies with Sparse Matrices Real datasets come with lots of empty values http://aimotion.blogspot.com/2011/05/evaluating-recommender-systems.htmlSolutions: scipy.sparse package Sharding operations Matrix Factorization techniques (SVD) Apontador Reviews Dataset
  • Studies with Sparse Matrices Real datasets come with lots of empty values http://aimotion.blogspot.com/2011/05/evaluating-recommender-systems.html Solutions: scipy.sparse package Sharding operations Matrix Factorization techniques (SVD) Crab implements a MatrixFactorization with Expectation Maximization algorithm Apontador Reviews Dataset
  • Studies with Sparse Matrices Real datasets come with lots of empty values http://aimotion.blogspot.com/2011/05/evaluating-recommender-systems.html Solutions: scipy.sparse package Sharding operations Matrix Factorization techniques (SVD) Crab implements a MatrixFactorization with Expectation Maximization algorithm scikits.crab.svd package Apontador Reviews Dataset
  • Why migrate ?Old Crab running only using Pure Python Recommendations demand heavy maths calculations and lots of processingCompatible with Numpy and Scipy libraries High Standard and popular scientific libraries optimized for scientific calculations in PythonScikits projects are amazing! Active Communities, Scientific Conferences and updated projects (e.g. scikit-learn)Turn the Crab framework visible for the community Join the scientific researchers and machine learning developers around the Globe coding with Python to help us in this project Be Fast and Furious
  • How are we working ? Sprints, Online Discussions and Issueshttps://github.com/muricoca/crab/wiki/UpcomingEvents
  • How are we working ? Our Project’s Home Pagehttp://muricoca.github.com/crab
  • Future Releases Planned Release 0.1 Collaborative Filtering Algorithms working, sample datasets to load and test Planned Release 0.11 Evaluation of Recommendation Algorithms and Database Models support Planned Release 0.12 Recommendation as Services with REST APIs....
  • Join us!1. Read our Wiki Page https://github.com/muricoca/crab/wiki/Developer-Resources2. Check out our current sprints and open issues https://github.com/muricoca/crab/issues3. Forks, Pull Requests mandatory4. Join us at irc.freenode.net #muricoca or at our discussion list http://groups.google.com/group/scikit-crab
  • Recommended booksToby Segaran, Programming Collective SatnamAlag, Collective Intelligence inIntelligence, OReilly, 2007 Action, Manning Publications, 2009 Sites such as TechCrunch and ReadWriteWeb
  • Recommended Conferences- ACM RecSys.–ICWSM: Weblogand Social Media–WebKDD: Web Knowledge Discovery and Data Mining–WWW: The original WWW conference–SIGIR: Information Retrieval–ACM KDD: Knowledge Discovery and Data Mining–ICML: Machine Learning
  • plore more the learning resource materials, one of the goals of the e-learning system. Conference, Bratislava, 2005. 229-234. [7] Romero, C., Ventura, S.: Educational Data Mining: a References 3.5.3 Better Recommendation Understanding The explanations can be quite helpful for users to better understand if the given resource is (or is not ) suitable for Survey from 1995 to 2005. Expert Systems with Applications. Elsevier 1:33, 2007. 135-146. [8] Romero, C., Ventura, S.: Data mining in e-learning. th the 5. REFERENCES Wit Press, 2006. them. Those explanations can be beneficial in how to for- er en- mulate Brusilovsky, P., Peylo, C.: Adaptive and to simply and [1] a better learning interaction and hence Intelligent [9] Atepassar. Available at: http://atepassar.comed the shorten the human/computer Systems. International Journal Web-based Educational interaction. [10] J. B. Schafer, J. A. Konstan, and J. Riedl.:rought of Artificial Intelligence in Education. 13, 2003, 156 - E-commerce recommendation applications. Data Miningquality 169. and Knowledge Discovery, 2001.nalized 4. [2] Conole, G. & Culver, J.: The design of Cloudworks: CONCLUSIONS [11] Shen, L.P., Shen, R.M.: Learning Content In this Applying social networking practice to foster the sys- paper, we presents a personalized recommender Recommendation Service Based-on Simple Sequencing tem that incorporate different types of sourcesand knowledge exchange of learning and teaching ideas of designs In: Specification. In Proceedings of Advanced in Web-based in a adaptative educational social network by3using a meta- Computers & Education , Vol. 54 , Nr. Elsevier, 2010 Learning 2004 363- recommender approach. We believe that this technique will , S. 679-692 . bring more personalized andE., Grigoriadou, M., Samarakou, [12] M. Montaner, B. L’opez, and J. Rosa.: A taxonomy of orking [3] Gogoulou, A., Gouli, better recommendations for the recommender agents on the internet. In Artificial students. and Chinou, D.: A Web-based Educational Setting M.,mmen- Intelligence Review, volume 19, pages 285-330. Kluwer ing to Supporting Individualized Learning, Collaborative Academic, 2003. Although we have integrated the In ProceedingsAtePassar, it Learning and Assessment. system in the oftion of [13] J. Herlocker, J. Konstan, and J. Riedl.: Explaining e that can, in Educational Technology & Society. 2007, 242-256. principle, also be used in other web-based educational systems. MoreJ.B.: The application of data-mining the area [4] Schafer, research need to be conducted in to collaborative filtering recommendations. In Ineduca- proceedings of ACM 2000 Conference on Computermprove of the recommendation systemsJ. Wang (Ed.), Encyclopedia recommender systems. In in social learning and how they can data warehousing and mining. Hershey, PA. Idea of influence the student satisfaction, motivation and Supported Cooperative Work, 2000. learning process. 44 - 48. Group 2005 [14] B. Schafer, J. Konstan, and J. Riedl.: [5] Farzan, R., Brusilovsky, P.: Social Navigation Support Meta-recommendation systems: user-controlled in a Course Recommendation System. In proceedings of Bringing machine learning and artificial intelligence to edu- integration of diverse recommendations. In Proceedingsrces in of the 11th conference on Information and knowledge of the cational social networks Conference on main contributions of 4th International is one of the Adaptive Hypermedia this work, Adaptive resulting on a new generation of web- and which is Web-based Systems. Dublin, 2006. management, pages 43-51. ACM Press, 2002. of the 91-100. educational systems. The task of recommendation in web- [15] Crab. Available at: http://muricoca.github.com/crabve themation based education A.: Recommender System for Adaptive col- [6] Ksristofic, can be considered as special type of [16] AtePassar Video. Available at:nd ex- laborative learning, due to the fact that they of Informatics Hypermedia Applications. In Proceeding help student http://www.youtube.com/watch?v=gzn7I1EJmb4e goals and Information Technology Student Research with similar interests to find an optimal path through the learning material and facilitate the 229-234. Conference, Bratislava, 2005. social network design to [7] Romero, C., Ventura, S.: Educational Data Mining: a be effectively be used to achieve these targets. We provide a videoSurvey from 1995 to recommender Systemsin action at demonstrating the 2005. Expert system with Applications. Elsevier 1:33, 2007. 135-146. AtePassar, which can be accessible at œ[16].better [8] Romero, C., Ventura, S.: Data mining in e-learning.ble for Wit Press, 2006.to for-ly and [9] Atepassar. Available at: http://atepassar.com [10] J. B. Schafer, J. A. Konstan, and J. Riedl.: E-commerce recommendation applications. Data Mining and Knowledge Discovery, 2001. [11] Shen, L.P., Shen, R.M.: Learning Contenter sys- Recommendation Service Based-on Simple Sequencingwledge
  • AtéPassar  -­‐  hOp://atepassar.com Data  available  for  researching  -­‐  marcel@atepassar.com
  • Where will you be ? Source: Hunch.com Thanks !!
  • Content  Recommenda,on  Based   on  Data  Mining  in  Adap,ve   Social  Networks Marcel  Pinheiro  Caraciolo marcelcaraciolo@acm.org