Recommendation engine


Published on

Recommendation Engine, Its base technologies and Its primary modules.

Published in: Technology, Education
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • The GraphLab model is defined in 4 parts. The Data Graph which is used to express sparse data dependencies in your computation.And the Shared Data Table which is used to express global data as well as global computationIn addition, we also have the scheduler which determines the order of computationAnd the scope system which provides thread safety and consistency.
  • 2 parts. A Map stage and a Reduce stage. The Map stage represents embarassingly parallel computation. That is, each computation is independent and can performed on different macheina without any communciation.
  • For instance, we could use MapReduce to perform feature extraction on a large number of pictures. For instance, .. To compute an attractiveness score.
  • The Reduce stage is essentially a “fold” or an aggregation operation over the results. This for instance can be used to compile summary statistics.
  • Recommendation engine

    1. 1. RecommendationEngine
    2. 2. Outlines Introduction Objectives Scope Problem with existing system Purpose of new system Proposed architecture Technologies to be used Modules of system Integration of technologies Implementation Issues to be solved Application Future Enhancement
    3. 3. Objectives Information Filtering System Recommendation engine recommends- User based- Item based- Slop based Run On Cloud Environment
    4. 4. Introduction Engine - Gives Suggestion Based onmovies,songs,videos,websites,books,images and alsosocial elements. Applicable for E-business. Useful for both Customers and online Retailers Recommendation engine is being used atAmazon, Youtube, Facebook,Twitter
    5. 5. Scope Our system will only provide Recommendation serviceonly. Recommendation will be genrated based on user’shistorical activity like purchase pattern as well asrating and like. Recommendation will be either stored on database,file or directly retrieved to retailers web application.
    6. 6. Problems with existing System Take more Time to generate recommendations No real time recommendation for large data
    7. 7. Purpose of new System Less time for generating recommendations Applicable for Bigdata Recommendations be several algorithms User based Item based Slop based Association rule mining Evaluation of recommendation
    8. 8. Recommendations-Type User Based Recommendation
    9. 9. Recommendations-Type Item Based Recommendation
    10. 10. Proposed System Architecture
    11. 11. Technologies to be used Hadoop Mahout Graphlab Google prediction Google Storage Google App engine
    12. 12. Modules of System User Module Admin Module Recommendation Module File management Module Search Module
    13. 13. Integration of Technologies Mahout based Recommendation Graph based Recommendation Google prediction Based Recommendation
    14. 14. Technology: HADOOP Hadoop is a top-level Apache project being builtand used by a global community of contributors. Hadoop project develops open-source software forreliable, scalable, distributed computing. It enables applications to work with thousands ofnodes and peta bytes of data. Hadoop also support Map/Reduce Algorithm. It provides HDFS file system that stores data onthe compute nodes.
    15. 15. Hadoop
    16. 16. Graphlab It is New Parallel Framework for MachineLearning Algorithm . Now a day ,Designing and implementing efficientand correct parallel machine learning (ML)algorithms can be very challenging. Designed specifically for ML needs Automatic data synchronization. Map phase like – Update Function . Reduce phase like – Sync Operation .
    17. 17. 17Data GraphShared Data TableSchedulingUpdate Functions andScopesGraphLabModel
    18. 18. CPU 1 CPU 2 CPU 3 CPU 4MapReduce – Map Phase18Embarrassingly Parallel independent computation12.942.321.325.8No Communication needed
    19. 19. CPU 1 CPU 2 CPU 3 CPU 4MapReduce – Map Phase19Embarrassingly Parallel independent computation12.942.321.325.824.184.318.484.4No Communication needed
    20. 20. CPU 1 CPU 2MapReduce – Reduce Phase2012.942.321.325.824.184.318.484.417.567.514.934.32226.261726.31Fold/Aggregation
    21. 21. Graphlab in Recommendation Graphlab provide better way in recommendationengine. Its just first load fits simple dataset file. In graphlab we can also implement various algortihmlike k-means clustering ,fuzzy logic, pagerank and etc. Its first translated dataset into Matrix form. And then according to different algorithm itgenerated recommendated output.
    22. 22. Google Prediction Service Google cloud service used for Building smartApplication. Having Machine learning Algorithms. Related to Artificial Intelligence.
    23. 23. Google Prediction Service Google Prediction API : Set of Methods for Data Analysis. Libraries support multiple languages. Google App Engine : Enable Application to Cloud environment Applicationserver Google Cloud Storage : Enable Data to store on Google Cloud database.
    24. 24. Google Prediction Service
    25. 25. Technology : MAHOUT• Apache Mahout is open source project by the ApacheSoftware Foundation (ASF).• The primary goal of Mahout is creating scalablemachine-learning algorithms.• Several Map-Reduce in Mahout enabled clusteringimplementations, including k-Means, fuzzy k-Means,Canopy, Dirichlet, and Mean-Shift.• Mahout have fix datasets which generally take as datainput.• Amzon EC2 are working with Hadoop and Mahout.
    26. 26. Implementation Issues to solved Lack of knowledge about hadoop,mahout,hive Memory issue Operating system support Load Balancing Configuration Data normalization Developing Clustering algorithm Configuring mahout with hadoop
    27. 27. Application of recommendation Yahoo! Facebook Twitter Baidu eBay LinkedIn New York Times Rackspace eHarmony PowersetRecommendationEngine
    28. 28. Future enhancement Integration with Web Application like Jsp , Servlet Integration with Database likeHive, Hbase, Mongodb, Couch db Cloud based recommendation Service Integration of Mahout , Graphlab and Google predictionbased recommendation services. Mobile application integration
    29. 29. Thank You