Building Search Engine in the Social Media Era

915
-1

Published on

Rising of social networks like Twitter and Facebook has been flooding the web with user-generated contents in the amount that was never seen before. As we become more depending on the social media for daily information gathering and social interaction, we have started to demand more from search engines, such as real time conversations and social connections. Such new environment is forcing search engine technology to evolve.



This presentation will first review the traditional search engine technologies and architecture. Then, we will talk about some of the new architectures for supporting real time search and social search. The last part of the presentation is devoted to discussion of the emerging real time web monitoring system, the cousin of search engine. Social media monitoring is predicted to become a mainstream tool for consumers and businesses because the inherent collective intelligece has the potential to further increase the efficiency of finding information and making decision as what search engine has done.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
915
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
27
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Building Search Engine in the Social Media Era

  1. 1. SDForum Event Building Search Engine in the Social Media Era AJ Chen ajchen@web2express.org twitter: @web2express http://web2express.org 3/24/2010
  2. 2. Agenda 1. The changing search environment 2. Traditional search engine architectures 3. New architectures for real time search and social search 4. Semantic search 5. Emerging real time web monitoring 6. Q/A
  3. 3. The Changing Search Environment
  4. 4. The Changing Search Environment
  5. 5. The Changing Search Environment
  6. 6. Agenda 1. The changing environment 2. Traditional search engine technologies and architecture 3. New architectures for supporting real time search and social search 4. Semantic search 5. Emerging real time web monitoring system 6. Q/A
  7. 7. Open Source Lucene/Solr from Grant Ingersoll
  8. 8. Lucene/Solr Search for Web App
  9. 9. Lucence Indexing/Searching
  10. 10. Solr Server Architecture from Yonik Seeley
  11. 11. Scaling Lucene and Solr from Mark Miller
  12. 12. Agenda 1. The changing environment 2. Traditional search engine technologies and architecture 3. New architectures for supporting real time search and social search 4. Semantic search 5. Emerging real time web monitoring system 6. Q/A
  13. 13. Near Real Time Search Lucene 2.9: IndexReader (old + new records) <-- IndexWriter Zoie (LinkedIn): Swap Indexes in memory
  14. 14. Google Real-time Search
  15. 15. Google Social Search
  16. 16. Google Aardvark Social Q/A Engine
  17. 17. Agenda 1. The changing environment 2. Traditional search engine technologies and architecture 3. New architectures for supporting real time search and social search 4. Semantic search 5. Emerging real time web monitoring system 6. Q/A
  18. 18. Bing/Powerset NLP Search Engine from Barney Pell
  19. 19. Sindice: Retrieving semantic data index API: sindice.com search engine: sig.ma
  20. 20. Google Semantic Search
  21. 21. Hybrid Search Engine
  22. 22. Hybrid (Universal) Search
  23. 23. Agenda 1. The changing environment 2. Traditional search engine technologies and architecture 3. New architectures for supporting real time search and social search 4. Semantic search 5. Emerging real time web monitoring system 6. Q/A
  24. 24. Social Media Monitoring Benefits for businesses: understand what people are talking about the company, brands, products, and competitions, etc. identify leads for marketing and sales; engage with customer and community conversations; support customers; cultivate product advocates; use customer feedback to improve products and brands; Differences from search engine: search engine for filtering real-time text analysis and semantic analysis auto-discovery of trends/patterns collective intelligence rule engine for routing right information to right person
  25. 25. Basic Monitoring
  26. 26. Scaling up Social CRM Problem: Today's social media program does not scale Solution: Build a pipeline of social context flow and then apply advanced text analytics to automate routing of social contexts.
  27. 27. Intelligent Monitoring Pipeline
  28. 28. Social CRM Integration in the Clouds
  29. 29. Turning CRM into Social CRM
  30. 30. Agenda 1. The changing environment 2. Traditional search engine technologies and architecture 3. New architectures for supporting real time search and social search 4. Semantic search 5. Emerging real time web monitoring system 6. Q/A

×