Solr@twitter                  (yats)	
  
http://pcod.no-ip.org/yats/

twitter.com/@PENGUINANA_

2010/3/11 solr     	
  
twitter                (yats)	
  

  2008/11

                    4.9
              xxxGB
       +1.2 /⽉月           ( ...
twitter               (yats)	
  

  2008/11

                    4.9
              125GB
       +1.2 /⽉月           (  ...
 

  yats             Solr          ⼯工
      
              ⾼高
       tips


                   (      ⽇日            ...
…	
  

  RSS

  twitter

  API       (JSONP   )



     solr

     http    ⾃自           	
  
⾼高…	
  

    42              /s

             ⼀一⽅方



      …	
  
Solr                	
  

                   6

       read only   4                                ++      	
  




   ...
Solr                      	
  

                     6                                	
  


       read only     4




...
tips	
  

  Jetty

  Daemon
       >solr[1-4…] start,restart,stop

 
       java -Xms1500M -Xmx1500M -XX:+UseConcMark...
tips	
  

 
                5
           

  SSD          OS
       100GB
        OS           1GB
       Solr      ...
 
       	
  
 
       	
  
 
  >              	
          :          	
  >260          	
  	
                    >          	
                     ...
 




         	
                    	
  
•  LDA                 •              	
  
                	
  
 

  LDA(Latent Dirichlet Allocation)
             2                   50                           …
                 ...
 

         48%          yats   	
  
300000
250000
200000
150000
100000
 50000
     0                              	
  
 




                       	
     Facet navigation	
  
• Solr   	
                   •         ⽰示	
  
Solr                               	
  

  Facet navigation
   

  Dynamic field
                                  …
 ...
Facet navigation	
  

	
  if facetquery:

           params["facet"] = "true"

           params[“facet.field”] = “tag_s”
...
Facet navigation	
  

                    …

<lst name="tag_s">

<int name="    ">2568</int>

<int name="      ">711</int...
Apache Solr⼊入⾨門              …	
  

  Facet query
                 …   …	
  
 

  Apache
     CPU : Core2Quad Q9650 / 4GB / HDD

  Solr
     CPU : Core2Quad Q9650 / 8GB / X25-M 160GB

  Solr2(20...
 


• Streaming API
• user_timeline
• user_timeline	
  

                                     ++   	
  



               ...
Upcoming SlideShare
Loading in …5
×

solr@twitter検索2010

2,805 views
2,683 views

Published on

solr勉強会2回目発表資料

solr@twitter検索2010

  1. 1. Solr@twitter (yats)   http://pcod.no-ip.org/yats/ twitter.com/@PENGUINANA_ 2010/3/11 solr  
  2. 2. twitter (yats)     2008/11   4.9   xxxGB   +1.2 /⽉月 ( …)   200 /   <1000ms(97%) <1500ms(99%)   204ms
  3. 3. twitter (yats)     2008/11   4.9   125GB   +1.2 /⽉月 ( …)   200 /   204ms   <1000ms(97%) <1500ms(99%)
  4. 4.     yats Solr ⼯工     ⾼高   tips   ( ⽇日 )   Solr ⽅方  
  5. 5. …     RSS   twitter   API (JSONP ) solr http ⾃自  
  6. 6. ⾼高…     42 /s   ⼀一⽅方 …  
  7. 7. Solr     6   read only 4 ++     1⽇日/ 1 1 /⽉月     100 / 1 360 400 /⽇日  
  8. 8. Solr     6     read only 4   24 / 1 ⼗十 …     100 / 1  
  9. 9. tips     Jetty   Daemon   >solr[1-4…] start,restart,stop     java -Xms1500M -Xmx1500M -XX:+UseConcMarkSweepGC -XX: +UseParNewGC -XX:CMSInitiatingOccupancyFraction=70   Project vordemort …
  10. 10. tips       5     SSD OS   100GB OS 1GB   Solr ⽤用
  11. 11.    
  12. 12.    
  13. 13.     >   :  >260       >   :  >194       (260) |       (194) |       @zanki   @coinlocker69   ⾒見見 ⾳音   ⽅方   ⾒見見     ⼀一 ⼈人 ⽵竹   @GorosBar   ⼭山 ⼀一 w   @yawaraka_bank AKB     2   AKB ⼤大 ⼦子 3 …  #akb48 ⽅方     15⽇日  
  14. 14.       •  LDA •     
  15. 15.     LDA(Latent Dirichlet Allocation)   2 50 …   50 ⼒力力   ⾷食                   1 3 6 10 19 21 26 31 39 41 43 46 49 41   46  
  16. 16.   48% yats   300000 250000 200000 150000 100000 50000 0  
  17. 17.     Facet navigation   • Solr   •  ⽰示  
  18. 18. Solr     Facet navigation     Dynamic field   …   String tag_s ⾔言 ⼊入  
  19. 19. Facet navigation    if facetquery: params["facet"] = "true" params[“facet.field”] = “tag_s” params[“facet.mincount”] = “10” params["fq"] = facetquery  
  20. 20. Facet navigation     … <lst name="tag_s"> <int name=" ">2568</int> <int name=" ">711</int> <int name=" ">615</int> <int name=" ⾦金金">584</int> <int name=" ">507</int> <int name=" ">353</int> …
  21. 21. Apache Solr⼊入⾨門 …     Facet query … …  
  22. 22.     Apache   CPU : Core2Quad Q9650 / 4GB / HDD   Solr   CPU : Core2Quad Q9650 / 8GB / X25-M 160GB   Solr2(2010/3 S-in)   CPU : Core2Quad Q9550 / 16GB / X25-M 160GB
  23. 23.   • Streaming API • user_timeline • user_timeline   ++   ⽤用  

×