Moving from fast to solr on atg

1,021 views

Published on

Presented by Ricardo Merizalde, Software Development Manager, Backcountry.com

The intent of this presentation is to describe an implementation of an open source framework for e-commerce sites. The presentation will focus on Oracle ATG but can be extended to other platforms. First, a brief introduction on what CommerceSearch is (an open source integration and framework for eCommerce sites). Review challenges we had with FAST Impulse. Review the main search artifacts merchandisers can manage through CommerceSearch. Review how CommerceSearch integrates with ATG to deploy changes in near real time. Review CommerceSearch integration test framework and automated test framework (Selenium). Finally, summarize the benefits we got by moving from FAST to Solr.

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,021
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
24
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Moving from fast to solr on atg

  1. 1. Moving from FAST to Solr on ATGRicardo MerizaldeSoftware Development ManagerBackcountry.com
  2. 2. Who We Are§  Online  retailer  of  high-­‐end  outdoor  gear  §  Founded  in  1996  by  former  Olympic  skier  §  7  Ecommerce  sites  §  Internet  Retailer  Top  100    “We use the gear we sell”
  3. 3. More Background§ Legacy  Perl/Postgres  ecommerce  plaEorm  § Using  FAST  Impulse/ESP  since  2008  § Implemented  Oracle/ATG  Commerce  in  2012  § 1.5MM  daily  search  engine  hits  during  Q4  2012    
  4. 4. Why change search engines?§ Product  End  of  Life    § Index  latency  § Hardware  footprint    § License  limitaUons    § Support  effort  is  high  § Not  a  good  fit  for  conUnuous  delivery    
  5. 5. Goals§ Low  index  latency  § High  availability  § Easy  to  scale  § ConUnuous  delivery  § Improve  user  experience  § Improve  merchandising  tools    
  6. 6. OpenCommerceSearch
  7. 7. What’s OpenCommerceSearch?§ Oracle  ATG/Solr  IntegraUon  § Search  Module:  Business  Tools  § Framework  to  ease  development    
  8. 8. Integration OverviewEach environment has its own collections
  9. 9. Search Module§  Manage  search  assets  •  Facets  •  Redirects  •  Synonym  Lists  &  Synonyms  •  Boosts  &  Blocks  •  Ranking  Rules  §  Rule  engine  uses  Solr  •  Has  its  own  index  (very  small)  •  Extra  search  call  to  get  rules  (fast  &  cacheable)    
  10. 10. Search Module
  11. 11. Search Module
  12. 12. Search Module§  Rule’s  context  •  Target  Page:  Search,  Category  or  All  Pages  •  Query:  shirts,  [the  north  face]  •  MulU-­‐site  aware  •  Catalog  aware  •  Category  aware    §  Rule’s  life  Ume  •  Start  &  End  Date  
  13. 13. Search Module§  Rule’s  custom  properUes  
  14. 14. Search Module§  Ranking  rules  •  Boost/demote  chunks  of  products  instead  of  individual  products  •  Less  manual  boosUng  •  Ideal  state  is  lidle/no  manual  boosUng.  Use  mulUple  data  points  to  automate  ranking  (clicks,  revenue,  etc.)    •  Uses  boost  funcUons    
  15. 15. Search Module
  16. 16. Search Module
  17. 17. Search Module
  18. 18. Search Module§  Rule  based  categories  •  PromoUon  landing  pages  •  Keyword  landing  pages  •  Price  range,  discount  off,  gender,  hand  pick  products,  etc.  
  19. 19. Search Module
  20. 20. Search Module§  Asset  MigraUon  •  Redirects:  imported  as  exact  matches  •  Synonyms:  manual  cleanup  •  Boosts  &  Blocks:  not  imported  •  Facets:    not  imported  §  Query  Synonyms  •  FAST  is  a  black  box  •  Analysis  Tool  •  Synonym-­‐Expanding  EDisMax  Parser  wriden  by  Nolan  Lawson  
  21. 21. Who We AreIndex Latency  
  22. 22. Index Latency§  Feed  limited  to  XML  feed/exlt  tool  §  Slow  process  with  mulUple  points  of  failure  1.  Generate  XML  files  2.  scp  XML  files  to  FAST  servers  3.  exlt  cronjob  loads  feed  into  SQL  schema  4.  JDBC  Connector  checking  for  changes  5.  Document  Processing  Pipeline  6.  Index  documents  
  23. 23. Index Latency§  Index  documents  using  API  calls  §  Different  SLAs  for  change  types  (price,  inventory,  content,  taxonomy)  §  Auto  soj  &  hard  commits  §  Products  &  Skus  are  not  versioned    
  24. 24. Index Latency
  25. 25. Who We AreConUnuous  Delivery  
  26. 26. Continuous Delivery§  Hard  to  do  integraUon  tesUng  §  Automated    tests  were  flaky  •  Data  dependencies  •  No  applicaUons  states  §  Leverage  EmbeddedSolrServer  •  RAM  index  •  Custom  JUnit  Test  Runner  for  IntegraUon  tests  •  Solr  XML  for  data  •  TestNG  &  Selenium  for  Automated  tests    
  27. 27. Continuous Delivery§  IntegraUon  Test  •  Shared  core  with  bootstrapped  data  •  Clone  core  &  index  test  data  •  Test  code  &  server  run  within  same  process  
  28. 28. Continuous Delivery§  Automated  Test  •  Cores  live  within  applicaUon  server  •  Test  data  sent  over  HTTP    
  29. 29. Continuous Delivery§  Deployments    
  30. 30. Continuous Delivery§  Rundeck  jobs    
  31. 31. User  Experience  
  32. 32. User Experience§  Search  results  not  consistent  with  product  detail  page  §  De-­‐normalize  data  &  index  skus  §  Group  by  product  id  
  33. 33. User Experience
  34. 34. User Experience
  35. 35. User Experience§  Orange  jacket  not  available  in  XXL  
  36. 36. User Experience
  37. 37. User Experience§  Orange  jacket  available  in  XXL  
  38. 38. User Experience§  Grouping  is  awesome  but  is  CPU  intensive  §  Monitor  query  response  Umes  §  Pay  adenUon  to  parUal  match  search  (use  mm  parameter)  §  Use  the  facets  you  really  need  
  39. 39. User ExperienceQuesUons  
  40. 40. CONTACTRicardo Merizaldermerizalde@backcountry.comhttps://github.com/rmerizalde/opencommercesearchhttp://ricardomerizalde.blogspot.com/

×