Your searchbox doesn’t need to be hidden in a corner. Put it in focus and allow people to quickly go where they can find the information they need. In this talk you learn how Neos allows you to offer autocompletion, suggestions and direct navigation to results while entering a searchterm. This talk shows you all the building blocks to integrate a powerful search with Neos.

1. Search as main navigation

2. Daniel Lienert
• Scrum Master / Software Architect
@ punkt.de / Karlsruhe
• Neos Core Team Member
• @dlienert

3. Sebastian Helzle
• Product Owner
@ punkt.de / Karlsruhe
• Neos Core Team Member
• @sebobo

4. Overview
• Motivation to integrate a great search
• The basics
• Indexing
• Search
• Result rendering
• Live search
• Outlook

5. Motivation

6. „Increase user interaction“

7. „Optimize real users workflows“

8. „Different users different goals“

9. „Choose the right tools“

10. „Don’t build or sell Google 2.0“

11. „Have an ongoing feedback loop“

12. „Think about search
in the design process“

18. Glossary

19. Glossary
• Terms & Phrases

20. Glossary
• „Did you mean“

21. Glossary
• Completions & Suggestions

22. Glossary
• Aggregations & Facets

23. Glossary
• Boosting & Weights

24. Building Blocks

25. Elasticsearch
Elasticsearch is an open source, distributed, scalable,
document-oriented, RESTful, full text search engine
with real-time search an analytics capabilities.
Based on Apache Lucene. Combines search and
powerful analytics.
Provides a HTTP REST and a Java interface.

26. Neos Content Repository
Flowpack.SearchPlugin
Neos.ContentRepository.
Search
Flowpack.ElasticSearch.ContentRepositoryAdaptor
Flowpack.ElasticSearch

27. Neos Content Repository
Flowpack.SearchPlugin
Neos.ContentRepository.
Search
Flowpack.ElasticSearch.ContentRepositoryAdaptor
Flowpack.ElasticSearch
‣ The frontend plugin
‣ Search input and result rendering
‣ Soon: Controller for real-time autocompletion
and suggestions

28. Neos Content Repository
Flowpack.SearchPlugin
Neos.ContentRepository.
Search
Flowpack.ElasticSearch.ContentRepositoryAdaptor
Flowpack.ElasticSearch‣ Indexing of nodes to ES documents
‣ Compiles Eel statements into ES queries
‣ ES driver for versions 1.x and 2.x

29. Flowpack.SearchPlugin
Neos Content Repository
Neos.ContentRepository.
Search
Flowpack.ElasticSearch.ContentRepositoryAdaptor
Flowpack.ElasticSearch
‣ Provides the indexing EelHelper
‣ Provides an abstract search helper interface.

30. Flowpack.SearchPlugin
Neos Content Repository
Neos.ContentRepository.
Search
Flowpack.ElasticSearch.ContentRepositoryAdaptor
Flowpack.ElasticSearch
‣ Provides an API for using ES with Flow
‣ Low Level Interface to ES

31. Start the Engine

32. Preparation - Installation
Step 1: Install Elasticsearch
•
wget https://download.elastic.co/.../elasticsearch-2.4.0.zip
unzip elasticsearch-2.4.0.zip
elasticsearch-2.4.0/bin/elasticsearch

33. Preparation - Installation
Step 2: Configure Elasticsearch
•
script.inline: true
config/elasticsearch.yml:

34. Indexing

35. Basic Index Configuration
• Configuration provider
• Neos CR Search package
• Elasticsearch CR Adaptor
• Flowpack Searchplugin
• Many use cases don’t need extra configuration

36. Node Types
• Package stack already provides many defaults
• Custom configurations can be added or extended
• Best practice
• Use mixins for common configurations

37. Node Type indexing example
'PunktDe.Website:EmployeeElement':
search:
fulltext:
isRoot: true
label: '${String.cropAtWord(q(node).property(''name'') || … }’
superTypes:
'Neos.Neos:Content': true
'Flowpack.SearchPlugin:SuggestableMixin': true
'Flowpack.SearchPlugin:AutocompletableMixin': true
…

38. Node Type indexing example
'PunktDe.Website:EmployeeElement':
search:
fulltext:
isRoot: true
label: '${String.cropAtWord(q(node).property(''name'') || … }’
superTypes:
'Neos.Neos:Content': true
'Flowpack.SearchPlugin:SuggestableMixin': true
'Flowpack.SearchPlugin:AutocompletableMixin': true
…

39. Node Type indexing example
'PunktDe.Website:EmployeeElement':
…
properties:
'__suggestions':
search:
indexing: "${Flowpack.SearchPlugin.Suggestion.build(q(node).property('name')
+ …, node.identifier, FusionRendering.render(node, 'suggestion'), 100)}"
'__completion':
search:
indexing: "${q(node).property('name')}"
name:
type: string
search:
fulltextExtractor: ${Indexing.extractInto('h1', value)}

40. Node Type indexing example
'PunktDe.Website:EmployeeElement':
…
properties:
'__suggestions':
search:
indexing: "${Flowpack.SearchPlugin.Suggestion.build(q(node).property('name')
+ …, node.identifier, FusionRendering.render(node, 'suggestion'), 100)}"
'__completion':
search:
indexing: "${q(node).property('name')}"
name:
type: string
search:
fulltextExtractor: ${Indexing.extractInto('h1', value)}

41. Node Type indexing example
'PunktDe.Website:EmployeeElement':
…
properties:
'__suggestions':
search:
indexing: "${Flowpack.SearchPlugin.Suggestion.build(q(node).property('name')
+ …, node.identifier, FusionRendering.render(node, 'suggestion'), 100)}"
'__completion':
search:
indexing: "${q(node).property('name')}"
name:
type: string
search:
fulltextExtractor: ${Indexing.extractInto('h1', value)}

42. Indexing Assets
bin/plugin install elasticsearch/elasticsearch-mapper-attachments/2.1.7
• Requires the attachment type for Elasticsearch
• Included for Elasticsearch > 2.1
• For 1.7 you need to install the plugin separately

43. Asset indexing configuration
Neos:
ContentRepository:
Search:
defaultConfigurationPerType:
'NeosMediaDomainModelAsset':
elasticSearchMapping:
type: attachment
include_in_all: true
indexing: ${Indexing.indexAsset(value)}
'array<NeosMediaDomainModelAsset>':
elasticSearchMapping:
type: attachment
include_in_all: true
indexing: ${Indexing.indexAsset(value)}

44. Searching
Search

45. Basic Search

46. Basic Search
baseQuery = ${Search.query(site).fulltext(this.searchTerm)}

47. •CaseStudies•BlogArticles
Node Type Based Rendering
•Pages

48. Node Type Based Rendering
prototype(Neos.Neos:DocumentSearchResult) {
}
prototype(Vendor.Site:CaseStudySearchResult) < prototype(Neos.Neos:DocumentSearchResult) {
templatePath = resource://Vendor.Site/.../CaseStudySearchResult.html
}
prototype(Vendor.Site:BlogPostSearchResult) < prototype(Neos.Neos:DocumentSearchResult) {
templatePath = resource://Vendor.Site/.../BlogPostSearchResult.html
}

49. Facets #1
aggregateQuery = ${this.baseQuery
.fieldBasedAggregation('types','_type')
.execute().getAggregations().types.buckets}

50. Facets #2
<f:for each="{types}" as="type">
<li>
<neos:link.node node="{searchPage}"
arguments="{'search' : ‚{searchTerm}‘, 'type': '{type.key}'}">
{type.key} ({type.doc_count})
</neos:link.node>
</li>
</f:for>

51. Facets #3
baseQuery.@process {
facets = ${value.queryFilter("term", {"_type": request.arguments.type})}
}

52. Highlighting
searchQuery.@process {
highlight = ${value.highlight(300, 1)}
}

53. Did You Mean
didYouMean = ${SearchResult.didYouMean(this.searchQuery)}

54. Live search

55. Autocompletion
• Offers possible word & phrase completions (like google)
• Helps when you’re unsure about spelling
• Very fast
• No correction
• Currently missing
• Doesn’t respect context (e.g. dimensions)
• No weighting

56. Suggestions
• Fuzzy search
• Respects search context (dimensions, workspaces, starting point)
• Slower if context is not cached (Solved in Searchplugin)
• Weighting
• Show alternative results
• Customizable payload during index time
• Type based suggestions
• Pre rendered output
• Direct linking

57. The Future

58. The future
• Elasticsearch 5.x driver
• Support multiple indices (e.g. multi-site installations)
• Better support for language specialities
• Indexqueue
• Improved documentation and example library

59. I also want to have a cool search
• Checkout the Flowpack/Searchplugin package
• Post feedback & ideas
• on Github
• the guild-search Slack channel
• Use the SEO package and index metadata
• Treat internal search like external search (index property)

60. Questions?

61. Links
• https://github.com/neos/typo3cr-search
• https://github.com/Flowpack/
Flowpack.ElasticSearch.ContentRepositoryAdaptor
• https://github.com/Flowpack/Flowpack.SearchPlugin
• https://github.com/Flowpack/Flowpack.ElasticSearch