The document discusses highlighting in legal search applications. It provides an overview of existing highlighters in Solr and their tradeoffs between accuracy, speed and memory usage. It then describes improvements made to the standard highlighter and the development of a new Unified Highlighter that aims to improve performance while maintaining accuracy. Benchmark results show the Unified Highlighter performs similarly or better than other highlighters. Future work is discussed to further improve accuracy and relevancy.

1. O C T O B E R 1 1 - 1 4 , 2 0 1 6 • B O S T O N , M A

2. Solr Highlighting at Full Speed
Timothy M. Rodriguez
Verticals Search Team Lead, Bloomberg
David Smiley
Search Developer/Consultant

3. 3
Agenda
§ Legal Search
§ Business Requirements
§ Highlighters Overview
§ Improving the Standard Highlighter
§ Unified Highlighter
§ Questions
01

4. Bloomberg Law
§ Suite of legal business tools
for lawyers and legal
professionals
§ Business development
§ Drafting
§ Analytics
§ Search

5. 5
Legal Search
§ Recall Matters
§ Large Documents
§ Citizens United is 130 pages
long
§ Some in the 100s of MB
§ Researchers rely on
highlighting to help them
decide if they should read a
document
01

6. Requirements
ü

7. Accuracy
§ Legal users issue detailed searches
§ “cafeteria plan” AND tax
§ Custom Span Queries
§ “insurance fraud” /s conviction

8. “Just Right” Digest Sizing

9. 9
Full Document Highlighting
01

10. Zone Highlighting

11. 11
Speed
01

12. 12
Solr Highlighters Overview
01
Highlighter Offset Source Accuracy Speed
Default/Standard Highlighter
Analysis
Better
Slowest
Term Vectors Slow
Fast Vector Highlighter Term Vectors Good Medium
Postings Highlighter Postings (+ Analysis for wildcards) Okay Fast*
But poor wildcard
performance

13. Offset Source & Index Size
§ Analysis requires no extra data on-disk
§ But analyzing text on the fly is expensive
§ Term vectors are heavy
§ Adding offsets to postings is much lighter than TVs
0 0.5 1 1.5 2 2.5 3
Multiples of Stored Value
StoredValue
Terms
Positions
Offsets
TV Terms
TV Positions
TV Offsets

14. 14
Initial Attempt
§ Chose the default highlighter and added in customizations as needed
§ Added payload support to the MemoryIndex - LUCENE-6155 v5.0
§ We investigated using the PostingsHighlighter and
FastVectorHighlighter but the accuracy trade-offs were not acceptable
for our users
§ Ran into performance problems as highlighting was taking the bulk of
our execution time
01

15. 15
Make it Faster
§ Improvements to the Standard Highlighter
§ Fast uninverting of term vectors to a token stream – LUCENE-6031
v5.0 (remove expensive sort)
§ Rely on Term Vectors for phrase highlighting instead of the analyzing
into the MemoryIndex – LUCENE-6034 v5.0
Still not fast enough…
01

16. 16
Multithreaded Highlighting
§ Highlighting each returned doc is easily parallelizable
§ Greatly improved performance
§ Greatly increased memory consumption
Still a sizeable fraction of our query times…
01

17. 17
Re-evaluating
§ Didn’t look like we could make the Standard Highlighter much faster
§ Perhaps we could federate to one of the highlighters based on the
query?
§ Our customizations would have to be ported to each of the
highlighters
§ Work would need to be repeated 3x
§ Increased disk utilization from adding postings to the main index
01

18. 18
Enhance the Postings Highlighter?
§ Fastest of the bunch
§ Add accuracy at least as good as the standard highlighter
§ Add support for the other offset sources too
§ (supports our full-doc-highlighting use-case)
§ But it’s a big job with major internal highlighter surgery…
Let’s do it!
01

19. Offsets Overview
§ Getting character offsets is key to highlighting. 3 ways:
§ Analysis:
§ Analyzer è TokenStream è OffsetAttribute,
oa.startOffset()
§ Term vectors:
§ IndexReader.getTermVector(docId,field) è Terms è
TermsEnum, te.postings(…, PostingsEnum.OFFSETS) è
PostingsEnum è pe.startOffset()
§ Postings:
§ IndexReader è LeafReader è Terms è … (see above)

20. PostingsHighlighter Algorithm
1. Fetches all stored-value content needed up-front
2. Highlight in field sorted order, then doc sorted order loop:
1. Get PostingsEnum from a Terms for each query term
2. MTQs: Fake PostingsEnum around filtered TokenStream
3. Process PostingsEnum[ ] into Passage[ ]
java.text.BreakIterator: for passage delineation
PassageScorer: for passage scoring (BM25 default)
4. PassageFormatter: for formatting/mark-up

21. UnifiedHighlighter
§ Forked PH; given new name agnostic of offset source
§ Mostly same PH API; internals re-arranged and expanded
§ Solr adapter is nearly identical too
§ Untouched: Passage, PassageScorer, PassageFormatter
§ Re-uses some standard-Highlighter code too:
§ WeightedSpanTermExtractor (for phrase accuracy)
§ TokenStreamFromTermVector (for wildcards/MTQs)

22. UH: Accurate Phrases
(including any SpanQuery)
§ Convert position-sensitive Queries to SpanQueries
§ Re-use WeightedSpanTermExtractor (WSTE) for this
§ Wrap PostingsEnum for position-sensitive words with one that filters
by position-span extracted from span queries
§ Custom: WSTE is not used for this, although it’s similar
§ Note: not 100% accurate with query but very good

23. UH: Analysis Offset Source
§ The most difficult offset source…
§ Honor positionIncrementGap for multi-valued data
§ Populates a MemoryIndex when query has phrases
§ But smartly filters irrelevant terms! (new trick)
§ Wildcards/MTQs too? Uninvert MemoryIndex with re-used
TokenStreamFromTermVector
§ If just terms, treat them like wildcards to avoid MemoryIndex usage

24. UH: Postings Plus Light TVs
§ Postings offset source is great, but not for MTQs (wildcards)
§ MTQs need to see all terms in just the document
§ A plain term vector (no offsets or postings) has that!
§ Trick:
§ Wrap the main index with a term vector’s TermsEnum
§ Then TokenStreamFromTermVector for MTQ

25. 25
Benchmark Results
01
§ Unified Highlighter performed
similarly or better than peers
§ Best performance: Postings
with “light” Term Vectors
§ No use case for full term
vectors anymore?
§ Caveats
§ Substantial variability in test
runs (YMMV)
§ Depends on the specifics of
your use case
§ Benchmark code available
Highlighter Offset Source Terms Phrases Wildcards
(search) N/A 1.0x! 1.0x! 1.0x
Standard Highlighter Analysis 4.6x! 4.7x! 7.4x
Unified Highlighter Analysis 2.8x! 2.4x! 3.7x
Standard Highlighter Term Vectors 2.7x! 2.3x! 3.7x
Fast Vector Highlighter Term Vectors 1.8x! 2.1x! 2.6x
Unified Highlighter Term Vectors 1.7x! 1.8x! 2.3x
Postings Highlighter Postings 1.8x! 1.5x! 3.8x
Unified Highlighter Postings 1.6x! 1.3x! 3.8x
Unified Highlighter Postings with Term Vectors* 1.5x! 1.3x! 2.2x
Times shown in multiples of the original
search time (top row).

26. 26
Future Potential Improvements
§ Accuracy
§ Switch from WSTE approach to SpanCollector API
§ Honor conjunctions “(X AND Y) OR Z”
§ Relevancy
§ Consider term diversity across top-X passages
§ Incorporate query boosts in passage scores
§ Support “requireFieldMatch=false”
01

27. Summary
§ Importance of highlighting in Legal Search
§ Overview of the existing Highlighters
§ Improvements to the Standard Highlighter
§ UnifiedHighlighter
§ Contributed to Lucene/Solr! LUCENE-7438
§ Your new favorite highlighter?

28. 28
Questions?
01
Timothy M. Rodriguez
Verticals Search Team Lead, Bloomberg
@Timothy055
David Smiley
Search Developer/Consultant
@DavidWSmiley