Simple Fuzzy Name Matching in Solr: Presented by Chris Mack, Basis Technology
•
1 like•1,857 views
This document describes a custom Solr plugin for fuzzy name matching. The plugin handles challenges like name variations and ambiguity. It creates a custom field type that scores name matches and supports multiple fields and values per document. At query time, it generates a custom Lucene query to find candidates, then uses Solr's rerank feature to rescore the top results based on the name matching algorithm. The plugin is configurable to trade off accuracy versus speed and supports multi-lingual name matching.
Report
Share
Report
Share
Download to read offline
Recommended
Streaming Aggregation in Solr - New Horizons for Search: Presented by Erick E...
Erick Erickson presented on Streaming Aggregation in Solr. SA allows processing very large result sets across Solr nodes in parallel. It enables SQL-like queries and arbitrary operations on result sets. SA uses docValues fields and tuples to export data for processing. Parallel SQL and Streaming Expressions provide additional interfaces to SA beyond Java code. SA complements Solr's search capabilities by enabling analytics on large result sets.
Data Engineering with Solr and Spark
Grant Ingersoll presented on using Apache Solr and Apache Spark for data engineering. He discussed how Solr can be used for indexing and searching large amounts of data, while Spark enables large-scale processing on the indexed data. Lucidworks' Fusion product combines Solr and Spark capabilities to allow search-driven applications and machine learning on indexed content.
Data Science with Solr and Spark
This document provides an overview of a data science conference where the keynote speaker will discuss using Apache Solr and Apache Spark together for data science applications. The speaker is the CTO of Lucidworks and will cover getting started with Solr and Spark, demoing how to index data, run analytics like clustering and classification, and more. Resources for learning more about Solr, Spark, and Lucidworks Fusion are also provided.
Webinar: What's New in Solr 6
Solr committer Cassandra Targett shares a preview of features and functionality included in the upcoming release of Apache Solr.
Your Big Data Stack is Too Big!: Presented by Timothy Potter, Lucidworks
Timothy Potter presented at a Big Data conference in Boston from October 11-14, 2016. He discussed how Lucidworks Fusion provides an alternative to traditional big data stacks that emphasizes fast access, agility and automation over integration. Fusion allows for common access patterns like fast lookups, ranked retrieval and distributed scans while integrating technologies like Solr, Spark, HDFS and more. It provides tools for data ingestion, time-based partitioning, analytics, machine learning and more to solve business problems rather than focus on infrastructure.
Efficient Scalable Search in a Multi-Tenant Environment: Presented by Harry H...
This document discusses efficient scalable search in a multi-tenant environment. It describes Bloomberg Vault, which hosts large volumes of enterprise communications and documents for compliance. The system uses a distributed architecture with shards that are loaded on demand to serve search queries. Security is ensured by dynamically generating field values that encapsulate access permissions for each user's view of a document.
Rackspace: Email's Solution for Indexing 50K Documents per Second: Presented ...
George Bailey and Cameron Baker of Rackspace presented their solution for indexing over 50,000 documents per second for Rackspace Email. They modernized their system using Apache Flume for event processing and aggregation and SolrCloud for real-time search. This reduced indexing time from over 20 minutes to under 5 seconds, reduced the number of physical servers needed from over 100 to 14, and increased indexing throughput from 1,000 to over 50,000 documents per second while supporting over 13 billion searchable documents.
Learning to Rank in Solr: Presented by Michael Nilsson & Diego Ceccarelli, Bl...
This document summarizes Bloomberg's use of machine learning for search ranking within their Solr implementation. It discusses how they process 8 million searches per day and need machine learning to automatically tune rankings over time as their index grows to 400 million documents. They use a Learning to Rank approach where features are extracted from queries and documents, training data is collected, and a ranking model is generated to optimize metrics like click-through rates. Their Solr Learning to Rank plugin allows this model to re-rank search results in Solr for improved relevance.
Recommended
Streaming Aggregation in Solr - New Horizons for Search: Presented by Erick E...
Erick Erickson presented on Streaming Aggregation in Solr. SA allows processing very large result sets across Solr nodes in parallel. It enables SQL-like queries and arbitrary operations on result sets. SA uses docValues fields and tuples to export data for processing. Parallel SQL and Streaming Expressions provide additional interfaces to SA beyond Java code. SA complements Solr's search capabilities by enabling analytics on large result sets.
Data Engineering with Solr and Spark
Grant Ingersoll presented on using Apache Solr and Apache Spark for data engineering. He discussed how Solr can be used for indexing and searching large amounts of data, while Spark enables large-scale processing on the indexed data. Lucidworks' Fusion product combines Solr and Spark capabilities to allow search-driven applications and machine learning on indexed content.
Data Science with Solr and Spark
This document provides an overview of a data science conference where the keynote speaker will discuss using Apache Solr and Apache Spark together for data science applications. The speaker is the CTO of Lucidworks and will cover getting started with Solr and Spark, demoing how to index data, run analytics like clustering and classification, and more. Resources for learning more about Solr, Spark, and Lucidworks Fusion are also provided.
Webinar: What's New in Solr 6
Solr committer Cassandra Targett shares a preview of features and functionality included in the upcoming release of Apache Solr.
Your Big Data Stack is Too Big!: Presented by Timothy Potter, Lucidworks
Timothy Potter presented at a Big Data conference in Boston from October 11-14, 2016. He discussed how Lucidworks Fusion provides an alternative to traditional big data stacks that emphasizes fast access, agility and automation over integration. Fusion allows for common access patterns like fast lookups, ranked retrieval and distributed scans while integrating technologies like Solr, Spark, HDFS and more. It provides tools for data ingestion, time-based partitioning, analytics, machine learning and more to solve business problems rather than focus on infrastructure.
Efficient Scalable Search in a Multi-Tenant Environment: Presented by Harry H...
This document discusses efficient scalable search in a multi-tenant environment. It describes Bloomberg Vault, which hosts large volumes of enterprise communications and documents for compliance. The system uses a distributed architecture with shards that are loaded on demand to serve search queries. Security is ensured by dynamically generating field values that encapsulate access permissions for each user's view of a document.
Rackspace: Email's Solution for Indexing 50K Documents per Second: Presented ...
George Bailey and Cameron Baker of Rackspace presented their solution for indexing over 50,000 documents per second for Rackspace Email. They modernized their system using Apache Flume for event processing and aggregation and SolrCloud for real-time search. This reduced indexing time from over 20 minutes to under 5 seconds, reduced the number of physical servers needed from over 100 to 14, and increased indexing throughput from 1,000 to over 50,000 documents per second while supporting over 13 billion searchable documents.
Learning to Rank in Solr: Presented by Michael Nilsson & Diego Ceccarelli, Bl...
This document summarizes Bloomberg's use of machine learning for search ranking within their Solr implementation. It discusses how they process 8 million searches per day and need machine learning to automatically tune rankings over time as their index grows to 400 million documents. They use a Learning to Rank approach where features are extracted from queries and documents, training data is collected, and a ranking model is generated to optimize metrics like click-through rates. Their Solr Learning to Rank plugin allows this model to re-rank search results in Solr for improved relevance.
Parallel SQL and Streaming Expressions in Apache Solr 6
Apache Solr is a powerful search and analytics engine with features such as full-text search, faceting, joins, sorting and capable of handling large amounts of data across a large number of servers. However, with all that power and scalability comes complexity. Solr 6 supports a Parallel SQL feature which provides a simplified, well-known interface to your data in Solr, performs key operations such as sorts and shuffling inside Solr for massive speedups, provides best-practices based query optimization and by leveraging the scalability of SolrCloud and a clever implementation, allows you to throw massive amounts of computation power behind analytical queries.
In this talk, we will explore the why, what and how of Parallel SQL and its building block Streaming Expressions in Solr 6 with a hint of the exciting new developments around this feature.
Building a Large Scale SEO/SEM Application with Apache Solr: Presented by Rah...
The document discusses building a large scale SEO/SEM application using Apache Solr. It describes some of the key challenges faced in indexing and searching over 40 billion records in the application's database each month. It discusses techniques used to optimize the data import process, create a distributed index across multiple tables, address out of memory errors, and improve search performance through partitioning, index optimization, and external caching.
Understanding Lucene Search Performance
This document provides an overview of a workshop on Lucene performance given by Lucid Imagination, Inc. It discusses common Lucene performance issues, introduces Lucid Gaze for Lucene (LG4L) as a tool for monitoring Lucene performance statistics and examples of using it to analyze indexing and search performance. LG4L provides statistics on indexing, analysis, searching and storage through logs, a persistent database and an API. It can help identify causes of poor performance and was shown to have low overhead.
Solr4 nosql search_server_2013
The document discusses Solr 4, an open source search platform built on Apache Lucene. Some key points:
- Solr 4 is a NoSQL search server that provides distributed indexing, fault tolerance, and real-time search capabilities.
- Solr Cloud is Solr's distributed architecture which uses Zookeeper for coordination to provide features like automatic sharding and replication of indexes across multiple servers.
- The document outlines Solr 4's capabilities including schema-less options, atomic updates, optimistic concurrency, and a REST API for managing the schema dynamically.
Integrating Spark and Solr-(Timothy Potter, Lucidworks)
This document discusses integrating Solr and Spark. It provides an example of using Solr as a sink for streaming data from Spark Streaming. It also describes reading data from Solr into Spark using SolrRDD and exposing it as a Spark SQL DataFrame. Additional capabilities covered include querying Solr from the Spark shell, document matching using stored queries, and reading term vectors from Solr for machine learning with MLLib.
ApacheCon NA 2015 Spark / Solr Integration
Apache Solr has been adopted by all major Hadoop platform vendors because of its ability to scale horizontally to meet even the most demanding big data search problems. Apache Spark has emerged as the leading platform for real-time big data analytics and machine learning. In this presentation, Timothy Potter presents several common use cases for integrating Solr and Spark.
Specifically, Tim covers how to populate Solr from a Spark streaming job as well as how to expose the results of any Solr query as an RDD. The Solr RDD makes efficient use of deep paging cursors and SolrCloud sharding to maximize parallel computation in Spark. After covering basic use cases, Tim digs a little deeper to show how to use MLLib to enrich documents before indexing in Solr, such as sentiment analysis (logistic regression), language detection, and topic modeling (LDA), and document classification.
H-Hypermap - Heatmap Analytics at Scale: Presented by David Smiley, D W Smile...
This document provides an agenda and overview for a presentation on H-Hypermap, a project to build a search platform called the Billion Object Platform (BOP) to index and search over billions of geo-tagged tweets in near real-time. The presentation will cover the architecture using Apache Kafka, Solr sharding, and techniques for fast geo-spatial queries and heatmaps. It will also discuss experiences using technologies like Kotlin, Dropwizard, Docker and Kontena.
Solr and Elasticsearch, a performance study
The document summarizes a performance comparison study conducted between Elasticsearch and SolrCloud. It found that SolrCloud was slightly faster at indexing and querying large datasets, and was able to support a significantly higher queries per second. However, the document notes limitations to the study and concludes that both Elasticsearch and SolrCloud showed acceptable performance, so the best option depends on the specific search application requirements.
Benchmarking Solr Performance at Scale
Organizations continue to adopt Solr because of its ability to scale to meet even the most demanding workflows. Recently, LucidWorks has been leading the effort to identify, measure, and expand the limits of Solr. As part of this effort, we've learned a few things along the way that should prove useful for any organization wanting to scale Solr. Attendees will come away with a better understanding of how sharding and replication impact performance. Also, no benchmark is useful without being repeatable; Tim will also cover how to perform similar tests using the Solr-Scale-Toolkit in Amazon EC2.
Lessons From Sharding Solr At Etsy: Presented by Gregg Donovan, Etsy
Gregg Donovan presented on lessons learned from sharding Solr at Etsy over three versions:
1) Initially, Etsy did not shard to avoid problems, but the single node approach did not scale.
2) The first sharding version used local sharding across multiple JVMs per host for better latency and manageability.
3) The current version uses distributed sharding across data centers for further latency gains, but this introduced challenges of partial failures, synchronization, and distributed queries.
TweetMogaz - The Arabic Tweets Platform: Presented by Ahmed Adel, BADR
The document summarizes TweetMogaz, an Arabic tweets platform developed by BADR. It describes the key modules of the system including tweets processing, indexing, event detection, archiving and analytics. The system collects and analyzes Arabic tweets in real-time using Apache Solr, identifies trending topics and events, and allows users to browse, search and visualize tweets and analytics. It addresses challenges of analyzing micro-blogs and Arabic language variations. Future work includes improving the adaptive classifier and integrating statistical processing with R.
Webinar: Fusion 2.3 Preview - Enhanced Features with Solr & Spark
Lucidworks CTO and Lucene/Solr committer Grant Ingersoll shows a preview of the new features coming up in Fusion 2.3.
Solr Indexing and Analysis Tricks
This session will introduce and demonstrate several techniques for enhancing the search experience by augmenting documents during indexing. First we'll survey the analysis components available in Solr, and then we'll delve into using Solr's update processing pipeline to modify documents on the way in. The session will build on Erik's "Poor Man's Entity Extraction" blog at http://www.searchhub.org/2013/06/27/poor-mans-entity-extraction-with-solr/
Webinar: Solr 6 Deep Dive - SQL and Graph
This document provides an agenda and overview for a conference session on Solr 6 and its new capabilities for parallel SQL and graph queries. The session will cover motivations for adding these features to Solr, how streaming expressions enable parallel SQL, graph capabilities through the new graph query parser and streaming expressions, and comparisons to other technologies. The document includes examples of SQL queries and graph streaming expressions in Solr.
Downtown SF Lucene/Solr Meetup: Developing Scalable User Search for PlayStati...
The document discusses developing a scalable user search feature for the PlayStation 4. It describes setting up a SolrCloud cluster with 300 million user documents distributed across 4 shards. Personalized search ranks results based on friendship connections by using a Lucene index to store close connections for each user. Challenges included instability in the initial Solr 4.8 cluster which was addressed through configuration changes. An upgrade to Solr 5.4 required fully reindexing the data due to schema changes.
Webinar: What's New in Solr 7
The next major release of Solr is right around the corner! Join Solr Committer Cassandra Targett and Lucidworks SVP of Engineering Trey Grainger for a first look into what’s included in the upcoming release.
Loading 350M documents into a large Solr cluster: Presented by Dion Olsthoorn...
This document summarizes a presentation about loading 350 million documents into a Solr cluster in 8 hours or less. It describes using an external cloud platform for preprocessing content before indexing into Solr. It also details using a queueing system to post content to Solr in batches to avoid overloading the cluster. The presentation recommends indexing large content sets on an isolated Solr environment before restoring indexes to the production cluster.
10 Keys to Solr's Future: Presented by Grant Ingersoll, Lucidworks
This document outlines 10 keys to the future of Solr, an open source search platform. It discusses improving ease of use, modularity, pluggability, APIs, scale, and being more open for development. It also announces new features for Lucidworks Fusion 1.1, including additional connectors for sources like Google Drive and Couchbase. The document promotes using Solr for a variety of use cases and integrating it with other technologies for big data, distributed computing, and security.
Real Time search using Spark and Elasticsearch
This document discusses using Spark Streaming and Elasticsearch to enable real-time search and analysis of streaming data. Spark Streaming processes and enriches streaming data and stores it in Elasticsearch for low-latency search and alerts. The elasticsearch-hadoop connector allows Spark jobs to read from and write to Elasticsearch, integrating the batch processing of Spark with the real-time search of Elasticsearch.
Solr and Spark for Real-Time Big Data Analytics: Presented by Tim Potter, Luc...
This document provides an overview of using Apache Spark with Apache Solr. It discusses using Solr as a data source for Spark SQL, reading data from Solr into Spark RDDs, querying Solr from the Spark shell, indexing data from Spark Streaming into Solr, and an example of using Solr as a sink for a Spark Streaming application that processes tweets in real-time.
Solr Indexing and Analysis Tricks
This document summarizes Erik Hatcher's presentation on using Solr's indexing and analysis capabilities to enhance search experiences. It discusses using analyzers to modify documents during indexing, such as extracting acronyms, key phrases, links and locations to add as separate searchable fields without storing the extracted content. It also provides examples of using copyField and script update processors to extract fields during indexing for faceting, searching and boosting search results.
Simple fuzzy name matching in elasticsearch paris meetup
Those are the slides that were presented during the Elasticsearch meetup in Paris on July 29th.
Normalization is crucial to high quality search results -- who wants irrelevant variations between queries and documents leading to missed hits (e.g., “celebrity” v. “celebrities”)? Normalizing dictionary words works, but what if your application focuses on names? Whether you’re tackling log analysis, e-commerce, watch list screening or other applications, names are often the key. Can you find “Abdul Jabbar, Karim” if you search for “Kareem AbdalJabar” or “كريم عبد الجبار”?
Applications using Elasticsearch provide some fuzziness by mixing its built-in edit-distance matching and phonetic analysis with more generic analyzers and filters. We’ve tried to go beyond that to provide both better matching and a simpler integration. We use a custom Mapper and Score Function so that linguistic nuances can be handled behind-the-scenes. We’ll talk about how we built this sort of plug-in for Rosette, its customization, and its connection to broader trend of entity-centric search.
More Related Content
What's hot
Parallel SQL and Streaming Expressions in Apache Solr 6
Apache Solr is a powerful search and analytics engine with features such as full-text search, faceting, joins, sorting and capable of handling large amounts of data across a large number of servers. However, with all that power and scalability comes complexity. Solr 6 supports a Parallel SQL feature which provides a simplified, well-known interface to your data in Solr, performs key operations such as sorts and shuffling inside Solr for massive speedups, provides best-practices based query optimization and by leveraging the scalability of SolrCloud and a clever implementation, allows you to throw massive amounts of computation power behind analytical queries.
In this talk, we will explore the why, what and how of Parallel SQL and its building block Streaming Expressions in Solr 6 with a hint of the exciting new developments around this feature.
Building a Large Scale SEO/SEM Application with Apache Solr: Presented by Rah...
The document discusses building a large scale SEO/SEM application using Apache Solr. It describes some of the key challenges faced in indexing and searching over 40 billion records in the application's database each month. It discusses techniques used to optimize the data import process, create a distributed index across multiple tables, address out of memory errors, and improve search performance through partitioning, index optimization, and external caching.
Understanding Lucene Search Performance
This document provides an overview of a workshop on Lucene performance given by Lucid Imagination, Inc. It discusses common Lucene performance issues, introduces Lucid Gaze for Lucene (LG4L) as a tool for monitoring Lucene performance statistics and examples of using it to analyze indexing and search performance. LG4L provides statistics on indexing, analysis, searching and storage through logs, a persistent database and an API. It can help identify causes of poor performance and was shown to have low overhead.
Solr4 nosql search_server_2013
The document discusses Solr 4, an open source search platform built on Apache Lucene. Some key points:
- Solr 4 is a NoSQL search server that provides distributed indexing, fault tolerance, and real-time search capabilities.
- Solr Cloud is Solr's distributed architecture which uses Zookeeper for coordination to provide features like automatic sharding and replication of indexes across multiple servers.
- The document outlines Solr 4's capabilities including schema-less options, atomic updates, optimistic concurrency, and a REST API for managing the schema dynamically.
Integrating Spark and Solr-(Timothy Potter, Lucidworks)
This document discusses integrating Solr and Spark. It provides an example of using Solr as a sink for streaming data from Spark Streaming. It also describes reading data from Solr into Spark using SolrRDD and exposing it as a Spark SQL DataFrame. Additional capabilities covered include querying Solr from the Spark shell, document matching using stored queries, and reading term vectors from Solr for machine learning with MLLib.
ApacheCon NA 2015 Spark / Solr Integration
Apache Solr has been adopted by all major Hadoop platform vendors because of its ability to scale horizontally to meet even the most demanding big data search problems. Apache Spark has emerged as the leading platform for real-time big data analytics and machine learning. In this presentation, Timothy Potter presents several common use cases for integrating Solr and Spark.
Specifically, Tim covers how to populate Solr from a Spark streaming job as well as how to expose the results of any Solr query as an RDD. The Solr RDD makes efficient use of deep paging cursors and SolrCloud sharding to maximize parallel computation in Spark. After covering basic use cases, Tim digs a little deeper to show how to use MLLib to enrich documents before indexing in Solr, such as sentiment analysis (logistic regression), language detection, and topic modeling (LDA), and document classification.
H-Hypermap - Heatmap Analytics at Scale: Presented by David Smiley, D W Smile...
This document provides an agenda and overview for a presentation on H-Hypermap, a project to build a search platform called the Billion Object Platform (BOP) to index and search over billions of geo-tagged tweets in near real-time. The presentation will cover the architecture using Apache Kafka, Solr sharding, and techniques for fast geo-spatial queries and heatmaps. It will also discuss experiences using technologies like Kotlin, Dropwizard, Docker and Kontena.
Solr and Elasticsearch, a performance study
The document summarizes a performance comparison study conducted between Elasticsearch and SolrCloud. It found that SolrCloud was slightly faster at indexing and querying large datasets, and was able to support a significantly higher queries per second. However, the document notes limitations to the study and concludes that both Elasticsearch and SolrCloud showed acceptable performance, so the best option depends on the specific search application requirements.
Benchmarking Solr Performance at Scale
Organizations continue to adopt Solr because of its ability to scale to meet even the most demanding workflows. Recently, LucidWorks has been leading the effort to identify, measure, and expand the limits of Solr. As part of this effort, we've learned a few things along the way that should prove useful for any organization wanting to scale Solr. Attendees will come away with a better understanding of how sharding and replication impact performance. Also, no benchmark is useful without being repeatable; Tim will also cover how to perform similar tests using the Solr-Scale-Toolkit in Amazon EC2.
Lessons From Sharding Solr At Etsy: Presented by Gregg Donovan, Etsy
Gregg Donovan presented on lessons learned from sharding Solr at Etsy over three versions:
1) Initially, Etsy did not shard to avoid problems, but the single node approach did not scale.
2) The first sharding version used local sharding across multiple JVMs per host for better latency and manageability.
3) The current version uses distributed sharding across data centers for further latency gains, but this introduced challenges of partial failures, synchronization, and distributed queries.
TweetMogaz - The Arabic Tweets Platform: Presented by Ahmed Adel, BADR
The document summarizes TweetMogaz, an Arabic tweets platform developed by BADR. It describes the key modules of the system including tweets processing, indexing, event detection, archiving and analytics. The system collects and analyzes Arabic tweets in real-time using Apache Solr, identifies trending topics and events, and allows users to browse, search and visualize tweets and analytics. It addresses challenges of analyzing micro-blogs and Arabic language variations. Future work includes improving the adaptive classifier and integrating statistical processing with R.
Webinar: Fusion 2.3 Preview - Enhanced Features with Solr & Spark
Lucidworks CTO and Lucene/Solr committer Grant Ingersoll shows a preview of the new features coming up in Fusion 2.3.
Solr Indexing and Analysis Tricks
This session will introduce and demonstrate several techniques for enhancing the search experience by augmenting documents during indexing. First we'll survey the analysis components available in Solr, and then we'll delve into using Solr's update processing pipeline to modify documents on the way in. The session will build on Erik's "Poor Man's Entity Extraction" blog at http://www.searchhub.org/2013/06/27/poor-mans-entity-extraction-with-solr/
Webinar: Solr 6 Deep Dive - SQL and Graph
This document provides an agenda and overview for a conference session on Solr 6 and its new capabilities for parallel SQL and graph queries. The session will cover motivations for adding these features to Solr, how streaming expressions enable parallel SQL, graph capabilities through the new graph query parser and streaming expressions, and comparisons to other technologies. The document includes examples of SQL queries and graph streaming expressions in Solr.
Downtown SF Lucene/Solr Meetup: Developing Scalable User Search for PlayStati...
The document discusses developing a scalable user search feature for the PlayStation 4. It describes setting up a SolrCloud cluster with 300 million user documents distributed across 4 shards. Personalized search ranks results based on friendship connections by using a Lucene index to store close connections for each user. Challenges included instability in the initial Solr 4.8 cluster which was addressed through configuration changes. An upgrade to Solr 5.4 required fully reindexing the data due to schema changes.
Webinar: What's New in Solr 7
The next major release of Solr is right around the corner! Join Solr Committer Cassandra Targett and Lucidworks SVP of Engineering Trey Grainger for a first look into what’s included in the upcoming release.
Loading 350M documents into a large Solr cluster: Presented by Dion Olsthoorn...
This document summarizes a presentation about loading 350 million documents into a Solr cluster in 8 hours or less. It describes using an external cloud platform for preprocessing content before indexing into Solr. It also details using a queueing system to post content to Solr in batches to avoid overloading the cluster. The presentation recommends indexing large content sets on an isolated Solr environment before restoring indexes to the production cluster.
10 Keys to Solr's Future: Presented by Grant Ingersoll, Lucidworks
This document outlines 10 keys to the future of Solr, an open source search platform. It discusses improving ease of use, modularity, pluggability, APIs, scale, and being more open for development. It also announces new features for Lucidworks Fusion 1.1, including additional connectors for sources like Google Drive and Couchbase. The document promotes using Solr for a variety of use cases and integrating it with other technologies for big data, distributed computing, and security.
Real Time search using Spark and Elasticsearch
This document discusses using Spark Streaming and Elasticsearch to enable real-time search and analysis of streaming data. Spark Streaming processes and enriches streaming data and stores it in Elasticsearch for low-latency search and alerts. The elasticsearch-hadoop connector allows Spark jobs to read from and write to Elasticsearch, integrating the batch processing of Spark with the real-time search of Elasticsearch.
Solr and Spark for Real-Time Big Data Analytics: Presented by Tim Potter, Luc...
This document provides an overview of using Apache Spark with Apache Solr. It discusses using Solr as a data source for Spark SQL, reading data from Solr into Spark RDDs, querying Solr from the Spark shell, indexing data from Spark Streaming into Solr, and an example of using Solr as a sink for a Spark Streaming application that processes tweets in real-time.
What's hot (20)
Parallel SQL and Streaming Expressions in Apache Solr 6
Parallel SQL and Streaming Expressions in Apache Solr 6
Building a Large Scale SEO/SEM Application with Apache Solr: Presented by Rah...
Building a Large Scale SEO/SEM Application with Apache Solr: Presented by Rah...
Integrating Spark and Solr-(Timothy Potter, Lucidworks)
Integrating Spark and Solr-(Timothy Potter, Lucidworks)
H-Hypermap - Heatmap Analytics at Scale: Presented by David Smiley, D W Smile...
H-Hypermap - Heatmap Analytics at Scale: Presented by David Smiley, D W Smile...
Lessons From Sharding Solr At Etsy: Presented by Gregg Donovan, Etsy
Lessons From Sharding Solr At Etsy: Presented by Gregg Donovan, Etsy
TweetMogaz - The Arabic Tweets Platform: Presented by Ahmed Adel, BADR
TweetMogaz - The Arabic Tweets Platform: Presented by Ahmed Adel, BADR
Webinar: Fusion 2.3 Preview - Enhanced Features with Solr & Spark
Webinar: Fusion 2.3 Preview - Enhanced Features with Solr & Spark
Downtown SF Lucene/Solr Meetup: Developing Scalable User Search for PlayStati...
Downtown SF Lucene/Solr Meetup: Developing Scalable User Search for PlayStati...
Loading 350M documents into a large Solr cluster: Presented by Dion Olsthoorn...
Loading 350M documents into a large Solr cluster: Presented by Dion Olsthoorn...
10 Keys to Solr's Future: Presented by Grant Ingersoll, Lucidworks
10 Keys to Solr's Future: Presented by Grant Ingersoll, Lucidworks
Solr and Spark for Real-Time Big Data Analytics: Presented by Tim Potter, Luc...
Solr and Spark for Real-Time Big Data Analytics: Presented by Tim Potter, Luc...
Viewers also liked
Solr Indexing and Analysis Tricks
This document summarizes Erik Hatcher's presentation on using Solr's indexing and analysis capabilities to enhance search experiences. It discusses using analyzers to modify documents during indexing, such as extracting acronyms, key phrases, links and locations to add as separate searchable fields without storing the extracted content. It also provides examples of using copyField and script update processors to extract fields during indexing for faceting, searching and boosting search results.
Simple fuzzy name matching in elasticsearch paris meetup
Those are the slides that were presented during the Elasticsearch meetup in Paris on July 29th.
Normalization is crucial to high quality search results -- who wants irrelevant variations between queries and documents leading to missed hits (e.g., “celebrity” v. “celebrities”)? Normalizing dictionary words works, but what if your application focuses on names? Whether you’re tackling log analysis, e-commerce, watch list screening or other applications, names are often the key. Can you find “Abdul Jabbar, Karim” if you search for “Kareem AbdalJabar” or “كريم عبد الجبار”?
Applications using Elasticsearch provide some fuzziness by mixing its built-in edit-distance matching and phonetic analysis with more generic analyzers and filters. We’ve tried to go beyond that to provide both better matching and a simpler integration. We use a custom Mapper and Score Function so that linguistic nuances can be handled behind-the-scenes. We’ll talk about how we built this sort of plug-in for Rosette, its customization, and its connection to broader trend of entity-centric search.
Building Client-side Search Applications with Solr
Presented by Daniel Beach, Search Application Developer, OpenSource Connections
Solr is a powerful search engine, but creating a custom user interface can be daunting. In this fast paced session I will present an overview of how to implement a client-side search application using Solr. Using open-source frameworks like SpyGlass (to be released in September) can be a powerful way to jumpstart your development by giving you out-of-the box results views with support for faceting, autocomplete, and detail views. During this talk I will also demonstrate how we have built and deployed lightweight applications that are able to be performant under large user loads, with minimal server resources.
Background Screening: Name Matching
The document discusses name matching in background screening. It explains that name matching ensures complete and correct background screening results by maximizing the amount of information found. However, name matching faces three issues: incomplete applicant information, protected personally identifiable information, and derivatives of names. The document provides tips for background screeners, including collecting all name variations from applicants and searching each one. It outlines a ten step process for effective name matching.
Testing instrument
This document provides an overview of an early childhood testing instrument. It outlines key details about the instrument's purpose, type, developmental domain assessed, target age group, scoring procedures, how results are used, reliability, validity, cultural sensitivity, practicality, and costs. The purpose is to measure specific developmental domains for a targeted age group. It is a norm-referenced test that provides standardized scores to evaluate children's performance compared to peers.
Cassandra Summit 2014: Fuzzy Entity Matching at Scale
Presenter: Ken Krugler, President of Scale Unlimited
Early Warning has information on hundreds of millions of people and companies. When a person wants to open a new bank account, they need to be able to accurately find similar entities in this large dataset, to provide a risk assessment. Using the combination of Cassandra & Solr via DSE, they can quickly find and evaluate all reasonable candidates.
Screening ppt
Screening, testing, and interviewing are key steps in the selection process. Selection involves identifying applicants that best meet the criteria for open positions. A variety of tools are used including application screening, employment tests that measure abilities and skills, and structured interviews. The goal is to select candidates that have the highest chance of performing well in the job. Care must be taken to ensure selection methods are valid, reliable, and avoid potential biases.
Viewers also liked (7)
Simple fuzzy name matching in elasticsearch paris meetup
Simple fuzzy name matching in elasticsearch paris meetup
Building Client-side Search Applications with Solr
Building Client-side Search Applications with Solr
Cassandra Summit 2014: Fuzzy Entity Matching at Scale
Cassandra Summit 2014: Fuzzy Entity Matching at Scale
Similar to Simple Fuzzy Name Matching in Solr: Presented by Chris Mack, Basis Technology
Fuzzy Name Matching in Solr
We all know normalization is crucial to delivering high quality search results. We don’t want uninteresting variations between the query and the document to lead to missed hits (e.g., “celebrity” v. “celebrities”). Normalization of dictionary words is well understood, but what if your application focuses on names? Whether you’re tackling patent examination, sports records, e-commerce, watchlist screening or many other topics, names are often the key. Can your users find “Abdul Jabbar, Karim” if they search for “Kareem AbdalJabar” or “كريم عبد الجبار”? Solr application architects have attempted to address this through custom integration of nickname lists, edit distance, case normalization, phonetic encoding and n-grams, but doing so requires significant effort and may not address all desired variations. A simpler approach is to use a Solr field type for names that handles these linguistic nuances behind-the-scenes. We’ll talk about how we built this sort of field type via a Solr plug-in for the Rosette Name Indexer. We’ll also discuss examples of use cases this has enabled, how it can be tuned if necessary, and how it connects to the broader trend of entity-centric search.
Fuzzy Name Matching with Rosette
Normalization is crucial to high quality search results -- who wants irrelevant variations between queries and documents leading to missed hits (e.g., “celebrity” v. “celebrities”)? Normalizing dictionary words works, but what if your application focuses on names? Whether you’re tackling log analysis, e-commerce, watch list screening or other applications, names are often the key. Can you find “Abdul Jabbar, Karim” if you search for “Kareem AbdalJabar” or “كريم عبد الجبار”?
Applications using Elasticsearch provide some fuzziness by mixing its built-in edit-distance matching and phonetic analysis with more generic analyzers and filters. We’ve tried to go beyond that to provide both better matching and a simpler integration. We use a custom Mapper and Score Function so that linguistic nuances can be handled behind-the-scenes. We’ll talk about how we built this sort of plug-in for Rosette, its customization, and its connection to broader trend of entity-centric search.
Simple fuzzy name matching in solr
We all know normalization is crucial to delivering high quality search results. We don’t want uninteresting variations between the query and the document to lead to missed hits (e.g., “celebrity” v. “celebrities”). Normalization of dictionary words is well understood, but what if your application focuses on names? Whether you’re tackling patent examination, sports records, e-commerce, watchlist screening or many other topics, names are often the key. Can your users find “Abdul Jabbar, Karim” if they search for “Kareem AbdalJabar” or “كريم عبد الجبار”? Solr application architects have attempted to address this through custom integration of nickname lists, edit distance, case normalization, phonetic encoding and n-grams (see example #1 or example #2), but doing so requires significant effort and may not address all desired variations. A simpler approach is to use a Solr field type for names that handles these linguistic nuances behind-the-scenes. We’ll talk about how we built this sort of field type via a Solr plug-in for the Rosette Name Indexer. We’ll also discuss examples of use cases this has enabled, how it can be tuned if necessary, and how it connects to the broader trend of entity-centric search.
Building a real time big data analytics platform with solr
Having “big data” is great, but turning that data into actionable intelligence is where the real value lies. This talk will demonstrate how you can use Solr to build a highly scalable data analytics engine to enable customers to engage in lightning fast, real-time knowledge discovery.
At CareerBuilder, we utilize these techniques to report the supply and demand of the labor force, compensation trends, customer performance metrics, and many live internal platform analytics. You will walk away from this talk with an advanced understanding of faceting, including pivot-faceting, geo/radius faceting, time-series faceting, function faceting, and multi-select faceting. You’ll also get a sneak peak at some new faceting capabilities just wrapping up development including distributed pivot facets and percentile/stats faceting, which will be open-sourced.
The presentation will be a technical tutorial, along with real-world use-cases and data visualizations. After this talk, you'll never see Solr as just a text search engine again.
Building a real time, big data analytics platform with solr
Trey Grainger discussed how CareerBuilder is using Apache Solr as a real-time big data analytics platform. Some key points:
1) CareerBuilder indexes over 1/2 billion documents including jobs and resumes to power search and analytics.
2) Solr allows ad-hoc querying, fast faceted searches, and aggregate calculations to explore billions of documents.
3) CareerBuilder is contributing enhancements like distributed pivot faceting and stats within facets.
4) Examples shown include workforce supply/demand analysis, customer analytics, A/B testing, and event stream analysis.
New-Age Search through Apache Solr
SolrTM is the popular, blazing fast open Source Enterprise search platform from the Apache LuceneTM project. Its major features include powerful full-text search, hit highlighting, faceted search, near real-time indexing, dynamic clustering, database integration, rich document (e.g., Word, PDF) handling, and geospatial search. Solr is highly reliable, scalable and fault tolerant, providing distributed indexing, replication and load-balanced querying, automated failover and recovery, centralized configuration and more. Solr powers the search and navigation features of many of the world's largest internet sites like (Aol, Yahoo, Buy.com, Cnet, CitySearch, Netflix, Zappos, Stubhub!, digg, eTrade, Disney, Apple, NASA and MTV).
Apache Solr 1.4 – Faster, Easier, and More Versatile than Ever
In this On-Demand Webinar, Erik Hatcher, co-founder of Lucid Imagination, co-author of Lucene in Action, and Lucene/Solr PMC member and committer, presents and discusess key features and innovations of Apache Solr 1.4
New-Age Search through Apache Solr
Solr is the popular, blazing fast open Source Enterprise search platform from the Apache LuceneTM project. Its major features include powerful full-text search, hit highlighting, faceted search, near real-time indexing, dynamic clustering, database integration, rich document (e.g., Word, PDF) handling, and geospatial search. Solr is highly reliable, scalable and fault tolerant, providing distributed indexing, replication and load-balanced querying, automated failover and recovery, centralized configuration and more. Solr powers the search and navigation features of many of the world's largest internet sites like (Aol, Yahoo, Buy.com, Cnet, CitySearch, Netflix, Zappos, Stubhub!, digg, eTrade, Disney, Apple, NASA and MTV).
Advanced Relevancy Ranking
Lucene and Solr provide a number of options for query parsing, and these are valuable tools for creating powerful search applications. This presentation given at the 2013 Lucene Revolution will review the role that advanced query parsing can play in building systems, including: Relevancy customization, taking input from user interface variables such as the position on a website or geographical indicators, which sources are to be searched and 3rd party data sources. Query parsing can also enhance data security. Best practices for building and maintaining complex query parsing rules will be discussed and illustrated. Chief Architect Paul Nelson provides this compelling presentation.
Search Technologies provides relevancy tuning services for Solr. For further information, see http://www.searchtechnologies.com/solr-lucene-relevancy.html
http://www.searchtechnologies.com
Advanced query parsing techniques
This document introduces the Query Processing Language (QPL) developed by Search Technologies to provide a more flexible and powerful way to construct search queries in Solr. Some key points:
- QPL is a domain-specific language built on Groovy that allows constructing complex queries through scripting rather than string manipulation.
- It includes Solr plugins that integrate QPL scripts into query parsing and search components to build queries programmatically and access Solr functionality.
- Examples show how QPL can tokenize queries, apply a thesaurus expansion, implement different relevancy operators, and execute embedded searches to enhance queries.
- QPL aims to provide a "4GL for text search query expressions" and
IT talk SPb "Full text search for lazy guys"
by Alexander Tokarev, DB expert, DataArt St.Petersburg
and Alexander Polushkin, Configuration Manager, DataArt St.Petersburg
Advanced full text searching techniques using Lucene
Presentation on advanced searching techniques using Lucene ( A java based opensource full text search library )
Simple fuzzy Name Matching in Elasticsearch - Graham Morehead
The document describes how Elasticsearch can be used for name matching through custom analyzers, mappings, and rescoring queries. Key points extracted:
- Names are indexed using a custom NameMapper that generates keys for name parts using different analyzers and stores them in separate fields.
- Queries generate analogous keys to find candidate matches. A rescore query then scores how well the query name matches names in candidate documents to reorder results.
- The rescore query uses a custom name_score function that retrieves the indexed name from a document and scores it against the query name using a cached scorer, returning a similarity score. This allows computationally expensive but high precision name matching.
Solr 101
Solr 101 was a presentation about the Solr search platform. It introduced Solr, explaining that it is an open-source enterprise search platform built on Lucene. It covered key Solr concepts like indexing, documents, fields, queries and facets. The presentation also discussed Solr features, how it works, and how to scale Solr through techniques like multicore, replication and sharding. Finally, it provided two case studies on how Sparebank1 and Komplett implemented Solr to improve their search capabilities.
Rapid Prototyping with Solr
This document summarizes a presentation about rapid prototyping with Solr. It discusses getting documents indexed into Solr quickly, adjusting Solr's schema to better match needs, and showcasing data in a flexible search UI. It outlines how to leverage faceting, highlighting, spellchecking and debugging in rapid prototyping. Finally, it discusses next steps in developing a search application and taking it to production.
Apache Solr for begginers
The presentation describes what is Apache Solr, how it could be used. There is apache solr overview, performance tuning tips and advanced features description
Building Search & Recommendation Engines
In this talk, you'll learn how to build your own search and recommendation engine based on the open source Apache Lucene/Solr project. We'll dive into some of the data science behind how search engines work, covering multi-lingual text analysis, natural language processing, relevancy ranking algorithms, knowledge graphs, reflected intelligence, collaborative filtering, and other machine learning techniques used to drive relevant results for free-text queries. We'll also demonstrate how to build a recommendation engine leveraging the same platform and techniques that power search for most of the world's top companies. You'll walk away from this presentation with the toolbox you need to go and implement your very own search-based product using your own data.
Solr Search Engine: Optimize Is (Not) Bad for You
This talk was given during Lucene Revolution 2017.
They say optimize is bad for you, they say you shouldn't do it, they say it will invalidate operating system caches and make your system suffer. This is all true, but is it true in all cases?
In this presentation we will look closer on what optimize or better called force merge does to your Solr search engine. You will learn what segments are, how they are built and how they are used by Lucene and Solr for searching. We will discuss real-life performance implications regarding Solr collections that have many segments on a single node and compare that to the Solr where the number of segments is moderate and low. We will see what we can do to tune the merging process to trade off indexing performance for better query performance and what pitfalls are there waiting for us. Finally, at the end of the talk we will discuss possibilities of running force merge to avoid system disruption and still benefit from query performance boost that single segment index provides.
Hacking Lucene for Custom Search Results
Search is everywhere, and therefore so is Apache Lucene. While providing amazing out-of-the-box defaults, there’s enough projects weird enough to require custom search scoring and ranking. In this talk, I’ll walk through how to use Lucene to implement your custom scoring and search ranking. We’ll see how you can achieve both amazing power (and responsibility) over your search results. We’ll see the flexibility of Lucene’s data structures and explore the pros/cons of custom Lucene scoring vs other methods of improving search relevancy.
Lifecycle of a Solr Search Request - Chris "Hoss" Hostetter, Lucidworks
The document provides a deep dive into the lifecycle of a Solr search request, from the initial HTTP request to the generation of the response. It describes each stage of processing, including how the request is routed through the Solr core, how the query and filters are parsed and executed against the index, how various caches and plugins can be leveraged, and how the final response is generated. It uses examples of simple and more complex queries to demonstrate how each component interacts throughout the processing pipeline.
Similar to Simple Fuzzy Name Matching in Solr: Presented by Chris Mack, Basis Technology (20)
Building a real time big data analytics platform with solr
Building a real time big data analytics platform with solr
Building a real time, big data analytics platform with solr
Building a real time, big data analytics platform with solr
Apache Solr 1.4 – Faster, Easier, and More Versatile than Ever
Apache Solr 1.4 – Faster, Easier, and More Versatile than Ever
Advanced full text searching techniques using Lucene
Advanced full text searching techniques using Lucene
Simple fuzzy Name Matching in Elasticsearch - Graham Morehead
Simple fuzzy Name Matching in Elasticsearch - Graham Morehead
Lifecycle of a Solr Search Request - Chris "Hoss" Hostetter, Lucidworks
Lifecycle of a Solr Search Request - Chris "Hoss" Hostetter, Lucidworks
More from Lucidworks
Search is the Tip of the Spear for Your B2B eCommerce Strategy
With ecommerce experiencing explosive growth, it seems intuitive that the B2B segment of that ecosystem is mirroring the same trajectory. That said, B2B has very different needs when it comes to transacting with the same style of experiences that we see in B2C. For instance, B2B ecommerce is about precision findability, whereas B2C customers can convert at higher rates when they’re just browsing online. In order for the B2B buying experience to be successful, search needs to be tuned to meet the unique needs of the segment.
In this webinar with Forrester senior analyst Joe Cicman, you’ll learn:
-Which verticals in B2B will drive the most growth, and how machine-learning powered personalization tactics can be deployed to support those specific verticals
-Why an omnichannel selling approach must be deployed in order to see success in B2B
-How deploying content search capabilities will support a longer sales cycle at scale
-What the next steps are to support a robust B2B commerce strategy supported by new technology
Speakers
Joe Cicman, Senior Analyst, Forrester
Jenny Gomez, VP of Marketing, Lucidworks
Drive Agent Effectiveness in Salesforce
Customer loyalty starts with quickly responding to your customer’s needs. When it comes to resolving open support cases, time is of the essence. Time spent searching for answers adds up and creates inefficiencies in resolving cases at scale. Relevant answers need to be a few clicks away and easily accessible for agents directly from their service console.
We will explore how Lucidworks’ Agent Insights application automatically connects agents with the correct answers and resources. You’ll learn how to:
-Configure a proactive widget in an agent’s case view page to access resources across third-party systems (such as Sharepoint, Confluence, JIRA, Zendesk, and ServiceNow).
-Easily set up query pipelines to autonomously route assets and resources that are relevant to the case-at-hand—directly to the right agent.
-Identify subject matter experts within your support data and access tribal knowledge with lightning-fast speed.
How Crate & Barrel Connects Shoppers with Relevant Products
Lunch and Learn during Retail TouchPoints #RIC21 virtual event.
***
Crate & Barrel’s previous search solution couldn’t provide its shoppers with an online search and browse experience consistent with the customer-centric Crate & Barrel brand. Meanwhile, Crate & Barrel merchandisers spent the bulk of their time manually creating and maintaining search rules. The search experience impacted customer retention, loyalty, and revenue growth.
Join this lunch & learn for an interactive chat on how Crate & Barrel partnered with Lucidworks to:
-Improve search and browse by modernizing the technology stack with ML-based personalization and merchandising solutions
-Enhance the experience for both shoppers and merchandisers
-Explore signals to transform the omnichannel shopping experience
Questions? Visit https://lucidworks.com/contact/
Lucidworks & IMRG Webinar – Best-In-Class Retail Product Discovery
Learn how to guide customers to relevant products using eCommerce search, hyper-personalisation, and recommendations in our ‘Best-In-Class Retail Product Discovery’ webinar.
Nowadays, shoppers want their online experience to be engaging, inspirational and fulfilling. They want to find what they’re looking for quickly and easily. If the sought after item isn’t available, they want the next best product or content surfaced to them. They want a website to understand their goals as though they were talking to a sales assistant in person, in-store.
In this webinar, we explore IMRG industry data insights and a best-in-class example of retail product discovery. You’ll learn:
- How AI can drive increased revenue through hyper-personalised experiences
- How user intent can be easily understood and results displayed immediately
- How merchandisers can be empowered to curate results and product placement – all without having to rely on IT.
Presented by:
Dave Hawkins, Principal Sales Engineer - Lucidworks
Matthew Walsh, Director of Data & Retail - IMRG
Connected Experiences Are Personalized Experiences
Many companies claim personalization and omnichannel capabilities are top priorities. Few are able to deliver on those experiences.
For a recent Lucidworks-commissioned study, Forrester Consulting surveyed 350+ global business decision-makers to see what gets in the way of achieving these goals. They discovered that inefficient technology, lack of behavioral insights, and failure to tie initiatives to enterprise-wide goals are some of the most frequent blockers to personalization success.
Join guest speaker, Forrester VP and Principal Analyst, Brendan Witcher, and Lucidworks CEO, Will Hayes, to hear the results of the Forrester Consulting study, how to avoid “digital blindness,” and how to apply VoC data in real-time to delight customers with personalized experiences connected across every touchpoint.
In this webinar, you’ll learn:
- Why companies who utilize real-time customer signals report more effective personalization
- How to connect employees and customers in a shared experience through search and browse
- How Lucidworks clients Lenovo, Morgan Stanley and Red Hat fast-tracked improvements in conversion, engagement and customer satisfaction
Featuring
- Will Hayes, CEO, Lucidworks
- Brendan Witcher, VP, Principal Analyst, Forrester
Intelligent Insight Driven Policing with MC+A, Toronto Police Service and Luc...
Intelligent Policing. Leveraging Data to more effectively Serve Communities.
Policing in the next decade is anticipated to be very different from historical methods. More data driven, more focused on the intricacies of communities they serve and more open and collaborative to make informed recommendations a reality. Whether its social populations, NIBRS or organization improvement that’s the driver, the IT requirement is largely the same. Provide 360 access to large volumes of siloed data to gain a full 360 understanding of existing connections and patterns for improved insight and recommendation.
Join us for a round table discussion of how the Toronto Police Service is better serving their community through deploying a unified intelligent data platform.
Data innovation improves officers' engagement with existing data and streamlines investigation workflows by enhancing collaboration. This improved visibility into existing police data allows for a more intelligent and responsive police force.
In this webinar, we'll cover:
-The technology needs of an intelligent police force.
-How a Global Search improves an officer's interaction with existing data.
Featuring:
-Simon Taylor, VP, Worldwide Channels & Alliances, Lucidworks
-Michael Cizmar, Managing Director, MC+A
-Ian Williams, Manager of Analytics & Innovation, Toronto Police Service
[Webinar] Intelligent Policing. Leveraging Data to more effectively Serve Com...
Policing in the next decade is anticipated to be very different from historical methods. More data driven, more focused on the intricacies of communities they serve and more open and collaborative to make informed recommendations a reality. Whether its social populations, NIBRS or organization improvement that’s the driver, the IT requirement is largely the same. Provide 360 access to large volumes of siloed data to gain a full 360 understanding of existing connections and patterns for improved insight and recommendation.
Join us for a round table discussion of how the Toronto Police Service is better serving their community through deploying a unified intelligent data platform.
Data innovation improves officers' engagement with existing data and streamlines investigation workflows by enhancing collaboration. This improved visibility into existing police data allows for a more intelligent and responsive police force.
In this webinar, we'll cover:
The technology needs of an intelligent police force.
How a Global Search improves an officer's interaction with existing data.
Featuring
-Simon Taylor, VP, Worldwide Channels & Alliances, Lucidworks
-Michael Cizmar, Managing Director, MC+A
-Ian Williams, Manager of Analytics & Innovation, Toronto Police Service
Preparing for Peak in Ecommerce | eTail Asia 2020
This document provides a framework for prioritizing onsite search problems and key performance indicators (KPIs) to measure for e-commerce search optimization. It recommends prioritizing fixing searches that yield no results, improving relevance of results, and reducing false positives. The most essential KPIs to measure include query latency, throughput, result relevance through click-through rates and NDCG scores. The document also provides tips for self-benchmarking search performance and examples of search performance benchmarks across nine e-commerce sites from various industries.
Accelerate The Path To Purchase With Product Discovery at Retail Innovation C...
Wish your conversion rates were higher? Can’t figure out how to efficiently and effectively serve all the visitors on your site? Embarrassed by the quality of your product discovery experience? The bar is high and the influx of online shopping over recent months has reminded us that the opportunities are real. We’re all deep in holiday prep, but let’s take a few minutes to think about January 2021 and beyond. How can we position ourselves for success with our customers and against our competition?
Grab your lunch and let’s dive into three strategies that need to be part of your 2021 roadmap. You don’t need an army to get there. But you do need to take action and capitalize on the shoppers abandoning the product discovery journey on your site.
In this session, attendees will find out how to:
-Take control of merchandising at scale;
-Implement hands-free search relevancy; and
-Address personalization challenges.
AI-Powered Linguistics and Search with Fusion and Rosette
For a personalized search experience, search curation requires robust text interpretation, data enrichment, relevancy tuning and recommendations. In order to achieve this, language and entity identification are crucial.
For teams working on search applications, advanced language packages allow them to achieve greater recall without sacrificing precision.
Join us for a guided tour of our new Advanced Linguistics packages, available in Fusion, thanks to the technology partnership between Lucidworks and Basistech.
We’ll explore the application of language identification and entity extraction in the context of search, along with practical examples of personalizing search and enhancing entity extraction.
In this webinar, we’ll cover:
-How Fusion uses the Rosette Basic Linguistics and Entity Extraction packages
-Tips for improving language identification and treatment as well as data enrichment for personalization
-Speech2 demo modeling Active Recommendation
-Use Rosette’s packages with Fusion Pipelines to build custom entities for specific domain use cases
Featuring:
-Radu Miclaus, Director of Product, AI and Cloud, Lucidworks, Lucidworks
-Robert Lucarini, Senior Software Engineer, Lucidworks
-Nick Belanger, Solutions Engineer, Basis Technology
The Service Industry After COVID-19: The Soul of Service in a Virtual Moment
Before COVID-19, almost 80% of the US workforce worked service in jobs that involve in-person interaction with strangers. Now, leaders of service organizations must reshape their offerings during the pandemic and prepare for whatever the new normal turns out to be. Our three panelists will share ideas for adapting their service businesses, now that closer-than-six-feet isn’t an option.
Join Lucidworks as we talk shop with 3 service business leaders, covering:
-Common impacts of the pandemic on service businesses (and what to do about them),
-How service teams can maintain a human touch across virtual channels, and
-Plans for the future, before and after the pandemic subsides.
Featuring
-Sara Nathan, President & CEO, AMIGOS
-Anthony Carruesco, Founder, AC Fly Fishing
-sara bradley, chef and proprietor, freight house
-Justin Sears, VP Product Marketing, Lucidworks
Webinar: Smart answers for employee and customer support after covid 19 - Europe
The COVID-19 pandemic has forced companies to support far more customers and employees through digital channels than ever before. Many are turning to chatbots to help meet increasing demand, but traditional rules-based approaches can’t keep up. Our new Smart Answers add-on to Lucidworks Fusion makes existing chatbots and virtual assistants more intelligent and more valuable to the people you serve.
Smart Answers for Employee and Customer Support After COVID-19
Watch our on-demand webinar showcasing Smart Answers on Lucidworks Fusion. This technology makes existing chatbots and virtual assistants more intelligent and more valuable to the people you serve.
In this webinar, we’ll cover off:
-How search and deep learning extend conversational frameworks for improved experiences
-How Smart Answers improves customer care, call deflection, and employee self-service
-A live demo of Smart Answers for multi-channel self-service support
Applying AI & Search in Europe - featuring 451 Research
In the current climate, it’s now more important than ever to digitally enable your workforce and customers.
Hear from Simon Taylor, VP Global Partners & Alliances, Lucidworks and Matt Aslett, Research Vice President, 451 Research to get the inside scoop on how industry leaders in Europe are developing and executing their digital transformation strategies.
In this webinar, we’ll discuss:
The top challenges and aspirations European business and technology leaders are solving using AI and search technology
Which search and AI use cases are making the biggest impact in industries such as finance, healthcare, retail and energy in Europe
What technology buyers should look for when evaluating AI and search solutions
Webinar: Accelerate Data Science with Fusion 5.1
This document introduces Fusion 5.1 and its new capabilities for integrating with data science tools like Tensorflow, Scikit-Learn, and Spacy.
It provides an overview of Fusion's capabilities for understanding content, users, and delivering insights at scale. The document then demonstrates Fusion's Jupyter Notebook integration for reading and writing data and running SQL queries.
Finally, it shows how Fusion integrates with Seldon Core to easily deploy machine learning models with tools like Tensorflow and Scikit-Learn. A live demo is provided of deploying a custom model and using it in Fusion's query and indexing pipelines.
Webinar: 5 Must-Have Items You Need for Your 2020 Ecommerce Strategy
In this webinar with 451 Research, you'll understand how retailers are using AI to predict customer intent and learn which key performance metrics are used by more than 120 online retailers in Lucidworks’ 2019 Retail Benchmark Survey.
In this webinar, you’ll learn:
● What trends and opportunities are facing the ecommerce industry in 2020
● Why search is the universal path to understanding customer intent
● How large online retailers apply AI to maximize the effectiveness of their personalization efforts
Where Search Meets Science and Style Meets Savings: Nordstrom Rack's Journey ...
Nordstrom Rack | Hautelook curates and serves customers a wide selection of on-trend apparel, accessories, and shoes at an everyday savings of up to 75 percent off regular prices. With over a million visitors shopping across different platforms every day, and a realization that customers have become accustomed to robust and personalized search interactions, Nordstrom Rack | Hautelook launched an initiative over a year ago to provide data science-driven digital experiences to their customers.
In this session, we’ll discuss Nordstrom Rack | Hautelook’s journey of operationalizing a hefty strategy, optimizing a fickle infrastructure, and rallying troops around a single vision of building an expansible machine-learning driven product discovery engine.
The audience will learn about:
-The key technical challenges and outcomes that come with onboarding a solution
-The lessons learned of creating and executing operational design
-The use of Lucidworks Fusion to plug custom data science models into search and browse applications to understand user intent and deliver personalized experiences
Apply Knowledge Graphs and Search for Real-World Decision Intelligence
Knowledge graphs and machine learning are on the rise as enterprises hunt for more effective ways to connect the dots between the data and the business world. With newer technologies, the digital workplace can dramatically improve employee engagement, data-driven decisions, and actions that serve tangible business objectives.
In this webinar, you will learn
-- Introduction to knowledge graphs and where they fit in the ML landscape
-- How breakthroughs in search affect your business
-- The key features to consider when choosing a data discovery platform
-- Best practices for adopting AI-powered search, with real-world examples
Webinar: Building a Business Case for Enterprise Search
The document discusses building a business case for enterprise search. It notes that 85% of information is unstructured data locked in various locations and applications. Many knowledge workers spend a significant portion of their day searching across multiple systems for information. The rise of unstructured data and AI capabilities can help organizations unlock value from their information assets. Effective enterprise search powered by AI can provide real-time intelligence, personalized information, and more efficient research to help knowledge workers.
Why Insight Engines Matter in 2020 and Beyond
Webinar: Why Insight Engines Matter in 2020 and Beyond
Date: December 12, 2019
Presenter: Justin Sears, VP Product Marketing, Lucidworks
More from Lucidworks (20)
Search is the Tip of the Spear for Your B2B eCommerce Strategy
Search is the Tip of the Spear for Your B2B eCommerce Strategy
How Crate & Barrel Connects Shoppers with Relevant Products
How Crate & Barrel Connects Shoppers with Relevant Products
Lucidworks & IMRG Webinar – Best-In-Class Retail Product Discovery
Lucidworks & IMRG Webinar – Best-In-Class Retail Product Discovery
Connected Experiences Are Personalized Experiences
Connected Experiences Are Personalized Experiences
Intelligent Insight Driven Policing with MC+A, Toronto Police Service and Luc...
Intelligent Insight Driven Policing with MC+A, Toronto Police Service and Luc...
[Webinar] Intelligent Policing. Leveraging Data to more effectively Serve Com...
[Webinar] Intelligent Policing. Leveraging Data to more effectively Serve Com...
Accelerate The Path To Purchase With Product Discovery at Retail Innovation C...
Accelerate The Path To Purchase With Product Discovery at Retail Innovation C...
AI-Powered Linguistics and Search with Fusion and Rosette
AI-Powered Linguistics and Search with Fusion and Rosette
The Service Industry After COVID-19: The Soul of Service in a Virtual Moment
The Service Industry After COVID-19: The Soul of Service in a Virtual Moment
Webinar: Smart answers for employee and customer support after covid 19 - Europe
Webinar: Smart answers for employee and customer support after covid 19 - Europe
Smart Answers for Employee and Customer Support After COVID-19
Smart Answers for Employee and Customer Support After COVID-19
Applying AI & Search in Europe - featuring 451 Research
Applying AI & Search in Europe - featuring 451 Research
Webinar: 5 Must-Have Items You Need for Your 2020 Ecommerce Strategy
Webinar: 5 Must-Have Items You Need for Your 2020 Ecommerce Strategy
Where Search Meets Science and Style Meets Savings: Nordstrom Rack's Journey ...
Where Search Meets Science and Style Meets Savings: Nordstrom Rack's Journey ...
Apply Knowledge Graphs and Search for Real-World Decision Intelligence
Apply Knowledge Graphs and Search for Real-World Decision Intelligence
Webinar: Building a Business Case for Enterprise Search
Webinar: Building a Business Case for Enterprise Search
Recently uploaded
Removing Uninteresting Bytes in Software Fuzzing
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
みなさんこんにちはこれ何文字まで入るの？40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの？えこ...
ここ3000字までしか入らないけどタイトルの方がたくさん文字入ると思います。
Infrastructure Challenges in Scaling RAG with Custom AI models
Building Retrieval-Augmented Generation (RAG) systems with open-source and custom AI models is a complex task. This talk explores the challenges in productionizing RAG systems, including retrieval performance, response synthesis, and evaluation. We’ll discuss how to leverage open-source models like text embeddings, language models, and custom fine-tuned models to enhance RAG performance. Additionally, we’ll cover how BentoML can help orchestrate and scale these AI components efficiently, ensuring seamless deployment and management of RAG systems in the cloud.
Presentation of the OECD Artificial Intelligence Review of Germany
Consult the full report at https://www.oecd.org/digital/oecd-artificial-intelligence-review-of-germany-609808d6-en.htm
Essentials of Automations: The Art of Triggers and Actions in FME
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
20240609 QFM020 Irresponsible AI Reading List May 2024
Everything I found interesting about the irresponsible use of machine intelligence in May 2024
Full-RAG: A modern architecture for hyper-personalization
Mike Del Balso, CEO & Co-Founder at Tecton, presents "Full RAG," a novel approach to AI recommendation systems, aiming to push beyond the limitations of traditional models through a deep integration of contextual insights and real-time data, leveraging the Retrieval-Augmented Generation architecture. This talk will outline Full RAG's potential to significantly enhance personalization, address engineering challenges such as data management and model training, and introduce data enrichment with reranking as a key solution. Attendees will gain crucial insights into the importance of hyperpersonalization in AI, the capabilities of Full RAG for advanced personalization, and strategies for managing complex data integrations for deploying cutting-edge AI solutions.
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-und-domino-lizenzkostenreduzierung-in-der-welt-von-dlau/
DLAU und die Lizenzen nach dem CCB- und CCX-Modell sind für viele in der HCL-Community seit letztem Jahr ein heißes Thema. Als Notes- oder Domino-Kunde haben Sie vielleicht mit unerwartet hohen Benutzerzahlen und Lizenzgebühren zu kämpfen. Sie fragen sich vielleicht, wie diese neue Art der Lizenzierung funktioniert und welchen Nutzen sie Ihnen bringt. Vor allem wollen Sie sicherlich Ihr Budget einhalten und Kosten sparen, wo immer möglich. Das verstehen wir und wir möchten Ihnen dabei helfen!
Wir erklären Ihnen, wie Sie häufige Konfigurationsprobleme lösen können, die dazu führen können, dass mehr Benutzer gezählt werden als nötig, und wie Sie überflüssige oder ungenutzte Konten identifizieren und entfernen können, um Geld zu sparen. Es gibt auch einige Ansätze, die zu unnötigen Ausgaben führen können, z. B. wenn ein Personendokument anstelle eines Mail-Ins für geteilte Mailboxen verwendet wird. Wir zeigen Ihnen solche Fälle und deren Lösungen. Und natürlich erklären wir Ihnen das neue Lizenzmodell.
Nehmen Sie an diesem Webinar teil, bei dem HCL-Ambassador Marc Thomas und Gastredner Franz Walder Ihnen diese neue Welt näherbringen. Es vermittelt Ihnen die Tools und das Know-how, um den Überblick zu bewahren. Sie werden in der Lage sein, Ihre Kosten durch eine optimierte Domino-Konfiguration zu reduzieren und auch in Zukunft gering zu halten.
Diese Themen werden behandelt
- Reduzierung der Lizenzkosten durch Auffinden und Beheben von Fehlkonfigurationen und überflüssigen Konten
- Wie funktionieren CCB- und CCX-Lizenzen wirklich?
- Verstehen des DLAU-Tools und wie man es am besten nutzt
- Tipps für häufige Problembereiche, wie z. B. Team-Postfächer, Funktions-/Testbenutzer usw.
- Praxisbeispiele und Best Practices zum sofortigen Umsetzen
UiPath Test Automation using UiPath Test Suite series, part 6
Welcome to UiPath Test Automation using UiPath Test Suite series part 6. In this session, we will cover Test Automation with generative AI and Open AI.
UiPath Test Automation with generative AI and Open AI webinar offers an in-depth exploration of leveraging cutting-edge technologies for test automation within the UiPath platform. Attendees will delve into the integration of generative AI, a test automation solution, with Open AI advanced natural language processing capabilities.
Throughout the session, participants will discover how this synergy empowers testers to automate repetitive tasks, enhance testing accuracy, and expedite the software testing life cycle. Topics covered include the seamless integration process, practical use cases, and the benefits of harnessing AI-driven automation for UiPath testing initiatives. By attending this webinar, testers, and automation professionals can gain valuable insights into harnessing the power of AI to optimize their test automation workflows within the UiPath ecosystem, ultimately driving efficiency and quality in software development processes.
What will you get from this session?
1. Insights into integrating generative AI.
2. Understanding how this integration enhances test automation within the UiPath platform
3. Practical demonstrations
4. Exploration of real-world use cases illustrating the benefits of AI-driven test automation for UiPath
Topics covered:
What is generative AI
Test Automation with generative AI and Open AI.
UiPath integration with generative AI
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Microsoft - Power Platform_G.Aspiotis.pdf
Revolutionizing Application Development
with AI-powered low-code, presentation by George Aspiotis, Sr. Partner Development Manager, Microsoft
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
As the digital landscape continually evolves, operating systems play a critical role in shaping user experiences and productivity. The launch of Nitrux Linux 3.5.0 marks a significant milestone, offering a robust alternative to traditional systems such as Windows 11. This article delves into the essence of Nitrux Linux 3.5.0, exploring its unique features, advantages, and how it stands as a compelling choice for both casual users and tech enthusiasts.
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
Imagine a world where machines not only perform tasks but also learn, adapt, and make decisions. This is the promise of Artificial Intelligence (AI), a technology that's not just enhancing our lives but revolutionizing entire industries.
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Discover how MongoDB Atlas and vector search technology can revolutionize your application's search capabilities. This comprehensive presentation covers:
* What is Vector Search?
* Importance and benefits of vector search
* Practical use cases across various industries
* Step-by-step implementation guide
* Live demos with code snippets
* Enhancing LLM capabilities with vector search
* Best practices and optimization strategies
Perfect for developers, AI enthusiasts, and tech leaders. Learn how to leverage MongoDB Atlas to deliver highly relevant, context-aware search results, transforming your data retrieval process. Stay ahead in tech innovation and maximize the potential of your applications.
#MongoDB #VectorSearch #AI #SemanticSearch #TechInnovation #DataScience #LLM #MachineLearning #SearchTechnology
Building Production Ready Search Pipelines with Spark and Milvus
Spark is the widely used ETL tool for processing, indexing and ingesting data to serving stack for search. Milvus is the production-ready open-source vector database. In this talk we will show how to use Spark to process unstructured data to extract vector representations, and push the vectors to Milvus vector database for search serving.
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
20240607 QFM018 Elixir Reading List May 2024
Everything I found interesting about the Elixir programming ecosystem in May 2024
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for life science domain, where you retriever information from biomedical knowledge graphs using LLMs to increase the accuracy and performance of generated answers
Recently uploaded (20)
みなさんこんにちはこれ何文字まで入るの？40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの？えこ...
みなさんこんにちはこれ何文字まで入るの？40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの？えこ...
Infrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI models
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Simple Fuzzy Name Matching in Solr: Presented by Chris Mack, Basis Technology
- 1. O C T O B E R 1 3 - 1 6 , 2 0 1 6 • A U S T I N , T X
- 2. Simple Fuzzy Name Matching in Solr Chris Mack Director Customer Engineering Basis Technology
- 5. 5 02 Why Match Names? Just a Name…. ...Right? 1. Security 2. Fraud 3. Commerce
- 6. 6 01 Quick survey: How many of you... • Regularly develop Solr applications? • Develop Solr applications that include names of… ...People? ...Places? ...Products? ...Organizations? • Have names in languages beside English?
- 7. 7 03 What Makes Name Matching Hard?
- 11. 11 01 How Would You Solve It?
- 12. 12 01 Best Practice: field per variation type?
- 13. 13 01 Idea: Create a Custom Solr Field • Contribute score that reflects phenomena. • Be part of queries using many field types. • Have multiple fields per document. • Have multiple values per field.
- 14. 14 01 But what if variations co-occur? “Jesus Alfonso Lopez Diaz” v. “LobezDias, Chuy” 1) Reordered. 2) Nickname for first name. 3) Missing 2nd Name. 4) Two spelling differences. 5) Missing space.
- 15. 15 01 Can We Do Better? • Incorporate our proprietary name matching • Provide similarity scores to name pairs • Use Solr’s Rerank feature • Allows for higher precision ranking and tresholding • Provides multi-lingual name search
- 16. 16 01 Simple to Configure • Plugin contains custom field type which does all the work behind the scenes • Simple addition to schema.xml to include new field type <fieldType name="rni_name" class="com.basistech.rni.solr.NameField"/> <field name="name" type="rni_name" indexed="true" stored="true" multiValued="false"/> <field name="aka" type="rni_name" indexed="true" stored="true" multiValued="true"/>
- 18. 18 01 What happens at query time? • Step #1: NameField generates analogous keys for a custom Lucene query that finds good candidates for re-ranking public Query getFieldQuery(QParser parser, SchemaField field, String val) { Name name = parseNameString(externalVal, parser.getParams()); QuerySpec querySpec = buildQuery(name); return querySpec.accept(new SolrQueryVisitor(field.getName())); }
- 19. 19 01 What else happens at query time? • Step #2: Uses Solr’s Rerank feature to rescore name s in top documents and reorder accordingly - Tuned for high precision - Simple addition to solrconfig.xml <queryParser name="rniRerank" class="com.basistech.rni.solr.RNIReRankQParserPlugin"/> <valueSourceParser name="rniMatch” class="com.basistech.rni.solr.NameMatchValueSourceParser"/>
- 21. 21 01 Ability to Tradeoff Accuracy vs. Speed • reRankScoreThreshold - Score threshold top doc must meet to be rescored. • reRankDocs - Controls how many of the top documents to rescore
- 22. 22 01 Summary: How it works • Custom field type - Splits a single field into multiple fields covering different phenomena - Supports multiple name fields in a document as well as multivalued fields - Intercepts the query to inject a custom Lucene query • Custom rerank function - Rescores documents with algorithm specific to name matching - Limits intense calculations to only top candidates - Highly configurable
- 23. 23 01 Suggested Questions: • What is names are in unstructured text? • What if the names are in other text fields? • How did you implement multi-valued fields? • How does it scale? • How do you handle names not in English? • How does this relate to the theme of Entity-Centric Search? • How do plug-in’s scores relate to Solr scores? • How can I learn more?
- 24. Simple Fuzzy Name Matching in Solr Chris Mack Director Customer Engineering Basis Technology