THE DB AND THE INDEX: WHY OUR NEED FOR SEARCH IS SHAPING OUR TECHNOLOGY

  • 199 views
Uploaded on

Presentation from Grant Ingersoll, LucidWorks …

Presentation from Grant Ingersoll, LucidWorks
#dataconf
More at http://event.gigaom.com/structuredata/

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
199
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
2
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. THE DB AND THE INDEX: WHY OUR NEED FOR SEARCH IS SHAPING OUR TECHNOLOGY SPEAKER: Grant Ingersoll CTO LucidWorksThursday, March 28, 13
  • 2. The DB and the Index Grant Ingersoll (@gsingers) CTO Confidential © Copyright 2012Thursday, March 28, 13
  • 3. Confidential and Proprietary 3 © 2012 LucidWorksThursday, March 28, 13
  • 4. Search: NoSQL before NoSQL was cool • Search is a system building block - Text is only a part of the story • If the algorithms fit, use them! • Embrace fuzziness! • Scoring features are everywhere Confidential and Proprietary © 2012 LucidWorksThursday, March 28, 13
  • 5. Reference Architecture Access APIs Search View Analytic Personalization & 1 2 Services Machine Learning Shards 3 N •Documents Discovery & Document •Users •Logs Classification Models Enrichment Store In memory Replicated Multi-tenant Content Acquisition ETL, batch or near real- time Data • LucidWorks Search connectors • Push Confidential and Proprietary © 2012 LucidWorksThursday, March 28, 13
  • 6. Search (R)evolution • Search use leads to search abuse - Denormalization frees your mind - Scoring is just a sparse matrix multiply • Lucene/Solr evolution - Non free text usages abound - Many DB-like features - Flexible indexing - Finite State Transducers FTW! • Scale • “This ain’t your father’s relevance anymore” Confidential and Proprietary © 2012 LucidWorksThursday, March 28, 13
  • 7. Resources • http://www.lucidworks.com • http://www.lucidworks.com/products/lucidworks-big-data • grant@lucidworks.com • @gsingers Confidential and Proprietary © 2012 LucidWorksThursday, March 28, 13
  • 8. Thursday, March 28, 13