DBSight Instant Scalable Full Text Search on Any Databases
Some Customers <ul><ul><li>Websites </li></ul></ul><ul><ul><ul><li>Workopolis.com, biggest job site in Canada </li></ul></...
Features <ul><ul><li>Add Search when you want to </li></ul></ul><ul><ul><ul><li>Database Independent </li></ul></ul></ul><...
The Pain to Add Search <ul><ul><li>Do you have it? Common, but slow SQL,  </li></ul></ul><ul><ul><ul><li>select * from tab...
DBSight Design Goals <ul><ul><li>Very fast to create and adjust </li></ul></ul><ul><ul><li>“ Knobs” to tune search </li></...
Why DBSight? – Simple! <ul><ul><li>Create Search as 1-2-3 </li></ul></ul><ul><ul><ul><li>Setup JDBC Connection </li></ul><...
Simple to Create <ul><ul><li>Web UI to </li></ul></ul><ul><ul><ul><li>set SQL to retrieve content </li></ul></ul></ul>
Simple to Create <ul><ul><li>Scaffolding to generate search result template </li></ul></ul>
Simple to Create <ul><ul><li>Keep it DRY – Do not Repeat Yourself </li></ul></ul><ul><ul><ul><li>Make full use of existing...
Why DBSight? – Not So Simple! <ul><ul><li>A whole Solution </li></ul></ul><ul><ul><ul><li>No consulting fees </li></ul></u...
DBSight covers SDLC <ul><ul><li>No existing production ready solutions in the market covers the whole software development...
DBSight – Loosely Coupled <ul><ul><li>Database Independent </li></ul></ul><ul><ul><li>Programming language Independent </l...
DBSight – Enterprise Ready <ul><ul><li>Easy to move deployment environments </li></ul></ul><ul><ul><ul><li>Development, Te...
DBSight – SQL Friendly <ul><ul><li>Incremental Indexing </li></ul></ul><ul><ul><ul><li>Handle New/Updated/Deleted records ...
Easy to maintain <ul><ul><li>Scheduled Jobs </li></ul></ul><ul><ul><ul><li>Incremental indexing </li></ul></ul></ul><ul><u...
Facet Search <ul><ul><li>Not all facet search are the same! </li></ul></ul><ul><ul><ul><li>Single Value </li></ul></ul></u...
Fast Facet Search <ul><ul><li>In Memory Facets </li></ul></ul><ul><ul><li>Several cache for More Speed </li></ul></ul><ul>...
Examples of Facet Search <ul><ul><li>Basic Facet Search </li></ul></ul><ul><ul><ul><li>Example: Category </li></ul></ul></...
Feature: Seperated Indexing and Searching <ul><ul><li>Problem: Search Pauses! </li></ul></ul><ul><ul><ul><li>Indexing is C...
DBSight Architecture <ul><ul><li>Crawl database via user defined SQL </li></ul></ul><ul><ul><ul><li>Multiple database tabl...
 
DBSight Common Setup <ul><ul><li>Embedded </li></ul></ul><ul><ul><li>Multiple Indexes on single node </li></ul></ul><ul><u...
 
 
 
 
DBSight Introduction
DBSight Introduction
DBSight Introduction
Upcoming SlideShare
Loading in …5
×

DBSight Introduction

3,664
-1

Published on

An excellent tool to create Database search quickly, with high performance and scalability, easy to maintain, flexible result rendering, api or javascript integration, etc.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
3,664
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
10
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

DBSight Introduction

  1. 1. DBSight Instant Scalable Full Text Search on Any Databases
  2. 2. Some Customers <ul><ul><li>Websites </li></ul></ul><ul><ul><ul><li>Workopolis.com, biggest job site in Canada </li></ul></ul></ul><ul><ul><ul><li>Twenga.com, busy </li></ul></ul></ul><ul><ul><ul><li>Current.com </li></ul></ul></ul><ul><ul><li>Corporate </li></ul></ul><ul><ul><ul><li>eBay </li></ul></ul></ul><ul><ul><ul><li>Costco </li></ul></ul></ul><ul><ul><ul><li>FactSet </li></ul></ul></ul><ul><ul><ul><li>Genetech </li></ul></ul></ul><ul><ul><li>Consulting </li></ul></ul><ul><ul><ul><li>Computer Science Corporation </li></ul></ul></ul><ul><ul><li>Federal </li></ul></ul><ul><ul><ul><li>Federal Procurement Data System </li></ul></ul></ul><ul><ul><li>Banking </li></ul></ul><ul><ul><ul><li>European Central Bank, International Counterfeit Deterrence Centre(ECB-ICDC) </li></ul></ul></ul>
  3. 3. Features <ul><ul><li>Add Search when you want to </li></ul></ul><ul><ul><ul><li>Database Independent </li></ul></ul></ul><ul><ul><ul><li>Language Independent </li></ul></ul></ul><ul><ul><ul><li>Very Easy to Create </li></ul></ul></ul><ul><ul><ul><li>Very Easy to Modify </li></ul></ul></ul><ul><ul><ul><li>Very Easy to Monitor/Maintain </li></ul></ul></ul><ul><ul><li>Feature Rich </li></ul></ul><ul><ul><ul><li>Facet Search </li></ul></ul></ul><ul><ul><ul><li>Real Time Search </li></ul></ul></ul><ul><ul><ul><li>Flexible ranking </li></ul></ul></ul><ul><ul><ul><li>… </li></ul></ul></ul><ul><ul><li>High Performance </li></ul></ul><ul><ul><li>Linearly Scalable </li></ul></ul>
  4. 4. The Pain to Add Search <ul><ul><li>Do you have it? Common, but slow SQL, </li></ul></ul><ul><ul><ul><li>select * from tableA </li></ul></ul></ul><ul><ul><ul><li>where column1 like ‘%abc%’ </li></ul></ul></ul><ul><ul><ul><li>or column2 like ‘%abc%’ </li></ul></ul></ul><ul><ul><ul><li>or … </li></ul></ul></ul><ul><ul><li>Some database has search features, but still </li></ul></ul><ul><ul><ul><li>Not easy to customize </li></ul></ul></ul><ul><ul><ul><li>No facet search </li></ul></ul></ul><ul><ul><ul><li>Database specific solution </li></ul></ul></ul><ul><ul><li>Some Lucene based search, but </li></ul></ul><ul><ul><ul><li>Not cover development life cycle </li></ul></ul></ul><ul><ul><ul><li>Hard to maintain. </li></ul></ul></ul><ul><ul><ul><li>Will it scale if your data grows? </li></ul></ul></ul><ul><ul><ul><li>Too closely coupled with your program </li></ul></ul></ul>
  5. 5. DBSight Design Goals <ul><ul><li>Very fast to create and adjust </li></ul></ul><ul><ul><li>“ Knobs” to tune search </li></ul></ul><ul><ul><li>Features beyond basic search </li></ul></ul><ul><ul><ul><li>Facet Search </li></ul></ul></ul><ul><ul><ul><li>Results ranking by attributes like “price”, “time”! </li></ul></ul></ul><ul><ul><li>Minimal administration </li></ul></ul><ul><ul><li>Low total cost of ownership </li></ul></ul><ul><ul><ul><li>Off-the-shelf, no expensive consulting fees </li></ul></ul></ul><ul><ul><li>Flexible </li></ul></ul><ul><ul><ul><li>Customizable analyzers, similarities, data retrieval, UI scaffolding, different output formats, APIs. </li></ul></ul></ul>
  6. 6. Why DBSight? – Simple! <ul><ul><li>Create Search as 1-2-3 </li></ul></ul><ul><ul><ul><li>Setup JDBC Connection </li></ul></ul></ul><ul><ul><ul><li>Select with SQL </li></ul></ul></ul><ul><ul><ul><li>Generate Search Configuration, Results Template </li></ul></ul></ul><ul><ul><li>Java knowledge is not necessary </li></ul></ul>
  7. 7. Simple to Create <ul><ul><li>Web UI to </li></ul></ul><ul><ul><ul><li>set SQL to retrieve content </li></ul></ul></ul>
  8. 8. Simple to Create <ul><ul><li>Scaffolding to generate search result template </li></ul></ul>
  9. 9. Simple to Create <ul><ul><li>Keep it DRY – Do not Repeat Yourself </li></ul></ul><ul><ul><ul><li>Make full use of existing metadata </li></ul></ul></ul><ul><ul><li>From your SQL </li></ul></ul><ul><ul><ul><li>Generate most Lucene configuration </li></ul></ul></ul><ul><ul><ul><li>Generate most search options </li></ul></ul></ul><ul><ul><ul><li>Generate most rendering templates </li></ul></ul></ul><ul><ul><li>UI to fine-tune customization </li></ul></ul>
  10. 10. Why DBSight? – Not So Simple! <ul><ul><li>A whole Solution </li></ul></ul><ul><ul><ul><li>No consulting fees </li></ul></ul></ul><ul><ul><ul><li>UI to manage everything </li></ul></ul></ul><ul><ul><ul><li>Set it up, and leave it run. No babysitting. </li></ul></ul></ul><ul><ul><li>Agnostic of programming languages or frameworks </li></ul></ul><ul><ul><li>Create and maintain, with basic SQL </li></ul></ul><ul><ul><li>Built-in Usage Statistics </li></ul></ul><ul><ul><li>Scalable </li></ul></ul><ul><ul><ul><li>Linearly Scalable Sharded Search </li></ul></ul></ul><ul><ul><ul><li>Separated Indexing and Searching </li></ul></ul></ul><ul><ul><li>Many Customization point </li></ul></ul><ul><ul><ul><li>Search Results templates easy to customize </li></ul></ul></ul><ul><ul><ul><li>API for deep integration </li></ul></ul></ul>
  11. 11. DBSight covers SDLC <ul><ul><li>No existing production ready solutions in the market covers the whole software development life cycle. </li></ul></ul><ul><ul><ul><li>DBSight make search a separated concern </li></ul></ul></ul><ul><ul><ul><li>Change easily when database schema changes </li></ul></ul></ul><ul><ul><ul><li>Enterprise Ready </li></ul></ul></ul><ul><ul><ul><ul><li>Monitoring </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Portable for Dev=>Test => Stage => Production environments </li></ul></ul></ul></ul>
  12. 12. DBSight – Loosely Coupled <ul><ul><li>Database Independent </li></ul></ul><ul><ul><li>Programming language Independent </li></ul></ul><ul><ul><li>Framework Independent </li></ul></ul><ul><ul><li>Works during the whole software life cycle </li></ul></ul><ul><ul><ul><li>Allows frequent adjustment </li></ul></ul></ul><ul><ul><ul><li>Easily re-create the whole index </li></ul></ul></ul><ul><ul><ul><li>Linearly scalable for high concurrency and for high data volume </li></ul></ul></ul>
  13. 13. DBSight – Enterprise Ready <ul><ul><li>Easy to move deployment environments </li></ul></ul><ul><ul><ul><li>Development, Testing(QA), Staging, Production </li></ul></ul></ul><ul><ul><li>Secure </li></ul></ul><ul><ul><ul><li>access control </li></ul></ul></ul><ul><ul><ul><li>sensitive database passwords </li></ul></ul></ul><ul><ul><li>Package-able Solution </li></ul></ul><ul><ul><ul><li>Import/Export configuration </li></ul></ul></ul><ul><ul><ul><li>Customizable enterprise-specific scaffolding </li></ul></ul></ul>
  14. 14. DBSight – SQL Friendly <ul><ul><li>Incremental Indexing </li></ul></ul><ul><ul><ul><li>Handle New/Updated/Deleted records </li></ul></ul></ul><ul><ul><ul><li>Find Deleted Records Efficiently! </li></ul></ul></ul><ul><ul><ul><ul><li>Support Hard Deleted Records </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Support Soft Deleted Records </li></ul></ul></ul></ul><ul><ul><li>Flexible User Defined SQL </li></ul></ul><ul><ul><ul><li>Star-schema like content retrieval </li></ul></ul></ul><ul><ul><ul><li>No too-smart auto discovery </li></ul></ul></ul><ul><ul><li>Efficient </li></ul></ul><ul><ul><ul><li>Multi-threaded data retrieval </li></ul></ul></ul><ul><ul><ul><li>Caching to minimize database load </li></ul></ul></ul><ul><ul><ul><li>Customizable number of SQL connections </li></ul></ul></ul>
  15. 15. Easy to maintain <ul><ul><li>Scheduled Jobs </li></ul></ul><ul><ul><ul><li>Incremental indexing </li></ul></ul></ul><ul><ul><ul><li>Re-Create indexing </li></ul></ul></ul><ul><ul><ul><li>Build spell checking, synonyms, stop words dictionaries </li></ul></ul></ul><ul><ul><li>Web UI to </li></ul></ul><ul><ul><ul><li>Monitor Indexing process </li></ul></ul></ul><ul><ul><ul><li>Monitor Search Usage </li></ul></ul></ul>
  16. 16. Facet Search <ul><ul><li>Not all facet search are the same! </li></ul></ul><ul><ul><ul><li>Single Value </li></ul></ul></ul><ul><ul><ul><li>Multiple Value </li></ul></ul></ul><ul><ul><ul><li>Range Value </li></ul></ul></ul><ul><ul><li>Fast! </li></ul></ul><ul><ul><li>Memory Efficient! </li></ul></ul><ul><ul><li>More Features! </li></ul></ul><ul><ul><ul><li>List most-used facet according to usage! </li></ul></ul></ul><ul><ul><ul><li>Sum()/Avg() functions </li></ul></ul></ul>
  17. 17. Fast Facet Search <ul><ul><li>In Memory Facets </li></ul></ul><ul><ul><li>Several cache for More Speed </li></ul></ul><ul><ul><ul><li>Cache for top facets </li></ul></ul></ul><ul><ul><ul><li>Cache for recent facets </li></ul></ul></ul><ul><ul><li>Automatically Pre-warm up </li></ul></ul>
  18. 18. Examples of Facet Search <ul><ul><li>Basic Facet Search </li></ul></ul><ul><ul><ul><li>Example: Category </li></ul></ul></ul><ul><ul><ul><ul><li>Classic ( 23 matches) </li></ul></ul></ul></ul><ul><ul><li>Multi-Valued Facet Search </li></ul></ul><ul><ul><ul><li>Example: Tag, or Tag Cloud. Several tags for one record </li></ul></ul></ul><ul><ul><li>Configurable dynamic facet grouping </li></ul></ul><ul><ul><ul><li>Example: Price Range </li></ul></ul></ul><ul><ul><ul><ul><li>$2 ~$3 (7 matches) </li></ul></ul></ul></ul><ul><ul><li>Average/Sum for each facet </li></ul></ul><ul><ul><ul><li>Example: Average price for the Year Range </li></ul></ul></ul><ul><ul><ul><ul><li>1970~1980(6 matches, average price $1,245.34) </li></ul></ul></ul></ul>
  19. 19. Feature: Seperated Indexing and Searching <ul><ul><li>Problem: Search Pauses! </li></ul></ul><ul><ul><ul><li>Indexing is CPU and Disk intensive </li></ul></ul></ul><ul><ul><ul><li>Searching is CPU, Disk, and Memory intensive </li></ul></ul></ul><ul><ul><ul><li>Resource competition </li></ul></ul></ul><ul><ul><li>Solution: Separated Indexing and Searching </li></ul></ul><ul><ul><ul><li>Different JVM processes </li></ul></ul></ul><ul><ul><ul><ul><li>Easier to manage resources via JVM settings </li></ul></ul></ul></ul><ul><ul><ul><li>Indexing and Searching can be on different machines. </li></ul></ul></ul><ul><ul><ul><ul><li>Improve performance </li></ul></ul></ul></ul><ul><ul><ul><ul><li>No hiccup because of CPU, memory, disk resources contension. </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Cluster of Searching nodes for better scalable performance! </li></ul></ul></ul></ul>
  20. 20. DBSight Architecture <ul><ul><li>Crawl database via user defined SQL </li></ul></ul><ul><ul><ul><li>Multiple database tables </li></ul></ul></ul><ul><ul><ul><li>Support star-schema like outer joins </li></ul></ul></ul><ul><ul><li>Create and Maintain Lucene index </li></ul></ul><ul><ul><ul><li>Incremental Indexing </li></ul></ul></ul><ul><ul><ul><li>Re-Creating Indexing </li></ul></ul></ul><ul><ul><li>Serve Search results via </li></ul></ul><ul><ul><ul><li>user defined templates </li></ul></ul></ul><ul><ul><ul><ul><li>XML/HTML/JSON/JSONP </li></ul></ul></ul></ul><ul><ul><ul><li>API, protocol buffer for Java and other languages </li></ul></ul></ul><ul><ul><li>Linear Scalability </li></ul></ul>
  21. 22. DBSight Common Setup <ul><ul><li>Embedded </li></ul></ul><ul><ul><li>Multiple Indexes on single node </li></ul></ul><ul><ul><li>Single Node, Indexing + Searching </li></ul></ul><ul><ul><li>Two Nodes, Separated Indexing and Searching </li></ul></ul><ul><ul><ul><li>Setup for LAN </li></ul></ul></ul><ul><ul><ul><li>Setup for WAN </li></ul></ul></ul><ul><ul><li>Cluster of Searching nodes via Replication, one Indexing nodes </li></ul></ul><ul><ul><li>Cluster of Sharded Nodes, each with Indexing +Searching </li></ul></ul><ul><ul><li>Cluster of Searching nodes via Sharding, one or several indexing nodes. </li></ul></ul>
  1. ¿Le ha llamado la atención una diapositiva en particular?

    Recortar diapositivas es una manera útil de recopilar información importante para consultarla más tarde.

×