Grails lucenecacherdfperformance

1,376 views
1,296 views

Published on

2 Comments
0 Likes
Statistics
Notes
  • Be the first to like this

No Downloads
Views
Total views
1,376
On SlideShare
0
From Embeds
0
Number of Embeds
14
Actions
Shares
0
Downloads
13
Comments
2
Likes
0
Embeds 0
No embeds

No notes for slide

Grails lucenecacherdfperformance

  1. 1. BLAZING FAST PERFORMANCE with Spring Cache, Lucene, GPars, RDF and Grails
  2. 2. WHAT’S A TRIPLE? subjectpredicate object
  3. 3. WHAT’S A TRIPLE? subject “Imatinib”predicate type object Pharmaceutical Drug
  4. 4. LINKING OPEN DRUG DATA http://esw.w3.org/HCLSIG/LODD
  5. 5. WHAT’S A TRIPLE?http://www4.wiwiss.fu-berlin.de/drugbank/page/drugs/DB00619 subject predicate object DB00619 label “Imatinib” DB00619 type drugbank:drugs DB00619 brandName “Gleevec” DB00619 drugbank:target targets:17
  6. 6. WHAT’S A TRIPLE?http://www4.wiwiss.fu-berlin.de/drugbank/page/targets/17 subject predicate object DB00619 drugbank:target targets/17 targets/17 type drugbank:drugs “Proto-oncogene tyrosine- targets/17 label protein kinase ABL1” targets/17 geneName “ABL1”
  7. 7. RELATIONSHIPS ABL1Imatinib target Legend associatedGene Gene possibleDrug Disease Compound Predicate Leukemia
  8. 8. PERFORMANCE Type-ahead search
  9. 9. select id, labelfrom targetswhere label like ‘%${queryValue}%’
  10. 10. select id, labelfrom targetswhere label like ‘%${queryValue}%’SELECT ?uri ?label WHERE { ?uri rdfs:label ?label . ?uri rdf:type drugbank:targets . FILTER regex(?label, Q${queryValue}E, i)"}
  11. 11. select id, labelfrom targetswhere label like ‘%${queryValue}%’SELECT ?uri ?label WHERE { ?uri rdfs:label ?label . ?uri rdf:type drugbank:targets . FILTER regex(?label, Q${queryValue}E, i)"} 40 million records
  12. 12. Index the data (up front performance hit)Search (really really really really fast)BuildConfig.groovy: runtime org.apache.lucene:lucene-core:3.0.1 -OR-grails install-plugin searchable
  13. 13. DE-DUPE
  14. 14. Lucene ExtensionIndex the (RDF) dataSearch (really really really really fast)
  15. 15. Understands Triples (or structured data) subject predicate object DB00619 label “Imatinib”SirenTupleQuery tupleQuery = new SirenTupleQuery()tupleQuery.add(createCellQuery(‘label’, SirenTupleConstraintPosition.PREDICATE), SirenTupleClause.Occur.MUST)tupleQuery.add(createCellQuery(‘imatinib’, SirenTupleConstraintPosition.OBJECT), SirenTupleClause.Occur.MUST)
  16. 16. GRAILS SPRING CACHE• grails install-plugin springcache• Add @Cacheable (and or @CacheFlush) annotation to services / controllers@Cacheable(somecachename)def slow(String name) { log.info "resolving $name" Thread.sleep(2000) return "took a long time to resolve ${name}"}
  17. 17. GPARS• BuildConfig.groovy: runtime org.codehaus.gpars:gpars:0.10 runtime org.coconut.forkjoin.jsr166y:jsr166y:070108• http://www.gpars.org/guide/index.html• Note:hibernate session is not available on GPars threads; you need to get one yourself (use DomainClass.withTransaction)• Data Parallelism• Map-Reduce, Fork-Join, and many more...
  18. 18. GPARSvoid resolve(scoreDocs){ scoreDocs.each {scoreDoc -> //do something in single thread }}void resolveWithPool(scoreDocs){ GParsPool.withPool { scoreDocs.eachParallel {scoreDoc -> //do something in parallel } }}
  19. 19. OTHER HANDY TOOLS• Grails Melody (Java Melody) (monitoring)• Perf4j (logging performance)• Solr (lucene search server)• Grails Console Plugin (web based console)

×