Searching for Better Code: Presented by Grant Ingersoll, Lucidworks
•
0 likes•963 views
Lucene/Solr Revolution 2015
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (15)
Similar to Searching for Better Code: Presented by Grant Ingersoll, Lucidworks
Similar to Searching for Better Code: Presented by Grant Ingersoll, Lucidworks (20)
More from Lucidworks
More from Lucidworks (20)
Recently uploaded
Recently uploaded (20)
Searching for Better Code: Presented by Grant Ingersoll, Lucidworks
- 1. O C T O B E R 1 3 - 1 6 , 2 0 1 6 • A U S T I N , T X
- 2. Searching for Better Code Grant Ingersoll CTO, Lucidworks @gsingers
- 3. 3
- 5. 5 • Languages: Java, JS/HTML/CSS, Python, Clojure • Tooling: Gradle, JIRA, Github, Jenkins, Docker, Slack • Tests: TestNG, Karma, Pytest, Manual • Lots of open source, downstream dependencies • Agile (mostly) • ~40 people, globally distributed Lucidworks Eng
- 7. Fusion: we wanted to call it Solr++, but our attorney said no! Connector Framework Index Pipelines (ETL) ( )Scale Fault Tolerance Real-Time Fusion APIs Recommendations Personalization Contextual Search Relevancy Tool Machine Learning / Signal Processing Analytics Security Apps Mobile Silk Database Web File Logs Hadoop
- 8. Make your data smarter. Ask more interesting questions. Massive scale. Real-time. Secure. Any data. Any source. Connector Framework Index Pipelines (ETL) ( )Scale Fault Tolerance Real-Time Fusion APIs Recommendations Personalization Contextual Search Relevancy Tool Machine Learning / Signal Processing Analytics Security Apps Mobile Silk Database Web File Logs Hadoop
- 9. Fusion: Solr and more Feature Solr Fusion Core Search/Faceting/… Alerting Connectors JDBC only (60+) Integrated ETL Limited Built-in Recommender Editorial Controls Limited Integrated NLP and Models Limited: UIMA Admin UI Fault tolerance/High Scale Security Limited Analytics Signal Capture and feedback Dashboards (reporting, monitoring) Big data Integrations Limited: HDFS Index Storage Log Search
- 10. Billions of Docs Optional REST Security woven throughout Proxy/LB Recs Worker Pipes Metrics NLP Sched. Blobs Msging Connectors Worker Cluster Mgr. Spark Shards Shards Solr HDFS Shared Config Mgmt Leader Election Load Balancing ZK 1 Zookeeper ZK N Signals Fusion Architecture Millions of Users
- 11. 11 • Code search • Operational problems • Download Stats Analysis • Heartbeat Use Cases: Basic Search
- 12. Demo I
- 13. 13 • Connectors • e.g. JIRA, Github/SVN, Logstash, Zendesk (2.2) • Pipelines for ETL, per doc analysis • Spark/Aggregations for large scale analysis Fusion for Code
- 14. 14 • Alerting • PagerDuty, Slack, email • Visualization: Silk built in! • Security • Kerberos, LDAP/AD, custom Fusion for Code
- 15. 15 • Code Review • Static Analysis • Headers, required configuration • Team/Developer/Commit/Bug Metrics • Trends and Hotspots • Legal issues Use Cases: Better Code
- 16. Demo II
- 17. 17 • OpenGrok Analysis coming soon (help wanted!) • http://github.com/gsingers/open-grok-analyzers • Findbugs, Linters • IT Operations • Realtime + Static • Sales/Support Operations: Zendesk, Apache Mailing Lists, JIRA Next Steps
- 18. 18 • Download Fusion 2.1: http://lucidworks.com/products/fusion/ • Check out http://github.com/LucidWorks/fusion-examples • Contact Me: • grant@lucidworks.com • @gsingers on Twitter More Info