Get involved with the Apache Software Foundation

•Download as ODP, PDF•

3 likes•1,467 views

Presented at Indian Institute of Information Technology (IIIT) Allahabad on 21 Oct 2009 to students about the Apache Software Foundation, Lucene, Solr, Hadoop and on the benefits of contributing to open source projects. The target audience was sophomore, junior and senior B.Tech students.

Get involved with the Apache Software Foundation Shalin Shekhar Mangar shalin [at] apache [dot] org

1999 – The Apache Software Foundation, Tomcat, Lucene

Lucene powers search on thousands of web sites

Hadoop powers AOL, Yahoo, Facebook. Runs on thousand node clusters

Why work on Apache/OSS? ,[object Object]

Processing petabytes of data on thousands of unreliable commodity servers

At Capital One, we use Spark to detect Fraud. Recently we have started implementing real-time fraud detection using machine learnt models. One of Capital One’s fraud detection micro services was an early adopter of Structured Streaming. As part of this implementation, the micro service ran into several roadblocks. In this talk, we describe those roadblocks and how we got around them. Caching of lookup data Dilemma: Store state in Spark vs store state in database Retrieve state from database efficiently Non-homogenous data sources Aggregations in the stream Checkpointing fumbles Checkpointing performance and instabilities

Understanding the Solr security framework - Lucene Solr Revolution 2015

Anshum Gupta

Ease of use in Apache Solr

Anshum Gupta

Cross Datacenter Replication in Apache Solr 6

Shalin Shekhar Mangar

Using Apache Solr

pittaya

What's hot

Efficient Scalable Search in a Multi-Tenant Environment: Presented by Harry H...

Lucidworks

Rackspace: Email's Solution for Indexing 50K Documents per Second: Presented ...

Lucidworks

Deploying and managing Solr at scale

Anshum Gupta

Building a Large Scale SEO/SEM Application with Apache Solr: Presented by Rah...

Lucidworks

Building a Vibrant Search Ecosystem @ Bloomberg: Presented by Steven Bower & ...

Lucidworks

What's new in Solr 5.0

Anshum Gupta

Real Time search using Spark and Elasticsearch

Sigmoid

H-Hypermap - Heatmap Analytics at Scale: Presented by David Smiley, D W Smile...

Lucidworks

Searching for Better Code: Presented by Grant Ingersoll, Lucidworks

Lucidworks

Lessons From Sharding Solr At Etsy: Presented by Gregg Donovan, Etsy

Lucidworks

Webinar: Replace Google Search Appliance with Lucidworks Fusion

Lucidworks

Cross Data Center Replication for the Enterprise: Presented by Adam Williams,...

Lucidworks

What's New in Apache Solr 4.10

Anshum Gupta

Data Engineering with Solr and Spark

Lucidworks

Never Stop Exploring - Pushing the Limits of Solr: Presented by Anirudha Jadh...

Lucidworks

Simple Fuzzy Name Matching in Solr: Presented by Chris Mack, Basis Technology

Lucidworks

Searching The Enterprise Data Lake With Solr - Watch Us Do It!: Presented by...

Lucidworks

Whirlpools in the Stream with Jayesh Lalwani

Databricks

Understanding the Solr security framework - Lucene Solr Revolution 2015

Anshum Gupta

Ease of use in Apache Solr

Anshum Gupta

What's hot (20)

Efficient Scalable Search in a Multi-Tenant Environment: Presented by Harry H...

Rackspace: Email's Solution for Indexing 50K Documents per Second: Presented ...

Deploying and managing Solr at scale

Building a Large Scale SEO/SEM Application with Apache Solr: Presented by Rah...

Building a Vibrant Search Ecosystem @ Bloomberg: Presented by Steven Bower & ...

What's new in Solr 5.0

Real Time search using Spark and Elasticsearch

H-Hypermap - Heatmap Analytics at Scale: Presented by David Smiley, D W Smile...

Searching for Better Code: Presented by Grant Ingersoll, Lucidworks

Lessons From Sharding Solr At Etsy: Presented by Gregg Donovan, Etsy

Webinar: Replace Google Search Appliance with Lucidworks Fusion

Cross Data Center Replication for the Enterprise: Presented by Adam Williams,...

What's New in Apache Solr 4.10

Data Engineering with Solr and Spark

Never Stop Exploring - Pushing the Limits of Solr: Presented by Anirudha Jadh...

Simple Fuzzy Name Matching in Solr: Presented by Chris Mack, Basis Technology

Searching The Enterprise Data Lake With Solr - Watch Us Do It!: Presented by...

Whirlpools in the Stream with Jayesh Lalwani

Understanding the Solr security framework - Lucene Solr Revolution 2015

Ease of use in Apache Solr

Viewers also liked

Cross Datacenter Replication in Apache Solr 6

Shalin Shekhar Mangar

Using Apache Solr

pittaya

Apache Solr crash courseTommaso Teofili

Improving the Solr Update Chain

Cominvent AS

Solr 4

Erik Hatcher

Solr 4.0 dramatically improves scalability, performance, and flexibility. An overhauled Lucene underneath sports near real-time (NRT) capabilities allowing indexed documents to be rapidly visible and searchable. Lucene’s improvements also include pluggable scoring, much faster fuzzy and wildcard querying, and vastly improved memory usage. These Lucene improvements automatically make Solr much better, and Solr magnifies these advances with “SolrCloud.” SolrCloud enables highly available and fault tolerant clusters for large scale distributed indexing and searching. There are many other changes that will be surveyed as well. This talk will cover these improvements in detail, comparing and contrasting to previous versions of Solr.

Apache Solr Changes the Way You Build Sites

Peter

Solr 6 Feature Preview

Yonik Seeley

Lucene for Solr DevelopersErik Hatcher

Сергей Моренец: "Gradle. Write once, build everywhere"

Provectus

Meet Solr For The Tirst Again

Varun Thacker

What's New in Solr 3.x / 4.0Erik Hatcher

"Solr Update" at code4lib '13 - Chicago

Erik Hatcher

Solr is continually improving. Solr 4 was recently released, bringing dramatic changes in the underlying Lucene library and Solr-level features. It's tough for us all to keep up with the various versions and capabilities. This talk will blaze through the highlights of new features and improvements in Solr 4 (and up). Topics will include: SolrCloud, direct spell checking, surround query parser, and many other features. We will focus on the features library coders really need to know about.

Open source applied: Real-world uses

Rogue Wave Software

Why I want to Kazan

Provectus

Lucene for Solr DevelopersErik Hatcher

Lucene's Latest (for Libraries)

Erik Hatcher

Solr Black Belt Pre-conferenceErik Hatcher

Gimme shelter: Tips on protecting proprietary and open source code

Rogue Wave Software

Presented at ESC Minneapolis - September 2016. This presentation aims to train and retain by walking through real examples of the top security defects and open source liability issues for embedded systems today. Based on data from the National Vulnerability Database and recent court decisions, attendees will be exposed to lesser known but vital research on security and licensing to better prepare their teams to combat risks.

Solr Masterclass Bangkok, June 2014

Alexandre Rafalovitch

Call me maybe: Jepsen and flaky networks

Shalin Shekhar Mangar

In the big data world, our data stores communicate over an asynchronous, unreliable network to provide a facade of consistency. However, to really understand the guarantees of these systems, we must understand the realities of networks and test our data stores against them. Jepsen is a tool which simulates network partitions in data stores and helps us understand the guarantees of our systems and its failure modes. In this talk, I will help you understand why you should care about network partitions and how can we test datastores against partitions using Jepsen. I will explain what Jepsen is and how it works and the kind of tests it lets you create. We will try to understand the subtleties of distributed consensus, the CAP theorem and demonstrate how different data stores such as MongoDB, Cassandra, Elastic and Solr behave under network partitions. Finally, I will describe the results of the tests I wrote using Jepsen for Apache Solr and discuss the kinds of rare failures which were found by this excellent tool.

Viewers also liked (20)

Cross Datacenter Replication in Apache Solr 6

Using Apache Solr

Apache Solr crash course

Improving the Solr Update Chain

Solr 4

Apache Solr Changes the Way You Build Sites

Solr 6 Feature Preview

Lucene for Solr Developers

Сергей Моренец: "Gradle. Write once, build everywhere"

Meet Solr For The Tirst Again

What's New in Solr 3.x / 4.0

"Solr Update" at code4lib '13 - Chicago

Open source applied: Real-world uses

Why I want to Kazan

Lucene for Solr Developers

Lucene's Latest (for Libraries)

Solr Black Belt Pre-conference

Gimme shelter: Tips on protecting proprietary and open source code

Solr Masterclass Bangkok, June 2014

Call me maybe: Jepsen and flaky networks

Similar to Get involved with the Apache Software Foundation

Architecting the Future of Big Data and Search

Hortonworks

963Annu Ahmed

How Hadoop Revolutionized Data Warehousing at Yahoo and Facebook

Amr Awadallah

Predictive Analytics and Machine Learning…with SAS and Apache Hadoop

Hortonworks

The other Apache Technologies your Big Data solution needs

gagravarr

Hw09 Cross Data Center Logs ProcessingCloudera, Inc.

Big Data and Hadoop

Flavio Vit

The other Apache technologies your big data solution needs!

gagravarr

Hadoop Demystified + MapReduce (Java and C#), Pig, and Hive Demos

Lester Martin

Hadoop

Zubair Arshad

Big Data Concepts

Ahmed Salman

TriHUG: Lucene Solr Hadoop

Grant Ingersoll

Big Data, Ingeniería de datos, y Data Lakes en AWS

javier ramirez

Intro to Apache Lucene and Solr

Grant Ingersoll

Discover.hdp2.2.storm and kafka.final

Hortonworks

Hadoop Frameworks Panel__HadoopSummit2010

Yahoo Developer Network

Atlanta Data Science Meetup | Qubole slides

Qubole

Apache Spark: killer or savior of Apache Hadoop?

rhatr

Data infrastructure at Facebook

AhmedDoukh

What is Apache Hadoop and its ecosystem?

tommychauhan

Similar to Get involved with the Apache Software Foundation (20)

Architecting the Future of Big Data and Search

963

How Hadoop Revolutionized Data Warehousing at Yahoo and Facebook

Predictive Analytics and Machine Learning…with SAS and Apache Hadoop

The other Apache Technologies your Big Data solution needs

Hw09 Cross Data Center Logs Processing

Big Data and Hadoop

The other Apache technologies your big data solution needs!

Hadoop Demystified + MapReduce (Java and C#), Pig, and Hive Demos

Hadoop

Big Data Concepts

TriHUG: Lucene Solr Hadoop

Big Data, Ingeniería de datos, y Data Lakes en AWS

Intro to Apache Lucene and Solr

Discover.hdp2.2.storm and kafka.final

Hadoop Frameworks Panel__HadoopSummit2010

Atlanta Data Science Meetup | Qubole slides

Apache Spark: killer or savior of Apache Hadoop?

Data infrastructure at Facebook

What is Apache Hadoop and its ecosystem?

More from Shalin Shekhar Mangar

Solr BoF (Birds of a Feather) session at Fifth Elephant 2018

Shalin Shekhar Mangar

Parallel SQL and Streaming Expressions in Apache Solr 6

Shalin Shekhar Mangar

Apache Solr is a powerful search and analytics engine with features such as full-text search, faceting, joins, sorting and capable of handling large amounts of data across a large number of servers. However, with all that power and scalability comes complexity. Solr 6 supports a Parallel SQL feature which provides a simplified, well-known interface to your data in Solr, performs key operations such as sorts and shuffling inside Solr for massive speedups, provides best-practices based query optimization and by leveraging the scalability of SolrCloud and a clever implementation, allows you to throw massive amounts of computation power behind analytical queries. In this talk, we will explore the why, what and how of Parallel SQL and its building block Streaming Expressions in Solr 6 with a hint of the exciting new developments around this feature.

Intro to Apache Solr

Shalin Shekhar Mangar

Scaling SolrCloud to a Large Number of Collections - Fifth Elephant 2014

Shalin Shekhar Mangar

The traditional and typical search use case is the one large search collection distributed among many nodes and shared by all users. However, there is a class of applications which need a large number of small or medium collections which can be used, managed and scaled separately. This talk will cover our effort in helping a client set up a large scale SolrCloud setup with thousands of collections running on hundreds of nodes. I will describe the bottlenecks that we found in SolrCloud when running a large number of collections. I will also take you through the multiple features and optimizations that we contributed to Apache Solr to reduce or remove the choke points in the system. Finally, I will talk about the benchmarking process and the lessons learned from supporting such an installation in production.

High Performance Solr

Shalin Shekhar Mangar

GIDS2014: SolrCloud: Searching Big Data

Shalin Shekhar Mangar

These slides were presented at the Great Indian Developer Summit 2014 at Bangalore. See http://www.developermarch.com/developersummit/session.html?insert=ShalinMangar2 "SolrCloud" is the name given to Apache Solr's feature set for fault tolerant, highly available, and massively scalable capabilities. SolrCloud has enabled organizations to scale, impressively, into the billions of documents with sub-second search!

Introduction to Apache Solr

Shalin Shekhar Mangar

More from Shalin Shekhar Mangar (7)

Solr BoF (Birds of a Feather) session at Fifth Elephant 2018

Parallel SQL and Streaming Expressions in Apache Solr 6

Intro to Apache Solr

Scaling SolrCloud to a Large Number of Collections - Fifth Elephant 2014

High Performance Solr

GIDS2014: SolrCloud: Searching Big Data

Introduction to Apache Solr

Recently uploaded

UiPath Test Automation using UiPath Test Suite series, part 4

DianaGray10

Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap. The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies. Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques What will you get from this session? 1. Insights into SAP testing best practices 2. Heatmap utilization for testing 3. Optimization of testing processes 4. Demo Topics covered: Execution from the test manager Orchestrator execution result Defect reporting SAP heatmap example with demo Speaker: Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP

Communications Mining Series - Zero to Hero - Session 1

DianaGray10

This session provides introduction to UiPath Communication Mining, importance and platform overview. You will acquire a good understand of the phases in Communication Mining as we go over the platform with you. Topics covered: • Communication Mining Overview • Why is it important? • How can it help today’s business and the benefits • Phases in Communication Mining • Demo on Platform overview • Q/A

Transcript: Selling digital books in 2024: Insights from industry leaders - T...

BookNet Canada

The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more. Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/ Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.

FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf

FIDO Alliance

Removing Uninteresting Bytes in Software Fuzzing

Aftab Hussain

Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process. In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds. - These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.

Microsoft - Power Platform_G.Aspiotis.pdf

Uni Systems S.M.S.A.

Video Streaming: Then, Now, and in the Future

Alpen-Adria-Universität

In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.

Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...

SOFTTECHHUB

The choice of an operating system plays a pivotal role in shaping our computing experience. For decades, Microsoft's Windows has dominated the market, offering a familiar and widely adopted platform for personal and professional use. However, as technological advancements continue to push the boundaries of innovation, alternative operating systems have emerged, challenging the status quo and offering users a fresh perspective on computing. One such alternative that has garnered significant attention and acclaim is Nitrux Linux 3.5.0, a sleek, powerful, and user-friendly Linux distribution that promises to redefine the way we interact with our devices. With its focus on performance, security, and customization, Nitrux Linux presents a compelling case for those seeking to break free from the constraints of proprietary software and embrace the freedom and flexibility of open-source computing.

Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf

Paige Cruz

Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack. While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack. I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:

20240607 QFM018 Elixir Reading List May 2024

Matthew Sinclair

SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdf

Peter Spielvogel

Building better applications for business users with SAP Fiori. • What is SAP Fiori and why it matters to you • How a better user experience drives measurable business benefits • How to get started with SAP Fiori today • How SAP Fiori elements accelerates application development • How SAP Build Code includes SAP Fiori tools and other generative artificial intelligence capabilities • How SAP Fiori paves the way for using AI in SAP apps

The Art of the Pitch: WordPress Relationships and Sales

Laura Byrne

Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes? All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.

Climate Impact of Software Testing at Nordic Testing Days

Kari Kakkonen

My slides at Nordic Testing Days 6.6.2024 Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.

Pushing the limits of ePRTC: 100ns holdover for 100 days

Adtran

Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf

91mobiles

FIDO Alliance Osaka Seminar: Overview.pdf

FIDO Alliance

UiPath Test Automation using UiPath Test Suite series, part 5

DianaGray10

Elizabeth Buie - Older adults: Are we really designing for our future selves?

Nexer Digital

GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...

Neo4j

Dr. Sean Tan, Head of Data Science, Changi Airport Group Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.

Generative AI Deep Dive: Advancing from Proof of Concept to Production

Aggregage