Shellsort is a sorting algorithm invented by Donald Shell in 1959 that was the first to break the quadratic time barrier of simpler sorting algorithms like insertion sort. It works by sorting elements with increasing proximity over multiple passes rather than just adjacent elements. The algorithm uses an increment sequence to determine the spacing between elements to compare and sort in each pass until the final pass sorts adjacent elements like an insertion sort. While faster than older quadratic algorithms, shellsort is still outperformed by more efficient algorithms like merge, heap, and quicksort for larger data sets.