K-Nearest Neighbor (KNN) is an instance-based learning algorithm where classification of new data points is based on the majority class of its k nearest neighbors. It works by storing all training examples and classifying new examples based on the majority class of its nearest neighbors, where distance between examples is measured using a metric like Euclidean distance. KNN can perform both classification and regression tasks, with classification being the majority class for discrete targets and regression being the average of the k nearest neighbors' values for continuous targets.