Finite automata are the simplest machines for recognizing patterns, consisting of states that determine if an input string is accepted or rejected. The formal definition includes a 5-tuple comprising states, input symbols, initial and final states, and a transition function. Differences between deterministic finite automata (DFA) and non-deterministic finite automata (NFA) highlight that all DFAs are NFAs, but the reverse is not true, with DFAs commonly used in lexical analysis.