Lex is a program that generates scanners or tokenizers to recognize lexical patterns in text. Yacc is a parser generator for Unix systems that generates code for a parser in C. Together, Lex tokenizes input strings while Yacc parses the tokenized input. A lexical analyzer breaks input into tokens, which are pairs of a token name and attribute. A syntax tree represents the abstract syntactic structure of source code with each node denoting a construct in the code.