sigmod08

This document introduces Pig Latin, a new language designed for analyzing extremely large datasets. Pig Latin aims to fill the gap between the declarative style of SQL and the procedural style of MapReduce. It compiles programs into physical plans executed over Hadoop. The language allows for a flexible data model, user-defined functions, and operates directly on files without requiring data import. Pig Latin is being used by engineers at Yahoo to more easily analyze terabytes of collected data.