● What is it ?
● How does it work ?
● Why use it ?
● PigLatin Data Types
● PigLatin Maths
● PigLatin Example
Pig – What is it ?
● A high level language
● Used to analyse large data sets
● Used to create MapReduce jobs
● Abstracts definition of jobs
● Uses Pig Latin to define jobs
● Less code needed
● Compiles to MapReduce code
Pig – How does it work ?
● Three ways to use it
– Grunt – Pig's interactive shell
– Write Pig Latin in a script file
– Embed Pig commands in another language
● Run modes
– Local mode – single machine
– Hadoop – run on a Hadoop/MapReduce cluster
● Creates MapReduce code automatically
Pig – Why use it ?
● It is quicker
● It is data omnivorous
● It is easy to learn
● It is widely used
● Minor performance loss
– Compared to native code
● It can be extended via user defined functions ( UDF )
PigLatin Data Types
Some of the built in maths functions
Example borrowed from Wikipedia
input_lines = LOAD '/tmp/my-copy-of-all-pages-on-internet' AS (line:chararray);
-- Extract words from each line and put them into a pig bag
-- datatype, then flatten the bag to get one word on each row
words = FOREACH input_lines GENERATE FLATTEN(TOKENIZE(line)) AS word;
-- filter out any words that are just white spaces
filtered_words = FILTER words BY word MATCHES 'w+';
-- create a group for each word
word_groups = GROUP filtered_words BY word;
-- count the entries in each group
word_count = FOREACH word_groups GENERATE COUNT(filtered_words) AS count, group AS word;
-- order the records by count
ordered_word_count = ORDER word_count BY count DESC;
STORE ordered_word_count INTO '/tmp/number-of-words-on-internet';
● Feel free to contact us at
● We offer IT project consultancy
● We are happy to hear about your problems
● You can just pay for those hours that you need
● To solve your problems
A particular slide catching your eye?
Clipping is a handy way to collect important slides you want to go back to later.