- Yahoo needed to secure its Hadoop clusters to isolate different users' data and prevent impersonation, as it had hundreds of users sharing 38,000 computers across ~20 clusters. - The solution introduced Kerberos authentication to Hadoop to provide distributed authentication, integrating it with HDFS, MapReduce, and other components. Authorization was also added through access control lists. - The changes were made largely transparent to users and backwards compatible to allow signing in once with Kerberos and maintaining access, while securing various communication paths and APIs. This allowed for secure sharing of clusters while keeping options for non-shared clusters.

1. Hadoop Security Hadoop Summit 2010 Owen O’Malley [email_address] Yahoo’s Hadoop Team

6. Kerberos Dataflow

9. Primary Communication Paths