Up-Armoring The Elephant: Adding Kerberos-based Security to Hadoop

•Download as PPTX, PDF•

6 likes•1,984 views

blueboxtraveler

Presented at HadoopDay, Seattle, 2010.

Technology

Up-Armoring the Elephant Secure Hadoop is Here Jakob Homan jhoman@yahoo-inc.com

As of 2009, 72% percent of patches going into the Hadoop source code were coming from Yahoo! Developing Hadoop at Yahoo! 8/14/10 4

Yahoo! provides extensive QE and QA resources to test Hadoop releases at scale. Developing Hadoop at Yahoo! 8/14/10 5

Developing Hadoop at Yahoo! 8/14/10 6 The Yahoo! distribution of Hadoop, available on Github, is the same code we run internally on our servers. Patches important to stability and performance and stability are applied here, as well as Apache.

Developing Hadoop at Yahoo! 8/14/10 7 The rest of the family

Why do we need a secure Hadoop? 8/14/10 9

Current state of security in Hadoop 8/14/10 10

Current state of security in Hadoop 8/14/10 11 Bowser copyright Nintendo

Which can let bad people do bad things 8/14/10 13

RPC upgraded to use SASL/GSSAPI 8/14/10 17

What does a secure Hadoop look like? 8/14/10 18

Additional security throughout system 8/14/10 21

How do I write a secure MapReduce job? 8/14/10 22

Significant user-facing changes 8/14/10 24

Secure web access is pluggable 8/14/10 25

DistCP works… in 3 out of 4 cases 8/14/10 26

Take security for a test drive 8/14/10 29

Or build a secure cluster at home 8/14/10 30

Similar to Up-Armoring The Elephant: Adding Kerberos-based Security to Hadoop

The Evolution and Future of Hadoop Storage （Hadoop Conference Japan 2016キーノート...

Hadoop / Spark Conference Japan

Hadoop's Impact on the Future of Data Management | Amr Awadallah

Cloudera, Inc.

Hortonworks Hadoop summit 2011 keynote - eric14

Hortonworks

Hortonworks for Financial Analysts Presentation

Hortonworks

The Big Picture on Hadoop

StackIQ

Hortonworks continues to innovate throughout all Hadoop related projects, packaging the most enterprise-ready components, such as Ambari, into the Hortonworks Data Platform (HDP). Please join us in this interactive webinar as we present real-world use cases of Enterprise customers that are finding success with HDP and their Big Data initiatives. We will also introduce new features from version 1.2 of the Hortonworks Data Platform and how it has become the leading 100% open source distribution choice for the Enterprise. In this webinar we will outline how enterprise customers are successfult with HDP and also review some of the newest features in version1.2 including: -How to provision a cluster -How to manage and monitor a cluster using completely open source tools -How to perform diagnostics to identify issues in a cluster

Hadoop Operations, Innovations and Enterprise Readiness with Hortonworks Data...

Hortonworks

2014 feb 24_big_datacongress_hadoopsession1_hadoop101

Adam Muise

Open Source examples from Adobe : Oscon kiosk

Dave McAllister

big data

Arohi Khandelwal

Introduction of Big data and Hadoop

Arohi Khandelwal

Apache Hadoop at 10

Cloudera, Inc.

Don't Let Security Be The 'Elephant in the Room'

Hortonworks

Hue: The Hadoop UI - Hadoop Singapore

gethue

Dallas TDWI Meeting Dec. 2012: Hadoop

lamont_lockwood

Unattended Apache BigTop installer CD using preseed

Jazz Yao-Tsung Wang

Hadoop Present - Open Enterprise Hadoop

Yifeng Jiang

In 2012, we released Hortonworks Data Platform powered by Apache Hadoop and established partnerships with major enterprise software vendors including Microsoft and Teradata that are making enterprise ready Hadoop easier and faster to consume. As we start 2013, we invite you to join us for this live webinar where Shaun Connolly, VP of Strategy at Hortonworks, will cover the highlights of 2012 and the road ahead in 2013 for Hortonworks and Apache Hadoop.

State of the Union with Shaun Connolly

Hortonworks

Hw09 Hadoop Applications At Yahoo!

Cloudera, Inc.

Hadoop at Yahoo! -- Hadoop World NY 2009

yhadoop

E2Matrix Jalandhar provides Best Big Data training based on current industry standards that helps attendees to secure placements in their dream jobs at MNCs. E2Matrix Provides Best Big Data Training in Jalandhar Amritsar Ludhiana Phagwara Mohali Chandigarh. E2Matrix is one of the best Big Data training institute offering hands on practical knowledge. At E2Matrix Big Data training is conducted by subject specialist corporate professionals best experience in managing real-time Big Data projects. E2Matrix implements a blend of academic learning and practical sessions to give the student optimum exposure. At E2Matrix’s well-equipped Big Data training Institute aspirants learn the skills for Big Data Overview, Use Cases, Data Analytics Process, Data Preparation, Tools for Data Preparation, Hands on Exercise : Using SQL and NoSql DB's, Hands on Exercise : Usage of Tools, Data Analysis Introduction, Classification, Data Visualization using R, Automation Testing Training on real time projects.

Big Data Training in Mohali

E2MATRIX

Similar to Up-Armoring The Elephant: Adding Kerberos-based Security to Hadoop (20)

The Evolution and Future of Hadoop Storage （Hadoop Conference Japan 2016キーノート...

Hadoop's Impact on the Future of Data Management | Amr Awadallah

Hortonworks Hadoop summit 2011 keynote - eric14

Hortonworks for Financial Analysts Presentation

The Big Picture on Hadoop

Hadoop Operations, Innovations and Enterprise Readiness with Hortonworks Data...

2014 feb 24_big_datacongress_hadoopsession1_hadoop101

Open Source examples from Adobe : Oscon kiosk

big data

Introduction of Big data and Hadoop

Apache Hadoop at 10

Don't Let Security Be The 'Elephant in the Room'

Hue: The Hadoop UI - Hadoop Singapore

Dallas TDWI Meeting Dec. 2012: Hadoop

Unattended Apache BigTop installer CD using preseed

Hadoop Present - Open Enterprise Hadoop

State of the Union with Shaun Connolly

Hw09 Hadoop Applications At Yahoo!

Hadoop at Yahoo! -- Hadoop World NY 2009

Big Data Training in Mohali

Recently uploaded

Data Cloud, More than a CDP by Matt Robison

Anna Loughnan Colquhoun

[2024]Digital Global Overview Report 2024 Meltwater.pdf

hans926745

Handwritten Text Recognition for manuscripts and early printed texts

Maria Levchenko

Enterprise Knowledge’s Urmi Majumder, Principal Data Architecture Consultant, and Fernando Aguilar Islas, Senior Data Science Consultant, presented "Driving Behavioral Change for Information Management through Data-Driven Green Strategy" on March 27, 2024 at Enterprise Data World (EDW) in Orlando, Florida. In this presentation, Urmi and Fernando discussed a case study describing how the information management division in a large supply chain organization drove user behavior change through awareness of the carbon footprint of their duplicated and near-duplicated content, identified via advanced data analytics. Check out their presentation to gain valuable perspectives on utilizing data-driven strategies to influence positive behavioral shifts and support sustainability initiatives within your organization. In this session, participants gained answers to the following questions: - What is a Green Information Management (IM) Strategy, and why should you have one? - How can Artificial Intelligence (AI) and Machine Learning (ML) support your Green IM Strategy through content deduplication? - How can an organization use insights into their data to influence employee behavior for IM? - How can you reap additional benefits from content reduction that go beyond Green IM?

Driving Behavioral Change for Information Management through Data-Driven Gree...

Enterprise Knowledge

Axa Assurance Maroc - Insurer Innovation Award 2024

The Digital Insurer

Building Digital Trust in a Digital Economy Veronica Tan, Director - Cyber Security Agency of Singapore Apidays Singapore 2024: Connecting Customers, Business and Technology (April 17 & 18, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...

apidays

Real Time Object Detection Using Open CV

Khem

As privacy and data protection regulations evolve rapidly, organizations operating in multiple jurisdictions face mounting challenges to ensure compliance and safeguard customer data. With state-specific privacy laws coming up in multiple states this year, it is essential to understand what their unique data protection regulations will require clearly. How will data privacy evolve in the US in 2024? How to stay compliant? Our panellists will guide you through the intricacies of these states' specific data privacy laws, clarifying complex legal frameworks and compliance requirements. This webinar will review: - The essential aspects of each state's privacy landscape and the latest updates - Common compliance challenges faced by organizations operating in multiple states and best practices to achieve regulatory adherence - Valuable insights into potential changes to existing regulations and prepare your organization for the evolving landscape

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

TrustArc

Presentation on how to chat with PDF using ChatGPT code interpreter

naman860154

CNv6 Instructor Chapter 6 Quality of Service

giselly40

Boost Fertility New Invention Ups Success Rates.pdf

sudhanshuwaghmare1

The Raspberry Pi 5 was announced on October 2023. This new version of the popular embedded device comes with a new iteration of Broadcom’s VideoCore GPU platform, and was released with a fully open source driver stack, developed by Igalia. The presentation will discuss some of the major changes required to support this new Video Core iteration, the challenges we faced in the process and the solutions we provided in order to deliver conformant OpenGL ES and Vulkan drivers. The talk will also cover the next steps for the open source Raspberry Pi 5 graphics stack. (c) Embedded Open Source Summit 2024 April 16-18, 2024 Seattle, Washington (US) https://events.linuxfoundation.org/embedded-open-source-summit/ https://eoss24.sched.com/event/1aBEx

Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...

Igalia

The presentation explores the development and application of artificial intelligence (AI) from its inception to its current status in the modern world. The term "artificial intelligence" was first coined by John McCarthy in 1956 to describe efforts to develop computer programs capable of performing tasks that typically require human intelligence. This concept was first introduced at a conference held at Dartmouth College, where programs demonstrated capabilities such as playing chess, proving theorems, and interpreting texts. In the early stages, Alan Turing contributed to the field by defining intelligence as the ability of a being to respond to certain questions intelligently, proposing what is now known as the Turing Test to evaluate the presence of intelligent behavior in machines. As the decades progressed, AI evolved significantly. The 1980s focused on machine learning, teaching computers to learn from data, leading to the development of models that could improve their performance based on their experiences. The 1990s and 2000s saw further advances in algorithms and computational power, which allowed for more sophisticated data analysis techniques, including data mining. By the 2010s, the proliferation of big data and the refinement of deep learning techniques enabled AI to become mainstream. Notable milestones included the success of Google's AlphaGo and advancements in autonomous vehicles by companies like Tesla and Waymo. A major theme of the presentation is the application of generative AI, which has been used for tasks such as natural language text generation, translation, and question answering. Generative AI uses large datasets to train models that can then produce new, coherent pieces of text or other media. The presentation also discusses the ethical implications and the need for regulation in AI, highlighting issues such as privacy, bias, and the potential for misuse. These concerns have prompted calls for comprehensive regulations to ensure the safe and equitable use of AI technologies. Artificial intelligence has also played a significant role in healthcare, particularly highlighted during the COVID-19 pandemic, where it was used in drug discovery, vaccine development, and analyzing the spread of the virus. The capabilities of AI in healthcare are vast, ranging from medical diagnostics to personalized medicine, demonstrating the technology's potential to revolutionize fields beyond just technical or consumer applications. In conclusion, AI continues to be a rapidly evolving field with significant implications for various aspects of society. The development from theoretical concepts to real-world applications illustrates both the potential benefits and the challenges that come with integrating advanced technologies into everyday life. The ongoing discussion about AI ethics and regulation underscores the importance of managing these technologies responsibly to maximize their their benefits while minimizing potential harms.

Artificial Intelligence: Facts and Myths

Joaquim Jorge

A Call to Action for Generative AI in 2024

Results

Automating Google Workspace (GWS) & more with Apps Script

wesley chun

BooK Now Call us at +918448380779 to hire a gorgeous and seductive call girl for sex. Take a Delhi Escort Service. The help of our escort agency is mostly meant for men who want sexual Indian Escorts In Delhi NCR. It should be noted that any impersonator will get 100 attention from our Young Girls Escorts in Delhi. They will assume the position of reliable allies. VIP Call Girl With Original Photos Book Tonight +918448380779 Our Cheap Price 1 Hour not available 2 Hours 5000 Full Night 8000 TAG: Call Girls in Delhi, Noida, Gurgaon, Ghaziabad, Connaught Place, Greater Kailash Delhi, Lajpat Nagar Delhi, Mayur Vihar Delhi, Chanakyapuri Delhi, New Friends Colony Delhi, Majnu Ka Tilla, Karol Bagh, Malviya Nagar, Saket, Khan Market, Noida Sector 18, Noida Sector 76, Noida Sector 51, Gurgaon Mg Road, Iffco Chowk Gurgaon, Rajiv Chowk Gurgaon All Delhi Ncr Free Home Deliver

08448380779 Call Girls In Greater Kailash - I Women Seeking Men

Delhi Call girls

How to Troubleshoot Apps for the Modern Connected Worker

ThousandEyes

Explore 'The Codex of Business: Writing Software for Real-World Solutions,' a compelling SlideShare presentation that delves into digital transformation in healthcare. Discover through a detailed case study how Agile methodologies empower healthcare providers to develop, iterate, and refine digital solutions that address real-world challenges. Learn how strategic planning, user feedback, and continuous improvement drive success in deploying technologies that enhance patient care and operational efficiency. Ideal for healthcare professionals, IT specialists, and digital transformation advocates seeking actionable insights and practical examples of technology making a real difference.

The Codex of Business Writing Software for Real-World Solutions 2.pptx

Malak Abu Hammad

🐬 The future of MySQL is Postgres 🐘

RTylerCroy

08448380779 Call Girls In Diplomatic Enclave Women Seeking Men

Delhi Call girls

Recently uploaded (20)

Data Cloud, More than a CDP by Matt Robison

[2024]Digital Global Overview Report 2024 Meltwater.pdf

Handwritten Text Recognition for manuscripts and early printed texts

Driving Behavioral Change for Information Management through Data-Driven Gree...

Axa Assurance Maroc - Insurer Innovation Award 2024

Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...

Real Time Object Detection Using Open CV

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

Presentation on how to chat with PDF using ChatGPT code interpreter

CNv6 Instructor Chapter 6 Quality of Service

Boost Fertility New Invention Ups Success Rates.pdf

Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...

Artificial Intelligence: Facts and Myths

A Call to Action for Generative AI in 2024

Automating Google Workspace (GWS) & more with Apps Script

08448380779 Call Girls In Greater Kailash - I Women Seeking Men

How to Troubleshoot Apps for the Modern Connected Worker

The Codex of Business Writing Software for Real-World Solutions 2.pptx

🐬 The future of MySQL is Postgres 🐘

08448380779 Call Girls In Diplomatic Enclave Women Seeking Men

Up-Armoring The Elephant: Adding Kerberos-based Security to Hadoop

1. Up-Armoring the Elephant Secure Hadoop is Here Jakob Homan jhoman@yahoo-inc.com

2. Who I am 8/14/10 2

3. Using Hadoop at Yahoo! 8/14/10 3

4. As of 2009, 72% percent of patches going into the Hadoop source code were coming from Yahoo! Developing Hadoop at Yahoo! 8/14/10 4

5. Yahoo! provides extensive QE and QA resources to test Hadoop releases at scale. Developing Hadoop at Yahoo! 8/14/10 5

6. Developing Hadoop at Yahoo! 8/14/10 6 The Yahoo! distribution of Hadoop, available on Github, is the same code we run internally on our servers. Patches important to stability and performance and stability are applied here, as well as Apache.

7. Developing Hadoop at Yahoo! 8/14/10 7 The rest of the family

8. Hadoop at Yahoo! Sunnyvale 8/14/10 8

9. Why do we need a secure Hadoop? 8/14/10 9

10. Current state of security in Hadoop 8/14/10 10

11. Current state of security in Hadoop 8/14/10 11 Bowser copyright Nintendo

12. The elephant is too trusting 8/14/10 12

13. Which can let bad people do bad things 8/14/10 13

14. Why is securing Hadoop hard? 8/14/10 14

15. Enter Kerberos! 8/14/10 15

16. Kerberos workflow 8/14/10 16

17. RPC upgraded to use SASL/GSSAPI 8/14/10 17

18. What does a secure Hadoop look like? 8/14/10 18

19. Like this 8/14/10 19

20. Everyone now authenticated 8/14/10 20

21. Additional security throughout system 8/14/10 21

22. How do I write a secure MapReduce job? 8/14/10 22

23. This is how 8/14/10 23 Nochanges!

24. Significant user-facing changes 8/14/10 24

25. Secure web access is pluggable 8/14/10 25

26. DistCP works… in 3 out of 4 cases 8/14/10 26

27. Out of scope 8/14/10 27

28. Impact on performance 8/14/10 28

29. Take security for a test drive 8/14/10 29

30. Or build a secure cluster at home 8/14/10 30

31. Other projects and security 8/14/10 31

32. Current state 8/14/10 32

33. Current state 8/14/10 33

34. Security list 8/14/10 34

35. Questions? 8/14/10 35

Editor's Notes

Discussion of how security was not a huge priority. File system permissions not added until 17.
Kerberos was chosen because it’s a tested, trusted solution. In use at Yahoo! already. Hadoop actors – users, servers such as NameNode, JobTracker, DNs, TTs all authenticate with Kerberos as principals. This allows Hadoop, for the first time to be able to trust the identity of its various components.
Kerberos provides single-sign-on serviceKinit, kdestroyCan be configured to automatically initialize via PAMBy default last 10 hours, renewable 7 days
Overall, the entire ship has been tightened. We believe that we’ve secured each of the data access points that were shown in the big scary picture previously.
* Secure Distributed Cache
For the majority of jobs, there will be no changes necessary to run under security. It was important to make the switch to security as painless as possible since there are already thousands of different jobs running on our clusters, hundreds of thousands around the world and those needed to continue to run. Also, user education is very difficult.

Up-Armoring The Elephant: Adding Kerberos-based Security to Hadoop

Recommended

Recommended

More Related Content

Similar to Up-Armoring The Elephant: Adding Kerberos-based Security to Hadoop

Similar to Up-Armoring The Elephant: Adding Kerberos-based Security to Hadoop (20)

Recently uploaded

Recently uploaded (20)

Up-Armoring The Elephant: Adding Kerberos-based Security to Hadoop

Editor's Notes