Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Kernel Development: Drawing Lessons from "Mistakes" (Japan Linux Symposium 2009)

6,511 views

Published on

Every kernel developer knows that Linux comes with plenty of precious documentation as an integral part. From coding style to how to post patches, almost everything has been documented. However, history shows that error is human nature. Sometimes developers do not well know Don’ts, but there are also cases when they make mistakes despite being aware of such rules. Why this happen is unsolved, but a documentation, so far missing, of the consequences of this misbehavior could discourage it. The presenter is project manager of TOMOYO Linux, a security enhancement feature merged in version 2.6.30. Thinking open-minded, he decided to share the errors his project made, wishing it could be a helpful warning to other projects, especially newcomers. In this presentation, it will try to explain the mistake circumstances in TOMOYO Linux project, highlighting the thoughts of project members and the community reactions. No prior kernel development experiences are necessary.

Published in: Technology, News & Politics
  • Hello my dear
    I am Modester by name good day. i just went to your profile this time true this site (www.slideshare.net) and i got your detail and your explanation in fact the way you explain your self shows me that you are innocent and maturity and also understand person i decided to have a contact with you so that we can explain to our self each other because God great everyone to make a friend with each other and from that we know that we are from thism planet God great for us ok my dear please try and reach me through my email address (modester4life4@yahoo.com) so that i can send you my picture true your reply we can know each other ok have a nice day and God bless you yours Modester
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Kernel Development: Drawing Lessons from "Mistakes" (Japan Linux Symposium 2009)

  1. 1. KERNEL DEVELOPMENT: DRAWING LESSONS FROM "MISTAKES" Japan Linux Symposium 2009 October 23, 2009 Toshiharu Harada haradats@nttdata.co.jp NTT DATA CORPORATION
  2. 2. ABSTRACTS Every kernel developer knows that Linux comes with plenty of precious documentation as an integral part. From coding style to how to post patches, almost everything has been documented. However, history shows that error is human nature. Sometimes developers do not well know Don’ts, but there are also cases when they make mistakes despite being aware of such rules. Why this happen is unsolved, but a documentation, so far missing, of the consequences of this misbehavior could discourage it. The presenter is project manager of TOMOYO Linux, a security enhancement feature merged in version 2.6.30. Thinking open-minded, he decided to share the errors his project made, wishing it could be a helpful warning to other projects, especially newcomers. In this presentation, it will try to explain the mistake circumstances in TOMOYO Linux project, highlighting the thoughts of project members and the community reactions.
  3. 3. “Experience is the name everyone gives to his mistakes” --- Oscar Wild
  4. 4. WHAT’S THIS ALL ABOUT
  5. 5. WHAT’S THIS ALL ABOUT • Linux comes with a set of great documentation
  6. 6. WHAT’S THIS ALL ABOUT • Linux comes with a set of great documentation • DOs and DON’Ts are there already
  7. 7. WHAT’S THIS ALL ABOUT • Linux comes with a set of great documentation • DOs and DON’Ts are there already • Yet we keep on making mistakes
  8. 8. WHAT’S THIS ALL ABOUT • Linux comes with a set of great documentation • DOs and DON’Ts are there already • Yet we keep on making mistakes • Finding a missing piece
  9. 9. WHAT IS MISSING?
  10. 10. WHAT IS MISSING? • In my humble opinion:
  11. 11. WHAT IS MISSING? • In my humble opinion: • Human nature (hard to fix)
  12. 12. WHAT IS MISSING? • In my humble opinion: • Human nature (hard to fix) • Most of us don’t really like readings (hard to fix)
  13. 13. WHAT IS MISSING? • In my humble opinion: • Human nature (hard to fix) • Most of us don’t really like readings (hard to fix) • Real-life examples taken from TOMOYO Linux project
  14. 14. WHO AM I?
  15. 15. WHO AM I? • Project manager of TOMOYO Linux
  16. 16. WHO AM I? • Project manager of TOMOYO Linux • What is “project manager”?
  17. 17. WHO AM I? • Project manager of TOMOYO Linux • What is “project manager”? • Something put in between an Open Source projects and a Company
  18. 18. WHO AM I? • Project manager of TOMOYO Linux • What is “project manager”? • Something put in between an Open Source projects and a Company • It’s an adventurous role (experimental stage)
  19. 19. OUCH, IS THIS “SECURE LINUX” TALK? • No. (so please remain seated, you are safe)
  20. 20. COVERED TOPICS • Chapter 1: Where to find DOs and DON’Ts • Chapter2: TOMOYO Linux posting history overview • Chapter 3: Step by step introduction of DOs and DON’Ts of the TOMOYO Linux
  21. 21. CHAPTER 1 Where to find DOs and DON’Ts
  22. 22. Gentle Reminder
  23. 23. Gentle Reminder Documentation is a part of Linux Kernel
  24. 24. Gentle Reminder Documentation is a part of Linux Kernel After checking out the kernel, cd to “Documentation”
  25. 25. Gentle Reminder Documentation is a part of Linux Kernel After checking out the kernel, cd to “Documentation” Problem is “there are just too many files and directories” and people prefer coding than reading
  26. 26. How Great is It?
  27. 27. How Great is It? $Documentation/ManagementStyle
  28. 28. How Great is It? $Documentation/ManagementStyle “Most people are idiots, and being a manager means you'll have to deal with it, and perhaps more importantly, that _they_ have to deal with _you_.”
  29. 29. How Great is It?
  30. 30. How Great is It? $Documentation/ManagementStyle
  31. 31. How Great is It? $Documentation/ManagementStyle “Thing will go wrong, and people want somebody to blame. Tag you’re it.”
  32. 32. Okay, I’ll do so
  33. 33. Okay, I’ll do so
  34. 34. Okay, I’ll do so Me Blame
  35. 35. The truth is people will blame you regardless of you are tagged or not (you can omit) Just these two statements illustrate the essential part of managements Linux documentation is so practical
  36. 36. Minimal Reading
  37. 37. Minimal Reading Entire Scheme
  38. 38. Minimal Reading Entire Scheme $Documentation/HOWTO
  39. 39. Minimal Reading Entire Scheme $Documentation/HOWTO Submitting Patches
  40. 40. Minimal Reading Entire Scheme $Documentation/HOWTO Submitting Patches $Documentation/SubmitChecklist
  41. 41. Minimal Reading Entire Scheme $Documentation/HOWTO Submitting Patches $Documentation/SubmitChecklist $Documentation/CodingStyle
  42. 42. References
  43. 43. References Note that the title is “How to Participate in the Linux Community”
  44. 44. References Note that the title is “How to Participate in the Linux Community” Making your code upstream means your participation in the Linux Community (Be nice!)
  45. 45. My favorite one http://www.linuxfoundation.jp/jp_uploads/ seminar20070710/Jon-Dev-Process.pdf
  46. 46. Picked up Two pages
  47. 47. I was laughing when I saw the slides for the first time (in 2007) When I came to realize that it was true, I couldn’t laugh any more ... They kept asking me “not yet?” ;-)
  48. 48. CHAPTER 2 TOMOYO Linux by Numbers
  49. 49. Leo Tolstoy said All Happy Families Resemble Each Other, Each Unhappy Family Is Unhappy in Its Own Way.
  50. 50. 2,700,000 9,230 2,700,000
  51. 51. Number of employees 2,700,000 of NTT DATA 2,700,000 CORPORATION
  52. 52. 2,700,000 3 2,700,000
  53. 53. 2,700,000 Number of project 2,700,000 members
  54. 54. 2,700,000 0.0325% 2,700,000
  55. 55. Possibilities to be 2,700,000 assigned to the 2,700,000 project
  56. 56. 2,700,000 3 2,700,000
  57. 57. TOMOYO is the 3rd 2,700,000 (and the latest) LSM 2,700,000 module merged upstream
  58. 58. 2,700,000 0 2,700,000
  59. 59. Number of people 2,700,000 who expected 2,700,000 TOMOYO would be merged
  60. 60. 2,700,000 15 2,700,000
  61. 61. 2,700,000 We posted patches 15 2,700,000 times
  62. 62. 2,700,000 716 2,700,000
  63. 63. 2,700,000 Merged since 716 days after the first 2,700,000 post
  64. 64. 2,700,000 162 2,700,000
  65. 65. Number of comments 2,700,000 from LKML 2,700,000 (0.2 comments/day)
  66. 66. Proposal History http://tomoyo.sourceforge.jp/wiki-e/?JLS2009
  67. 67. CHAPTER 3 Drawing Lessons from the “Mistakes” of TOMOYO Linux Project
  68. 68. CHAPTER 3 Drawing Lessons from the “Mistakes” of TOMOYO Linux Project B lame Me
  69. 69. IN THE 1ST POSTING I wrote: snip All right, that's almost everything. Please visit the following URL for the code and documents:   http://tomoyo.sourceforge.jp/wiki-e/ If you want to see the code first, then:   http://tomoyo.sourceforge.jp/cgi-bin/lxr/ source/security/tomoyo/?v=linux-2.6.21.3- tomoyo-2.0
  70. 70. DON’T Send URL Send patches
  71. 71. DON’T Send URL Send patches
  72. 72. WHAT HAPPENED? •Igot a personal message from Stephen Smalley, a maintainer of that famous SELinux –“If you really want feedback or to get your code into the kernel, you need to do more than post a URL to the code - you need to break your code down into a number of patches and post them, just like the AppArmor folks have been doing. “
  73. 73. SO WE RUSHED TO POSTED PATCHES NEXT DAY • Pavel Machek gave a comment •“Looks whitespace-damaged to me.”
  74. 74. DON’T Ignore the Linux standard coding style Always apply checkpatch.pl
  75. 75. DON’T Ignore the Linux standard coding style Always apply checkpatch.pl
  76. 76. DO
  77. 77. DO • Carefully read the $Document/CodingStyle
  78. 78. DO • Carefully read the $Document/CodingStyle • Check your code with $scripts/checkpatch.pl
  79. 79. DO • Carefully read the $Document/CodingStyle • Check your code with $scripts/checkpatch.pl • Also use other $scripts/check*.pl
  80. 80. • Jiri Kosina pointed us to make patches bisectable • “Justa trivial minor nitpick - IMHO this breaks bisectability. It might be better to add the Kconfig/ Makefile patch at the end of the whole series, so that bisect doesn't end up in the tree in which Makefile references non-existing files/directories.”
  81. 81. DO • Add the Kconfig/Makefile patch at the end of the whole series, so that bisect doesn't end up in the tree in which Makefile references non-existing files/directories.”
  82. 82. DO • Add the Kconfig/Makefile patch at the end of the whole series, so that bisect doesn't end up in the tree in which Makefile references non-existing files/directories.”
  83. 83. IN THE 3RD PATCH • James Morris taught us series of patches should form a thread •“I'dalso suggest making all of the patches a reply to the first email, so they can be threaded.”
  84. 84. LIKE THIS
  85. 85. DO • Send series of patches as children of the first message so that they can form a thread or you want people to read your messages
  86. 86. DO • Send series of patches as children of the first message so that they can form a thread or you want people to read your messages
  87. 87. • James Morris said: • “Please use standard kernel list handling, per include/linux/list.h” • YOSHIFUJI Hideaki also mentioned: • You'reintroducing a custom API, which is open-coded repeatedly throughout your module. • All linked lists (at least, new ones) must use the standard kernel list API.
  88. 88. DON’T Propose new data structure Use existing one
  89. 89. DON’T Propose new data structure Use existing one
  90. 90. IN THE 5TH POSTING • James Morris suggested to CC netdev mailing list • “Youshould send anything which touches core networking to netdev, too, and get an ack from one of the core developers there.”
  91. 91. DO • Carefully choose CCs and get a review from them
  92. 92. DO • Carefully choose CCs and get a review from them
  93. 93. IN THE 6TH POSTING • Tetsuo posted 30 series of messages with the subject, “Subject: [TOMOYO #7 00/30] TOMOYO Linux 1.6.0 released” • The problem was “TOMOYO 1.6.0” did not use LSM and implemented different hooks
  94. 94. DON’T Try to invent a new API Respect a standard and follow one
  95. 95. DON’T Try to invent a new API Respect a standard and follow one
  96. 96. • Tetsuoand I knew that posting such patches will never be accepted • However, we had been stuck and we couldn’t find another way • Our posting was thoughtless, but we were so serious to make our code upstream
  97. 97. IN THE 7TH POSTING I changed my mind and wrote: We apologize for the confusion we caused in the last posting, but we don't want to give up returning our work to the mainline. We cordially request LSM changes to pass vfsmount parameters.
  98. 98. • Stephen Smalley kindly responded on the list –“Don't cordially request it - submit patches to make it happen.  Or work with others who have been submitting such patches. “
  99. 99. DON’T Cordially ask to merge your code ;-) Make good patches and post them
  100. 100. DON’T Cordially ask to merge your code ;-) Make good patches and post them
  101. 101. IN THE 16TH POSTING • Tetsuoposted 25series of messages with the subject, “Subject: [TOMOYO #16 00/25] Starting TOMOYO 2.3” • The patches included enhancements as well as garbage collector functionality (so Tetsuo had a reason for 25)
  102. 102. • Pavel Machek commented: •You are expected to submit diffs in smaller steps, not here it is, totally rewritten, take it or leave it.
  103. 103. IT HAPPENED IN THE 7TH • Serge E. Hallyn once gave us a same comment: • First let me point out that reviewing patches is always a lot of work. What you've done here by posting an entirely new 30-patch implementation of tomoyo when (I hope) you're not even serious about that is to basically tell us our time means nothing to you...
  104. 104. CONCLUSION
  105. 105. • Our mistakes, presented in this slides, are merely the tip of the iceberg • We made many failures and we sometimes behaved very badly • Nevertheless, this Linux community reacted and even merged our code
  106. 106. • Feedbacks were not limited by mailing list replies • There were people who sent off-list messages and left advices in face
  107. 107. WITH ALL MY EXPERIENCES I CAN SAY
  108. 108. WITH ALL MY EXPERIENCES I CAN SAY •Linux is not just free code
  109. 109. WITH ALL MY EXPERIENCES I CAN SAY •Linux is not just free code • Linux is great because people are great
  110. 110. WITH ALL MY EXPERIENCES I CAN SAY •Linux is not just free code • Linux is great because people are great • Sending your code is a conversation with people
  111. 111. WITH ALL MY EXPERIENCES I CAN SAY •Linux is not just free code • Linux is great because people are great • Sending your code is a conversation with people • Theymight not appear friendly for the first time, but try to speak them first
  112. 112. ACKNOWLEDGMENTS
  113. 113. Al Viro, Albert Cahalan, Alexey Dobriyan, Andi Kleen, Andrew Morton, Bodo Eggert, Casey Schaufler, Chris Wright, Christoph Hellwig, Crispin Cowan, Daniel Walker, David Howells, David Lang, David P. Quigley, Greg KH, James Morris, Jamie Lokier, Jiri Kosina, Jonathan Corbet, Joshua Brindle, KaiGai Kohei, Kamezawa Hiroyuki, KOSAKI Motohiro, Kyle Moffett, Linus Torvalds, Matthew Wilcox, Miklos Szeredi, Paul E. McKenney, Paul Moore, Pavel Machek, Peter Dolding, Peter Zijlstra, Rik van Riel, Serge E. Hallyn, Seth Arnorld, Shaya Potter, Stephen Smalley, Tim Bird, Trond Myklebust, Valdis Kletnieks, William Leibzon, YOSHIFUJI Hideaki
  114. 114. WE COULDN’T HAVE DONE IT WITHOUT YOUR HELP Al Viro, Albert Cahalan, Alexey Dobriyan, Andi Kleen, Andrew Morton, Bodo Eggert, Casey Schaufler, Chris Wright, Christoph Hellwig, Crispin Cowan, Daniel Walker, David Howells, David Lang, David P. Quigley, Greg KH, James Morris, Jamie Lokier, Jiri Kosina, Jonathan Corbet, Joshua Brindle, KaiGai Kohei, Kamezawa Hiroyuki, KOSAKI Motohiro, Kyle Moffett, Linus Torvalds, Matthew Wilcox, Miklos Szeredi, Paul E. McKenney, Paul Moore, Pavel Machek, Peter Dolding, Peter Zijlstra, Rik van Riel, Serge E. Hallyn, Seth Arnorld, Shaya Potter, Stephen Smalley, Tim Bird, Trond Myklebust, Valdis Kletnieks, William Leibzon, YOSHIFUJI Hideaki
  115. 115. AND OF COURSE • We thank NTT DATA CORPORATION for their support of the project
  116. 116. RELATED PRESENTATIONS Realities of Mainlining - Case of the TOMOYO Linux Project - Toshiharu Harada haradats@nttdata.co.jp haradats@gmail.com Time to Glean !#$%'$()*+,-$.)/0'1$2*3$0.4$%+0+'4 NTT DATA CORPORATION July 9, 2008 July 25, 2008 Toshiharu Harada haradats@nttdata.co.jp Kentaro Takeda Tetsuo Handa NTT DATA CORPORATION
  117. 117. What does it mean being an Open Source project manager in Enterprise enterprise edition What does it mean being an Open Source project manager LinuxCon 2009 (Business) September 23, 2009 in Enterprise Toshiharu Harada haradats@nttdata.co.jp Open Source Edition NTT DATA CORPORATION LinuxCon2009 (Business) September 23, 2009 Toshiharu Harada haradats@gmail.com TOMOYO Linux Project
  118. 118. TRADEMARKS • Linuxis a registered trademark of Linus Torvalds in Japan and other countries • TOMOYOis a registered trademark of NTT DATA CORPORATION in Japan

×