Your SlideShare is downloading. ×
0
KERNEL DEVELOPMENT:
DRAWING LESSONS FROM
      "MISTAKES"
    Japan Linux Symposium 2009
          October 23, 2009
      ...
ABSTRACTS
Every kernel developer knows that Linux comes with plenty of precious
documentation as an integral part. From co...
“Experience is the name everyone gives to
his mistakes” --- Oscar Wild
WHAT’S THIS ALL ABOUT


• Linux   comes with a set of great documentation

• DOs     and DON’Ts are there already

• Yet  ...
WHAT IS MISSING?


• In   my humble opinion:

  • Human       nature (hard to fix)

  • Most   of us don’t really like read...
WHO AM I?



• Project   manager of TOMOYO Linux
OUCH, IS THIS “SECURE LINUX” TALK?




• No! Thistalk has no “security”, no “Policy” or “MAC” (so, please
  remain seated)...
COVERED TOPICS

•   Chapter 1:

    •   Where to find DOs and DON’Ts

•   Chapter 2:

    •   TOMOYO Linux posting history ...
CHAPTER 1
Where to find DOs and DON’Ts
Gentle Reminder

Documentation is a part of Linux Kernel

After checking out the kernel, cd to
“Documentation”

Problem is...
How Great is It?

$Documentation/ManagementStyle

  “Most people are idiots, and being a
  manager means you'll have to de...
Minimal Reading

Entire Scheme

  $Documentation/HOWTO

Submitting Patches

  $Documentation/SubmitChecklist

  $Documenta...
References




Note that the title is “How to Participate in
the Linux Community”

Making your code upstream means your
pa...
My favorite one
http://www.linuxfoundation.jp/jp_uploads/
seminar20070710/Jon-Dev-Process.pdf
CHAPTER 2
TOMOYO Linux by Numbers
All Happy Families Resemble Each
Other, Each Unhappy Family Is
Unhappy in Its Own Way. - Leo Tolstoy
3
We were 3
3
3rd LSM
module
15
Posted 15
  times
716
Merged since 716
days after the first
       post
162
Received 162
comments from
    LKML
Proposal History
http://tomoyo.sourceforge.jp/wiki-e/?JLS2009
CHAPTER 3
Drawing Lessons from the “Mistakes”
    of TOMOYO Linux Project
IN THE 1ST POSTING

I wrote:

All right, that's almost everything. Please
visit the following
URL for the code and documen...
DON’T
Send URL




Send patches
WHAT HAPPENED?


•Igot a personal message from Stephen Smalley, a maintainer
 of that famous SELinux

     –“If you really...
SO WE RUSHED TO POSTED
     PATCHES NEXT DAY


• Pavel   Machek gave a comment

  • “Looks   whitespace-damaged to me.”
DON’T
Ignore the Linux standard coding style




         Always apply checkpatch.pl
DO




• Read   the $Document/CodingStyle

• Check   your code with $scripts/checkpatch.pl

• Also   use other $scripts/ch...
• Jiri   Kosina pointed us to make patches bisectable

• “Just a trivial minor nitpick - IMHO this breaks bisectability. I...
DO




• Add the Kconfig/Makefile patch at the end of the whole
 series, so that bisect doesn't end up in the tree in which
...
IN THE 3RD PATCH



• James   Morris taught us series of patches should form a thread

 • “I'd
      also suggest making a...
DO




• Send series of patches as children of the first message so that
 they can form a thread
• James   Morris said:

 • “Please   use standard kernel list handling, per include/linux/list.h”

• YOSHIFUJI    Hideaki ...
DON’T
Propose new data structure




       Use existing one
IN THE 5TH POSTING


• James   Morris suggested to CC netdev mailing list

 • “You should send anything which touches core...
DO




• Carefully   choose CCs and get a review from them
IN THE 6TH POSTING


• Tetsuoposted 30 series of messages with the subject,
 “Subject: [TOMOYO #7 00/30] TOMOYO Linux 1.6....
DON’T
Try to invent a new API




Respect a standard and follow one
• Tetsuoand I knew that posting such patches will never be
 accepted

• However, we   had been stuck and we couldn’t find a...
IN THE 7TH POSTING

I changed my mind and wrote:

We apologize for the confusion we caused in
the last posting,
but we don...
DON’T
Cordially ask to merge your code ;-)




     Make good patches and post them
• Stephen   Smalley kindly responded on the list

  –“Don't cordially request it - submit patches to make it happen. 
   O...
• Serge   E. Hallyn wrote:
CONCLUSION
• Our mistakes, presented in this slides, are merely the tip of the
 iceberg

• We made many failures and we sometimes beh...
• Feedbacks    were not limited by mailing list replies

• There    were people who sent off-list messages and left advice...
WITH ALL MY EXPERIENCES
                      I CAN SAY


• Linux   is not just free code

• Linux   is great because peop...
ACKNOWLEDGMENTS
Al Viro, Albert Cahalan, Alexey Dobriyan, Andi Kleen, Andrew
Morton, Bodo Eggert, Casey Schaufler, Chris Wright, Christoph
...
WE COULDN’T HAVE DONE IT
           WITHOUT YOUR HELP

Al Viro, Albert Cahalan, Alexey Dobriyan, Andi Kleen, Andrew
Morton...
AND OF COURSE
• We thank NTT DATA CORPORATION for their support of
 the project
RELATED PRESENTATIONS

Realities of Mainlining
 - Case of the TOMOYO Linux Project -     Time to Glean
                   ...
What does it mean being an
 What does it mean being an
Open Source project manager        Open Source project manager
    ...
TRADEMARKS


• Linuxis a registered trademark of Linus Torvalds in Japan and
 other countries

• TOMOYOis a registered tra...
Kernel Development: Drawing Lessons From "Mistakes" (Japan Linux Symposium 2009)
Upcoming SlideShare
Loading in...5
×

Kernel Development: Drawing Lessons From "Mistakes" (Japan Linux Symposium 2009)

1,381

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,381
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
10
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Kernel Development: Drawing Lessons From "Mistakes" (Japan Linux Symposium 2009)"

  1. 1. KERNEL DEVELOPMENT: DRAWING LESSONS FROM "MISTAKES" Japan Linux Symposium 2009 October 23, 2009 Toshiharu Harada haradats@nttdata.co.jp NTT DATA CORPORATION
  2. 2. ABSTRACTS Every kernel developer knows that Linux comes with plenty of precious documentation as an integral part. From coding style to how to post patches, almost everything has been documented. However, history shows that error is human nature. Sometimes developers do not well know Don’ts, but there are also cases when they make mistakes despite being aware of such rules. Why this happen is unsolved, but a documentation, so far missing, of the consequences of this misbehavior could discourage it. The presenter is project manager of TOMOYO Linux, a security enhancement feature merged in version 2.6.30. Thinking open-minded, he decided to share the errors his project made, wishing it could be a helpful warning to other projects, especially newcomers. In this presentation, it will try to explain the mistake circumstances in TOMOYO Linux project, highlighting the thoughts of project members and the community reactions.
  3. 3. “Experience is the name everyone gives to his mistakes” --- Oscar Wild
  4. 4. WHAT’S THIS ALL ABOUT • Linux comes with a set of great documentation • DOs and DON’Ts are there already • Yet we keep on making mistakes • Finding a missing piece
  5. 5. WHAT IS MISSING? • In my humble opinion: • Human nature (hard to fix) • Most of us don’t really like readings (hard to fix) • Real-life examples taken from TOMOYO Linux project
  6. 6. WHO AM I? • Project manager of TOMOYO Linux
  7. 7. OUCH, IS THIS “SECURE LINUX” TALK? • No! Thistalk has no “security”, no “Policy” or “MAC” (so, please remain seated) • If you are interested in them, contact me personally
  8. 8. COVERED TOPICS • Chapter 1: • Where to find DOs and DON’Ts • Chapter 2: • TOMOYO Linux posting history overview • Chapter 3: • Step by step introDOs and DON’Ts of the TOMOYO Linux
  9. 9. CHAPTER 1 Where to find DOs and DON’Ts
  10. 10. Gentle Reminder Documentation is a part of Linux Kernel After checking out the kernel, cd to “Documentation” Problem is “there are just too many files and directories” and people prefer coding than reading
  11. 11. How Great is It? $Documentation/ManagementStyle “Most people are idiots, and being a manager means you'll have to deal with it, and perhaps more importantly, that _they_ have to deal with _you_.” “Thing will go wrong, and people want somebody to blame. Tag you’re it.”
  12. 12. Minimal Reading Entire Scheme $Documentation/HOWTO Submitting Patches $Documentation/SubmitChecklist $Documentation/CodingStyle
  13. 13. References Note that the title is “How to Participate in the Linux Community” Making your code upstream means your participation in the Linux Community
  14. 14. My favorite one http://www.linuxfoundation.jp/jp_uploads/ seminar20070710/Jon-Dev-Process.pdf
  15. 15. CHAPTER 2 TOMOYO Linux by Numbers
  16. 16. All Happy Families Resemble Each Other, Each Unhappy Family Is Unhappy in Its Own Way. - Leo Tolstoy
  17. 17. 3
  18. 18. We were 3
  19. 19. 3
  20. 20. 3rd LSM module
  21. 21. 15
  22. 22. Posted 15 times
  23. 23. 716
  24. 24. Merged since 716 days after the first post
  25. 25. 162
  26. 26. Received 162 comments from LKML
  27. 27. Proposal History http://tomoyo.sourceforge.jp/wiki-e/?JLS2009
  28. 28. CHAPTER 3 Drawing Lessons from the “Mistakes” of TOMOYO Linux Project
  29. 29. IN THE 1ST POSTING I wrote: All right, that's almost everything. Please visit the following URL for the code and documents:   http://tomoyo.sourceforge.jp/wiki-e/ If you want to see the code first, then:   http://tomoyo.sourceforge.jp/cgi-bin/lxr/ source/security/tomoyo/?v=linux-2.6.21.3- tomoyo-2.0
  30. 30. DON’T Send URL Send patches
  31. 31. WHAT HAPPENED? •Igot a personal message from Stephen Smalley, a maintainer of that famous SELinux –“If you really want feedback or to get your code into the kernel, you need to do more than post a URL to the code - you need to break your code down into a number of patches and post them, just like the AppArmor folks have been doing. “
  32. 32. SO WE RUSHED TO POSTED PATCHES NEXT DAY • Pavel Machek gave a comment • “Looks whitespace-damaged to me.”
  33. 33. DON’T Ignore the Linux standard coding style Always apply checkpatch.pl
  34. 34. DO • Read the $Document/CodingStyle • Check your code with $scripts/checkpatch.pl • Also use other $scripts/check*.pl
  35. 35. • Jiri Kosina pointed us to make patches bisectable • “Just a trivial minor nitpick - IMHO this breaks bisectability. It might be better to add the Kconfig/Makefile patch at the end of the whole series, so that bisect doesn't end up in the tree in which Makefile references non-existing files/directories.”
  36. 36. DO • Add the Kconfig/Makefile patch at the end of the whole series, so that bisect doesn't end up in the tree in which Makefile references non-existing files/directories.”
  37. 37. IN THE 3RD PATCH • James Morris taught us series of patches should form a thread • “I'd also suggest making all of the patches a reply to the first email, so they can be threaded.”
  38. 38. DO • Send series of patches as children of the first message so that they can form a thread
  39. 39. • James Morris said: • “Please use standard kernel list handling, per include/linux/list.h” • YOSHIFUJI Hideaki also mentioned: • You'reintroducing a custom API, which is open-coded repeatedly throughout your module. • All linked lists (at least, new ones) must use the standard kernel list API.
  40. 40. DON’T Propose new data structure Use existing one
  41. 41. IN THE 5TH POSTING • James Morris suggested to CC netdev mailing list • “You should send anything which touches core networking to netdev, too, and get an ack from one of the core developers there.”
  42. 42. DO • Carefully choose CCs and get a review from them
  43. 43. IN THE 6TH POSTING • Tetsuoposted 30 series of messages with the subject, “Subject: [TOMOYO #7 00/30] TOMOYO Linux 1.6.0 released” • Theproblem was “TOMOYO 1.6.0” did not use LSM and implemented different hooks
  44. 44. DON’T Try to invent a new API Respect a standard and follow one
  45. 45. • Tetsuoand I knew that posting such patches will never be accepted • However, we had been stuck and we couldn’t find another way • Our posting was thoughtless, but we were so serious to make our code upstream
  46. 46. IN THE 7TH POSTING I changed my mind and wrote: We apologize for the confusion we caused in the last posting, but we don't want to give up returning our work to the mainline. We cordially request LSM changes to pass vfsmount parameters.
  47. 47. DON’T Cordially ask to merge your code ;-) Make good patches and post them
  48. 48. • Stephen Smalley kindly responded on the list –“Don't cordially request it - submit patches to make it happen.  Or work with others who have been submitting such patches. “
  49. 49. • Serge E. Hallyn wrote:
  50. 50. CONCLUSION
  51. 51. • Our mistakes, presented in this slides, are merely the tip of the iceberg • We made many failures and we sometimes behaved very badly • Nevertheless, this Linux community reacted and even merged our code
  52. 52. • Feedbacks were not limited by mailing list replies • There were people who sent off-list messages and left advices in face
  53. 53. WITH ALL MY EXPERIENCES I CAN SAY • Linux is not just free code • Linux is great because people are great • Sending your code is a conversation with people • Theymight not appear friendly for the first time, but try to speak them first
  54. 54. ACKNOWLEDGMENTS
  55. 55. Al Viro, Albert Cahalan, Alexey Dobriyan, Andi Kleen, Andrew Morton, Bodo Eggert, Casey Schaufler, Chris Wright, Christoph Hellwig, Crispin Cowan, Daniel Walker, David Howells, David Lang, David P. Quigley, Greg KH, James Morris, Jamie Lokier, Jiri Kosina, Jonathan Corbet, Joshua Brindle, KaiGai Kohei, Kamezawa Hiroyuki, KOSAKI Motohiro, Kyle Moffett, Linus Torvalds, Matthew Wilcox, Miklos Szeredi, Paul E. McKenney, Paul Moore, Pavel Machek, Peter Dolding, Peter Zijlstra, Rik van Riel, Serge E. Hallyn, Seth Arnorld, Shaya Potter, Stephen Smalley, Tim Bird, Trond Myklebust, Valdis Kletnieks, William Leibzon, YOSHIFUJI Hideaki
  56. 56. WE COULDN’T HAVE DONE IT WITHOUT YOUR HELP Al Viro, Albert Cahalan, Alexey Dobriyan, Andi Kleen, Andrew Morton, Bodo Eggert, Casey Schaufler, Chris Wright, Christoph Hellwig, Crispin Cowan, Daniel Walker, David Howells, David Lang, David P. Quigley, Greg KH, James Morris, Jamie Lokier, Jiri Kosina, Jonathan Corbet, Joshua Brindle, KaiGai Kohei, Kamezawa Hiroyuki, KOSAKI Motohiro, Kyle Moffett, Linus Torvalds, Matthew Wilcox, Miklos Szeredi, Paul E. McKenney, Paul Moore, Pavel Machek, Peter Dolding, Peter Zijlstra, Rik van Riel, Serge E. Hallyn, Seth Arnorld, Shaya Potter, Stephen Smalley, Tim Bird, Trond Myklebust, Valdis Kletnieks, William Leibzon, YOSHIFUJI Hideaki
  57. 57. AND OF COURSE • We thank NTT DATA CORPORATION for their support of the project
  58. 58. RELATED PRESENTATIONS Realities of Mainlining - Case of the TOMOYO Linux Project - Time to Glean !#$%'$()*+,-$.)/0'1$2*3$0.4$%+0+'4 Toshiharu Harada haradats@nttdata.co.jp July 25, 2008 haradats@gmail.com Toshiharu Harada haradats@nttdata.co.jp Kentaro Takeda NTT DATA CORPORATION Tetsuo Handa NTT DATA CORPORATION July 9, 2008
  59. 59. What does it mean being an What does it mean being an Open Source project manager Open Source project manager in Enterprise in Enterprise enterprise edition Open Source Edition LinuxCon 2009 (Business) LinuxCon2009 (Business) September 23, 2009 Toshiharu Harada September 23, 2009 haradats@nttdata.co.jp Toshiharu Harada NTT DATA CORPORATION haradats@gmail.com TOMOYO Linux Project
  60. 60. TRADEMARKS • Linuxis a registered trademark of Linus Torvalds in Japan and other countries • TOMOYOis a registered trademark of NTT DATA CORPORATION in Japan
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×