Photo (cc) Horia VarlanVirtual Separation of ConcernsChristian Kästner
Feature-OrientedProduct Lines
Database Engine
PrinterFirmware
LinuxKernel
7
Variability = Complexity
33 features       optional, independenta unique configuration for everyperson on this planet
320 features     optional, independent                      more configurations than estimated                    atoms in...
Correctness?
Maintenance?Comprehension?
Conditional Compilationstatic int _rep_queue_filedone(...)   DB_ENV *dbenv;   REP *rep;   __rep_fileinfo_args *rfp; {#ifnd...
Objections / Criticism  Designed in the 70th and hardly evolved since    “#ifdef considered harmful”                      ...
Academia: Compositional ApproachesBase / Platformclass Stack {  void push(Object o) {    elementData[size++] = o;  }  ...}...
A CLOSER LOOK AT PREPROCESSORS
Separation of Concerns
class Stack {   void push(Object o#ifdef SYNC                      Obfuscation   , Transaction txn#endif   ) {      if (o=...
Obfuscation       Femto OS
static int _rep_queue_filedone(...)   DB_ENV *dbenv;   REP *rep;                            Error-Prone   __rep_fileinfo_a...
Advantages• Easy to use  • “annotate and remove”  • Already part of many languages / simple tools  • Most developers alrea...
IMPROVING TOOL SUPPORT
[ICSE‟08, ViSPLE‟08]Views
View on Feature Eval
View on Configuration       “Add and Eval”
[ICSE‟08, ICSE„10, EASE„11]
FeatureCommander
Controlled3 Experiments                    Faster?Scaling?         How Used?
ERROR DETECTION / PREVENTION
Disciplined          Annotations#ifdef WORLDint main() {   printf(“Hello World”);}#endifint main() {#ifdef WORLD   printf(...
Disciplined    Undisciplined          Annotations#ifdef WORLD                   int main() {int main() {                  ...
static int _rep_queue_filedone(...)   DB_ENV *dbenv;   REP *rep;   __rep_fileinfo_args *rfp; {#ifndef HAVE_QUEUE   COMPQUI...
ParserVariability-Aware   Type System                    Static Analysis                    Bug Finding                   ...
References             Conflicts
Presence Conditions       WORLD               BYEtrue                     true
Reachability: pc(caller) -> pc(target) Conflicts: ¬(pc(def1) ˄pc(def2))                                      ¬ (WORLD ˄BYE...
P              Variability Model:                                          WORLD       BYE                                ...
AST with Variability Information                                             true -> true                         greet.c ...
[ASE‟08, TOSEM„11]Formalization: CFJ
Surface Complexity                          Inherent                          Complexity                SAT               ...
Variability-Aware Analysis                 Type System                 Static Analysis                 Bug Finding        ...
CIDE                         open source, fosd.net/cide       http://fosd.de/cide
[GPCE‟09]                Automated Refactorings       • Feature Module                • Annotationen       class Stack {  ...
THE REAL-WORLD CHALLENGE
Parse and Type check      all configurations of the           entire Linux              kernel   10,000 features, 6 millio...
[OOPSLA„11]                      greet.c         …printf       VWORLD        VBYE       main         msg     ε      msg   ...
Macro expansion               Undisciplinedneeded for parsing             annotations                                     ...
Parsing C without Preprocessing
Previous SolutionsDisciplined Subset  Requires Code PreparationHeuristics and Partial Analysis  Inaccurate, False Positive...
TypeChef                                    (                                                           +                 ...
4A   (¬A 4¬A˄B +¬A˄ 6¬A                                            B        )¬A       true(            3       +          ...
Variability-Aware   ParsersGNU C + cpp     Java + Antenna
Parsing LinuxVariability in Build System (kbuild)         and with #ifdef                                       2.6.33.3  ...
353 included header files per C file      0   8590 macros per C file      0   1387 conditional macros per C file      0   ...
7665 C files (x86)   0  30 seconds per file (median)   0  85 hours total   0 4.1 % overhead (undiscipl.)   0     0     0 s...
Type Checking Linux                  2.6.33.3                  X86
SUMMARY AND PERSPECTIVE
Summary• Problems:                  • Improvements:  • Separation of Concerns      • Views  • Obfuscation                 ...
Perspective •   Preprocessors common in practice, despite criticism •   Neglected by researchers •   Tool support can addr...
Questions?             http://fosd.de/cide
Picture Credits© Stuck in Customs (cc by-nc-sa 2.0)© Horia Varlan (cc by 2.0)
Virtual Separation of Concerns (2011 Update)
Virtual Separation of Concerns (2011 Update)
Virtual Separation of Concerns (2011 Update)
Virtual Separation of Concerns (2011 Update)
Upcoming SlideShare
Loading in …5
×

Virtual Separation of Concerns (2011 Update)

5,098 views
5,054 views

Published on

Revised and extended version of the "Virtual Separation of Concerns" talk, prepared for the GI-Dissertationspreis colloquium at Dagstuhl and for the invited talk at Oregon State University in Corvallis.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
5,098
On SlideShare
0
From Embeds
0
Number of Embeds
4,462
Actions
Shares
0
Downloads
11
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Virtual Separation of Concerns (2011 Update)

  1. 1. Photo (cc) Horia VarlanVirtual Separation of ConcernsChristian Kästner
  2. 2. Feature-OrientedProduct Lines
  3. 3. Database Engine
  4. 4. PrinterFirmware
  5. 5. LinuxKernel
  6. 6. 7
  7. 7. Variability = Complexity
  8. 8. 33 features optional, independenta unique configuration for everyperson on this planet
  9. 9. 320 features optional, independent more configurations than estimated atoms in the universe
  10. 10. Correctness?
  11. 11. Maintenance?Comprehension?
  12. 12. Conditional Compilationstatic int _rep_queue_filedone(...) DB_ENV *dbenv; REP *rep; __rep_fileinfo_args *rfp; {#ifndef HAVE_QUEUE COMPQUIET(rep, NULL); COMPQUIET(rfp, NULL); return (__db_no_queue_am(dbenv));#else db_pgno_t first, last; u_int32_t flags; int empty, ret, t_ret;#ifdef DIAGNOSTIC DB_MSGBUF mb;#endif ... Excerpt from Oracle’s Berkeley DB
  13. 13. Objections / Criticism Designed in the 70th and hardly evolved since “#ifdef considered harmful” “#ifdef hell” “maintenance becomes a „hit or miss‟ process” “is difficult to determine if the code being viewed is actually compiled into the system” “incomprehensible source texts” “programming errors are easy “CPP makes maintenance difficult” to make and difficult to detect” “source code rapidly“preprocessor diagnostics are poor” becomes a maze”
  14. 14. Academia: Compositional ApproachesBase / Platformclass Stack { void push(Object o) { elementData[size++] = o; } ...} class Stack { void push(Object o) { Lock l = lock(o);Feature: Queue elementData[size++] = o;refines class Stack { Composition } l.unlock(); ... void push(Object o) { Lock l = lock(o); } Super.push(o); l.unlock(); } ...} Module ComponentsFeature: Diagnostic Frameworks, Plug-insaspect Diagnostics { ... Feature-Modules / Mixin Layers / …} Aspects / Subjects, Hyper/J, Deltas
  15. 15. A CLOSER LOOK AT PREPROCESSORS
  16. 16. Separation of Concerns
  17. 17. class Stack { void push(Object o#ifdef SYNC Obfuscation , Transaction txn#endif ) { if (o==null#ifdef SYNC || txn==null#endif ) return;#ifdef SYNC Lock l=txn.lock(o);#endif elementData[size++] = o;#ifdef SYNC l.unlock();#endif fireStackChanged(); }}
  18. 18. Obfuscation Femto OS
  19. 19. static int _rep_queue_filedone(...) DB_ENV *dbenv; REP *rep; Error-Prone __rep_fileinfo_args *rfp; {#ifndef HAVE_QUEUE COMPQUIET(rep, NULL); COMPQUIET(rfp, NULL); return (__db_no_queue_am(dbenv));#else db_pgno_t first, last; #ifdef TABLES u_int32_t flags; class Table { int empty, ret,void insert(Object data, Txn txn) { t_ret;#ifdef DIAGNOSTIC storage.set(data, txn.getLock()); DB_MSGBUF mb; }#endif } // over 100 lines of add. code #endif} class Storage {#endif #ifdef WRITE boolean set(…) { ... } #endif }
  20. 20. Advantages• Easy to use • “annotate and remove” • Already part of many languages / simple tools • Most developers already familiar• Flexible / Expressive• Language-independent / Uniform• Low Adoption Barrier • esp. with legacy code
  21. 21. IMPROVING TOOL SUPPORT
  22. 22. [ICSE‟08, ViSPLE‟08]Views
  23. 23. View on Feature Eval
  24. 24. View on Configuration “Add and Eval”
  25. 25. [ICSE‟08, ICSE„10, EASE„11]
  26. 26. FeatureCommander
  27. 27. Controlled3 Experiments Faster?Scaling? How Used?
  28. 28. ERROR DETECTION / PREVENTION
  29. 29. Disciplined Annotations#ifdef WORLDint main() { printf(“Hello World”);}#endifint main() {#ifdef WORLD printf(“Hello World”);#endif} greet.c printf VBYE mainAlign with variability model VWORLD msg ε msg ε printf Align with code structure msg
  30. 30. Disciplined Undisciplined Annotations#ifdef WORLD int main() {int main() { #ifdef DYNAMIC printf(“Hello World”); if (enabled) {} #endif#endif printf(“Hello World”); #ifdef DYNAICint main() { }#ifdef WORLD #endif printf(“Hello World”); }#endif} int a = #ifdef DYNAMICAlign with variability model enabled? 3 : 4 + #endif Align with code structure 5;
  31. 31. static int _rep_queue_filedone(...) DB_ENV *dbenv; REP *rep; __rep_fileinfo_args *rfp; {#ifndef HAVE_QUEUE COMPQUIET(rep, NULL); COMPQUIET(rfp, NULL); return (__db_no_queue_am(dbenv));#else db_pgno_t first, last; u_int32_t flags; int empty, ret, t_ret;#ifdef DIAGNOSTIC DB_MSGBUF mb;#endif // over 100 lines of add. code}#endif
  32. 32. ParserVariability-Aware Type System Static Analysis Bug Finding Testing Model Checking Theorem Proving …
  33. 33. References Conflicts
  34. 34. Presence Conditions WORLD BYEtrue true
  35. 35. Reachability: pc(caller) -> pc(target) Conflicts: ¬(pc(def1) ˄pc(def2)) ¬ (WORLD ˄BYE) true -> (WORLD v BYE)true -> true
  36. 36. P Variability Model: WORLD BYE VM ->¬ (WORLD ˄BYE) VM -> (true -> (WORLD v BYE))VM -> (true -> true)
  37. 37. AST with Variability Information true -> true greet.c …printf VWORLD VBYE main msg ε msg ε printf¬ (WORLD ˄BYE) true -> (WORLD v BYE) msg WORLD BYE 40 Extended Lookup Mechanism
  38. 38. [ASE‟08, TOSEM„11]Formalization: CFJ
  39. 39. Surface Complexity Inherent Complexity SAT Problem
  40. 40. Variability-Aware Analysis Type System Static Analysis Bug Finding Testing Model Checking Theorem Proving …
  41. 41. CIDE open source, fosd.net/cide http://fosd.de/cide
  42. 42. [GPCE‟09] Automated Refactorings • Feature Module • Annotationen class Stack { class Stack { int[] data; int[] data;Base void push(int o) { … } #ifdef UNDO int pop() { /*...*/ } int backup; } void undo() { /*...*/ } #endif refines class Stack { #ifdef TOP Automated TransformationTop int top() { /*...*/ } int top() { /*...*/ } } #endif void push(int o) { refines class Stack { #ifdef UNDO int backup; backup = top(); void undo() { /*...*/ } #endifUndo void push(int o) { /*...*/ backup = top(); } original(v); int pop() { /*...*/ } } }
  43. 43. THE REAL-WORLD CHALLENGE
  44. 44. Parse and Type check all configurations of the entire Linux kernel 10,000 features, 6 million lines of C code
  45. 45. [OOPSLA„11] greet.c …printf VWORLD VBYE main msg ε msg ε printf msg AST with Variability Information
  46. 46. Macro expansion Undisciplinedneeded for parsing annotations Alternative macros ? ? ? greet.c + printf VWORLD VBYE + main + msg ε + msg ε printf msg
  47. 47. Parsing C without Preprocessing
  48. 48. Previous SolutionsDisciplined Subset Requires Code PreparationHeuristics and Partial Analysis Inaccurate, False PositivesBrute Force Infeasible Effort
  49. 49. TypeChef ( + 2 Variability-Aware * Variability-Aware 3 * VA Lexer ) + Parser 4A 5¬A 2 3 4 5 Variability-Aware Analysishttps://github.com/ckaestne/TypeChef
  50. 50. 4A (¬A 4¬A˄B +¬A˄ 6¬A B )¬A true( 3 + ) 4¬A˄ +¬A˄ 6¬A B B + 4A (¬A )¬A3 VA 4¬A˄B +¬A˄ 6¬A B 4 VB + 6 4 6
  51. 51. Variability-Aware ParsersGNU C + cpp Java + Antenna
  52. 52. Parsing LinuxVariability in Build System (kbuild) and with #ifdef 2.6.33.3 X86Variability Model (kconfig)
  53. 53. 353 included header files per C file 0 8590 macros per C file 0 1387 conditional macros per C file 0 340 alternative macros per C file 0335490 token per C file 0 2.6.33.3 X86 72 % conditional 0
  54. 54. 7665 C files (x86) 0 30 seconds per file (median) 0 85 hours total 0 4.1 % overhead (undiscipl.) 0 0 0 syntax errors 2.6.33.3 X86
  55. 55. Type Checking Linux 2.6.33.3 X86
  56. 56. SUMMARY AND PERSPECTIVE
  57. 57. Summary• Problems: • Improvements: • Separation of Concerns • Views • Obfuscation • Visual representation • Error-proneness • Variability-aware analysis• Retained advantages: • Remaining Problems: • Easy to use • Modular Type Checking • Flexible • Separate Compilation • Language-independent • Black-box Reuse • Low adoption barrier “Virtual Separation of Concerns”
  58. 58. Perspective • Preprocessors common in practice, despite criticism • Neglected by researchers • Tool support can address most problems • Currently scaling to Linux • Interim solution or serious alternative?Give preprocessors a second chance!
  59. 59. Questions? http://fosd.de/cide
  60. 60. Picture Credits© Stuck in Customs (cc by-nc-sa 2.0)© Horia Varlan (cc by 2.0)

×