Virtual Separation of Concerns

Photo (cc) Horia Varlan

Virtual Separation of Concerns
Toward preprocessors 2.0

Christian Kästner

Christian Kästner: Virtual Separation of Concerns 2

Software Product Lines & Features

• Set of related software systems (variants)
in a domain
• Generated from common code base
• Variants distinguished in terms of features


Conditional Compilation with
Preprocessors
● Annotated code static int _rep_queue_filedone(...
DB_ENV *dbenv;
fragments REP *rep;
__rep_fileinfo_args *rfp; {
● Variants with and #ifndef HAVE_QUEUE
COMPQUIET(rep, NULL);
without features COMPQUIET(rfp, NULL);
return (__db_no_queue_am(dbenv
#else
db_pgno_t first, last;
● Common for product- u_int32_t flags;
int empty, ret, t_ret;
line implementation #ifdef DIAGNOSTIC
DB_MSGBUF mb;
in industry #endif
...
Excerpt from Oracle’s Berkeley DB

Preprocessors in C, C++, Java ME, Pascal, Erlang, C#, Visual Basic, …
GPP, GNU M4, SPLET, Frames/XVCL, Gears, pure::variants


Objections / Criticism

Designed in the 70th and hardly evolved since

“#ifdef considered harmful”
“#ifdef hell”
“maintenance becomes a ‘hit or miss’ process”

“is difficult to determine if the code being
viewed is actually compiled into the system”

“incomprehensible source texts”
“programming errors are easy “CPP makes maintenance difficult”
to make and difficult to detect”
“source code rapidly
“preprocessor diagnostics are poor” becomes a maze”


Academia: Compositional Approaches

Base / Platform
class Stack {
class Stack {
void push(Object o) {
elementData[size++] = o;
}
}
...
...
}
}

class Stack {
class Stack {
Lock l = lock(o);
Feature: Queue Lock l = lock(o);
refines class Stack {
Composition }
}
l.unlock();
l.unlock();
...
void push(Object o) { ...
Lock l = lock(o);
Lock l = lock(o); }
}
Super.push(o);
Super.push(o);
l.unlock();
l.unlock();
}
}
...
...
}
}
Module
Components
Feature: Diagnostic Frameworks, Plug-ins
aspect Diagnostics {
aspect Diagnostics {
...
Feature-Modules / Mixin Layers / …
...
}
} Aspects / Subjects, Hyper/J


Agenda

• Problems and Advantages of Preprocessors
• 4 Improvements
• Views
• Visual Representation
• Disciplined Annotations
• Product-Line-Aware Type System
• Summary and Perspective


Problem 1: Lack of Separation of Concerns

• Scattered and tangled source code
• Global search to understand a feature
• Deleting obsolete feature as challenge
• Difficult distributed development
• Difficult reuse

Beispiel Berkeley DB:
Feature Sperren mit 1835 LOC
in 28 von 300 Dateien
an 155 Stellen


Problem 2: Obfuscation

• Mixture of two class Stack {
class Stack {
void push(Object o
languages void push(Object o
#ifdef SYNC
#ifdef SYNC
, Transaction txn
(C und #ifdef, …) , Transaction txn
#endif
#endif
) {
• Control flow hard to ) {
if (o==null
if (o==null
#ifdef SYNC
#ifdef SYNC
follow || txn==null
|| txn==null
#endif
#endif
• Additional line breaks ) return;
) return;
#ifdef SYNC
#ifdef SYNC
destroy code layout Lock l=txn.lock(o);
Lock l=txn.lock(o);
#endif
#endif
• Long annotations elementData[size++] = o;
#ifdef SYNC
#ifdef SYNC
difficult to recognize l.unlock();
l.unlock();
#endif
#endif
fireStackChanged();
fireStackChanged();
}
}
}
}


Preprocessor in Femto OS


Problem 3: Error-Prone

• Syntax Errors • Type Errors
static int _rep_queue_filedone(...) #ifdef TABLES
DB_ENV *dbenv; class Table {
REP *rep; void insert(Object data,
__rep_fileinfo_args *rfp; { Txn txn) {
#ifndef HAVE_QUEUE storage.set(data,
COMPQUIET(rep, NULL); txn.getLock());
COMPQUIET(rfp, NULL); }
return (__db_no_queue_am(dbenv)); }
#else #endif
db_pgno_t first, last; class Storage {
u_int32_t flags; #ifdef WRITE
int empty, ret, t_ret; boolean set(…) { ... }
#ifdef DIAGNOSTIC #endif
DB_MSGBUF mb; }
#endif
// over 100 lines of additional code
}
#endif



#else #endif
DB_MSGBUF mb; }
#endif
}
#endif


Advantages

• Easy to use
• “annotate and remove”
• Already part of many languages / simple tools
• Most developers already familiar

• Flexible / Expressive
• Language-independent / Uniform

• Low Adoption Barrier
• esp. with legacy code


Agenda

• 4 Improvements
• Views


[ICSE’08, ViSPLE’08]

Views

• Emulate modularity
• Hide irrelevant source code
• Hides entire files
• Hides code inside files

Related work:
• C-CLR
• Version Editor
• FeatureMapper
• pure::variants
• Effective Views
• on-demand remodularization
…


Expression Problem


View on Feature Eval

• Shows all files containing Eval code
• Shows context information (kursiv)
• Hides code without annotations


View on the Variant “Add and Eval”

• Entirely hides file Power
• Hides method print in Add ([…])
• Code without annotation remains visible


Agenda

• 4 Improvements
• Views


[ICSE’08, ViSPLE’08]

Visual Representation: Background Colors

class Stack {
class Stack {
void push(Object o
void push(Object o
#ifdef SYNC
#ifdef SYNC
, Transaction txn
, Transaction txn
#endif
#endif
) {
) {
if (o==null
if (o==null
#ifdef SYNC
#ifdef SYNC
|| txn==null
|| txn==null
#endif
#endif
) return;
) return;
#ifdef SYNC
#ifdef SYNC
Lock l=txn.lock(o);
Lock l=txn.lock(o);
#endif
#endif
#ifdef SYNC
#ifdef SYNC Related Work:
l.unlock();
l.unlock();
#endif • Spotlight
#endif
fireStackChanged();
fireStackChanged(); • NetBeans
}} • fmp2rsm
}
} • FeatureMapper
• AspectBrowser
…


Alternative Visual Representations

● NetBeans ● Spotlight


Scaling Visual Representations

• Focus on few features at a time
• Repeating colors / manual assignment
sufficient
• Analysis of 4 Java ME and 40 C programs:
• 96 % pages of source code with ≤ 3 colors
• 99 % pages of source code with ≤ 7 colors


Program Comprehension: An Experiment

• #ifdef vs. colors
• 43 subjects in 2
groups
• S1-2: search tasks
faster with colors
(43% & 23%)
• M1-3: maintenance
tasks same perform.
• M4: maintenance
task with red back-
ground color -37%
• No influence on correctness
• Subjects prefer colors


Agenda

• 4 Improvements
• Views


[TOOLS‘09]

Disciplined Annotations
class Foo { }
• Syntax errors caused by
annotations on plain text class Foo { }

• Solution: Use underlying artifact structure
• Enforce disciplined annotations on entire classes,
methods, or statements
• Annotations of isolated brackets or ClassDeclaration
Name=C
keywords regarded as undisciplined MethodDeclaration
Name=m

ReturnType Parameter
class C { Type-void Name=p
Block
void m(int p) {
s1();
MethodInvk MethodInvk
s2(p, true); Name=s1 Name=s2
}
} Parameter Parameter
Name=p Name=true


Variant Generation by AST Transformation
ClassDeclaration ClassDeclaration
Name=C Name=C

MethodDeclaration
MethodDeclaration
Name=m
Name=m

ReturnType Parameter
Block
Type-void Name=p ReturnType Parameter
Block
Type-void Name=p
MethodInvk MethodInvk
Name=s1 Name=s2
variant MethodInvk
Name=s1
Parameter Parameter generation
Name=p Name=true
print /
parsing unparse
class C { class C {
void m(int p) { void m(int p) {
s1(); s1();
s2(p, true); }
} }
}

Variant generation cannot cause syntax errors.


Expressiveness of Disciplined Annotations

• Not all annotations are possible
• Sometimes workarounds required
• Experience: not a significant restriction in
practice
• ~90% of all annotations in 40 C programs is
disciplined already
high flexibility no flexibility

unrestricted disciplined annotations classes no
annotations (underlying structure) only annotations
on any character


Agenda

• 4 Improvements
• Views


[ASE’08]

Product-Line-Aware Type System
class Database {
Storage storage;
• Base: Type correct without void insert(Object data
Txn txn) {
annotations (for tool support) storage.set(data,
txn.getLock())
• Detecting all pairs: }
}
• Method invocation and class Storage {
#ifdef WRITE
method declaration boolean set(…) { ... }
• Reference and declaration #endif
}
of class, variable, etc.
• … Related Work:
• Comparison of annotations for • SafeGen
• fmp2rsm
each pair • Safe Composition
• FFJPL
Declaration annotated + Error in some • Conditional
Reference not annotated variants Methods


Type Checking in the Context
of a Feature Model
¬Read
• Handling dependencies class Database {
between features void insert(…) {
storage.set(…);

• Can all variants with
}
}
class Storage {
reference reach a boolean set(…) { ... }
corresp. declaration? }
Write

FM = API ∧ ( API ⇒
• Implementation: Propositional (read ∨ write))
logic and SAT solvers:
In all variants in which code fragment A is
included also code fragment B is included:
FM ⇒ ( AT (a ) ⇒ AT (b))


[ASE’08]

Formalization: CFJ

● On top of Featherweight Java
● Theorem: Generation preserves typing
● Formal proof with Coq


Implementation: CIDE

http://fosd.de/cide


[GPCE’09]

Automated Refactorings

• Feature Module ● Annotationen
class Stack {
class Stack { class Stack {
class Stack {
int[] data;
int[] data; int[] data;
int[] data;
Base

void push(int o) { … }
void push(int o) { … } #ifdef UNDO
#ifdef UNDO
int pop() { /*...*/ }
int pop() { /*...*/ } int backup;
int backup;
}
} void undo() { /*...*/ }
void undo() { /*...*/ }
#endif
#endif
refines class Stack { #ifdef TOP
#ifdef TOP
Automated Transformationtop() { /*...*/ }
Top

int top() { /*...*/ }
int top() { /*...*/ } int top() { /*...*/ }
int
}
} #endif
#endif
void push(int o) {
void push(int o) {
refines class Stack { #ifdef UNDO
#ifdef UNDO
int backup;
int backup; backup = top();
backup = top();
void undo() { /*...*/ }
void undo() { /*...*/ } #endif
#endif
Undo

void push(int o) {
void push(int o) { /*...*/
/*...*/
backup = top();
backup = top(); }
}
original(v);
original(v); int pop() { /*...*/ }
int pop() { /*...*/ }
}
} }
}


Agenda

• 4 Improvements
• Views


Summary

• Problems: • Improvements:
• Separation of Concerns • Views
• Obfuscation • Visual representation
• Error-proneness • Disciplined annotations
• Product-line-aware type
system
• Retained advantages:
• Easy to use
• Flexible
• Language-independent
• Low adoption barrier

“Virtual Separation of Concerns”


Summary

system
• Easy to use
• Flexible
• Language-independent



Summary

system
• Easy to use • Remaining Problems:
• Flexible • Separate Compilation
• Language-independent • Black-box Reuse



Perspective

• Preprocessors common in practice,
despite strong criticism
• Neglected by researchers
• Tool support can address most problems

• Interim solution (with migration path) or
serious alternative?

• Give preprocessors a second chance!


Questions?

http://fosd.de/cide

Virtual Separation of Concerns

More Related Content

What's hot

Viewers also liked

Similar to Virtual Separation of Concerns

Recently uploaded

Virtual Separation of Concerns