1.
Generic Programming Galore Using D
Andrei Alexandrescu, PhD
Research Scientist
Facebook
c 2011 Andrei Alexandrescu 1 / 33

2.
37% off
c 2011 Andrei Alexandrescu 2 / 33

3.
Generic Programming
c 2011 Andrei Alexandrescu 3 / 33

4.
What is Generic Programming?
c 2011 Andrei Alexandrescu 4 / 33

5.
What is Generic Programming?
• Find most general algorithm representation
◦ Narrowest requirements
◦ Widest guarantees
◦ Big-O() encapsulation should be a crime
◦ No need to regress to hand-written code
c 2011 Andrei Alexandrescu 4 / 33

6.
What is Generic Programming?
• Find most general algorithm representation
◦ Narrowest requirements
◦ Widest guarantees
◦ Big-O() encapsulation should be a crime
◦ No need to regress to hand-written code
• Define types to implement said requirements
c 2011 Andrei Alexandrescu 4 / 33

7.
What is Generic Programming?
• Find most general algorithm representation
◦ Narrowest requirements
◦ Widest guarantees
◦ Big-O() encapsulation should be a crime
◦ No need to regress to hand-written code
• Define types to implement said requirements
• Leverage the algorithm for ultimate reuse
c 2011 Andrei Alexandrescu 4 / 33

8.
What is Generic Programming?
• Find most general algorithm representation
◦ Narrowest requirements
◦ Widest guarantees
◦ Big-O() encapsulation should be a crime
◦ No need to regress to hand-written code
• Define types to implement said requirements
• Leverage the algorithm for ultimate reuse
• ...
• Profit!
c 2011 Andrei Alexandrescu 4 / 33

9.
What is Generic Programming?
• Find most general algorithm representation
◦ Narrowest requirements
◦ Widest guarantees
◦ Big-O() encapsulation should be a crime
◦ No need to regress to hand-written code
• Define types to implement said requirements
• Leverage the algorithm for ultimate reuse
• ...
• Profit!
´
• Arguably one of the noblest endeavors of our metier
c 2011 Andrei Alexandrescu 4 / 33

10.
Generic Programming
• “Write once, instantiate anywhere”
◦ Specialized implementations welcome
• Prefers static type information and static expansion
(macro style)
• Fosters strong mathematical underpinnings
◦ Minimizing assumptions common theme in math
• Tenuous relationship with binary interfaces
• Starts with algorithms, not interfaces or objects
• “Premature encapsulation is the root of some
derangement”
c 2011 Andrei Alexandrescu 5 / 33

11.
Warmup: the “dream min”
• Should work at efficiency comparable to hand-written
code
• Take variadic arguments: min(a, b, ...)
◦ Avoid nonsensical calls min() and min(a)
• Work for all ordered types and conversions
• Decline incompatible types, without prejudice
c 2011 Andrei Alexandrescu 6 / 33

12.
First prototype
auto min(L, R)(L lhs, R rhs) {
return rhs < lhs ? rhs : lhs;
}
auto a = min(x, y);
• This function is as efficient as any specialized,
handwritten version (true genericity)
• Deduction for argument types and result type
c 2011 Andrei Alexandrescu 7 / 33

13.
Variadic arguments
auto min(T...)(T x) {
static if (x.length > 2)
return min(min(x[0], x[1]), x[2 .. $]);
else
return x[0] > x[1] ? x[1] : x[0];
}
...
auto m = min(a + b, 100, c);
• x is not an array
• This is not classic recursion
c 2011 Andrei Alexandrescu 8 / 33

14.
Reject nonsensical calls
auto min(T...)(T x) if (x.length > 1) {
...
}
• Rejects calls with 0 or 1 arguments
• Allows other overloads to take over, e.g. min element
over a collection
• More work needed
◦ Only accept types with a valid intersection
◦ Only accept types comparable with ”<”
c 2011 Andrei Alexandrescu 9 / 33

15.
Common type
• Task: Given a list of types, find the common type of all
template CommonType(T...)
{
static if (T.length == 1)
alias T[0] CommonType;
else
static if (is(typeof(1 ? T[0].init : T[1].init) U))
alias CommonType!(U, T[2 .. $]) CommonType;
else
alias void CommonType;
}
// Usage
static assert(is(CommonType!(int, short, long) == long));
c 2011 Andrei Alexandrescu 10 / 33

16.
Using CommonType
auto min(T...)(T x)
if (x.length > 1
&& is(typeof(CommonType!T.init < CommonType!T.init)
== bool))
{
static if (x.length > 2)
return min(min(x[0], x[1]), x[2 .. $]);
else
return x[0] > x[1] ? x[1] : x[0];
}
c 2011 Andrei Alexandrescu 11 / 33

17.
How about min over many elements?
auto min(R)(R range)
if (isInputRange!R &&
is(typeof(range.front < range.front) == bool))
{
auto result = range.front;
range.popFront();
foreach (e; range) {
if (e < result) result = e;
}
return result;
}
auto m = [ 1, 5, 2, 0, 7, 9 ].min();
• Works over anything that can be iterated
c 2011 Andrei Alexandrescu 12 / 33

18.
How about argmin now?
auto argmin(alias fun, R)(R r)
if (isInputRange!R &&
is(typeof(fun(r.front) < fun(r.front)) == bool))
{
auto result = r.front;
auto cache = fun(result);
r.popFront();
foreach (e; r) {
auto cand = fun(e);
if (cand > cache) continue;
result = e;
cache = cand;
}
return result;
}
c 2011 Andrei Alexandrescu 13 / 33

19.
argmin
auto s = ["abc", "a", "xz"];
auto m = s.argmin!((x) => x.length);
• Works on anything iterable and any predicate
• Predicate is passed by alias
• No loss of efficiency
c 2011 Andrei Alexandrescu 14 / 33

20.
The Generative Connection
c 2011 Andrei Alexandrescu 15 / 33

21.
Generative programming
• In brief: code that generates code
• Generic programming often requires algorithm
specialization
• Specification often present in a DSL
c 2011 Andrei Alexandrescu 16 / 33

22.
Embedded DSLs
Force into host language’s syntax?
c 2011 Andrei Alexandrescu 17 / 33

23.
Embedded DSLs
• Formatted printing?
c 2011 Andrei Alexandrescu 18 / 33

24.
Embedded DSLs
• Formatted printing?
• Regular expressions?
c 2011 Andrei Alexandrescu 18 / 33

25.
Embedded DSLs
• Formatted printing?
• Regular expressions?
• EBNF?
c 2011 Andrei Alexandrescu 18 / 33

26.
Embedded DSLs
• Formatted printing?
• Regular expressions?
• EBNF?
• PEG?
c 2011 Andrei Alexandrescu 18 / 33

27.
Embedded DSLs
• Formatted printing?
• Regular expressions?
• EBNF?
• PEG?
• SQL?
c 2011 Andrei Alexandrescu 18 / 33

28.
Embedded DSLs
• Formatted printing?
• Regular expressions?
• EBNF?
• PEG?
• SQL?
• . . . Pasta for everyone!
c 2011 Andrei Alexandrescu 18 / 33

29.
Embedded DSLs
Here: use with native grammar
Process during compilation
Generate D code accordingly
c 2011 Andrei Alexandrescu 19 / 33

30.
Compile-Time Evaluation
• A large subset of D available for compile-time evaluation
ulong factorial(uint n) {
ulong result = 1;
foreach (i; 2 .. n) result *= i;
return result;
}
...
auto f1 = factorial(10); // run-time
static f2 = factorial(10); // compile-time
c 2011 Andrei Alexandrescu 20 / 33

31.
Code injection with mixin
mixin("writeln("hello, world");");
mixin(generateSomeCode());
• Not as glamorous as AST manipulation but darn
effective
• Easy to understand and debug
• Now we have compile-time evaluation AND mixin. . .
c 2011 Andrei Alexandrescu 21 / 33

32.
Wait a minute!
c 2011 Andrei Alexandrescu 22 / 33

33.
Example: bitfields in library
struct A {
int a;
mixin(bitfields!(
uint, "x", 2,
int, "y", 3,
uint, "z", 2,
bool, "flag", 1));
}
A obj;
obj.x = 2;
obj.z = obj.x;
c 2011 Andrei Alexandrescu 23 / 33

34.
Parser
import pegged.grammar; // by Philippe Sigaud
mixin(grammar("
Expr < Factor AddExpr*
AddExpr < (’+’/’-’) Factor
Factor < Primary MulExpr*
MulExpr < (’*’/’/’) Primary
Primary < Parens / Number / Variable
/ ’-’ Primary
Parens < ’(’ Expr ’)’
Number <~ [0-9]+
Variable <- Identifier
"));
c 2011 Andrei Alexandrescu 24 / 33

35.
Usage
// Parsing at compile-time:
static parseTree1 = Expr.parse(
"1 + 2 - (3*x-5)*6");
pragma(msg, parseTree1.capture);
// Parsing at run-time:
auto parseTree2 = Expr.parse(readln());
writeln(parseTree2.capture);
c 2011 Andrei Alexandrescu 25 / 33

36.
Scaling up
1000 lines of D grammar →
3000 lines D parser
c 2011 Andrei Alexandrescu 26 / 33

37.
Highly integrated lex+yacc
c 2011 Andrei Alexandrescu 27 / 33

38.
What about regexen?
c 2011 Andrei Alexandrescu 28 / 33

39.
Compile-time regular expressions
• GSoC project by Dmitry Olshansky: FReD
• Fully UTF capable, no special casing for ASCII
• Two modes sharing the same backend:
auto r1 = regex("^.*/([^/]+)/?$");
static r2 = ctRegex!("^.*/([^/]+)/?$");
• Run-time version uses intrinsics in a few places
• Static version generates specialized automaton, then
compiles it
c 2011 Andrei Alexandrescu 29 / 33

40.
Summary
c 2011 Andrei Alexandrescu 31 / 33

41.
Summary
• Generic/generative programming—long unattained
promise
• D’s angle
◦ Powerful base language
◦ Type manipulation
◦ Compile-time evaluation
◦ Code generation
c 2011 Andrei Alexandrescu 32 / 33

42.
Grill the Speaker!
More about ranges? • Thought the talk was
boring • Intriguing! • He lost me at mixin •
Awesome • Went over my head • Meh • I wonder
how I can implement binary search • What’s for
dinner? • Accent is bothersome • Real
programmers use C • Must. . . control. . . fist. . .
of. . . death. . .
c 2011 Andrei Alexandrescu 33 / 33