1.
Generic Programming Galore Using D Andrei Alexandrescu, PhD Research Scientist Facebookc 2011 Andrei Alexandrescu 1 / 33

2.
37% offc 2011 Andrei Alexandrescu 2 / 33

3.
Generic Programmingc 2011 Andrei Alexandrescu 3 / 33

4.
What is Generic Programming?c 2011 Andrei Alexandrescu 4 / 33

5.
What is Generic Programming? • Find most general algorithm representation ◦ Narrowest requirements ◦ Widest guarantees ◦ Big-O() encapsulation should be a crime ◦ No need to regress to hand-written codec 2011 Andrei Alexandrescu 4 / 33

6.
What is Generic Programming? • Find most general algorithm representation ◦ Narrowest requirements ◦ Widest guarantees ◦ Big-O() encapsulation should be a crime ◦ No need to regress to hand-written code • Define types to implement said requirementsc 2011 Andrei Alexandrescu 4 / 33

7.
What is Generic Programming? • Find most general algorithm representation ◦ Narrowest requirements ◦ Widest guarantees ◦ Big-O() encapsulation should be a crime ◦ No need to regress to hand-written code • Define types to implement said requirements • Leverage the algorithm for ultimate reusec 2011 Andrei Alexandrescu 4 / 33

8.
What is Generic Programming? • Find most general algorithm representation ◦ Narrowest requirements ◦ Widest guarantees ◦ Big-O() encapsulation should be a crime ◦ No need to regress to hand-written code • Define types to implement said requirements • Leverage the algorithm for ultimate reuse • ... • Profit!c 2011 Andrei Alexandrescu 4 / 33

9.
What is Generic Programming? • Find most general algorithm representation ◦ Narrowest requirements ◦ Widest guarantees ◦ Big-O() encapsulation should be a crime ◦ No need to regress to hand-written code • Define types to implement said requirements • Leverage the algorithm for ultimate reuse • ... • Profit! ´ • Arguably one of the noblest endeavors of our metierc 2011 Andrei Alexandrescu 4 / 33

10.
Generic Programming • “Write once, instantiate anywhere” ◦ Specialized implementations welcome • Prefers static type information and static expansion (macro style) • Fosters strong mathematical underpinnings ◦ Minimizing assumptions common theme in math • Tenuous relationship with binary interfaces • Starts with algorithms, not interfaces or objects • “Premature encapsulation is the root of some derangement”c 2011 Andrei Alexandrescu 5 / 33

11.
Warmup: the “dream min” • Should work at efficiency comparable to hand-written code • Take variadic arguments: min(a, b, ...) ◦ Avoid nonsensical calls min() and min(a) • Work for all ordered types and conversions • Decline incompatible types, without prejudicec 2011 Andrei Alexandrescu 6 / 33

12.
First prototype auto min(L, R)(L lhs, R rhs) { return rhs < lhs ? rhs : lhs; } auto a = min(x, y); • This function is as efficient as any specialized, handwritten version (true genericity) • Deduction for argument types and result typec 2011 Andrei Alexandrescu 7 / 33

13.
Variadic arguments auto min(T...)(T x) { static if (x.length > 2) return min(min(x[0], x[1]), x[2 .. $]); else return x[0] > x[1] ? x[1] : x[0]; } ... auto m = min(a + b, 100, c); • x is not an array • This is not classic recursionc 2011 Andrei Alexandrescu 8 / 33

14.
Reject nonsensical calls auto min(T...)(T x) if (x.length > 1) { ... } • Rejects calls with 0 or 1 arguments • Allows other overloads to take over, e.g. min element over a collection • More work needed ◦ Only accept types with a valid intersection ◦ Only accept types comparable with ”<”c 2011 Andrei Alexandrescu 9 / 33

15.
Common type • Task: Given a list of types, find the common type of all template CommonType(T...) { static if (T.length == 1) alias T[0] CommonType; else static if (is(typeof(1 ? T[0].init : T[1].init) U)) alias CommonType!(U, T[2 .. $]) CommonType; else alias void CommonType; } // Usage static assert(is(CommonType!(int, short, long) == long));c 2011 Andrei Alexandrescu 10 / 33

16.
Using CommonType auto min(T...)(T x) if (x.length > 1 && is(typeof(CommonType!T.init < CommonType!T.init) == bool)) { static if (x.length > 2) return min(min(x[0], x[1]), x[2 .. $]); else return x[0] > x[1] ? x[1] : x[0]; }c 2011 Andrei Alexandrescu 11 / 33

17.
How about min over many elements? auto min(R)(R range) if (isInputRange!R && is(typeof(range.front < range.front) == bool)) { auto result = range.front; range.popFront(); foreach (e; range) { if (e < result) result = e; } return result; } auto m = [ 1, 5, 2, 0, 7, 9 ].min(); • Works over anything that can be iteratedc 2011 Andrei Alexandrescu 12 / 33

18.
How about argmin now? auto argmin(alias fun, R)(R r) if (isInputRange!R && is(typeof(fun(r.front) < fun(r.front)) == bool)) { auto result = r.front; auto cache = fun(result); r.popFront(); foreach (e; r) { auto cand = fun(e); if (cand > cache) continue; result = e; cache = cand; } return result; }c 2011 Andrei Alexandrescu 13 / 33

19.
argmin auto s = ["abc", "a", "xz"]; auto m = s.argmin!((x) => x.length); • Works on anything iterable and any predicate • Predicate is passed by alias • No loss of efficiencyc 2011 Andrei Alexandrescu 14 / 33

20.
The Generative Connectionc 2011 Andrei Alexandrescu 15 / 33

21.
Generative programming • In brief: code that generates code • Generic programming often requires algorithm specialization • Specification often present in a DSLc 2011 Andrei Alexandrescu 16 / 33

22.
Embedded DSLs Force into host language’s syntax?c 2011 Andrei Alexandrescu 17 / 33

23.
Embedded DSLs • Formatted printing?c 2011 Andrei Alexandrescu 18 / 33

24.
Embedded DSLs • Formatted printing? • Regular expressions?c 2011 Andrei Alexandrescu 18 / 33

25.
Embedded DSLs • Formatted printing? • Regular expressions? • EBNF?c 2011 Andrei Alexandrescu 18 / 33

26.
Embedded DSLs • Formatted printing? • Regular expressions? • EBNF? • PEG?c 2011 Andrei Alexandrescu 18 / 33

27.
Embedded DSLs • Formatted printing? • Regular expressions? • EBNF? • PEG? • SQL?c 2011 Andrei Alexandrescu 18 / 33

28.
Embedded DSLs • Formatted printing? • Regular expressions? • EBNF? • PEG? • SQL? • . . . Pasta for everyone!c 2011 Andrei Alexandrescu 18 / 33

29.
Embedded DSLs Here: use with native grammar Process during compilation Generate D code accordinglyc 2011 Andrei Alexandrescu 19 / 33

30.
Compile-Time Evaluation • A large subset of D available for compile-time evaluation ulong factorial(uint n) { ulong result = 1; foreach (i; 2 .. n) result *= i; return result; } ... auto f1 = factorial(10); // run-time static f2 = factorial(10); // compile-timec 2011 Andrei Alexandrescu 20 / 33

31.
Code injection with mixin mixin("writeln("hello, world");"); mixin(generateSomeCode()); • Not as glamorous as AST manipulation but darn effective • Easy to understand and debug • Now we have compile-time evaluation AND mixin. . .c 2011 Andrei Alexandrescu 21 / 33

32.
Wait a minute!c 2011 Andrei Alexandrescu 22 / 33

33.
Example: bitfields in library struct A { int a; mixin(bitfields!( uint, "x", 2, int, "y", 3, uint, "z", 2, bool, "flag", 1)); } A obj; obj.x = 2; obj.z = obj.x;c 2011 Andrei Alexandrescu 23 / 33

34.
Parser import pegged.grammar; // by Philippe Sigaud mixin(grammar(" Expr < Factor AddExpr* AddExpr < (’+’/’-’) Factor Factor < Primary MulExpr* MulExpr < (’*’/’/’) Primary Primary < Parens / Number / Variable / ’-’ Primary Parens < ’(’ Expr ’)’ Number <~ [0-9]+ Variable <- Identifier "));c 2011 Andrei Alexandrescu 24 / 33

35.
Usage // Parsing at compile-time: static parseTree1 = Expr.parse( "1 + 2 - (3*x-5)*6"); pragma(msg, parseTree1.capture); // Parsing at run-time: auto parseTree2 = Expr.parse(readln()); writeln(parseTree2.capture);c 2011 Andrei Alexandrescu 25 / 33

36.
Scaling up 1000 lines of D grammar → 3000 lines D parserc 2011 Andrei Alexandrescu 26 / 33

37.
Highly integrated lex+yaccc 2011 Andrei Alexandrescu 27 / 33

38.
What about regexen?c 2011 Andrei Alexandrescu 28 / 33

39.
Compile-time regular expressions • GSoC project by Dmitry Olshansky: FReD • Fully UTF capable, no special casing for ASCII • Two modes sharing the same backend: auto r1 = regex("^.*/([^/]+)/?$"); static r2 = ctRegex!("^.*/([^/]+)/?$"); • Run-time version uses intrinsics in a few places • Static version generates specialized automaton, then compiles itc 2011 Andrei Alexandrescu 29 / 33

40.
Summaryc 2011 Andrei Alexandrescu 31 / 33

41.
Summary • Generic/generative programming—long unattained promise • D’s angle ◦ Powerful base language ◦ Type manipulation ◦ Compile-time evaluation ◦ Code generationc 2011 Andrei Alexandrescu 32 / 33

42.
Grill the Speaker! More about ranges? • Thought the talk was boring • Intriguing! • He lost me at mixin • Awesome • Went over my head • Meh • I wonder how I can implement binary search • What’s for dinner? • Accent is bothersome • Real programmers use C • Must. . . control. . . fist. . . of. . . death. . .c 2011 Andrei Alexandrescu 33 / 33