Generic Programming Galore Using D                             Andrei Alexandrescu, PhD                                 Re...
37% offc 2011 Andrei Alexandrescu   2 / 33
Generic Programmingc 2011 Andrei Alexandrescu                         3 / 33
What is Generic Programming?c 2011 Andrei Alexandrescu      4 / 33
What is Generic Programming?         • Find most general algorithm representation             ◦ Narrowest requirements    ...
What is Generic Programming?         • Find most general algorithm representation             ◦ Narrowest requirements    ...
What is Generic Programming?         • Find most general algorithm representation             ◦ Narrowest requirements    ...
What is Generic Programming?         • Find most general algorithm representation             ◦ Narrowest requirements    ...
What is Generic Programming?         • Find most general algorithm representation             ◦ Narrowest requirements    ...
Generic Programming         • “Write once, instantiate anywhere”             ◦ Specialized implementations welcome        ...
Warmup: the “dream min”         • Should work at efficiency comparable to hand-written           code         • Take variad...
First prototype       auto min(L, R)(L lhs, R rhs) {          return rhs < lhs ? rhs : lhs;       }       auto a = min(x, ...
Variadic arguments       auto min(T...)(T x) {         static if (x.length > 2)           return min(min(x[0], x[1]), x[2 ...
Reject nonsensical calls       auto min(T...)(T x) if (x.length > 1) {         ...       }         • Rejects calls with 0 ...
Common type         • Task: Given a list of types, find the common type of all       template CommonType(T...)       {     ...
Using CommonType       auto min(T...)(T x)       if (x.length > 1         && is(typeof(CommonType!T.init < CommonType!T.in...
How about min over many elements?       auto min(R)(R range)       if (isInputRange!R &&           is(typeof(range.front <...
How about argmin now?       auto argmin(alias fun, R)(R r)       if (isInputRange!R &&           is(typeof(fun(r.front) < ...
argmin       auto s = ["abc", "a", "xz"];       auto m = s.argmin!((x) => x.length);         • Works on anything iterable ...
The Generative Connectionc 2011 Andrei Alexandrescu                           15 / 33
Generative programming         • In brief: code that generates code         • Generic programming often requires algorithm...
Embedded DSLs              Force into host language’s syntax?c 2011 Andrei Alexandrescu                         17 / 33
Embedded DSLs         •   Formatted printing?c 2011 Andrei Alexandrescu         18 / 33
Embedded DSLs         • Formatted printing?         • Regular expressions?c 2011 Andrei Alexandrescu        18 / 33
Embedded DSLs         • Formatted printing?         • Regular expressions?         • EBNF?c 2011 Andrei Alexandrescu      ...
Embedded DSLs         • Formatted printing?         • Regular expressions?         • EBNF?         • PEG?c 2011 Andrei Ale...
Embedded DSLs         •   Formatted printing?         •   Regular expressions?         •   EBNF?         •   PEG?         ...
Embedded DSLs         •   Formatted printing?         •   Regular expressions?         •   EBNF?         •   PEG?         ...
Embedded DSLs                    Here: use with native grammar                             Process during compilation     ...
Compile-Time Evaluation         • A large subset of D available for compile-time evaluation                 ulong factoria...
Code injection with mixin       mixin("writeln("hello, world");");       mixin(generateSomeCode());         • Not as glamo...
Wait a minute!c 2011 Andrei Alexandrescu                    22 / 33
Example: bitfields in library       struct A {         int a;         mixin(bitfields!(           uint, "x",    2,         ...
Parser       import pegged.grammar; // by Philippe Sigaud       mixin(grammar("         Expr     <    Factor AddExpr*     ...
Usage       // Parsing at compile-time:       static parseTree1 = Expr.parse(         "1 + 2 - (3*x-5)*6");       pragma(m...
Scaling up       1000 lines of D grammar →          3000 lines D parserc 2011 Andrei Alexandrescu         26 / 33
Highly integrated lex+yaccc 2011 Andrei Alexandrescu           27 / 33
What about regexen?c 2011 Andrei Alexandrescu              28 / 33
Compile-time regular expressions         • GSoC project by Dmitry Olshansky: FReD         • Fully UTF capable, no special ...
Summaryc 2011 Andrei Alexandrescu             31 / 33
Summary         • Generic/generative programming—long unattained           promise         • D’s angle             ◦ Power...
Grill the Speaker!   More about ranges? • Thought the talk was   boring • Intriguing! • He lost me at mixin •   Awesome • ...
Generic Programming Galore Using D
Upcoming SlideShare
Loading in...5
×

Generic Programming Galore Using D

6,615

Published on

Published in: Technology, Education
1 Comment
2 Likes
Statistics
Notes
No Downloads
Views
Total Views
6,615
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
50
Comments
1
Likes
2
Embeds 0
No embeds

No notes for slide

Generic Programming Galore Using D

  1. 1. Generic Programming Galore Using D Andrei Alexandrescu, PhD Research Scientist Facebookc 2011 Andrei Alexandrescu 1 / 33
  2. 2. 37% offc 2011 Andrei Alexandrescu 2 / 33
  3. 3. Generic Programmingc 2011 Andrei Alexandrescu 3 / 33
  4. 4. What is Generic Programming?c 2011 Andrei Alexandrescu 4 / 33
  5. 5. What is Generic Programming? • Find most general algorithm representation ◦ Narrowest requirements ◦ Widest guarantees ◦ Big-O() encapsulation should be a crime ◦ No need to regress to hand-written codec 2011 Andrei Alexandrescu 4 / 33
  6. 6. What is Generic Programming? • Find most general algorithm representation ◦ Narrowest requirements ◦ Widest guarantees ◦ Big-O() encapsulation should be a crime ◦ No need to regress to hand-written code • Define types to implement said requirementsc 2011 Andrei Alexandrescu 4 / 33
  7. 7. What is Generic Programming? • Find most general algorithm representation ◦ Narrowest requirements ◦ Widest guarantees ◦ Big-O() encapsulation should be a crime ◦ No need to regress to hand-written code • Define types to implement said requirements • Leverage the algorithm for ultimate reusec 2011 Andrei Alexandrescu 4 / 33
  8. 8. What is Generic Programming? • Find most general algorithm representation ◦ Narrowest requirements ◦ Widest guarantees ◦ Big-O() encapsulation should be a crime ◦ No need to regress to hand-written code • Define types to implement said requirements • Leverage the algorithm for ultimate reuse • ... • Profit!c 2011 Andrei Alexandrescu 4 / 33
  9. 9. What is Generic Programming? • Find most general algorithm representation ◦ Narrowest requirements ◦ Widest guarantees ◦ Big-O() encapsulation should be a crime ◦ No need to regress to hand-written code • Define types to implement said requirements • Leverage the algorithm for ultimate reuse • ... • Profit! ´ • Arguably one of the noblest endeavors of our metierc 2011 Andrei Alexandrescu 4 / 33
  10. 10. Generic Programming • “Write once, instantiate anywhere” ◦ Specialized implementations welcome • Prefers static type information and static expansion (macro style) • Fosters strong mathematical underpinnings ◦ Minimizing assumptions common theme in math • Tenuous relationship with binary interfaces • Starts with algorithms, not interfaces or objects • “Premature encapsulation is the root of some derangement”c 2011 Andrei Alexandrescu 5 / 33
  11. 11. Warmup: the “dream min” • Should work at efficiency comparable to hand-written code • Take variadic arguments: min(a, b, ...) ◦ Avoid nonsensical calls min() and min(a) • Work for all ordered types and conversions • Decline incompatible types, without prejudicec 2011 Andrei Alexandrescu 6 / 33
  12. 12. First prototype auto min(L, R)(L lhs, R rhs) { return rhs < lhs ? rhs : lhs; } auto a = min(x, y); • This function is as efficient as any specialized, handwritten version (true genericity) • Deduction for argument types and result typec 2011 Andrei Alexandrescu 7 / 33
  13. 13. Variadic arguments auto min(T...)(T x) { static if (x.length > 2) return min(min(x[0], x[1]), x[2 .. $]); else return x[0] > x[1] ? x[1] : x[0]; } ... auto m = min(a + b, 100, c); • x is not an array • This is not classic recursionc 2011 Andrei Alexandrescu 8 / 33
  14. 14. Reject nonsensical calls auto min(T...)(T x) if (x.length > 1) { ... } • Rejects calls with 0 or 1 arguments • Allows other overloads to take over, e.g. min element over a collection • More work needed ◦ Only accept types with a valid intersection ◦ Only accept types comparable with ”<”c 2011 Andrei Alexandrescu 9 / 33
  15. 15. Common type • Task: Given a list of types, find the common type of all template CommonType(T...) { static if (T.length == 1) alias T[0] CommonType; else static if (is(typeof(1 ? T[0].init : T[1].init) U)) alias CommonType!(U, T[2 .. $]) CommonType; else alias void CommonType; } // Usage static assert(is(CommonType!(int, short, long) == long));c 2011 Andrei Alexandrescu 10 / 33
  16. 16. Using CommonType auto min(T...)(T x) if (x.length > 1 && is(typeof(CommonType!T.init < CommonType!T.init) == bool)) { static if (x.length > 2) return min(min(x[0], x[1]), x[2 .. $]); else return x[0] > x[1] ? x[1] : x[0]; }c 2011 Andrei Alexandrescu 11 / 33
  17. 17. How about min over many elements? auto min(R)(R range) if (isInputRange!R && is(typeof(range.front < range.front) == bool)) { auto result = range.front; range.popFront(); foreach (e; range) { if (e < result) result = e; } return result; } auto m = [ 1, 5, 2, 0, 7, 9 ].min(); • Works over anything that can be iteratedc 2011 Andrei Alexandrescu 12 / 33
  18. 18. How about argmin now? auto argmin(alias fun, R)(R r) if (isInputRange!R && is(typeof(fun(r.front) < fun(r.front)) == bool)) { auto result = r.front; auto cache = fun(result); r.popFront(); foreach (e; r) { auto cand = fun(e); if (cand > cache) continue; result = e; cache = cand; } return result; }c 2011 Andrei Alexandrescu 13 / 33
  19. 19. argmin auto s = ["abc", "a", "xz"]; auto m = s.argmin!((x) => x.length); • Works on anything iterable and any predicate • Predicate is passed by alias • No loss of efficiencyc 2011 Andrei Alexandrescu 14 / 33
  20. 20. The Generative Connectionc 2011 Andrei Alexandrescu 15 / 33
  21. 21. Generative programming • In brief: code that generates code • Generic programming often requires algorithm specialization • Specification often present in a DSLc 2011 Andrei Alexandrescu 16 / 33
  22. 22. Embedded DSLs Force into host language’s syntax?c 2011 Andrei Alexandrescu 17 / 33
  23. 23. Embedded DSLs • Formatted printing?c 2011 Andrei Alexandrescu 18 / 33
  24. 24. Embedded DSLs • Formatted printing? • Regular expressions?c 2011 Andrei Alexandrescu 18 / 33
  25. 25. Embedded DSLs • Formatted printing? • Regular expressions? • EBNF?c 2011 Andrei Alexandrescu 18 / 33
  26. 26. Embedded DSLs • Formatted printing? • Regular expressions? • EBNF? • PEG?c 2011 Andrei Alexandrescu 18 / 33
  27. 27. Embedded DSLs • Formatted printing? • Regular expressions? • EBNF? • PEG? • SQL?c 2011 Andrei Alexandrescu 18 / 33
  28. 28. Embedded DSLs • Formatted printing? • Regular expressions? • EBNF? • PEG? • SQL? • . . . Pasta for everyone!c 2011 Andrei Alexandrescu 18 / 33
  29. 29. Embedded DSLs Here: use with native grammar Process during compilation Generate D code accordinglyc 2011 Andrei Alexandrescu 19 / 33
  30. 30. Compile-Time Evaluation • A large subset of D available for compile-time evaluation ulong factorial(uint n) { ulong result = 1; foreach (i; 2 .. n) result *= i; return result; } ... auto f1 = factorial(10); // run-time static f2 = factorial(10); // compile-timec 2011 Andrei Alexandrescu 20 / 33
  31. 31. Code injection with mixin mixin("writeln("hello, world");"); mixin(generateSomeCode()); • Not as glamorous as AST manipulation but darn effective • Easy to understand and debug • Now we have compile-time evaluation AND mixin. . .c 2011 Andrei Alexandrescu 21 / 33
  32. 32. Wait a minute!c 2011 Andrei Alexandrescu 22 / 33
  33. 33. Example: bitfields in library struct A { int a; mixin(bitfields!( uint, "x", 2, int, "y", 3, uint, "z", 2, bool, "flag", 1)); } A obj; obj.x = 2; obj.z = obj.x;c 2011 Andrei Alexandrescu 23 / 33
  34. 34. Parser import pegged.grammar; // by Philippe Sigaud mixin(grammar(" Expr < Factor AddExpr* AddExpr < (’+’/’-’) Factor Factor < Primary MulExpr* MulExpr < (’*’/’/’) Primary Primary < Parens / Number / Variable / ’-’ Primary Parens < ’(’ Expr ’)’ Number <~ [0-9]+ Variable <- Identifier "));c 2011 Andrei Alexandrescu 24 / 33
  35. 35. Usage // Parsing at compile-time: static parseTree1 = Expr.parse( "1 + 2 - (3*x-5)*6"); pragma(msg, parseTree1.capture); // Parsing at run-time: auto parseTree2 = Expr.parse(readln()); writeln(parseTree2.capture);c 2011 Andrei Alexandrescu 25 / 33
  36. 36. Scaling up 1000 lines of D grammar → 3000 lines D parserc 2011 Andrei Alexandrescu 26 / 33
  37. 37. Highly integrated lex+yaccc 2011 Andrei Alexandrescu 27 / 33
  38. 38. What about regexen?c 2011 Andrei Alexandrescu 28 / 33
  39. 39. Compile-time regular expressions • GSoC project by Dmitry Olshansky: FReD • Fully UTF capable, no special casing for ASCII • Two modes sharing the same backend: auto r1 = regex("^.*/([^/]+)/?$"); static r2 = ctRegex!("^.*/([^/]+)/?$"); • Run-time version uses intrinsics in a few places • Static version generates specialized automaton, then compiles itc 2011 Andrei Alexandrescu 29 / 33
  40. 40. Summaryc 2011 Andrei Alexandrescu 31 / 33
  41. 41. Summary • Generic/generative programming—long unattained promise • D’s angle ◦ Powerful base language ◦ Type manipulation ◦ Compile-time evaluation ◦ Code generationc 2011 Andrei Alexandrescu 32 / 33
  42. 42. Grill the Speaker! More about ranges? • Thought the talk was boring • Intriguing! • He lost me at mixin • Awesome • Went over my head • Meh • I wonder how I can implement binary search • What’s for dinner? • Accent is bothersome • Real programmers use C • Must. . . control. . . fist. . . of. . . death. . .c 2011 Andrei Alexandrescu 33 / 33
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×