Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Oleksii Holub "Expression trees in C#"

585 views

Published on

Expression trees is an obscure, although very interesting feature in .NET. Most people probably think of it as something synonymous with object-relational mapping frameworks, but despite being its most common use case, it’s not the only one. There are a lot of creative things you can do with expression trees, including code generation, transpilation, metaprogramming, and more. During this talk, I will explain what this feature really is about and guide you through some examples where it provides real-life benefits.

We'll talk about:

What is an expression tree
How to compile code at runtime
How to make reflection faster
How to implement generic operators
How to turn DSLs into expression trees
How to make metaprogramming type-safe
How to translate a lambda into a different language...and more

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Oleksii Holub "Expression trees in C#"

  1. 1. Speaker: Alexey Golub @Tyrrrz Expression Trees in C# I heard you like code, so we put code in your code so you can code while you code
  2. 2. /whois ${speaker} Speaker: Alexey Golub @Tyrrrz • Open-source developer ✨ • Conference speaker & blogger 🌐️ • C#, F#, JavaScript 💻 • Cloud & web ☁️ • Automation & DevOps ⚙️
  3. 3. What is an expression tree? Speaker: Alexey Golub @Tyrrrz
  4. 4. Speaker: Alexey Golub @Tyrrrz +Constant (2) Constant (3) Plus operator 2 3 Binary expression
  5. 5. Speaker: Alexey Golub @Tyrrrz !string.IsNullOrWhiteSpace(personName) ? "Greetings, " + personName : null; string? GetGreeting(string personName) { return }
  6. 6. Speaker: Alexey Golub @Tyrrrz "Greetings, " ! personName string.IsNullOrWhiteSpace( ) null;: ? + personName OPERATOR "NOT" METHOD CALL PARAMETER PARAMETERCONSTANT OPERATOR "ADD" CONSTANT
  7. 7. Speaker: Alexey Golub @Tyrrrz { Ternary conditional } { + } TRUE { null } FALSE { Method call } CONDITION { string.IsNullOrWhiteSpace } { personName } { personName } { "Greetings, " } { ! }
  8. 8. Speaker: Alexey Golub @Tyrrrz Expression Tree describes the structure of an expression
  9. 9. Expression Trees in .NET • Object model represented by types deriving from System.Linq.Expressions.Expression • Can be constructed manually using factory methods • Can be inferred from lambdas by the compiler • Lambda expression trees can be converted into delegates Speaker: Alexey Golub @Tyrrrz
  10. 10. Constructing Expression Trees Manually Speaker: Alexey Golub @Tyrrrz
  11. 11. Speaker: Alexey Golub @Tyrrrz
  12. 12. Speaker: Alexey Golub @Tyrrrz Expression.Constant(...) ConstantExpression Expression.New(...) NewExpression Expression.Assign(...) BinaryExpression Expression.Equal(...) BinaryExpression Expression.Call(...) MethodCallExpression Expression.Condition(...) ConditionalExpression Expression.Loop(...) LoopExpression ...
  13. 13. Speaker: Alexey Golub @Tyrrrz !string.IsNullOrWhiteSpace(personName) ? "Greetings, " + personName : null; Let’s recreate our expression dynamically
  14. 14. Speaker: Alexey Golub @Tyrrrz public Func<string, string?> ConstructGreetingFunction() { var personNameParameter = Expression.Parameter(typeof(string), "personName"); var isNullOrWhiteSpaceMethod = typeof(string) .GetMethod(nameof(string.IsNullOrWhiteSpace)); var condition = Expression.Not( Expression.Call(isNullOrWhiteSpaceMethod, personNameParameter)); var trueClause = Expression.Add( Expression.Constant("Greetings, "), personNameParameter); var falseClause = Expression.Constant(null, typeof(string)); var conditional = Expression.Condition(condition, trueClause, falseClause); var lambda = Expression.Lambda<Func<string, string?>>(conditional, personNameParameter); return lambda.Compile(); }
  15. 15. Speaker: Alexey Golub @Tyrrrz var getGreeting = ConstructGreetingFunction(); var greetingForJohn = getGreeting("John"); The binary operator Add is not defined for the types 'System.String' and 'System.String'.
  16. 16. Speaker: Alexey Golub @Tyrrrz We need to call string.Concat() directly var concatMethod = typeof(string) .GetMethod(nameof(string.Concat), new[] {typeof(string), typeof(string)}); var trueClause = Expression.Call( concatMethod, Expression.Constant("Greetings, "), personNameParameter);
  17. 17. Speaker: Alexey Golub @Tyrrrz var getGreetings = ConstructGreetingFunction(); var greetingsForJohn = getGreetings("John"); var greetingsForNobody = getGreetings(" "); // "Greetings, John" // <null>
  18. 18. Optimizing Reflection-Heavy Code Speaker: Alexey Golub @Tyrrrz
  19. 19. Speaker: Alexey Golub @Tyrrrz How can we invoke Execute() from the outside? public class Command { private int Execute() /> 42; } public static int CallExecute(Command command) => (int) typeof(Command) .GetMethod("Execute", BindingFlags.NonPublic | BindingFlags.Instance) .Invoke(command, null);
  20. 20. Speaker: Alexey Golub @Tyrrrz public static class ReflectionCached { private static MethodInfo ExecuteMethod { get; } = typeof(Command) .GetMethod("Execute", BindingFlags.NonPublic | BindingFlags.Instance); public static int CallExecute(Command command) => (int) ExecuteMethod.Invoke(command, null); } public static class Re/lectionDelegate { private static MethodInfo ExecuteMethod { get; } = typeof(Command) .GetMethod("Execute", Binding/lags.NonPublic | Binding/lags.Instance); private static Func<Command, int> Impl { get; } = (Func<Command, int>) Delegate .CreateDelegate(typeof(Func<Command, int>), ExecuteMethod); public static int CallExecute(Command command) /> Impl(command); } Using cached MethodInfo Using Delegate.CreateDelegate
  21. 21. public static class ExpressionTrees { private static MethodInfo ExecuteMethod { get; } = typeof(Command) .GetMethod("Execute", Binding/lags.NonPublic | Binding/lags.Instance); private static Func<Command, int> Impl { get; } static ExpressionTrees() { var instance = Expression.Parameter(typeof(Command)); var call = Expression.Call(instance, ExecuteMethod); Impl = Expression.Lambda<Func<Command, int/>(call, instance).Compile(); } public static int CallExecute(Command command) /> Impl(command); } Speaker: Alexey Golub @Tyrrrz Lazy thread-safe initialization via static constructor
  22. 22. Speaker: Alexey Golub @Tyrrrz public class Benchmarks { [Benchmark(Description = "Reflection", Baseline = true)] public int Reflection() => (int) typeof(Command) .GetMethod("Execute", BindingFlags.NonPublic | BindingFlags.Instance) .Invoke(new Command(), null); [Benchmark(Description = "Reflection (cached)")] public int Cached() => ReflectionCached.CallExecute(new Command()); [Benchmark(Description = "Reflection (delegate)")] public int Delegate() => ReflectionDelegate.CallExecute(new Command()); [Benchmark(Description = "Expressions")] public int Expressions() => ExpressionTrees.CallExecute(new Command()); public static void Main() => BenchmarkRunner.Run<Benchmarks>(); } | Method | Mean | Error | StdDev | Ratio | |---------------------- |-----------:|----------:|----------:|------:| | Reflection | 192.975 ns | 1.6802 ns | 1.4895 ns | 1.00 | | Reflection (cached) | 123.762 ns | 1.1063 ns | 1.0349 ns | 0.64 | | Reflection (delegate) | 6.419 ns | 0.0646 ns | 0.0605 ns | 0.03 | | Expressions | 5.383 ns | 0.0433 ns | 0.0383 ns | 0.03 |
  23. 23. Implementing Generic Operators Speaker: Alexey Golub @Tyrrrz
  24. 24. Speaker: Alexey Golub @Tyrrrz What can we do to support multiple types? public int ThreeFourths(int x) => 3 * x / 4; public int ThreeFourths(int x) => 3 * x / 4; public long ThreeFourths(long x) => 3 * x / 4; public float ThreeFourths(float x) => 3 * x / 4; public double ThreeFourths(double x) => 3 * x / 4; public decimal ThreeFourths(decimal x) => 3 * x / 4; public T ThreeFourths<T>(T x) /> 3 * x / 4; But we actually want something like this instead
  25. 25. Speaker: Alexey Golub @Tyrrrz public T ThreeFourths<T>(T x) { var param = Expression.Parameter(typeof(T)); var three = Expression.Convert(Expression.Constant(3), typeof(T)); var four = Expression.Convert(Expression.Constant(4), typeof(T)); var operation = Expression.Divide(Expression.Multiply(param, three), four); var lambda = Expression.Lambda<Func<T, T>>(operation, param); var func = lambda.Compile(); return func(x); } var a = ThreeFourths(18); // 13 var b = ThreeFourths(6.66); // 4.995 var c = ThreeFourths(100M); // 75M
  26. 26. Speaker: Alexey Golub @Tyrrrz public dynamic ThreeFourths(dynamic x) /> 3 * x / 4; Wait, how is it different from this?
  27. 27. public static class ThreeFourths { private static class Impl<T> { public static Func<T, T> Of { get; } static Impl() { var param = Expression.Parameter(typeof(T)); var three = Expression.Convert(Expression.Constant(3), typeof(T)); var four = Expression.Convert(Expression.Constant(4), typeof(T)); var operation = Expression.Divide(Expression.Multiply(param, three), four); var lambda = Expression.Lambda<Func<T, T>>(operation, param); Of = lambda.Compile(); } } public static T Of<T>(T x) => Impl<T>.Of(x); } Speaker: Alexey Golub @Tyrrrz Generic, thread-safe lazy initialization
  28. 28. Speaker: Alexey Golub @Tyrrrz public class Benchmarks { [Benchmark(Description = "Static", Baseline = true)] [Arguments(13.37)] public double Static(double x) /> 3 * x / 4; [Benchmark(Description = "Expressions")] [Arguments(13.37)] public double Expressions(double x) /> ThreeFourths.Of(x); [Benchmark(Description = "Dynamic")] [Arguments(13.37)] public dynamic Dynamic(dynamic x) /> 3 * x / 4; public static void Main() /> BenchmarkRunner.Run<Benchmarks>(); } | Method | x | Mean | Error | StdDev | Ratio | RatioSD | |------------ |------ |-----------:|----------:|----------:|------:|--------:| | Static | 13.37 | 0.6077 ns | 0.0176 ns | 0.0147 ns | 1.00 | 0.00 | | Expressions | 13.37 | 1.9510 ns | 0.0163 ns | 0.0145 ns | 3.21 | 0.08 | | Dynamic | 13.37 | 19.3267 ns | 0.1512 ns | 0.1340 ns | 31.82 | 0.78 |
  29. 29. Parsing DSLs into Expression Trees Speaker: Alexey Golub @Tyrrrz
  30. 30. Speaker: Alexey Golub @Tyrrrz public static class SimpleCalculator { private static readonly Parser<Expression> Constant = Parse.DecimalInvariant .Select(n => double.Parse(n, CultureInfo.InvariantCulture)) .Select(n => Expression.Constant(n, typeof(double))) .Token(); private static readonly Parser<ExpressionType> Operator = Parse.Char('+').Return(ExpressionType.Add) .Or(Parse.Char('-').Return(ExpressionType.Subtract)) .Or(Parse.Char('*').Return(ExpressionType.Multiply)) .Or(Parse.Char('/').Return(ExpressionType.Divide)); private static readonly Parser<Expression> Operation = Parse.ChainOperator(Operator, Constant, Expression.MakeBinary); private static readonly Parser<Expression> FullExpression = Operation.Or(Constant).End(); public static double Run(string expression) { var operation = FullExpression.Parse(expression); var func = Expression.Lambda<Func<double>>(operation).Compile(); return func(); } }
  31. 31. Speaker: Alexey Golub @Tyrrrz var a = SimpleCalculator.Run("2 + 2"); var b = SimpleCalculator.Run("3.15 * 5 + 2"); var c = SimpleCalculator.Run("1 / 2 * 3"); // 4 // 17.75 // 1.5
  32. 32. Obtaining Expression Trees from Lambdas Speaker: Alexey Golub @Tyrrrz
  33. 33. Speaker: Alexey Golub @Tyrrrz Func<int, int, int> div = (a, b) => a / b; Expression<Func<int, int, int>> divExpr = (a, b) => a / b; Same value, different type Console.WriteLine(divExpr.Type); // System.Func`3[System.Int32,System.Int32,System.Int32]
  34. 34. Speaker: Alexey Golub @Tyrrrz Func<int, int, int> div = (a, b) /> a / b; Expression<Func<int, int, int/> divExpr = (a, b) /> a / b; Same value, different type foreach (var param in divExpr.Parameters) Console.WriteLine($"Param: {param.Name} ({param.Type.Name})"); // Param: a (Int32) // Param: b (Int32)
  35. 35. Speaker: Alexey Golub @Tyrrrz Func<int, int, int> div = (a, b) /> a / b; Expression<Func<int, int, int/> divExpr = (a, b) /> a / b; Same value, different type var div = divExpr.Compile(); var c = div(10, 2); // 5
  36. 36. Speaker: Alexey Golub @Tyrrrz Func<int, int, int> div = (a, b) => a / b; Expression<Func<int, int, int>> divExpr = (a, b) => a / b; Product Recipe
  37. 37. Speaker: Alexey Golub @Tyrrrz Limitations Func<int, int, int> div = (a, b) /> a / b; Expression<Func<int, int, int/> divExpr = div; Compilation error
  38. 38. Speaker: Alexey Golub @Tyrrrz Limitations Expression<Func<int, int, int/> divExpr = (a, b) /> { var result = a / b; return result; }; Compilation error Expression<Action> writeToConsole = () => { Console.Write("Hello "); Console.WriteLine("world!"); }; Compilation error
  39. 39. Speaker: Alexey Golub @Tyrrrz Limitations • Null-coalescing operator (obj?.Prop) • Dynamic variables (dynamic) • Asynchronous code (async/await) • Default or named parameters (func(a, b: 5), func(a)) • Parameters passed by reference (int.TryParse("123", out var i)) • Multi-dimensional array initializers (new int[2, 2] { { 1, 2 }, { 3, 4 } }) • Assignment operations (a = 5) • Increment and decrement (a++, a--, --a, ++a) • Base type access (base.Prop) • Dictionary initialization (new Dictionary<string, int> { ["foo"] = 100 }) • Unsafe code (via unsafe) • Throw expressions (throw new Exception()) • Tuple literals ((5, x)) Can’t use any of the following:
  40. 40. Identifying Type Members Speaker: Alexey Golub @Tyrrrz
  41. 41. Speaker: Alexey Golub @Tyrrrz How can we get PropertyInfo of Dto.Id? public class Dto { public Guid Id { get; set; } public string Name { get; set; } } var idProperty = typeof(Dto).GetProperty(nameof(Dto.Id)); Console.WriteLine($"Type: {idProperty.DeclaringType.Name}"); Console.WriteLine($"Property: {idProperty.Name} ({idProperty.PropertyType.Name})"); // Type: Dto // Property: Id (Guid)
  42. 42. Speaker: Alexey Golub @Tyrrrz public class Validator<T> { // Add validation predicate to the list public void AddValidation<TProp>(string propertyName, Func<TProp, bool> predicate) { var propertyInfo = typeof(T).GetProperty(propertyName); if (propertyInfo is null) throw new InvalidOperationException("Please provide a valid property name."); // //. } // Evaluate all predicates public bool Validate(T obj) { /* //. // } /* //. // } var validator = new Validator<Dto>(); validator.AddValidation<Guid>(nameof(Dto.Id), id => id != Guid.Empty); validator.AddValidation<string>(nameof(Dto.Name), name => !string.IsNullOrWhiteSpace(name)); var isValid = validator.Validate(new Dto { Id = Guid.NewGuid() }); // false
  43. 43. Speaker: Alexey Golub @Tyrrrz What if we wanted to change the property type? public class Dto { public int Id { get; set; } public string Name { get; set; } } validator.AddValidation<Guid>(nameof(Dto.Id), id => id != Guid.Empty); Still compiles, even though there is now an error
  44. 44. Speaker: Alexey Golub @Tyrrrz public class Validator<T> { public void AddValidation<TProp>( Expression<Func<T, TProp>> propertyExpression, Func<TProp, bool> predicate) { var propertyInfo = (propertyExpression.Body as MemberExpression)?.Member as PropertyInfo; if (propertyInfo is null) throw new InvalidOperationException("Please provide a valid property expression."); // ... } public bool Validate(T obj) { /* ... */ } /* ... */ } Expression is used to identify a property var validator = new Validator<Dto>(); validator.AddValidation(dto => dto.Id, id => id != Guid.Empty); validator.AddValidation(dto => dto.Name, name => !string.IsNullOrWhiteSpace(name)); var isValid = validator.Validate(new Dto { Id = Guid.NewGuid() }); // false
  45. 45. Preserving Source Code in Assertions Speaker: Alexey Golub @Tyrrrz
  46. 46. Speaker: Alexey Golub @Tyrrrz [Test] public void IntTryParse_Test() { // Arrange const string s = "123"; // Act var result = int.TryParse(s, out var value); // Assert Assert.That(result, Is.True, "Parsing was unsuccessful"); Assert.That(value, Is.EqualTo(124), "Parsed value is incorrect"); } X IntTryParse_Test [60ms] Error Message: Parsed value is incorrect Expected: 124 But was: 123
  47. 47. Speaker: Alexey Golub @Tyrrrz public static class AssertEx { public static void Express(Expression<Action> expression) { var act = expression.Compile(); try { act(); } catch (AssertionException ex) { throw new AssertionException( expression.Body.ToReadableString() + Environment.NewLine + ex.Message); } } } Extension method from ReadableExpressions package
  48. 48. Speaker: Alexey Golub @Tyrrrz X IntTryParse_Test [60ms] Error Message: Assert.That(value, Is.EqualTo(124)) Expected: 124 But was: 123 [Test] public void IntTryParse_Test() { // Arrange const string s = "123"; // Act var result = int.TryParse(s, out var value); // Assert AssertEx.Express(() => Assert.That(result, Is.True)); AssertEx.Express(() => Assert.That(value, Is.EqualTo(124))); }
  49. 49. Traversing and Rewriting Expression Trees Speaker: Alexey Golub @Tyrrrz
  50. 50. Speaker: Alexey Golub @Tyrrrz
  51. 51. Speaker: Alexey Golub @Tyrrrz public class Visitor : ExpressionVisitor { protected override Expression VisitMethodCall(MethodCallExpression node) { Console.WriteLine($"Visited method call: {node}"); return base.VisitMethodCall(node); } protected override Expression VisitBinary(BinaryExpression node) { Console.WriteLine($"Visited binary expression: {node}"); return base.VisitBinary(node); } }
  52. 52. Speaker: Alexey Golub @Tyrrrz Expression<Func<double>> expr = () => Math.Sin(Guid.NewGuid().GetHashCode()) / 10; new Visitor().Visit(expr); // Visited binary expression: (Sin(Convert(NewGuid().GetHashCode(), Double)) / 10) // Visited method call: Sin(Convert(NewGuid().GetHashCode(), Double)) // Visited method call: NewGuid().GetHashCode() // Visited method call: NewGuid()
  53. 53. Speaker: Alexey Golub @Tyrrrz public class Visitor : ExpressionVisitor { protected override Expression VisitMethodCall(MethodCallExpression node) { var newMethodCall = node.Method == typeof(Math).GetMethod(nameof(Math.Sin)) ? typeof(Math).GetMethod(nameof(Math.Cos)) : node.Method; return Expression.Call(newMethodCall, node.Arguments); } }
  54. 54. Speaker: Alexey Golub @Tyrrrz Expression<Func<double/> expr = () /> Math.Sin(Guid.NewGuid().GetHashCode()) / 10; var result = expr.Compile()(); Console.WriteLine($"Old expression: {expr.ToReadableString()}"); Console.WriteLine($"Old result: {result}"); var newExpr = (Expression<Func<double/>) new Visitor().Visit(expr); var newResult = newExpr.Compile()(); Console.WriteLine($"New expression: {newExpr.ToReadableString()}"); Console.WriteLine($"New result value: {newResult}"); // Old expression: () => Math.Sin((double)Guid.NewGuid().GetHashCode()) / 10d // Old result: 0.09489518488876232 // New expression: () => Math.Cos((double)Guid.NewGuid().GetHashCode()) / 10d // New result value: 0.07306426748550407
  55. 55. Transpiling Code into a Different Language Speaker: Alexey Golub @Tyrrrz
  56. 56. Speaker: Alexey Golub @Tyrrrz Expression<Action<int, int>> expr = (a, b) => Console.WriteLine("a + b = {0}", a + b)); var fsharpCode = FSharpTranspiler.Convert(expr);
  57. 57. Speaker: Alexey Golub @Tyrrrz public static class FSharpTranspiler { private class Visitor : ExpressionVisitor { private readonly StringBuilder _buffer; public Visitor(StringBuilder buffer) { _buffer = buffer; } // ... } public static string Convert<T>(Expression<T> expression) { var buffer = new StringBuilder(); new Visitor(buffer).Visit(expression); return buffer.ToString(); } }
  58. 58. Speaker: Alexey Golub @Tyrrrz protected override Expression VisitLambda<T>(Expression<T> node) { _buffer.Append("fun ("); _buffer.AppendJoin(", ", node.Parameters.Select(p => p.Name)); _buffer.Append(") ->"); return base.VisitLambda(node); } protected override Expression VisitMethodCall(MethodCallExpression node) { if (node.Method.DeclaringType == typeof(Console) && node.Method.Name == nameof(Console.WriteLine)) { _buffer.Append("printfn "); if (node.Arguments.Count > 1) { var format = (string) ((ConstantExpression) node.Arguments[0]).Value; var formatValues = node.Arguments.Skip(1).ToArray(); _buffer.Append(""").Append(Regex.Replace(format, @"{d+}", "%O")).Append("" "); _buffer.AppendJoin(" ", formatValues.Select(v => $"({v.ToReadableString()})")); } } return base.VisitMethodCall(node); }
  59. 59. Speaker: Alexey Golub @Tyrrrz var fsharpCode = FSharpTranspiler.Convert<Action<int, int>>( (a, b) => Console.WriteLine("a + b = {0}", a + b)); > let foo = fun (a, b) -> printfn "a + b = %O" (a + b) val foo : a:int * b:int -> unit > foo (3, 5) a + b = 8 val it : unit = () // fun (a, b) > printfn "a + b = %O" (a + b)
  60. 60. Summary • Expression trees are fun • We can make reflection-heavy code much faster • We can do late-binding with almost no performance penalties • We can write our own runtime-compiled DSL • We can provide refactor-safe identification for type members • We can analyze specified lambdas and reflect on their structure • We can rewrite existing expressions to behave differently • We can transpile code into other languages Speaker: Alexey Golub @Tyrrrz
  61. 61. Useful packages • FastExpressionCompiler https://github.com/dadhi/FastExpressionCompiler Speeds up LambdaExpression.Compile() • ReadableExpressions https://github.com/agileobjects/ReadableExpressions Converts expression to readable C# code Speaker: Alexey Golub @Tyrrrz
  62. 62. Learn more • Working with expression trees in C# (by me) https://tyrrrz.me/blog/expression-trees • Introduction to expression trees (MS docs) https://docs.microsoft.com/en-us/dotnet/csharp/expression-trees • Expression trees in enterprise software (Maksim Arshinov) https://youtube.com/watch?v=J2XzsCoJM4o Speaker: Alexey Golub @Tyrrrz
  63. 63. Thank you! Speaker: Alexey Golub @Tyrrrz

×