Successfully reported this slideshow.
Your SlideShare is downloading. ×

Lowering in C#: What really happens with your code?, from NDC Oslo 2019

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Loading in …3
×

Check these out next

1 of 39 Ad
Advertisement

More Related Content

Advertisement
Advertisement

Lowering in C#: What really happens with your code?, from NDC Oslo 2019

  1. 1. @davidwengier Lowering in C# What’s really going on in your code? David Wengier Microsoft NDC { Oslo } 2019
  2. 2. @davidwengier foreach (int item in listOfInts) { // do something with item } for (int i = 0; i < listOfInts.Count; i++) { int item = listOfInts[i]; // do something with item } int i = 0; while (i < listOfInts.Count) { int item = listOfInts[i]; // do something with item i++; } int i = 0; again: int item = listOfInts[i]; // do something with item i++; if (i < listOfInts.Count) { goto again; } IL_0024: ldloc.0 IL_0025: ldloc.1 IL_0026: callvirt instance !0 class [mscorlib]List`1<int32>::get_Item(int32) IL_002b: pop IL_002c: ldloc.1 IL_002d: ldc.i4.1 IL_002e: add IL_002f: stloc.1 IL_0030: ldloc.1 IL_0031: ldloc.0 IL_0032: callvirt instance int32 class [mscorlib]List`1<int32>::get_Count() IL_0037: blt.s IL_0024 What is lowering? IL foreach for while gotogoto while for foreach
  3. 3. @davidwengier What is lowering? “A common technique … is to have the compiler “lower” from high-level language features to low-level language features in the same language.” Eric Lippert https://ericlippert.com/2014/04/28/lowering-in-language-design-part-one/
  4. 4. @davidwengier LINQ from c in customers where c.Country == “AU” select c;
  5. 5. @davidwengier LINQ customers.Where(c => c.Country == “AU”) .Select(c => c);
  6. 6. @davidwengier LINQ Enumerable.Select( Enumerable.Where(customers, c => c.Country == “AU”), c => c);
  7. 7. @davidwengier LINQ Enumerable.Select( Enumerable.Where(customers, FilterCustomers), SelectCustomer); bool FilterCustomers(Customer c) { return c.Country == “AU”; } Customer SelectCustomer(Customer c) { return c; }
  8. 8. @davidwengier var message = "NDC Oslo"; decimal message = 5M; string message = "NDC " + "Oslo"; string message = part1 + part2;
  9. 9. @davidwengier Why do I want to know? string message = "You have " + count + " items"; string message = $"You have {count} items"; string message = string.Format("You have {0} items", count);
  10. 10. @davidwengier foreach foreach (int m in values) { Console.WriteLine(m); }
  11. 11. @davidwengier IEnumerable interface IEnumerable { IEnumerator GetEnumerator(); } interface IEnumerator { object Current { get; } bool MoveNext(); void Reset(); } interface IEnumerable<T> : IEnumerable { new IEnumerator<T> GetEnumerator(); } interface IEnumerator<T> : IEnumerator, IDisposable { new T Current { get; } }
  12. 12. @davidwengier foreach { var e = values.GetEnumerator(); try { int m; while (e.MoveNext()) { m = (int)(int)e.Current; Console.WriteLine(m); } } finally { if (e != null && e is IDisposable) { ((IDisposable)e).Dispose(); } } } object[] v = new [] { 1, 2 }; Write(v); void Write(object[] arr) { foreach (int s in arr) { Console.WriteLine(s); } } Person[] v = new [] { Empl(),..}; Write(v); void Write(Person[] arr) { foreach (Customer c in arr) { Handle(c); } }
  13. 13. @davidwengier foreach (C# 5+) { var e = values.GetEnumerator(); try { while (e.MoveNext()) { int m; m = (int)(int)e.Current; Console.WriteLine(m); } } finally { if (e != null && e is IDisposable) { ((IDisposable)e).Dispose(); } } }
  14. 14. @davidwengier Lambdas public class C { public void M() { Action<string> act = x => Console.WriteLine(x); act(“hello”); } }
  15. 15. @davidwengier Lambdas public class C { public void M() { Action<string> act = _act; act(“hello”); } private void _act(string x) { Console.WriteLine(x); } }
  16. 16. @davidwengier Lambdas public class C { public void M() { ActHelper c = new ActHelper(); Action<string> act = c.act; act(“hello”); } private class ActHelper { internal void act(string x) { Console.WriteLine(x); } } }
  17. 17. @davidwengier Lambdas public class C { public void M() { if (ActHelper.Instance._act == null) { ActHelper.Instance._act = new Action<string>(ActHelper.Instance.act); } Action<string> act = ActHelper.Instance._act; act(“hello”); } private sealed class ActHelper { public static readonly ActHelper Instance = new ActHelper(); public static Action<string> _act; internal void act(string x) { Console.WriteLine(x); } } }
  18. 18. @davidwengier public class C { public void M() { Action<string> act = x => Console.WriteLine(x); act(“Hello”); } } public class C { public void M() { ActHelper c = new ActHelper(); Action<string> act = c.act; act(“Hello”); } private sealed class ActHelper { internal void act(string x) { Console.WriteLine(x); } } }
  19. 19. @davidwengier public class C { public void M() { string y = “ World”; Action<string> act = x => Console.WriteLine(x + y); act(“Hello”); } } public class C { public void M() { ActHelper c = new ActHelper(); Action<string> act = c.act; act(“Hello”); } private sealed class ActHelper { internal void act(string x) { Console.WriteLine(x + y); } } } public class C { public void M() { ActHelper c = new ActHelper(); Action<string> act = c.act; act(“Hello”); } private sealed class ActHelper { public string y; internal void act(string x) { Console.WriteLine(x + y); } } } public class C { public void M() { ActHelper c = new ActHelper(); c.y = “ World”; Action<string> act = c.act; act(“Hello”); } private sealed class ActHelper { public string y; internal void act(string x) { Console.WriteLine(x + y); } } }
  20. 20. @davidwengier public class C { public void M() { string y = “ World”; Action<string> act = x => Console.WriteLine(x + y); y = “ Fish”; act(“Hello”); } } public class C { public void M() { ActHelper c = new ActHelper(); c.y = “ World”; Action<string> act = c.act; act(“Hello”); } private sealed class ActHelper { public string y; internal void act(string x) { Console.WriteLine(x + y); } } }
  21. 21. @davidwengier public class C { public void M() { string y = “ World”; Action<string> act = x => Console.WriteLine(x + y); y = “ Fish”; act(“Hello”); } } public class C { public void M() { ActHelper c = new ActHelper(); c.y = “ World”; Action<string> act = c.act; c.y = “ Fish”; act(“Hello”); } private sealed class ActHelper { public string y; internal void act(string x) { Console.WriteLine(x + y); } } }
  22. 22. @davidwengier Foreach and lambdas List<Action> things = new List<Action>(); foreach (int m in values) { things.Add(() => Console.WriteLine(m)); }
  23. 23. @davidwengier Foreach and lambdas List<Action> things = new List<Action>(); { var e = values.GetEnumerator(); try { ActHelper c = new ActHelper(); while (e.MoveNext()) { c.m = (int)(int)e.Current; things.Add(new Action(c.act)); } } finally { if (e != null && e is IDisposable) ((IDisposable)e).Dispose(); } }
  24. 24. @davidwengier Foreach and lambdas (C# 5+) List<Action> things = new List<Action>(); { var e = values.GetEnumerator(); try { while (e.MoveNext()) { ActHelper c = new ActHelper(); c.m = (int)(int)e.Current; things.Add(new Action(c.act)); } } finally { if (e != null && e is IDisposable) ((IDisposable)e).Dispose(); } }
  25. 25. @davidwengier public class C { private int z; public void M() { string y = “ World”; Action<string> act = x => Console.Write(x + y + z); act(“Hello”); } } public class C { private int z; public void M() { ActHelper c = new ActHelper(); c.y = “ World”; c._this = this; c.act(“Hello”); } private sealed class ActHelper { public C _this; public string y; internal void act(string x) { Console.Write(x + y + _this.z); } } }
  26. 26. @davidwengier yield foreach (int x in GetInts()) { Console.WriteLine(x); } public IEnumerable<int> GetInts() { yield return 1; yield return 2; yield return 3; yield return 4; yield return 5; }
  27. 27. @davidwengier yield using (IEnumerator<int> enumerator = this.GetInts().GetEnumerator()) { while (enumerator.MoveNext()) { Console.WriteLine(enumerator.Current); } } public IEnumerable<int> GetInts() { return new GetIntsHelper(-2); }
  28. 28. @davidwengier yield private class GetIntsHelper : IEnumerable<int>, IEnumerable, IEnumerator<int>, IDisposable, IEnumerator { private int _state; private int _current; private int _initialThreadId; int IEnumerator<int>.Current { get { return this._current; } } object IEnumerator.Current { get { return this._current; } } public GetIntsHelper(int initialState) { this._state = initialState; this._initialThreadId = Environment.CurrentManagedThreadId; } void IDisposable.Dispose() { }
  29. 29. @davidwengier yield IEnumerator<int> IEnumerable<int>.GetEnumerator() { GetIntsHelper result; if (this._state == -2 && this._initialThreadId == Environment.CurrentManagedThreadId) { this._state = 0; result = this; } else { result = new GetIntsHelper(0); } return result; }
  30. 30. @davidwengier yield bool IEnumerator.MoveNext() { switch (this._state) { case 0: this._state = -1; this._current = 1; this._state = 1; return true; case 1: this._state = -1; this._current = 2; this._state = 2; return true; case 2: this._state = -1; this._current = 3; this._state = 3; return true; case 3: this._state = -1; this._current = 4; this._state = 4; return true; case 4: this._state = -1; this._current = 5; this._state = 5; return true; case 5: this._state = -1; return false; default: return false;
  31. 31. @davidwengier Yield states -2 : GetEnumerator() hasn’t been called -1 : Running – Getting the next value 0 : Before – MoveNext() hasn’t been called 1- 4 : Suspended – Waiting for a MoveNext() call 5 : After – Finished.
  32. 32. @davidwengier yield public IEnumerable<int> GetInts() { foreach (int x in Enumerable.Range(1, 10)) { yield return x; } }
  33. 33. @davidwengier yield private IEnumerator<int> _wrap; void IDisposable.Dispose() { if (this._state == -3 || this._state == 1) { this._Finally(); } } private void _Finally() { this._state = -1; if (this._wrap != null) { this._wrap.Dispose(); } }
  34. 34. @davidwengier yield bool IEnumerator.MoveNext() { try { // ... Next slide } catch { this.Dispose(); throw; } }
  35. 35. @davidwengier yield if (this._state == 0) { this._state = -1; this._wrap = Enumerable.Range(1, 10).GetEnumerator(); this._state = -3; } else { if (this._state != 1) return false; this._state = -3; } if (this._wrap.MoveNext()) { this._current = (int)this._wrap.Current; this._state = 1; return true; } else { this._Finally(); this._wrap = null; return false; }
  36. 36. @davidwengier Yield states -3 : Running – Getting the next value -2 : GetEnumerator() hasn’t been called -1 : Running – Getting the range enumerator 0 : Before – MoveNext() hasn’t been called 1 : Suspended (and After) – Waiting for a MoveNext() call
  37. 37. @davidwengier Captain planet private int _min; public void M() { int max = 5; foreach (int x in GetInts(max)) { Console.WriteLine(x); } } public IEnumerable<int> GetInts(int max) { yield return 1; foreach (int x in Enumerable.Range(this._min, max).OrderBy(i => i * this._min + max)) { yield return x; } } ???
  38. 38. @davidwengier Want to know more? • Roslyn source code • Your favourite decompiler • http://sharplab.io
  39. 39. @davidwengier Thank you! Questions? Comments? Quemments? @davidwengier

Editor's Notes

  • Invented a language, having a language design meeting. Someone suggests “foreach”. Everyone agrees, except the guy in the corner. Lowering. He’s lazy. “why not just use a for loop”
  • Because I love it!
    One abstraction layer deeper
    Debugging
    Performance
  • Lets look at one of those previous examples in detail.
  • Lets look at one of those previous examples in detail.
  • Simplified generic version. There are specific overloads for arrays, stirngs etc.
    No type on GetEnumerator. Duck typing. Not really var, the compiler works it out, I just can’t express it. Also checks for implicit implementations and casts to IEnumerable. If necessary.
    Two casts. One for item type, one for the type of the loop variable (because designed before generics, eg ArrayList). Means you can loop through objects, and ask for strings. EG ANIMATIONS!! Those casts could fail.
    Disposable is optional. Compiler will work out whether to include it (and will leave off the try..finally entirely if it can
    Note the brackets to introduce a new scope
    This is the C# 4 and below version.
  • Subtle difference. Variable declaration inside while loop. This is a breaking change. Why? Answer is closures, which brings us to our next bit of lowering.
  • To talk about closures, I think its easiest to talk about lambdas.
  • This is what it logically does, however this has a problem. Without knowing what Console.WriteLine does, and depending on what is passed in, we can’t guarantee this doesn’t hold a reference to class C. So instead this is what the compiler does:
  • By using a new class, even if an instance is held in memory the compiler knows its as small as it can be. Obviously these method and class names would be different. In fact, compiler deliberately uses names that are invalid C#
  • This is _really_ what the compiler does though. Apologies for the size.. Various optimizations.
    So far so good? Cool.
    The other thing we can do wth lambdas and delegates is create closures.
  • Lets go back to our original lamba, but make a small change. This action now creates a closure over y. Now the compiler has to put y in the new class, and store a reference to it. This looks like this.
  • Lets go back to our original lamba, but make a small change. This action now creates a closure over y. Now the compiler has to put y in the new class, and store a reference to it. This looks like this.
  • So, can you see the problem? It doesn’t matter when you create the delegate, we’ve capture the variable y, not the value of the variable y. So the code uses the value of the variable y as at the time it is executed.
  • So, can you see the problem? It doesn’t matter when you create the delegate, we’ve capture the variable y, not the value of the variable y. So the code uses the value of the variable y as at the time it is executed.
  • So lets revisit our foreach loop, but now add a lambda in the middle. We’ll just collect a list of things to do later.. Call it poor mans async 
  • Now this gets expanded as we know. I’ve highlighted the new bits.
  • In C# 5 however, we now move the declaration of m inside the loop, therefore we don’t have a problem.. Essentially when C# 4 came out everyone was so lambda happy that it exposed this quirk.
    Why did they do it the other way first? Matches the “for” semantics, of having one loop variable that is redefined. Just that with “for” its much more obvious, because the user is writing that redefinition.
  • One last word on closures, is what the difference is when we close over a class level field, property, method etc. The field is not hoisted, instead the helper class has a reference to the original object. This can potentially lead to memory leaks. Solved by introducing local variables to capture the value, though does change the semantics (value capture not variable capture)
  • Kinda still on foreaches, lets look at the yield statement. Things get pretty tricky with this one.. So any questions before we continue?
  • The foreach part we know, and GetInts is still a method that returns an enumerable. The content of GetInts has been moved to a new class though, like we saw with lambdas. Lets look at that class
  • So, it implements a few interfaces. And mose of these things are obvious.
    It captures the thread id, which we’ll see later is used for thread safety. It implements Idisposable as a just in case, and in this case doesn’t need to dispose anything. Will see more later.
    Current we’ve seen before from Ienumerable, and there is a local variable to track it.
    And it stores state. As you might know, or have guessed, the yield enumerator uses a state machine.
    The intial state, if you remember, is -2
  • Here is the GetEnumerator method. This is where we do some thread checking. So if GetEnumerator() is called from the same thread, and we have our initial state, then this is the thing we use. This is what allows our class to be an enumerable, that can be returned from GetInts(), and an enumerator, that can do the enumeration. This sets the state to 0.
    If something else calls GetEnumerator(), or we’re in a different state, then return a new instance with a 0 initial state.
  • Pretty basic state machine. Essentially its an unrolled loop, which makes sense because if you think about our yield statement, that was too. 4 states: Before (-2, 0), Running (-1), Suspended and After (positive integers)
    Questions?
    More realistic uses get a bit harder, so lets look at one.
  • This is a bit more typical, where in your method you’d be looping through something else, and yield returning. If the yield return unrolls the loop, well what happens when you put a loop in your loop unrolling?

    Lets look at the differences.
  • A new Enumerator field, called wrap. In Dispose we call Finally, and that disposes of the wrapped enumerator.
    The weird empty try..finally block, I cannot explain!. Could be a decompilation problem.
  • Just to save space on the next slide, MoveNext has nothing new that’s interesting.. Just a dispose call in a catch
  • Hopefully still big enough.. This looks very different! But its still a state machine. Initial state is 0, so first we set state to -3 (another type of “Running” - about get next), and get a reference to the range enumerator. Then we call MoveNext, set state to 1. Each call to this MoveNext calls the wrapped MoveNext, until we run out. At the end we call Finally() again. -1 is running the GetEnumerator, -3 is running the MoveNext
  • This instance doesn’t really have an “After” because it doesn’t know when to stop on its own, it simply passes calls on to the wrapped enumerator, and hence relies on that enumerators After state.
  • And of course you can combine these things, by having multiple yield returns, the iterator can close over variables, and fields, and can include a lambda that does the same!

    This is why lowering is good. No need to implement all of this lot in IL. We know this bit will become a class, closing over some things.. And this…

    At the end it will just be classes with straight forward code, containing pretty much just gotos

×