IronPython and Dynamic Languages on .NET Mahesh Prakriya [email_address] Principal PM Lead, DLR

Dynamic Languages on .NET Popular Powerful Simple Intuitive Interactive Inspiring Fun

Popular

Powerful

Simple

Intuitive

Interactive

Inspiring

Fun

DLR Dynamic Languages VBx JScript IronPython IronRuby 3rd Party langs CLR Frameworks VS Integration Tools WinForms, WPF Applications ASP.net, Silverlight Robotics, XNA

IronPython – four pillars True Python Implementation Seamless integration with .NET; built on DLR Open source under Microsoft Public License Fastest implementation of Python

True Python Implementation

Seamless integration with .NET; built on DLR

Open source under Microsoft Public License

Fastest implementation of Python

IronPython Demos: Compatability Winforms/Tablet Silverlight WebServices Mahesh Prakriya [email_address]

Standard Pystone Benchmark IronPython 0.1 Python 2.3 IronPython 1.0 IronPython 2.0 alpha1 Python 2.5 Python 2.1

Rough Roadmap

FePy and IPCE Sourceforge project led by Seo Sanghyeon “ FePy project aims to provide enhancements and add-ons for IronPython” IronPython Community Edition (IPCE) Files under Src/ are licensed under Shared Source License for IronPython. Files under Lib/ are licensed under Python Software Foundation License Version 2, except the following: ctypes, hashlib, md5, pyexpat, select, sha, site, socket, ssl, unicodedata, wsgi, zlib modules are part of FePy library MySQLdb, dbapi, odbc, psycopg, sqlclient, sqlite, sqlite3 modules are part of FePy DB-API library FePy library and FePy DB-API library are licensed under MIT License. Files under Lib/elementtree are licensed under ElementTree License. Files under Lib/Crypto are licensed under Python Cryptography Toolkit License. Files under Lib/paramiko are licensed under GNU Lesser General Public License. Click the link for the exact terms

Sourceforge project led by Seo Sanghyeon

“ FePy project aims to provide enhancements and add-ons for IronPython”

IronPython Community Edition (IPCE)

Files under Src/ are licensed under Shared Source License for IronPython.

Files under Lib/ are licensed under Python Software Foundation License Version 2, except the following:

ctypes, hashlib, md5, pyexpat, select, sha, site, socket, ssl, unicodedata, wsgi, zlib modules are part of FePy library

MySQLdb, dbapi, odbc, psycopg, sqlclient, sqlite, sqlite3 modules are part of FePy DB-API library

FePy library and FePy DB-API library are licensed under MIT License.

Files under Lib/elementtree are licensed under ElementTree License.

Files under Lib/Crypto are licensed under Python Cryptography Toolkit License.

Files under Lib/paramiko are licensed under GNU Lesser General Public License. Click the link for the exact terms

Visual Studio support for IronPython VS support for IPy through VS SDK Supports both VS 2005 and VS 2008 Key features Editor – color coding, Project system, Debugger, Rich client (WPF, Winforms) designers emit IronPython WPF designer support only in VS2008 http://blogs.msdn.com/aaronmar/

VS support for IPy through VS SDK

Supports both VS 2005 and VS 2008

Key features

Editor – color coding,

Project system,

Debugger,

Rich client (WPF, Winforms) designers emit IronPython

WPF designer support only in VS2008

http://blogs.msdn.com/aaronmar/

Copyright Microsoft Corporation

Copyright Microsoft Corporation

Copyright Microsoft Corporation

A framework for building 2d and 3d games Targets Windows and XBox 360 Free for Windows, $100/year to run on 360 Encouraging hobbyist game builder community Security and type-safety is vital for hobbyist games Let’s you comfortably run other’s games without virus risk Focus is on C# as development language This is a dramatic step away from C++ What if I don’t want to use C#?

A framework for building 2d and 3d games

Targets Windows and XBox 360

Free for Windows, $100/year to run on 360

Encouraging hobbyist game builder community

Security and type-safety is vital for hobbyist games

Let’s you comfortably run other’s games without virus risk

Focus is on C# as development language

This is a dramatic step away from C++

What if I don’t want to use C#?

Microsoft Robotics Studio

IronPython - customer examples 3 rd party languages running on DLR, some examples IronLisp, Nua, ColdFusion, SmallTalk, … MySpace Currently using IronPython on their web server farm of ~2500 Nvidia FXComposer 2 (IDE for shader authoring) Resolver Systems Startup based out of London doing dynamic spreadsheets 120K LOC (IronPython) Many Others: Manifold 8.0, Multiverse.net, … Miguel ships Ipy (and will ship DLR & IronRuby) on Mono FePy on Source Forge

3 rd party languages running on DLR, some examples

IronLisp, Nua, ColdFusion, SmallTalk, …

MySpace

Currently using IronPython on their web server farm of ~2500

Nvidia

FXComposer 2 (IDE for shader authoring)

Resolver Systems

Startup based out of London doing dynamic spreadsheets

120K LOC (IronPython)

Many Others:

Manifold 8.0, Multiverse.net, …

Miguel ships Ipy (and will ship DLR & IronRuby) on Mono

FePy on Source Forge

DLR Dynamic Languages VBx JScript IronPython IronRuby 3rd Party langs CLR Frameworks VS Integration Tools WinForms, WPF Applications ASP.net, Silverlight Robotics, XNA

Resources: www.codeplex.com/ironpython Mahesh Prakriya [email_address] IronPython Samples, Videos etc. blogs.msdn.com/ironpython Silverlight silverlight.net and codeplex.com/dynamicsilverlight ASP.NET asp.net/downloads/futures Web Services blogs.msdn.com/dmitryr

Questions? Image By Little Bitty Tam http://www.flickr.com/photos/tammra/

Dynamic Language Runtime Platform for building dynamic languages Built on top of Microsoft .NET Framework Garbage collector Just-in-time compiler (JIT) Rich libraries Tools http://www.codeplex.com/IronPython

Platform for building dynamic languages

Built on top of Microsoft .NET Framework

Garbage collector

Just-in-time compiler (JIT)

Rich libraries

Tools

http://www.codeplex.com/IronPython

Compiler Overview Frontend return Syntax Tree Return def add2(a) { return 2 + a; } Scan 2 + a ; Parse Token stream Add Named(a) Const(2)

Compiler Overview Traditional Backend Syntax Tree Return Add Named(a) Const(2) Generate IL ldc.i4.2 // load 2 box [mscorlib]System.Int32 ldarg.0 // load “a” call object ToyHelpers::Add(object, object) ret IL public static object Add (object x, object y) { ... } Runtime Library

Compiler Overview DLR Backend Syntax Tree Return Add Named(a) Const(2) Generate DLR Tree public static object Add (object x, object y) { ... } Runtime Library DLR Tree Return MethodCall ToyHelpers.Add BoundExpression ConstantExpression 2 ConvertTo Object Variable a: Object

Why DLR? Focus on your language Scanner Parser Runtime semantics DLR Code generation Dynamic operations Extension methods for .NET Type customization Common hosting for all DLR languages

Focus on your language

Scanner

Parser

Runtime semantics

DLR

Code generation

Dynamic operations

Extension methods for .NET Type customization

Common hosting for all DLR languages

DLR Trees DLR representation of programs Similar to LINQ expression trees Expressions: Constant , Unary, Binary, Method call, Property value, Field value, Assignment, … Statements: If, While, Try, Return, Switch, Throw, … Dynamic behavior support: ActionExpression Factory methods (Ast.…)

DLR representation of programs

Similar to LINQ expression trees

Expressions: Constant , Unary, Binary, Method call, Property value, Field value, Assignment, …

Statements: If, While, Try, Return, Switch, Throw, …

Dynamic behavior support: ActionExpression

Factory methods (Ast.…)

Targeting the DLR Implement scanner and parser Translate your AST to the DLR Tree Implement your custom types Implement customization to .NET types Via extension methods Tune performance Runtime library Dynamic types

Implement scanner and parser

Translate your AST to the DLR Tree

Implement your custom types

Implement customization to .NET types

Via extension methods

Tune performance

Runtime library

Dynamic types

Generating DLR Trees def add2(a) { return 2 + a; } Return MethodCall ToyHelpers.Add BoundExpression ConstantExpression 2 ConvertTo Object CodeBlock “ add2” Parameter a: Object public static object Add (object x, object y) { ... } Runtime Library

def add2(a) {

return 2 + a;

}

Generating DLR Trees CodeBlock cb = Ast.CodeBlock("add2"); Variable a = cb.CreateParameter( SymbolTable.StringToId( "a" ), typeof(object) ); cb.Body = Ast.Return( Ast.Call( typeof(ToyHelpers).GetMethod("Add"), Ast.Convert(Ast.Constant(2), typeof(object)), Ast.Read(a) ) );

CodeBlock cb = Ast.CodeBlock("add2");

Variable a = cb.CreateParameter(

SymbolTable.StringToId( "a" ),

typeof(object)

);

cb.Body = Ast.Return(

Ast.Call(

typeof(ToyHelpers).GetMethod("Add"),

Ast.Convert(Ast.Constant(2), typeof(object)),

Ast.Read(a)

)

);

Demo DLR Trees Image By Ryan Forsythe: http://flickr.com/photos/fweez/

Operators Method Call ToyHelpers.Add(a, b) BoundExpression Variable a MethodCall ToyHelpers.Add BoundExpression Variable b

ToyHelpers.Add(a, b)

Dynamic Actions a + b

a + b

Operators Action Expression BoundExpression Variable a Action Add BoundExpression Variable b

Dynamic Actions Call Site static DynamicSite a_plus_b_site = new DynamicSite(Add); // a + b a_plus_b_site.Invoke(a, b) // DynamicSite.Invoke object Invoke(object a0, object a1) { this._handler(this, a0, a1); }

static DynamicSite a_plus_b_site =

new DynamicSite(Add);

// a + b

a_plus_b_site.Invoke(a, b)

// DynamicSite.Invoke

object Invoke(object a0, object a1) {

this._handler(this, a0, a1);

}

Dynamic Actions Target Delegate object Handler(DynamicSite s, object a0, object a1) { // HELP !!! return s.UpdateMe(a0, a1); }

object Handler(DynamicSite s, object a0, object a1)

{

// HELP !!!

return s.UpdateMe(a0, a1);

}

Dynamic Actions Updated Target Delegate - Int object Handler(DynamicSite s, object a0, object a1) { if (a0 is int && a1 is int) { return (int)a0 + (int)a1; } // HELP !!! return s.UpdateMe(a0, a1); }

object Handler(DynamicSite s, object a0, object a1)

{

if (a0 is int && a1 is int) {

return (int)a0 + (int)a1;

}

// HELP !!!

return s.UpdateMe(a0, a1);

}

object Handler(DynamicSite s, object a0, object a1) { if (a0 is int && a1 is int) { return (int)a0 + (int)a1; } if (a0 is double && a1 is double) { return (double)a0 + (double)a1; } // HELP !!! return s.UpdateMe(a0, a1); } Dynamic Actions Updated Target Delegate - Double Rule

object Handler(DynamicSite s, object a0, object a1)

{

if (a0 is int && a1 is int) {

return (int)a0 + (int)a1;

}

if (a0 is double && a1 is double) {

return (double)a0 + (double)a1;

}

// HELP !!!

return s.UpdateMe(a0, a1);

}

Rules Rule = Test + Target Test Condition examining the arguments Target An operation to perform if the test succeeds Who makes the rules ??? The Language The DLR

Rule = Test + Target

Test

Condition examining the arguments

Target

An operation to perform if the test succeeds

Who makes the rules ???

The Language

The DLR

Rules Language Action Binder DLR requests: “Tell me how to perform this operation with these arguments!” KEY: “Tell me how!” NOT: “Do it!” Language responds: “Here is the Tree” “I don’t know” (DLR tries its own built-in behaviors)

DLR requests:

“Tell me how to perform this operation with these arguments!”

KEY: “Tell me how!” NOT: “Do it!”

Language responds:

“Here is the Tree”

“I don’t know” (DLR tries its own built-in behaviors)

Rules Adding Strings - Test (a0 is string) && (a1 is string) BoundExpression Parameter a0 string TypeIs BinaryExpression AndAlso BoundExpression Parameter a1 string TypeIs

(a0 is string) && (a1 is string)

Rules Adding Strings - Target String.Concat((string)a0, (string)a1) BoundExpression Parameter a0 string Convert MethodCall String.Concat(string, string) BoundExpression Parameter a1 string Convert

String.Concat((string)a0, (string)a1)

Demo Actions Image by Kim S http://flickr.com/photos/dearbarbie

Customizing .NET Types Extension Methods "Kathryn".Greet()

"Kathryn".Greet()

Customizing .NET Types Extension Methods – C# static class StringExtensions { public static string Greet( this string self) { return "Hello " + self + "!!!"; } } class Program { static void Main(string[] args) { "Kathryn".Greet(); } }

static class StringExtensions {

public static string Greet( this string self) {

return "Hello " + self + "!!!";

}

}

class Program {

static void Main(string[] args) {

"Kathryn".Greet();

}

}

Customizing .NET Types Extension Methods – DLR [assembly:ExtensionType( typeof(string), typeof(StringExtensions))] static class StringExtensions { public static string Greet(string self) { return "Hello " + self + "!!!"; } } >>> "Kathryn".Greet()

[assembly:ExtensionType( typeof(string), typeof(StringExtensions))]

static class StringExtensions {

public static string Greet(string self) {

return "Hello " + self + "!!!";

}

}

>>> "Kathryn".Greet()

Customizing .NET Types Extension Methods – DLR Similar to C# extension methods Support for extension properties and fields To use: [assembly:ExtensionTypeAttribute( … )] RuntimeHelpers.RegisterAssembly( typeof(StringExtensions).Assembly );

Similar to C# extension methods

Support for extension properties and fields

To use:

[assembly:ExtensionTypeAttribute( … )]

RuntimeHelpers.RegisterAssembly( typeof(StringExtensions).Assembly

);