Programming
Languages
Telefónica Digital DevCon 2013

Iván Montes - @drslump
Disclaimer
•

Talk is focused on mainstream languages

•

Many of the stuff we will talk about is already available

•

I ...
What is the best?
•

Good news! You’re probably using it already

•

Check list:
•

You can focus on solving your business...
But…
Can it be improved?
•

Given:
•
•

•

Expected functionality is ever growing
Solutions complexity is ever growing

When:
•...
The problem
•

We can separate a language in:
•

Specification, Compiler, Runtime and Libraries

•

Of those we are only in...
What’s wrong with Libraries?
•

They are meant for code reuse

•

We (ab)use them as language
customisation and extensibil...
Language design
•

Disclaimer: It’s a hard discipline, requiring lots of talent
and resources.

•

Basically the current s...
Taking control
•

Premise: Any non-tech-savvy person assumes
that a developer is someone intelligent.

•

Think highly of ...
but I already can!
•

No, you don’t.

•

Some compilers support writing add ons for them.

•

The problem is neither being...
Extensibility
•

We should be able to extend pretty much anything in our
language.

•

Extending the syntax and semantics ...
A Compiler
•

A compiler basically does three tasks:
•
•

Resolve (name resolution, type binding)

•
•

Parse (Ast constru...
Info!

Code
Parse
Resolve
Emit

•

Compilers produce a lot of information because they need
to understand our code.

•

Un...
Compiler Service
•

Almost a reality (llvm, mono mcs, roslyn, …).

•

Note: You know that an IDE basically implements an i...
Compiler Service (II)
•

But we want more!

•

If only we could hook our logic into the compiler like we
do with build sys...
Compiler Service (III)
•

All of the sudden, half our
libraries functionality is now
managed by the compiler!

•

That mea...
Macros
•

Extending the compiler is nice but could be made
more user friendly.

•

When you have a pattern coming again an...
Example
macro using(expr as Expression):

yield [|

try:

$(using.Body)

ensure:

$(expr).close()

|]
fp = open(‘path/to/f...
Quasi-Quotations
•

Normal language syntax constructs that are
somehow quoted (`[| .. |]` in the example).

•

Their purpo...
Splicing
•

Injection points in a quasi-quotation (`$(..)` in the
example).

•

Their purpose is to parametrize quasi-quot...
Why are they better?
•

Syntax is integrated and validated as normal code.

•

Most of the lexical information is kept, th...
Can’t libraries do this?
•

Yes, but they are not the right tool for the job.

•

Lots of stuff can be done at compile tim...
Syntax
A very quick overview
Parsers
•

Disclaimer: Not saying it’s a good
idea for a general language

•

The technology allows it and
current systems...
Making the language ours
•

Use cases:
•
•

Units: delay = 10h + 30m, weight = 20Kg

•
•

Any sort of DSL would benefit fro...
User eXperience
Have you checked clang recently?
Language design
•

Syntax of a language is the first and
most important point of interaction.

•

Language design is still ...
Reporting

•

•

The compiler is our main tool
but still, it’s generally
speaking, a bitch reporting
errors.
Error reporti...
Parsing errors
•

Language newcomers tend to have syntax errors until they get fluent.

•

The common approach is to extend...
Type Safety
Wishing it could be extended
Type Safety
•

Major families:
•

Structural (Go, TypeScript, …)

•

Pattern matching solves a lot of
type safety derived ...
API Versioning
•

If your language uses nominal typing you actually have to embrace
it. Carefully plan your Interfaces to ...
Cool stuff
Old tech sold as a bright future
Live Coding
•

Imagine you could tap into the
runtime environment and modify
the executing instructions on
realtime.

•

W...
Type Providers
•

Popularised recently by F#

•

They are basically specialised macros, whose purpose is to
generate new t...
What now?
Is your responsibility
To get better tools for our job
we have to earn it. Advocate
these concepts to gain traction
in the...
Conclusions
Conclusions
•

Languages should be
designed for extensibility.

•

Anyone that can create a
library should be also able to...
“The limits of my language are
the limits of my world.”

–Ludwig Wittgenstein
Upcoming SlideShare
Loading in …5
×

Programming Languages #devcon2013

988 views

Published on

Published in: Technology
2 Comments
1 Like
Statistics
Notes
  • Thanks a lot Fer. I'll have a look at your book recommendation, even for someone that dislikes c++ there is certainly a lot to learn from it :)
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Btw, if you haven't got a chance, give this a book chance:

    http://www.pearson.ch/1471/9780321227256/C-Template-Metaprogramming-Concepts.aspx
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
988
On SlideShare
0
From Embeds
0
Number of Embeds
21
Actions
Shares
0
Downloads
4
Comments
2
Likes
1
Embeds 0
No embeds

No notes for slide

Programming Languages #devcon2013

  1. 1. Programming Languages Telefónica Digital DevCon 2013 Iván Montes - @drslump
  2. 2. Disclaimer • Talk is focused on mainstream languages • Many of the stuff we will talk about is already available • I know your favourite language probably does something similar, or you know a hack to make it do it. • Scripting languages do have a compiler, they just happen to include an evaluator too :-) • C++ is purposely left out of any comparison (Guys you need to simplify the spec!) • If you don’t understand some terminology, ask!
  3. 3. What is the best? • Good news! You’re probably using it already • Check list: • You can focus on solving your business problem • You are proficient with it • You can deploy your product without fear • You have tooling available (editors, testing, packaging…) • You are not the only soul who knows how it works :-)
  4. 4. But…
  5. 5. Can it be improved? • Given: • • • Expected functionality is ever growing Solutions complexity is ever growing When: • • • We add layers and layers to keep it under control Those layers are usually libraries and frameworks Then: • Your really nice language is not that important anymore • You end up programming for the framework (does it pass the check list?)
  6. 6. The problem • We can separate a language in: • Specification, Compiler, Runtime and Libraries • Of those we are only in control of the Libraries • So we abuse them to solve our problems • But we can end up creating others: • Support • Versioning • Troubleshooting
  7. 7. What’s wrong with Libraries? • They are meant for code reuse • We (ab)use them as language customisation and extensibility tools • • Cycle ends with lots of abandonware even if you don’t notice it: • Check SourceForge, GitHub and similar sites • The guy now maintaining your 2009 project has megabytes of legacy source code, with no expertise to hack it and nobody to offer support. The cycle of doom then begins: • Our cool language toolchain breaks • We expend time modifying editors, build tools, … • The library gets updated or we want to support a new tool
  8. 8. Language design • Disclaimer: It’s a hard discipline, requiring lots of talent and resources. • Basically the current situation in the mainstream market sucks. • It’s slowly changing though (guys at Microsoft Research and Universities) • Designers are realising (again) that we are not totally stupid! • What we need is a very old tech, we just need it refined.
  9. 9. Taking control • Premise: Any non-tech-savvy person assumes that a developer is someone intelligent. • Think highly of yourself! • We want/need more control over our primary tool. • Make yourself this question: What do you think has evolved more in the last 25 years: compilers or cad software?
  10. 10. but I already can! • No, you don’t. • Some compilers support writing add ons for them. • The problem is neither being able to write an extension in C to be called from the host language. • The Language itself must define how this extension mechanisms work. It must be part of its specification. • This way we get extensibility without sacrificing portability and risking vendor lock-in. • It helps with versioning, you extend a version of the language not a version of a given compiler.
  11. 11. Extensibility • We should be able to extend pretty much anything in our language. • Extending the syntax and semantics (specification) is doable but not very mature yet. • Extending the runtime is expensive. You wan’t something tested and battle tested and then tested again. • However extending the compiler is ready for prime time! • Preprocessors and lispy macros are cool, don’t resent me. • But we need something more refined, almost fool-proof.
  12. 12. A Compiler • A compiler basically does three tasks: • • Resolve (name resolution, type binding) • • Parse (Ast construction) Emit (machine code, bytecode). Keep in mind that a bit is a measure for information! Memory usage (Kb) Code Parse Resolve Emit 0 300 600 900 1200
  13. 13. Info! Code Parse Resolve Emit • Compilers produce a lot of information because they need to understand our code. • Unfortunately this pretty cool information is trashed once the compilation is done. • Some languages support reflection, at least not everything is lost (re Java fan boys, type erasure is not cool). • Given that a lot of very clever people put a lot of effort over several years creating a compiler. • It seems like a huge waste of resources to not use it more.
  14. 14. Compiler Service • Almost a reality (llvm, mono mcs, roslyn, …). • Note: You know that an IDE basically implements an incremental compiler nowadays, right? Wonder why they feel slow now? • Use cases: • Better editor support • Optimised/Incremental builds • Documentation generation • Quality/Security audits
  15. 15. Compiler Service (II) • But we want more! • If only we could hook our logic into the compiler like we do with build systems… • The following would be dirty cheap to have: • Injection of cross-concerns (logging, aspects, …) • Statically checked/expanded DSLs • Quality, Security and Business specific enforcements
  16. 16. Compiler Service (III) • All of the sudden, half our libraries functionality is now managed by the compiler! • That means for example: • Build systems will notify errors generated by our custom enforcements Wait! won’t we suffer the “cycle of doom” too? • • Editors will autocomplete and report errors for expanded DSLs • • • No! :-) The logic is encapsulated under the compiler public interface. All the tooling using the compiler will automatically support our use cases! • Besides, we reduced some heavy weight from the runtime, pretty cool for restrained mobile apps for example.
  17. 17. Macros • Extending the compiler is nice but could be made more user friendly. • When you have a pattern coming again and again in your code, it will probably be better solved with a macro. • Macros are a pretty old concept but they mostly refer to “text macros”. What we need are AST Macros. • Instead of replacing chunks of text we operate on the syntax tree of the compiler. Much safer and powerful!
  18. 18. Example macro using(expr as Expression):
 yield [|
 try:
 $(using.Body)
 ensure:
 $(expr).close()
 |] fp = open(‘path/to/file.ext’)
 using fp:
 fp.write(‘foobar')
 # Note that the file resource is closed automatically
  19. 19. Quasi-Quotations • Normal language syntax constructs that are somehow quoted (`[| .. |]` in the example). • Their purpose is to make creating complex AST structures a breeze. • The compiler generates a separate AST for them. It’s like a template. • When we inject them somewhere the compiler will apply that template in that AST node.
  20. 20. Splicing • Injection points in a quasi-quotation (`$(..)` in the example). • Their purpose is to parametrize quasi-quotations to dynamically build arbitrary AST structures. • Resolves to a syntax node, the compiler will insert that node in the template tree. • They keep their lexical information once inserted.
  21. 21. Why are they better? • Syntax is integrated and validated as normal code. • Most of the lexical information is kept, thus reported errors after expansion contain valid information. • Debuggable! • Toolchain friendly. They are transparent to consuming tools. • DSL friendly. Capture common patterns to solve your problems and offer a statically checked interface to fit your needs!
  22. 22. Can’t libraries do this? • Yes, but they are not the right tool for the job. • Lots of stuff can be done at compile time, no need to bloat the runtime with stuff that just makes developing easier. • Unified error reporting. Find out incorrect usages at compile time instead of uncontrolled runtime exceptions. • The `assert` example shows pretty well the difference: #!/bin/env python ! foo = 10 assert foo != 10 # AssertionError: #!/bin/env booi ! foo = 10 assert foo != 10 # AssertionError: foo != 10
  23. 23. Syntax A very quick overview
  24. 24. Parsers • Disclaimer: Not saying it’s a good idea for a general language • The technology allows it and current systems have enough memory to pull it. • Traditionally parsing involved a lexer generating tokens and a parser (either a hand-coded recursive descent or a generated table based). • PEG style parsers (and other pattern matching techniques) operate the same on a stream of chars (source code in a file), a tree of XML nodes or any other object structure. • They can be made to have constant parsing time using memoization. • OMeta is an example of PEG parser with a built in extension model. • Parsing rules (productions) can be inherited and modified at will using a pretty simple syntax.
  25. 25. Making the language ours • Use cases: • • Units: delay = 10h + 30m, weight = 20Kg • • Any sort of DSL would benefit from having its specific syntax Reduce verbosity: for i in 1..10 Short comings: • Editors won’t know how to colorise the new syntax • It’s tempting to abuse it • Makes your code less portable • Tightly coupled to the standard grammar
  26. 26. User eXperience Have you checked clang recently?
  27. 27. Language design • Syntax of a language is the first and most important point of interaction. • Language design is still mainly driven by the target audience expectations (oh those curly braces everywhere). • • The syntax must be very carefully drafted to avoid ambiguities. Even if designers still care very much about the aesthetics. When asked if they performed user tests when designing C#, Hejlsberg answered "only for the integration in Visual Studio”. More generally, check how it integrates with the tools you’re going to use. • Very terse syntaxes tend to be good for DSLs and small and targeted projects. Code is read and reasoned about much more often than written! • Semantics are as important as the syntax, don’t overlook its implications. • Stay away from languages that release new keywords and constructs every other minor version. A syntax either works or doesn’t but can target every use case.
  28. 28. Reporting • • The compiler is our main tool but still, it’s generally speaking, a bitch reporting errors. Error reporting is the main interface, besides the language syntax, between the compiler and the developer. IDEs partially solve this by implementing their own heuristics on the code. • But it should be the compiler! I don’t want my IDE bloated with these things. • • clang has done a great job with this. Other compilers should follow their lead. • Compilers ought to be extensible in this aspect too, if we can extend them with our own logic we must have a clear way to give feedback to the user.
  29. 29. Parsing errors • Language newcomers tend to have syntax errors until they get fluent. • The common approach is to extend the grammar to include invalid productions with custom errors. • Grammars tend to get out of hand and difficult to refactor. • A pretty cool way to improve the errors is to use example snippets of incorrect syntax. • The snippets are fed to the parser and the state of it when it errors is recorded. • Recorded states are then used to build the final parser, so when hits an error that matches a custom message can be displayed. class Foo: def foo() # Note: I forgot the colon ! pass Before: <stdin>(3,13): BCE0043: Unexpected token: <INDENT>. <stdin>(3,13): BCE0044: expecting "DEDENT", found '<EOL>'. <stdin>(3,13): BCE0044: expecting "EOF", found '<DEDENT>'. After: <stdin>(3,13): BCE0043: Methods must use a ‘:’ for their body.
  30. 30. Type Safety Wishing it could be extended
  31. 31. Type Safety • Major families: • Structural (Go, TypeScript, …) • Pattern matching solves a lot of type safety derived issues (bye bye Visitor) • The problem with extending them is that the semantics of the language change. • People can easily reason and adapt to syntax changes but for semantic ones… • In the current state of the art is probably better switching languages completely if the problem at hand requires a different model of type safety. Nominal (Java, C#, …) • • Duck typing (Python, JavaScript, …) • Some languages offer a mix of them (ie. Scala, ObjC, Boo, …) • Many allow for runtime duck typing.
  32. 32. API Versioning • If your language uses nominal typing you actually have to embrace it. Carefully plan your Interfaces to get the job done. • C# does it right. If you want to be confident about exposing an API you can’t make every method virtual (*cough* Java *cough*). • public, protected and private are visibility modifiers, not extension point indicators. • Structural typing really shines in this regard. But it must allow it from the call site not only the definition site (Go vs Scala). The library author doesn’t have a clue if his API works as designed or not :-) • Dynamic languages are flawed on this. Rapid prototyping but zero confidence when upgrading.
  33. 33. Cool stuff Old tech sold as a bright future
  34. 34. Live Coding • Imagine you could tap into the runtime environment and modify the executing instructions on realtime. • We ask the compiler to regenerate the changes in our function, the result is a chunk of byte code to fed into the VMs. • Many projects deploy nowadays into a Virtual Machine, which technically simplify it. • No need to run an interpreter and very few limitations. • Still, this is the kind of thing that would flourish with an extensible compiler design. No big vendor is going to release something like this. • Still, the VM expects byte code, we want to write in a higher level syntax. • Extensible compilers would allow bringing Live Coding to languages that are not interpreted.
  35. 35. Type Providers • Popularised recently by F# • They are basically specialised macros, whose purpose is to generate new types in the program. • Remember when you had to preprocess some XML schema files to generate Java classes? What if the compiler could do that for you automatically? • Using Json? Just feed an example message file to the type provider macro and it will generate a typed interface for it. • This is available in many languages at runtime via Reflection. What’s cool is that now you have it at compile time. Type checking and auto completion as obvious benefits.
  36. 36. What now?
  37. 37. Is your responsibility To get better tools for our job we have to earn it. Advocate these concepts to gain traction in the profession. • If you have to choose between an extensible language and one that’s not, weight the former appropriately in the comparison. Don’t be afraid of moulding a language to your needs. Know your tool and help it help you! If deploying into a VM (JVM, CLR, JavaScript) there is little risk in mixing languages in a project that boost our productivity. • Forget the idea that only functional languages are extensible. Imperative language can do it too! Language designers do listen to its users even if for the mainstream languages they are slow reacting to it. • • • •
  38. 38. Conclusions
  39. 39. Conclusions • Languages should be designed for extensibility. • Anyone that can create a library should be also able to extend the compiler/language. Not everything must be done at runtime! • Statically typed languages are closing the gap with dynamic ones in ease of use and user experience. We have to review our current stand points on this. If you have the chance to learn a new framework or a new language, chose the later, it’s much more rewarding. • Extensible compilers will foster innovation, no need to wait for Oracle or Microsoft to come up with new ideas every 3 years. People will be able to shape the future trying new things and introducing those that work as part of the standard languages. • Good drivers don’t need to know about the internals of the car, but it certainly doesn’t hurt :-) We need better compilers in every aspect. • •
  40. 40. “The limits of my language are the limits of my world.” –Ludwig Wittgenstein

×