A talk exploring the challenges of programming language composition and how the team at King's College London propose to overcome them.

1. Language Composition
A Case Study in Cross-Language Tracing
Edd Barrett
Carl
Friedrich
Bolz
Laurence
Tratt
Naveneetha
Krishnan
Vasudevan
Lukas
Diekmann
Software Development Team
2013-11-13
1 / 25
http://soft-dev.org/

2. Life is Good
2 / 25
http://soft-dev.org/

3. Life is Good
3D Engine
2 / 25
http://soft-dev.org/

4. Life is Good
Mobile App
2 / 25
http://soft-dev.org/

5. Life is Good
Quick Script
2 / 25
http://soft-dev.org/

6. That is, until...
// A load of Android Java code...
// find pairs where i + j < 10
Arraylist<int> result = new ArrayList();
for (int i : ints1) {
for (int j : ints2) {
if (i + j < 10) {
ArrayList<int> n = new ArrayList();
n.add(i);
n.add(j);
result.add(n);
}
}
}
// More Android Java code...
3 / 25
http://soft-dev.org/

7. That is, until...
// A load of Android Java code...
# find pairs where i + j < 10 in *Python*
result = [
(i, j) for i in ints1 for j in ints2
if i + j < 10
]
// More Android Java code...
3 / 25
http://soft-dev.org/

8. Or until...
# Some Python code...
if first == 1:
if second == 1:
return 999
elif second == 2:
return 666
else:
raise TrollException("naughty")
elif first == 2:
if second == 1:
return 1337
elif second == 2:
return 42
else:
raise TrollException("naughty")
else:
raise TrollException("naughty")
# More Python code...
4 / 25
http://soft-dev.org/

9. Or until...
# Some Python code...
(* Much easier to use pattern matching from functional
world, e.g. Ocaml *)
match (first, second) with
| (1, 1) -> 999
| (1, 2) -> 666
| (2, 1) -> 1337
| (2, 2) -> 42
| _ -> raise (TrollException "naughty");;
# More Python code...
4 / 25
http://soft-dev.org/

10. So what is my point?
THE POINT:
5 / 25
http://soft-dev.org/

11. So what is my point?
THE POINT:
PL Wars!
5 / 25
http://soft-dev.org/

12. So what is my point?
THE POINT:
PL Wars!
5 / 25
http://soft-dev.org/

13. So what is my point?
THE POINT:
It would be sweet to compose
programming languages.
5 / 25
http://soft-dev.org/

14. Language Composition
PL X
PL Z
PL Y
6 / 25
http://soft-dev.org/

15. Not a New Idea
Existing composition methods:
Poor syntactic integration.
Stringly-typed.
Languages not born equal.
One language hosts the other.
Poor performance
No cross-language optimisations.
Too much engineering effort.
7 / 25
http://soft-dev.org/

16. What do we want from a language
composition?
8 / 25
http://soft-dev.org/

17. What do we want from a language composition?
High performance
PL
>>>
9 / 25
http://soft-dev.org/

18. What do we want from a language composition?
Good syntactic integration
pl x
pl z
pl y
9 / 25
http://soft-dev.org/

19. What do we want from a language composition?
Little effort
<
9 / 25
http://soft-dev.org/

20. Breaking down the problem
10 / 25
http://soft-dev.org/

21. Breaking Down the Problem
PL X
PL Z
PL Y
11 / 25
http://soft-dev.org/

22. Breaking Down the Problem
syntax
PL X
syntax
runtime
PL Z
syntax
runtime
PL Y
runtime
11 / 25
http://soft-dev.org/

23. Challenges: Syntactic Composition
12 / 25
http://soft-dev.org/

24. Parsing
PL X
<grammar>
expr::= ...
term::= ...
| ...
| ...
func ::= ...
13 / 25
http://soft-dev.org/

25. Parsing
PL X
<grammar>
expr::= ...
term::= ...
| ...
| ...
func ::= ...
PL X
<program>
for (j : js) {
doStuff();
}
.
.
.
13 / 25
http://soft-dev.org/

26. Parsing
PL X
<grammar>
expr::= ...
term::= ...
| ...
| ...
func ::= ...
Parsing
PL X
<program>
for (j : js) {
doStuff();
}
.
.
.
13 / 25
http://soft-dev.org/

27. Parsing
PL X
<grammar>
expr::= ...
term::= ...
| ...
| ...
func ::= ...
Parsing
PL X
<program>
Parse Tree
for (j : js) {
doStuff();
}
.
.
.
13 / 25
http://soft-dev.org/

28. Parsing
PL X
<grammar>
expr::= ...
term::= ...
| ...
| ...
func ::= ...
14 / 25
http://soft-dev.org/

29. Parsing
PL X
<grammar>
expr::= ...
term::= ...
| ...
| ...
func ::= ...
PL Y
<grammar>
expr::= ...
term::= ...
| ...
| ...
func ::= ...
14 / 25
http://soft-dev.org/

30. Parsing
PL X
<grammar>
expr::= ...
term::= ...
| ...
| ...
func ::= ...
PL Y
<grammar>
PL Z
<grammar>
expr::= ...
term::= ...
| ...
| ...
func ::= ...
expr::= ...
term::= ...
| ...
| ...
func ::= ...
14 / 25
http://soft-dev.org/

31. Parsing
PL X
<grammar>
expr::= ...
term::= ...
| ...
| ...
func ::= ...
PL Y
<grammar>
AMBI
GUOU
S
PL Z
<grammar>
expr::= ...
term::= ...
| ...
| ...
func ::= ...
expr::= ...
term::= ...
| ...
| ...
func ::= ...
14 / 25
http://soft-dev.org/

32. Parsing
PL Z
<grammar>
AM
expr::= ...
term::= ...
| ...
| ...
func ::= ...
BI
GU
OU
S
PL Z
<program>
Parsing
for (j : js) {
doStuff();
}
.
.
.
14 / 25
?
http://soft-dev.org/

33. Challenges: Runtime Composition
15 / 25
http://soft-dev.org/

34. Challenges: Runtime Composition
PL X
PL Y
Interpreter
Interpreter
C/C++
16 / 25
http://soft-dev.org/

35. Challenges: Runtime Composition
PL X
To
o
Interpreter
PL Y
sl Interpreter
ow
C/C++
16 / 25
http://soft-dev.org/

36. Challenges: Runtime Composition
JIT Compiler
JIT Compiler
PL X
PL Y
Interpreter
Interpreter
C/C++
16 / 25
http://soft-dev.org/

37. Challenges: Runtime Composition
JIT Compiler
JIT Compiler
PL e Too
X
n
PL Y
gi
mu Interpreter
ne ch
er
in
g
Interpreter
C/C++
16 / 25
http://soft-dev.org/

38. Challenges: Runtime Composition
PL X
PL Y
Interpreter
Interpreter
JVM/CLR
16 / 25
JIT Compiler
http://soft-dev.org/

39. Challenges: Runtime Composition
fo Poo
r
r
dy pe
na
Interpreter rfo
Interpreter
mi
rm
c
la anc
ng e
ua
ge
s
JIT Compiler
PL X
PL Y
JVM/CLR
16 / 25
http://soft-dev.org/

40. Our Proposed Solution
17 / 25
http://soft-dev.org/

41. Proposed solution
Meta-tracing + Language Boxes
18 / 25
http://soft-dev.org/

42. Meta-tracing
Meta-tracing
19 / 25
http://soft-dev.org/

43. PL
Interpreter
Meta-tracing
Meta-tracing
20 / 25
PL
Interpreter
Tracing JIT
http://soft-dev.org/

44. PL X
PL Y
Interpreters
Glue
Meta-tracing
RPython
Meta-tracing
20 / 25
PL Z
Interpreter
Tracing JIT
http://soft-dev.org/

45. PL X
PL Y
Meta-tracing
Meta-tracing
PL Z
Good perfo
rmance
Interpreters
Glue
20 / 25
Interpreter
Tracing JIT
http://soft-dev.org/

46. PL X
PL Y
Meta-tracing
Meta-tracing
PL Z
Good perfo
rmance
Interpreters
Glue
Interpreter
Tracing JIT
Little effort
20 / 25
http://soft-dev.org/

47. Meta-tracing
Language Boxes
21 / 25
http://soft-dev.org/

48. Language Boxes Editor
Suppose we want to write a Java + SQL program.
22 / 25
http://soft-dev.org/

49. Language Boxes Editor
Suppose we want to write a Java + SQL program.
Begin writing Java code
for (string s :
22 / 25
http://soft-dev.org/

50. Language Boxes Editor
Suppose we want to write a Java + SQL program.
for (string s :
Open SQL language box
22 / 25
http://soft-dev.org/

51. Language Boxes Editor
Suppose we want to write a Java + SQL program.
Write SQL code
for (string s :
SELECT * FROM tbl WHERE
22 / 25
http://soft-dev.org/

52. Language Boxes Editor
Suppose we want to write a Java + SQL program.
SELECT * FROM tbl WHERE
name = this.name;) {
for (string s :
Java code
22 / 25
http://soft-dev.org/

53. Language Boxes Editor
Suppose we want to write a Java + SQL program.
Good Syntact
ic Integrati
on
SELECT * FROM tbl WHERE
name = this.name;) {
for (string s :
Java code
22 / 25
http://soft-dev.org/

54. Language Boxes Editor
Suppose we want to write a Java + SQL program.
Good Syntact
ic Integrati
on
SELECT * FROM tbl WHERE
name = this.name;) {
y
for (string s :
Avoids Ambiguit
Java code
22 / 25
http://soft-dev.org/

55. Language Boxes Editor
Suppose we want to write a Java + SQL program.
Good Syntact
ic Integrati
on
NeedSELECT omFROM tbl WHERE
for (string s : s Cust *
Editor
name = this.name;) {
Avoids Ambiguity
Java code
22 / 25
http://soft-dev.org/

56. Our First Composition
23 / 25
http://soft-dev.org/

57. Unipycation
Interpreters
Glue
Unipycation
Prolog
RPython
Python
Interpreter
Tracing JIT
+ our language box editor (eco)
24 / 25
http://soft-dev.org/

58. Thanks
SELECT * FROM tbl WHERE
name = this.name;) {
for (string s :
Java code
Interpreters
Glue
Unipycation
Prolog
RPython
Python
Interpreter
Tracing JIT
http://soft-dev.org
25 / 25
http://soft-dev.org/