Towards a flexible Pharo Compiler

Marcus Denker
Marcus DenkerResearch Scientist at INRIA
Towards a flexible
Pharo Compiler
Clement Bera and Marcus Denker
Three Problems
• Architecture is not reusable
• Compiler can not be parametrized
• The mapping between source code and
bytecode is overly complex.
Reusability
• AST is special for the Compiler
• Tools use own AST (RB)
• AST is destroyed when compiling
• No reusable backend/parser...
No Parametrization
• No pluggable archicture
• Parser, code generator fixed
• No infrastructure for compiler options
Mapping bc2source
• For the Debugger, we need to map
bytecode to source offsets
• With closures, we need to map temp
offsets to real temps.
Old Compiler: Encoder builds
complex table structure
Solution: OPAL
• New compiler framework for Pharo
• Default compiler in Pharo3
• Old Compiler will be a loadable package
Design
• RB AST
• Visitors
• Bytecode
level IR
Smalltalk source code
Refactoring browser abstract
syntax tree
Intermediate representation
Bytecode
RBParser
ASTTranslator + IRBuilder
IRByteCodeGenerator
OCSemanticAnnotator
Reusability
• AST is unchanged
• Backend independent
AST Interpreter
Oz/Hazelnut
Reflectivity
Metalinks
Class Builder
Smart suggestions
Node navigation
Parametrization
• Explicit
compiler
context
• All visitors
are pluggable
Smalltalk source code
Refactoring browser
abstract syntax tree
Intermediate representation
Bytecode
Compilation context
Compiler Options
• Turn off inlining of ifTrue: and friends
MyClass>>foo
<compilerOptions: - optionInlineIf>
^ #myNonBooleanObject
ifTrue: [ 1 ]
ifFalse: [ 0 ]
Mapping
• Mapping
uses AST
directly
source codeBytecode
foo
^ 1 + 2 + 3
76 77 B0
20 B0 7C
IRMethod RBMethodNode
RBSequenceNode
RBReturnNode
RBMessageNode 1
RBLiteral
ValueNode 1
IRPushLiteral
IRSequence
RBLiteral
ValueNode 1
IRPushLiteral
RBLiteral
ValueNode 2
IRReturn
RBReturnNode
IRSend
RBMessageNode 1
IRPushLiteral
RBLiteral
ValueNode 3
RBMessageNode 2 RBLiteral
ValueNode 3
IRSend
RBMessageNode 2
B0
+ 2
1 2
3
+ 2
+ 3
^ 1 + 2 + 3
^ 1 + 2 + 3
foo
^ 1 + 2 + 3
RBLiteral
ValueNode 2
Performance
• Visitors and IR do cost a bit of speed
• But not much
Conclusion
• Opal solves the problems of the old
compiler
• Important basis for many features you will
see in Pharo3 and Pharo4
Conclusion
• Opal solves the problems of the old
compiler
• Important basis for many features you will
see in Pharo3 and Pharo4
Questions?
1 of 14

Recommended

Network layer Part 5 by
Network layer Part 5Network layer Part 5
Network layer Part 5Tutun Juhana
829 views14 slides
Compilation by
CompilationCompilation
CompilationDavid Halliday
700 views4 slides
Introduction to spartakus and how it can help fight linux kernel ABI breakages by
Introduction to spartakus and how it can help fight linux kernel ABI breakagesIntroduction to spartakus and how it can help fight linux kernel ABI breakages
Introduction to spartakus and how it can help fight linux kernel ABI breakagesSamikshan Bairagya
1.6K views21 slides
Linker and loader upload by
Linker and loader   uploadLinker and loader   upload
Linker and loader uploadBin Yang
1K views33 slides
A hands-on introduction to the ELF Object file format by
A hands-on introduction to the ELF Object file formatA hands-on introduction to the ELF Object file format
A hands-on introduction to the ELF Object file formatrety61
3K views15 slides
Dynamic pricing of Lyft rides using streaming by
Dynamic pricing of Lyft rides using streamingDynamic pricing of Lyft rides using streaming
Dynamic pricing of Lyft rides using streamingAmar Pai
871 views25 slides

More Related Content

Viewers also liked

Talk: An Instrument whose Music is Ideas by
Talk: An Instrument whose Music is IdeasTalk: An Instrument whose Music is Ideas
Talk: An Instrument whose Music is IdeasMarcus Denker
738 views31 slides
Unstuck by
UnstuckUnstuck
UnstuckMarcus Denker
612 views21 slides
Lecture: Reflection by
Lecture: ReflectionLecture: Reflection
Lecture: ReflectionMarcus Denker
846 views41 slides
Pharo3 at Fosdem by
Pharo3 at FosdemPharo3 at Fosdem
Pharo3 at FosdemMarcus Denker
3.4K views24 slides
Pharo: Syntax in a Nutshell by
Pharo: Syntax in a NutshellPharo: Syntax in a Nutshell
Pharo: Syntax in a NutshellMarcus Denker
2K views25 slides
Pharo ESUG by
Pharo ESUGPharo ESUG
Pharo ESUGMarcus Denker
1.1K views40 slides

Viewers also liked(9)

Talk: An Instrument whose Music is Ideas by Marcus Denker
Talk: An Instrument whose Music is IdeasTalk: An Instrument whose Music is Ideas
Talk: An Instrument whose Music is Ideas
Marcus Denker738 views
Pharo: Syntax in a Nutshell by Marcus Denker
Pharo: Syntax in a NutshellPharo: Syntax in a Nutshell
Pharo: Syntax in a Nutshell
Marcus Denker2K views
Pharo Status (from PharoDays 2015) by Marcus Denker
Pharo Status (from PharoDays 2015)Pharo Status (from PharoDays 2015)
Pharo Status (from PharoDays 2015)
Marcus Denker1.3K views
Business Diagram V2 by dankirk
Business Diagram V2Business Diagram V2
Business Diagram V2
dankirk166 views
Practical Object-Oriented Back-in-Time Debugging by lienhard
Practical Object-Oriented Back-in-Time DebuggingPractical Object-Oriented Back-in-Time Debugging
Practical Object-Oriented Back-in-Time Debugging
lienhard455 views

Similar to Towards a flexible Pharo Compiler

assem.ppt by
assem.pptassem.ppt
assem.pptSonaliAjankar
1 view36 slides
Inside Python by
Inside PythonInside Python
Inside PythonAlexey Ivanov
567 views31 slides
Inside Python [OSCON 2012] by
Inside Python [OSCON 2012]Inside Python [OSCON 2012]
Inside Python [OSCON 2012]Tom Lee
4.8K views31 slides
Compiler Construction by
Compiler ConstructionCompiler Construction
Compiler ConstructionAhmed Raza
1K views39 slides
Alchemist: An Apache Spark <=> MPI Interface with Michael Mahoney and Kai Rot... by
Alchemist: An Apache Spark <=> MPI Interface with Michael Mahoney and Kai Rot...Alchemist: An Apache Spark <=> MPI Interface with Michael Mahoney and Kai Rot...
Alchemist: An Apache Spark <=> MPI Interface with Michael Mahoney and Kai Rot...Databricks
451 views25 slides
C# 8 and .NET Core 3 by
C# 8 and .NET Core 3C# 8 and .NET Core 3
C# 8 and .NET Core 3Christian Nagel
664 views29 slides

Similar to Towards a flexible Pharo Compiler(20)

Inside Python [OSCON 2012] by Tom Lee
Inside Python [OSCON 2012]Inside Python [OSCON 2012]
Inside Python [OSCON 2012]
Tom Lee4.8K views
Compiler Construction by Ahmed Raza
Compiler ConstructionCompiler Construction
Compiler Construction
Ahmed Raza1K views
Alchemist: An Apache Spark <=> MPI Interface with Michael Mahoney and Kai Rot... by Databricks
Alchemist: An Apache Spark <=> MPI Interface with Michael Mahoney and Kai Rot...Alchemist: An Apache Spark <=> MPI Interface with Michael Mahoney and Kai Rot...
Alchemist: An Apache Spark <=> MPI Interface with Michael Mahoney and Kai Rot...
Databricks451 views
C Programming for ARM.pptx by Pratik Gohel
C Programming for ARM.pptxC Programming for ARM.pptx
C Programming for ARM.pptx
Pratik Gohel4 views
The reasons why 64-bit programs require more stack memory by PVS-Studio
The reasons why 64-bit programs require more stack memoryThe reasons why 64-bit programs require more stack memory
The reasons why 64-bit programs require more stack memory
PVS-Studio220 views
Valuable Information on Lexical Analysis in Compiler Design by Lesa Cote
Valuable Information on Lexical Analysis in Compiler DesignValuable Information on Lexical Analysis in Compiler Design
Valuable Information on Lexical Analysis in Compiler Design
Lesa Cote23 views
Design for Scalability in ADAM by fnothaft
Design for Scalability in ADAMDesign for Scalability in ADAM
Design for Scalability in ADAM
fnothaft1.5K views
Scalable up genomic analysis with ADAM by fnothaft
Scalable up genomic analysis with ADAMScalable up genomic analysis with ADAM
Scalable up genomic analysis with ADAM
fnothaft1.2K views
Writing High Peformance C# 7 Code by Fons Sonnemans
Writing High Peformance C# 7 CodeWriting High Peformance C# 7 Code
Writing High Peformance C# 7 Code
Fons Sonnemans370 views
Database in Kubernetes: Diagnostics and Monitoring by Sveta Smirnova
Database in Kubernetes: Diagnostics and MonitoringDatabase in Kubernetes: Diagnostics and Monitoring
Database in Kubernetes: Diagnostics and Monitoring
Sveta Smirnova76 views
So You Want to Write a Connector? by confluent
So You Want to Write a Connector? So You Want to Write a Connector?
So You Want to Write a Connector?
confluent4K views
Performance improvements in etcd 3.5 release by LibbySchulze
Performance improvements in etcd 3.5 releasePerformance improvements in etcd 3.5 release
Performance improvements in etcd 3.5 release
LibbySchulze280 views
Embedded computing platform design by RAMPRAKASHT1
Embedded computing platform designEmbedded computing platform design
Embedded computing platform design
RAMPRAKASHT11.1K views
Compiler Design Basics by Akhil Kaushik
Compiler Design BasicsCompiler Design Basics
Compiler Design Basics
Akhil Kaushik7.2K views

More from Marcus Denker

Soil And Pharo by
Soil And PharoSoil And Pharo
Soil And PharoMarcus Denker
47 views12 slides
ConstantBlocks in Pharo11 by
ConstantBlocks in Pharo11ConstantBlocks in Pharo11
ConstantBlocks in Pharo11Marcus Denker
50 views26 slides
Demo: Improved DoIt by
Demo: Improved DoItDemo: Improved DoIt
Demo: Improved DoItMarcus Denker
4 views3 slides
First Class Variables as AST Annotations by
First Class Variables as AST AnnotationsFirst Class Variables as AST Annotations
First Class Variables as AST AnnotationsMarcus Denker
5 views42 slides
Supporting Pharo / Getting Pharo Support by
Supporting Pharo / Getting Pharo SupportSupporting Pharo / Getting Pharo Support
Supporting Pharo / Getting Pharo SupportMarcus Denker
8 views19 slides
Lecture: "Advanced Reflection: MetaLinks" by
Lecture: "Advanced Reflection: MetaLinks"Lecture: "Advanced Reflection: MetaLinks"
Lecture: "Advanced Reflection: MetaLinks"Marcus Denker
11 views54 slides

More from Marcus Denker(20)

First Class Variables as AST Annotations by Marcus Denker
First Class Variables as AST AnnotationsFirst Class Variables as AST Annotations
First Class Variables as AST Annotations
Marcus Denker5 views
Supporting Pharo / Getting Pharo Support by Marcus Denker
Supporting Pharo / Getting Pharo SupportSupporting Pharo / Getting Pharo Support
Supporting Pharo / Getting Pharo Support
Marcus Denker8 views
Lecture: "Advanced Reflection: MetaLinks" by Marcus Denker
Lecture: "Advanced Reflection: MetaLinks"Lecture: "Advanced Reflection: MetaLinks"
Lecture: "Advanced Reflection: MetaLinks"
Marcus Denker11 views
thisContext in the Debugger by Marcus Denker
thisContext in the DebuggerthisContext in the Debugger
thisContext in the Debugger
Marcus Denker413 views
Lecture. Advanced Reflection: MetaLinks by Marcus Denker
Lecture. Advanced Reflection: MetaLinksLecture. Advanced Reflection: MetaLinks
Lecture. Advanced Reflection: MetaLinks
Marcus Denker11 views
Improving code completion for Pharo by Marcus Denker
Improving code completion for PharoImproving code completion for Pharo
Improving code completion for Pharo
Marcus Denker292 views
VUB Brussels Lecture 2019: Advanced Reflection: MetaLinks by Marcus Denker
VUB Brussels Lecture 2019: Advanced Reflection: MetaLinksVUB Brussels Lecture 2019: Advanced Reflection: MetaLinks
VUB Brussels Lecture 2019: Advanced Reflection: MetaLinks
Marcus Denker213 views
Lecture: Advanced Reflection. MetaLinks by Marcus Denker
Lecture: Advanced Reflection. MetaLinksLecture: Advanced Reflection. MetaLinks
Lecture: Advanced Reflection. MetaLinks
Marcus Denker1.4K views
Open-Source: An Infinite Game by Marcus Denker
Open-Source: An Infinite GameOpen-Source: An Infinite Game
Open-Source: An Infinite Game
Marcus Denker1.2K views
PharoTechTalk: Contributing to Pharo by Marcus Denker
PharoTechTalk: Contributing to PharoPharoTechTalk: Contributing to Pharo
PharoTechTalk: Contributing to Pharo
Marcus Denker684 views
Feedback Loops in Practice by Marcus Denker
Feedback Loops in PracticeFeedback Loops in Practice
Feedback Loops in Practice
Marcus Denker860 views

Recently uploaded

Spesifikasi Lengkap ASUS Vivobook Go 14 by
Spesifikasi Lengkap ASUS Vivobook Go 14Spesifikasi Lengkap ASUS Vivobook Go 14
Spesifikasi Lengkap ASUS Vivobook Go 14Dot Semarang
35 views1 slide
Architecting CX Measurement Frameworks and Ensuring CX Metrics are fit for Pu... by
Architecting CX Measurement Frameworks and Ensuring CX Metrics are fit for Pu...Architecting CX Measurement Frameworks and Ensuring CX Metrics are fit for Pu...
Architecting CX Measurement Frameworks and Ensuring CX Metrics are fit for Pu...NUS-ISS
32 views54 slides
RADIUS-Omnichannel Interaction System by
RADIUS-Omnichannel Interaction SystemRADIUS-Omnichannel Interaction System
RADIUS-Omnichannel Interaction SystemRADIUS
14 views21 slides
Black and White Modern Science Presentation.pptx by
Black and White Modern Science Presentation.pptxBlack and White Modern Science Presentation.pptx
Black and White Modern Science Presentation.pptxmaryamkhalid2916
14 views21 slides
STPI OctaNE CoE Brochure.pdf by
STPI OctaNE CoE Brochure.pdfSTPI OctaNE CoE Brochure.pdf
STPI OctaNE CoE Brochure.pdfmadhurjyapb
12 views1 slide
SAP Automation Using Bar Code and FIORI.pdf by
SAP Automation Using Bar Code and FIORI.pdfSAP Automation Using Bar Code and FIORI.pdf
SAP Automation Using Bar Code and FIORI.pdfVirendra Rai, PMP
19 views38 slides

Recently uploaded(20)

Spesifikasi Lengkap ASUS Vivobook Go 14 by Dot Semarang
Spesifikasi Lengkap ASUS Vivobook Go 14Spesifikasi Lengkap ASUS Vivobook Go 14
Spesifikasi Lengkap ASUS Vivobook Go 14
Dot Semarang35 views
Architecting CX Measurement Frameworks and Ensuring CX Metrics are fit for Pu... by NUS-ISS
Architecting CX Measurement Frameworks and Ensuring CX Metrics are fit for Pu...Architecting CX Measurement Frameworks and Ensuring CX Metrics are fit for Pu...
Architecting CX Measurement Frameworks and Ensuring CX Metrics are fit for Pu...
NUS-ISS32 views
RADIUS-Omnichannel Interaction System by RADIUS
RADIUS-Omnichannel Interaction SystemRADIUS-Omnichannel Interaction System
RADIUS-Omnichannel Interaction System
RADIUS14 views
Black and White Modern Science Presentation.pptx by maryamkhalid2916
Black and White Modern Science Presentation.pptxBlack and White Modern Science Presentation.pptx
Black and White Modern Science Presentation.pptx
maryamkhalid291614 views
STPI OctaNE CoE Brochure.pdf by madhurjyapb
STPI OctaNE CoE Brochure.pdfSTPI OctaNE CoE Brochure.pdf
STPI OctaNE CoE Brochure.pdf
madhurjyapb12 views
SAP Automation Using Bar Code and FIORI.pdf by Virendra Rai, PMP
SAP Automation Using Bar Code and FIORI.pdfSAP Automation Using Bar Code and FIORI.pdf
SAP Automation Using Bar Code and FIORI.pdf
Emerging & Future Technology - How to Prepare for the Next 10 Years of Radica... by NUS-ISS
Emerging & Future Technology - How to Prepare for the Next 10 Years of Radica...Emerging & Future Technology - How to Prepare for the Next 10 Years of Radica...
Emerging & Future Technology - How to Prepare for the Next 10 Years of Radica...
NUS-ISS15 views
Business Analyst Series 2023 - Week 3 Session 5 by DianaGray10
Business Analyst Series 2023 -  Week 3 Session 5Business Analyst Series 2023 -  Week 3 Session 5
Business Analyst Series 2023 - Week 3 Session 5
DianaGray10165 views
Attacking IoT Devices from a Web Perspective - Linux Day by Simone Onofri
Attacking IoT Devices from a Web Perspective - Linux Day Attacking IoT Devices from a Web Perspective - Linux Day
Attacking IoT Devices from a Web Perspective - Linux Day
Simone Onofri15 views
.conf Go 2023 - Data analysis as a routine by Splunk
.conf Go 2023 - Data analysis as a routine.conf Go 2023 - Data analysis as a routine
.conf Go 2023 - Data analysis as a routine
Splunk90 views
Special_edition_innovator_2023.pdf by WillDavies22
Special_edition_innovator_2023.pdfSpecial_edition_innovator_2023.pdf
Special_edition_innovator_2023.pdf
WillDavies2214 views
Data-centric AI and the convergence of data and model engineering: opportunit... by Paolo Missier
Data-centric AI and the convergence of data and model engineering:opportunit...Data-centric AI and the convergence of data and model engineering:opportunit...
Data-centric AI and the convergence of data and model engineering: opportunit...
Paolo Missier29 views
Upskilling the Evolving Workforce with Digital Fluency for Tomorrow's Challen... by NUS-ISS
Upskilling the Evolving Workforce with Digital Fluency for Tomorrow's Challen...Upskilling the Evolving Workforce with Digital Fluency for Tomorrow's Challen...
Upskilling the Evolving Workforce with Digital Fluency for Tomorrow's Challen...
NUS-ISS23 views
[2023] Putting the R! in R&D.pdf by Eleanor McHugh
[2023] Putting the R! in R&D.pdf[2023] Putting the R! in R&D.pdf
[2023] Putting the R! in R&D.pdf
Eleanor McHugh38 views
Five Things You SHOULD Know About Postman by Postman
Five Things You SHOULD Know About PostmanFive Things You SHOULD Know About Postman
Five Things You SHOULD Know About Postman
Postman25 views
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV by Splunk
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV
Splunk86 views

Towards a flexible Pharo Compiler

  • 1. Towards a flexible Pharo Compiler Clement Bera and Marcus Denker
  • 2. Three Problems • Architecture is not reusable • Compiler can not be parametrized • The mapping between source code and bytecode is overly complex.
  • 3. Reusability • AST is special for the Compiler • Tools use own AST (RB) • AST is destroyed when compiling • No reusable backend/parser...
  • 4. No Parametrization • No pluggable archicture • Parser, code generator fixed • No infrastructure for compiler options
  • 5. Mapping bc2source • For the Debugger, we need to map bytecode to source offsets • With closures, we need to map temp offsets to real temps. Old Compiler: Encoder builds complex table structure
  • 6. Solution: OPAL • New compiler framework for Pharo • Default compiler in Pharo3 • Old Compiler will be a loadable package
  • 7. Design • RB AST • Visitors • Bytecode level IR Smalltalk source code Refactoring browser abstract syntax tree Intermediate representation Bytecode RBParser ASTTranslator + IRBuilder IRByteCodeGenerator OCSemanticAnnotator
  • 8. Reusability • AST is unchanged • Backend independent AST Interpreter Oz/Hazelnut Reflectivity Metalinks Class Builder Smart suggestions Node navigation
  • 9. Parametrization • Explicit compiler context • All visitors are pluggable Smalltalk source code Refactoring browser abstract syntax tree Intermediate representation Bytecode Compilation context
  • 10. Compiler Options • Turn off inlining of ifTrue: and friends MyClass>>foo <compilerOptions: - optionInlineIf> ^ #myNonBooleanObject ifTrue: [ 1 ] ifFalse: [ 0 ]
  • 11. Mapping • Mapping uses AST directly source codeBytecode foo ^ 1 + 2 + 3 76 77 B0 20 B0 7C IRMethod RBMethodNode RBSequenceNode RBReturnNode RBMessageNode 1 RBLiteral ValueNode 1 IRPushLiteral IRSequence RBLiteral ValueNode 1 IRPushLiteral RBLiteral ValueNode 2 IRReturn RBReturnNode IRSend RBMessageNode 1 IRPushLiteral RBLiteral ValueNode 3 RBMessageNode 2 RBLiteral ValueNode 3 IRSend RBMessageNode 2 B0 + 2 1 2 3 + 2 + 3 ^ 1 + 2 + 3 ^ 1 + 2 + 3 foo ^ 1 + 2 + 3 RBLiteral ValueNode 2
  • 12. Performance • Visitors and IR do cost a bit of speed • But not much
  • 13. Conclusion • Opal solves the problems of the old compiler • Important basis for many features you will see in Pharo3 and Pharo4
  • 14. Conclusion • Opal solves the problems of the old compiler • Important basis for many features you will see in Pharo3 and Pharo4 Questions?