Implementing Refactorings in IntelliJ IDEA Dmit ry Jemerov Development Lead JetBrains, Inc. ht tp://www.jetbrains.com/

Agenda Demo of new refactoring features Refactoring building blocks Structural search and replace Problems and future directions

Demo of new refactoring features

Refactoring building blocks

Structural search and replace

Problems and future directions

Agenda Demo of new refactoring features Refactoring building blocks Structural search and replace Problems and future directions

Demo of new refactoring features

Refactoring building blocks

Structural search and replace

Problems and future directions

Building Blocks Not a set of Lego bricks A workflow framework into which you can plug your code And a set of utility functions and classes to implement the essence of refactoring Every refactoring is somewhat different

Not a set of Lego bricks

A workflow framework into which you can plug your code

And a set of utility functions and classes to implement the essence of refactoring

Every refactoring is somewhat different

Lexing and Parsing Lexer converts text into stream of lexems Recursive-descent parser builds PSI (Program Structure Interface) tree Contains all tokens including whitespace Error markers at places of syntax errors Supports high-level semantic operations Supports write operations (add, replace, delete)

Lexer converts text into stream of lexems

Recursive-descent parser builds PSI (Program Structure Interface) tree

Contains all tokens including whitespace

Error markers at places of syntax errors

Supports high-level semantic operations

Supports write operations (add, replace, delete)

References Usages of an element declared elsewhere Can cross languages (e.g. XML to Java) Key operations: Resolve (get declaration by usage) Callbacks on move/rename of target

Usages of an element declared elsewhere

Can cross languages (e.g. XML to Java)

Key operations:

Resolve (get declaration by usage)

Callbacks on move/rename of target

General Refactoring Workflow Show UI for entering options Find usages of target element Check for conflicts (accessibility etc.) Show list of affected usages, allow user to exclude some Modify target element and each usage as appropriate

Show UI for entering options

Find usages of target element

Check for conflicts (accessibility etc.)

Show list of affected usages, allow user to exclude some

Modify target element and each usage as appropriate

Find Usages Finds all references to PSI element Based on inverted index of all identifiers All words that look like a reference are checked if they indeed resolve to target

Finds all references to PSI element

Based on inverted index of all identifiers

All words that look like a reference are checked if they indeed resolve to target

Code Modification Stub of new element created from text Targets of references in fragment to be moved are remembered Stub contents is replaced with fragment to be moved Modifiers etc. are updated via PSI References are updated so that they point to the same targets Updated fragment is reformatted

Stub of new element created from text

Targets of references in fragment to be moved are remembered

Stub contents is replaced with fragment to be moved

Modifiers etc. are updated via PSI

References are updated so that they point to the same targets

Updated fragment is reformatted

Formatter Defines a set of constraints on layout of pairs of elements Minimum spacing / line breaks Indentation Alignment Whitespace not violating constraints is preserved

Defines a set of constraints on layout of pairs of elements

Minimum spacing / line breaks

Indentation

Alignment

Whitespace not violating constraints is preserved

Agenda Demo of new refactoring features Refactoring building blocks Structural search and replace Problems and future directions

Demo of new refactoring features

Refactoring building blocks

Structural search and replace

Problems and future directions

Structural Search QBE interface for navigating and modifying Java and XML/HTML code Code fragment with some variables Constraints for variables Replacement pattern specifying how occurrences should be changed (optional) Allows to perform custom refactoring operations

QBE interface for navigating and modifying Java and XML/HTML code

Code fragment with some variables

Constraints for variables

Replacement pattern specifying how occurrences should be changed (optional)

Allows to perform custom refactoring operations

Structural Search Template

Structural Search Problems Very hard to use by non-expert users Patterns are written in a non-obvious way Unintuitive UI Doesn’t cover all operations No façade around internal API for writing scripts

Very hard to use by non-expert users

Patterns are written in a non-obvious way

Unintuitive UI

Doesn’t cover all operations

No façade around internal API for writing scripts

Agenda Demo of new refactoring features Refactoring building blocks Structural search and replace Problems and future directions

Demo of new refactoring features

Refactoring building blocks

Structural search and replace

Problems and future directions

Context Dependence in Java Java language is not uniform Many contexts in which only certain code is allowed Constructors, super constructor calls Enum constants Anonymous classes Initializer blocks Lots of special-case code needed to handle weird context changes

Java language is not uniform

Many contexts in which only certain code is allowed

Constructors, super constructor calls

Enum constants

Anonymous classes

Initializer blocks

Lots of special-case code needed to handle weird context changes

Refactoring Quality Users typically don’t review the results of applying an automated refactoring Expect their tool to be 100% correct Solutions which provide 90% correctness with simple implementation aren’t acceptable No way to escape hand-coding corner cases

Users typically don’t review the results of applying an automated refactoring

Expect their tool to be 100% correct

Solutions which provide 90% correctness with simple implementation aren’t acceptable

No way to escape hand-coding corner cases

DSL for Code Transformations If the complexity can’t be avoided, maybe it can be compressed? Desire to replace verbose Java code with a domain-specific language Internal DSL Groovy/Ruby JetBrains MPS

If the complexity can’t be avoided, maybe it can be compressed?

Desire to replace verbose Java code with a domain-specific language

Internal DSL

Groovy/Ruby

JetBrains MPS

Patterns Internal DSL for matching AST fragments

Internal DSL for matching AST fragments

Language Independent Refactorings Currently only the basic refactorings (Rename and Move) are mostly language-agnostic All other refactorings are reimplemented from scratch for every new language Desire to extract language-independent essence to common part

Currently only the basic refactorings (Rename and Move) are mostly language-agnostic

All other refactorings are reimplemented from scratch for every new language

Desire to extract language-independent essence to common part

Summary Refactorings in IntelliJ IDEA involve a lot of complex code with limited reuse Complexity caused by large number of concepts in Java language Necessity of 100% correctness means that complexity is essential Main hope for improvement is building powerful code transformation DSLs

Refactorings in IntelliJ IDEA involve a lot of complex code with limited reuse

Complexity caused by large number of concepts in Java language

Necessity of 100% correctness means that complexity is essential

Main hope for improvement is building powerful code transformation DSLs

Q&A Dmitry Jemerov [email_address]

Implementing Refactorings in IntelliJ IDEA Dmit ry Jemerov Development Lead JetBrains, Inc. ht tp://www.jetbrains.com/