tiny.cloud
PORTING 100K LINES OF
CODE TO TYPESCRIPT
MILLIE MACDONALD
SOFTWARE ENGINEER @ TINY
The Talk
● About Tiny and me
● The Conversion Project:
○ Motivation
○ Why TypeScript?
○ How we converted our JavaScript to TypeScript
○ Benefits, lessons learned and ongoing work
TL;DR: Types help. Gradual typing really helps.
About Tiny
● TinyMCE is a popular open source project
● A rich text editing platform that helped launch
and scale the adoption of Medium, WordPress,
Shopify, Tumblr, Atlassian, EventBrite & more
● In 2015, we launched a startup with offices in
Palo Alto, Australia and Sweden
By the numbers:
● 1 million+ users on Cloud
● 125,000 downloads/week
● 1,000 paying customers
● 232 contributors
About Me
● From Brisbane, Australia
● Bachelor of Engineering
(Honours), Software Engineering
● Intern then QA Engineer then
Software Engineer at Tiny
● Admin of the Australian gaming
community for Splatoon
“100k Lines of Code”?
TinyMCE is:
● Core is ~30k LoC
● User Interface is ~30k LoC
● 44 core plugins + several
premium plugins with ~40k
LoC
Plus various other projects
and libraries…The point of
conversion!
The Conversion Project
Before
● Over the years, we tried and rejected a lot of frameworks
● We also use a functional programming style
● So we wrote our own libraries and frameworks
○ Avoided upgrade problems and framework churn
○ Easier for us to fix problems
○ FP + JS means defensive code, which resulted in code size and runtime performance
issues
● Then ES2015 appeared...
Motivation for the Conversion Project
● ES2015 features - especially a standard module system - were
very enticing, so in 2017 we decided to convert our code.
● We had also been considering switching to an alternative JS
language. Our wish list was:
○ static typing
○ pattern matching
Why TypeScript?
● We looked at ReasonML, PureScript and TypeScript
● We considered:
○ how much of the functional programming feature set we needed
○ the learning curve for each language
○ the upfront development cost to switch
Why TypeScript?
● ReasonML
○ Rewrote our PowerPaste plugin for TinyMCE, replacing complicated regex with strong types and
pattern matching
○ Syntax is similar to JS so easy to learn
● PureScript:
○ Used for some of our internal prototypes
○ Side-effect tracking speeds up code reviews and training, and optics improve runtime efficiency
○ Lots of FP features we don’t need on the client side
● Neither is an easy drop-in replacement for the editor code
Why TypeScript?
● Unfortunately...
○ No pattern matching
● But...
○ Vanilla JS is valid TypeScript
○ Due to already using a module system with strict syntax, converting our code
could be mostly done with a script
○ Types!
○ Gradual types!
What is Gradual Typing?
● Process of starting with minimal types and adding more over time
● Requires a mix of static and dynamic types - like TypeScript
● Requires minimal initial work
● Can invest as however much time and developers as you like in
increasing types each week/month/etc.
● Can start with simple types then increase complexity as you learn and
adjust to the type system
● Low initial cost of adding types, flexible ongoing cost
The How
Conversion Process for each Library
1. Run script to convert modules to ES2015 and TypeScript
2. Manually check the script converted each file correctly
3. Update with changes from already-converted dependencies
4. Fix problems found by TypeScript
5. Run tests and fix any problems
6. Update build process
The Conversion Script
1. Transpile all modules using an Abstract Syntax Tree (AST)
a. Change import syntax
b. Pull code from inside the wrapping function to the root level
c. Change export syntax and apply export default <any>
2. Move files and rename to .ts
3. Copy templates for configuration files
4. Generate Main.ts from api folder
5. Update package.json and .gitignore
Module Conversion Example
define(
'Example Module',
[ ‘example.Dependency' ],
function (Dependency) {
var bar = 0;
var foo = function () { … };
return {
foo,
bar
};
}
);
import { Dependency } from 'example';
var bar = 0;
var foo = function () { … };
export default <any> {
foo,
bar
};
Adding Types
● The script doesn’t add types
● We started adding types to libraries, starting with the libraries
most commonly depended on
● We add types with each PR
○ New code must be at least partially typed
○ “If you touch a file, type it” was the rule for a while
● Now, many of our libraries are at least partially typed
Benefits of TypeScript and
Gradual Typing
Types
● Gained type-based tools e.g. find all usages and
automated refactoring
● Compile time type checking
● Even just adding simple types caught some serious bugs
● Typing our most common dependencies caught many
more
Example Bugs
Incorrect function argument list length
Missing comment resulted in accidental global
Gradual Typing
● Most of the team was able to go back to normal development work
quickly
○ Sometimes devs take a couple days to type a library
○ We add some types with most PRs
● Conversion script used export default <any> so minimal errors
and developer work upfront
● Gradually removing any from modules allowed us to type modules
slowly and methodically
Increased Readability
● Types make it easier to follow code
○ Especially in our giant projects where variables sometimes get passed around a lot
● Types make it easier to debug code
○ Type errors are generally nicely worded
○ Sometimes easier to match weird values to types, then track down which variable or function
they come from
● Types make it easier for new developers to get up to speed and learn
the code
Lessons Learned
Convert Common Libraries First
● Because of our modular architecture we have a lot of
libraries, and many depend on other libraries
● We mostly converted the most common dependencies first,
so we could cascade their types and bug fixes through other
libraries
● But some other libraries were done at the same time, and
they have duplicate type definitions, clashing types, etc.
Gradually Introduce Compiler Features
A couple times we enabled compiler features without testing them
properly on every library...
Write a Script to Test Everything
● Write a script that does every kind of check it can - types,
linting, build, tests, etc.
● Run it when you make a big change
● Ours is called the Really Cool ScriptTM
○ Contains a list of all our libraries
○ Clones each one, installs its dependencies, runs tsc and npm test
Developer Adjustments
● Discuss changes in coding style, tools and processes
● Communicate changes
● Script pre-commit or pre-build type checks to help developers
remember what to check
Continuing Work
Continuing Work
● More types!
● Enabling more compiler features
● Building a type definition file for TinyMCE’s editor API
Check out TinyMCE
● TinyMCE is open source
○ https://github.com/tinymce/tinymce
○ Contains TS files and most config and build files
○ Master branch is TinyMCE 4
○ 5.x branch for TinyMCE 5
● Found a bug? Win some swag!
https://go.tiny.cloud/blog/tinymce-5-developer-challenge/
Thank you!
Questions?

Porting 100k Lines of Code to TypeScript

  • 1.
    tiny.cloud PORTING 100K LINESOF CODE TO TYPESCRIPT MILLIE MACDONALD SOFTWARE ENGINEER @ TINY
  • 2.
    The Talk ● AboutTiny and me ● The Conversion Project: ○ Motivation ○ Why TypeScript? ○ How we converted our JavaScript to TypeScript ○ Benefits, lessons learned and ongoing work TL;DR: Types help. Gradual typing really helps.
  • 3.
    About Tiny ● TinyMCEis a popular open source project ● A rich text editing platform that helped launch and scale the adoption of Medium, WordPress, Shopify, Tumblr, Atlassian, EventBrite & more ● In 2015, we launched a startup with offices in Palo Alto, Australia and Sweden By the numbers: ● 1 million+ users on Cloud ● 125,000 downloads/week ● 1,000 paying customers ● 232 contributors
  • 4.
    About Me ● FromBrisbane, Australia ● Bachelor of Engineering (Honours), Software Engineering ● Intern then QA Engineer then Software Engineer at Tiny ● Admin of the Australian gaming community for Splatoon
  • 5.
    “100k Lines ofCode”? TinyMCE is: ● Core is ~30k LoC ● User Interface is ~30k LoC ● 44 core plugins + several premium plugins with ~40k LoC Plus various other projects and libraries…The point of conversion!
  • 6.
  • 7.
    Before ● Over theyears, we tried and rejected a lot of frameworks ● We also use a functional programming style ● So we wrote our own libraries and frameworks ○ Avoided upgrade problems and framework churn ○ Easier for us to fix problems ○ FP + JS means defensive code, which resulted in code size and runtime performance issues ● Then ES2015 appeared...
  • 8.
    Motivation for theConversion Project ● ES2015 features - especially a standard module system - were very enticing, so in 2017 we decided to convert our code. ● We had also been considering switching to an alternative JS language. Our wish list was: ○ static typing ○ pattern matching
  • 9.
    Why TypeScript? ● Welooked at ReasonML, PureScript and TypeScript ● We considered: ○ how much of the functional programming feature set we needed ○ the learning curve for each language ○ the upfront development cost to switch
  • 10.
    Why TypeScript? ● ReasonML ○Rewrote our PowerPaste plugin for TinyMCE, replacing complicated regex with strong types and pattern matching ○ Syntax is similar to JS so easy to learn ● PureScript: ○ Used for some of our internal prototypes ○ Side-effect tracking speeds up code reviews and training, and optics improve runtime efficiency ○ Lots of FP features we don’t need on the client side ● Neither is an easy drop-in replacement for the editor code
  • 11.
    Why TypeScript? ● Unfortunately... ○No pattern matching ● But... ○ Vanilla JS is valid TypeScript ○ Due to already using a module system with strict syntax, converting our code could be mostly done with a script ○ Types! ○ Gradual types!
  • 12.
    What is GradualTyping? ● Process of starting with minimal types and adding more over time ● Requires a mix of static and dynamic types - like TypeScript ● Requires minimal initial work ● Can invest as however much time and developers as you like in increasing types each week/month/etc. ● Can start with simple types then increase complexity as you learn and adjust to the type system ● Low initial cost of adding types, flexible ongoing cost
  • 13.
  • 14.
    Conversion Process foreach Library 1. Run script to convert modules to ES2015 and TypeScript 2. Manually check the script converted each file correctly 3. Update with changes from already-converted dependencies 4. Fix problems found by TypeScript 5. Run tests and fix any problems 6. Update build process
  • 15.
    The Conversion Script 1.Transpile all modules using an Abstract Syntax Tree (AST) a. Change import syntax b. Pull code from inside the wrapping function to the root level c. Change export syntax and apply export default <any> 2. Move files and rename to .ts 3. Copy templates for configuration files 4. Generate Main.ts from api folder 5. Update package.json and .gitignore
  • 16.
    Module Conversion Example define( 'ExampleModule', [ ‘example.Dependency' ], function (Dependency) { var bar = 0; var foo = function () { … }; return { foo, bar }; } ); import { Dependency } from 'example'; var bar = 0; var foo = function () { … }; export default <any> { foo, bar };
  • 17.
    Adding Types ● Thescript doesn’t add types ● We started adding types to libraries, starting with the libraries most commonly depended on ● We add types with each PR ○ New code must be at least partially typed ○ “If you touch a file, type it” was the rule for a while ● Now, many of our libraries are at least partially typed
  • 18.
    Benefits of TypeScriptand Gradual Typing
  • 19.
    Types ● Gained type-basedtools e.g. find all usages and automated refactoring ● Compile time type checking ● Even just adding simple types caught some serious bugs ● Typing our most common dependencies caught many more
  • 20.
    Example Bugs Incorrect functionargument list length Missing comment resulted in accidental global
  • 21.
    Gradual Typing ● Mostof the team was able to go back to normal development work quickly ○ Sometimes devs take a couple days to type a library ○ We add some types with most PRs ● Conversion script used export default <any> so minimal errors and developer work upfront ● Gradually removing any from modules allowed us to type modules slowly and methodically
  • 22.
    Increased Readability ● Typesmake it easier to follow code ○ Especially in our giant projects where variables sometimes get passed around a lot ● Types make it easier to debug code ○ Type errors are generally nicely worded ○ Sometimes easier to match weird values to types, then track down which variable or function they come from ● Types make it easier for new developers to get up to speed and learn the code
  • 23.
  • 24.
    Convert Common LibrariesFirst ● Because of our modular architecture we have a lot of libraries, and many depend on other libraries ● We mostly converted the most common dependencies first, so we could cascade their types and bug fixes through other libraries ● But some other libraries were done at the same time, and they have duplicate type definitions, clashing types, etc.
  • 25.
    Gradually Introduce CompilerFeatures A couple times we enabled compiler features without testing them properly on every library...
  • 26.
    Write a Scriptto Test Everything ● Write a script that does every kind of check it can - types, linting, build, tests, etc. ● Run it when you make a big change ● Ours is called the Really Cool ScriptTM ○ Contains a list of all our libraries ○ Clones each one, installs its dependencies, runs tsc and npm test
  • 27.
    Developer Adjustments ● Discusschanges in coding style, tools and processes ● Communicate changes ● Script pre-commit or pre-build type checks to help developers remember what to check
  • 28.
  • 29.
    Continuing Work ● Moretypes! ● Enabling more compiler features ● Building a type definition file for TinyMCE’s editor API
  • 30.
    Check out TinyMCE ●TinyMCE is open source ○ https://github.com/tinymce/tinymce ○ Contains TS files and most config and build files ○ Master branch is TinyMCE 4 ○ 5.x branch for TinyMCE 5 ● Found a bug? Win some swag! https://go.tiny.cloud/blog/tinymce-5-developer-challenge/
  • 31.
  • 32.