Recommended
More Related Content
What's hot
Similar to Making Windows Reasonable
Similar to Making Windows Reasonable (20)
Recently uploaded
Recently uploaded (20)
Making Windows Reasonable
- 1. Bryan Phelps ReasonConf 2019 Making Windows Reasonable
- 2. @bryphe
- 3. Overview 1) Why Native Reason? 2) Esy on Windows 3) Building with Reason
- 5. Why Native Reason? Is performance still an issue in this era of cheap 486 computers and super-fast Pentium computers? You bet. How many programs that you use really run so fast that you wouldn't be happier if they ran faster? We're so used to slow software that when a compile- and-link sequence that took two minutes on a PC takes just ten seconds on a 486 computer, we're ecstatic—when in truth we should be settling for nothing less than instantaneous response. - Michael Abrash’s Graphics Programming Black Book, 1997
- 7. Desktop Applications Today ReactJS Application Chromium Node Harfbuzz FreeType Platform APIs NPM Modules Electron
- 8. Desktop Applications Today ReactJS Application Chromium Node Harfbuzz FreeType Platform APIs NPM Modules Electron
- 9. Desktop Applications Today Native React Application Harfbuzz FreeType Platform APIs Native Dependencies???
- 10. Why Native ReasonML? UI = ƒ(state) ...without all the extra layers!
- 12. Why Native ReasonML? • Benefits from OCaml: • PPX • Flambda • Multicore
- 13. Esy on Windows
- 14. Why Windows?
- 15. Why Windows?
- 16. Esy Easy package management for native Reason, OCaml, and more.
- 17. Esy • Source package manager • Built in Reason / OCaml • Uses OPAM: • Codebase • Protocol • Other sources: • NPM • Github • Local links • Like NPM, but for native code. @andreypop p
- 20. Esy on Windows
- 21. Esy on Windows: The Challenges POSIX Line Endings Path Differences Ecosystem General
- 22. Esy on Windows The OCaml compiler builds and works great on Windows!
- 23. Esy on Windows: POSIX OPAM packages needed: make m4 patch sed grep awk tar curl shasum Esy used:
- 24. Esy on Windows: WSL?
- 25. Esy on Windows: Cygwin • "Happy Path” for Windows • POSIX-Compatible Environment • MingW cross-compilers • Silent, unattended install
- 26. Esy on Windows: Cygwin
- 27. Esy on Windows: esy-bash DEMO
- 29. Esy on Windows: CRLF
- 30. Esy on Windows: CRLF • Carriage Return: Horizontal • Line Feed: Vertical • Multics -> Unix: LF • DOS -> Windows: CRLF • Commodore, Apple II: CR Line Endings Line Feed (LF) Carriage Return (CR)
- 31. Esy on Windows: Path Differences Start Path: “C:UsersBryanmy-projectpackage.json” …save it to a file… and read it… Result: "C:UsersBryanmy-projectpackage.json"
- 32. Esy on Windows: Success! + =
- 33. Enjoy that warm, fuzzy feeling…
- 36. • UI + Application Framework • React-like concepts • Compiles to native code • …and JavaScript • Desktop-only • Platform-agnostic UI
- 39. Native Demo
- 41. JavaScript + WebGL Demo
- 43. Revery: Next Steps Component Model Simplification Skia <Canvas /> Custom Layout Native Integration Developer Experience Improvements Mobile Support
- 47. Onivim 2: Timeline Prototyping 1/2019 Development 3/2019 Pre-Alpha Release 7/2019 Alpha Release 11/2019 Beta Release 3/2020 V1 ???
- 48. Demo
- 49. Onivim 2: Demo
- 51. Thank you! @CrossR @KevinSjoberg @glennsl @szbergeron @tcoopman OCaml Labs Jane Street @dra27 @fdopen @def @andreypopp @jordwalke @ulrikstrid @wokalski @Et7f3 @manuhornung @prometheansacrifice @jchavarri @kitten …and many others!
- 52. Thank you! • Website: https://outrunlabs.com • Pre-order: https://onivim.io • Twitter: @bryphe
- 53. Bryan Phelps ReasonConf 2019 Making Windows Reasonable
Editor's Notes
- Hey ReasonConf! My name’s Bryan Phelps, and I’m so excited to be here. My talk is titled - Making Windows Reasonable - but there’s a few things I want to talk to you about today.
- First a quick intro for those who don’t know me - I’m founder of a company called outrun labs, and native reason is what I hope is our secret sauce. Previously, I worked at Microsoft, which might explain why making things work on Windows is important to me. Even though it might not look like it from my Macbook… I’m primarily a Windows developer. You can find me on twitter, github, discord as Bryphe.
- First, I want to share with you why I’m excited about Reason - and especially native Reason. Second, I want to talk about making esy work on Windows - and some of the hurdles and challenges we faced. Finally, I want to share with you the way we’re using this native Reason tech to build and ship cross-platform applications.
- So let’s dive in to ‘Why Native Reason’? For that - I’d like to share a quote.
- And this quote is from a famous graphics programmer, Michael Abrash - it’s from a book he wrote called the Graphics Programming Black Book. it’s now freely available on GitHub: https://github.com/jagregory/abrash-black-book/blob/master/src/chapter-01.md I highly recommend it - even if you’re not interested in graphics, it’s a fun read and has generally applicable lessons. -Here’s the quote - What blows me away is that this quote is from over 20 years ago. In that time, processors have gotten 100s of times faster, our memory and disks have gotten faster with SSDs and flash drives - and yet we’re still struggling with this same exact challenge. I would argue that my PC doesn’t feel any faster than it did 10 years ago - in fact, it feels slower - its normal for apps to take 5, 10, even 20 seconds to startup.
- And when we think about performance - it’s worth looking at how we build our applications. The most popular tool today for building a cross-platform desktop application is Electron. And it’s popular for good reason - it’s never been easier for a small team to build and ship a cross-platform application. Its strength is that it allows you to develop cross-platform with confidence - what I mean by that is you can write your code, test it one platform, and be like 99% sure it’s going to work and look consistent everywhere. That’s really powerful! Not only that - you can bring to bear the full web development ecosystem - and tools like React mean it has never been easier to build UI!
- If you’re not familiar with Electron, it’s essentially bundling up Chromium and Node into a package that runs a JavaScript app. Of course - those aren’t the only ingredients - there’s always going to be some platform APIs underneath, some low-level libraries for things like text rendering or OpenGL, Vulkan, Metal, etc Above Node and Chromium, you have all your NPM modules - and if you’re using React, ReactJS would be one of those dependencies. It’s not uncommon to have thousands of these dependencies. Then, finally, you get your application code on top
- It’s a lot of layers. And we do get something for it - in the sense that Electron has made it easier than ever to build apps., But these layers and abstractions come at a cost. And part of that is a performance cost - a build time cost - and sometimes a complexity cost. Not only do you have to be aware of your application layer, but ultimately you’re responsible for everything below it as well. The fact is there just isn’t a great alternative today - especially not if you want to develop your application in a functional React, style.
- But what if we could streamline it? Remove the JavaScript, Node, and Chromium - and replace it with a compile-to-native react implementation. Replace our NPM modules with a smaller, more streamlined set of compile-to-native dependencies. What would that look like? This is where we could get our performance back! It’s kind of like going back in time - except we’re applying all the great learnings from React back.
- With Reason - it’s like we have a language purpose built for expressing this idea of React - a functional UI - your UI expressed as a pure function of state. It’s paradigm has been battle-tested in the fierce, ever evolving arena of web frameworks - and now we have a language that feels like it was built for React. Oh, and it compiles to native code.
- And when you compare ReasonML to TypeScript - this is the key strength, the differentiator in my mind - the ability to not only work on the web - but the ability to compile to streamlined, high-performance, native code. And I think there is still so much untapped potential here.
- On top of that, it’s not like native Reason is happening from scratch. We’re leveraging all the great investments companies like OCamlLabs and Janestreet are pouring into OCaml - continual compiler improvements and optimizations, flambda, multicore. It’s a cool hybrid of academia exploring new ideas, and then having them applied in real-world, intense environments like Jane Street. In a sense we’re benefitting from all that sweat and hard work - and really just repackaging for developers coming from JavaScript and React.
- So now I want to dive more into building cross-platform with confidence, and how that’s happening -let’s talk about esy on windows.
- A natural question to start with might be - why windows? Can I see a show of hands - how many people are primarily windows developers? Like 2 out of 100 :) So it really makes sense to ask why?
- Well #1 - Windows is a huge market of developers. And obviously an underrepresented set of users in our conference and community. And it’s sort of a chicken-and-the-egg kind of problem - if we don’t have the tooling, or seamless experience in place - how are we going to get more Windows developers to help us improve the tooling and get a better experience?
- To be able to work cross-platform, you need something that can help you install dependencies, build, and run everywhere. And esy is designed for that. We want to be also be able to leverage all the great work happening in the OCaml ecosystem, from the package manager - because there is so much high quality code there.
- Esy is a source packmanager, which means it builds from source code. This is desirable, because you can audit the sources your building instead of getting black-box libraries. But it also adds a lot of complexity - figuring out when a recompilation is needed, sometimes the environment can affect the build output. There are several sources esy can work with - the ocaml package repository, the node package repository, github, it can pull sources locally. Really I think the ‘esy-est’ way to think about is like NPM, but for native code. Another way you can look at it though, from the architecture, is that it’s really like react-the-idea - expressed as a package manager. Your build output as a pure function of your source code and your dependencies. @andreypopp is the primary developer and has done the vast majority of work on it.
- I just want to show a quick demo of how this works on Windows. This is using the `hello-reason` project on GitHub - the simplest starter project for Reason. https://github.com/esy-ocaml/hello-reason
- This is windows powershell, and I’m in the hello-reason starter project (https://github.com/esy-ocaml/hello-reason). I’m going to install the dependencies, build, and run. Using it is a lot like NPM - I say ‘esy install’ and ‘esy build’ Now the first you build - it isn’t going to be this fast - all your dependencies have to be built. But subsequent builds are fast.
- Here’s the `package.json` for this project - what’s cool is it shows us using sort of the full-extent of the capabilities. It’s picking up `dune`, `merlin`, and `odoc` from the ocaml package manager repo as well as several NPM dependencies. It can also get dependencies from github, or locally - which is great when we you need to test a fix across several packages! — There’s another cool feature that esy provides called Sandboxes - which let you have mini-environments in a project. This is great for testing & benchmarking. Here’s a demo of that in action.
- It wasn’t always this “easy” - there were really two classes of challenges we had in getting esy to work on Windows. The first were ecosystem challenges - challenges that are specific to the OCaml or OPAM ecosystem. And this is primarily that a lot of packages depend on ‘bashisms’ bash or POSIX utilities like grep and awk.
- The core OCaml compiler builds and runs great on Windows. That’s a testament to a lot of work from OCaml Labs and other contributors like Jane Street. And there’s a build tool for native Reason and OCaml called dune that’s quickly becoming the standard - and it also works great cross-platform, and often helps us sidestep needing these bash utilities. So things are getting better! But there are still a lot of packages and dependencies that need help to run on Windows.
- It turns out some older, but still core, OCaml packages depend on having POSIX utilities - sometimes as part of their build step, or they need make to install native code, they do some string replacement with awk - all kinds of random things that are just expected to work. Now dune - the basically-standard build system for OCaml - is actually making this a lot better. But there are still a lot of libraries that haven’t been upgraded to use Dune, yet. So how we do fix this?
- The first solution we looked at, that seemed like a perfect fit, was to use this feature of Windows - called the windows subsystem for Linux (or WSL short). This is a new feature where you can essentially host a Linux distro on Windows. It sounds perfect; but as we investigated, at least at the time, there was no one-click install - meaning we wouldn’t be able to have that simple flow of install esy, esy install, esy build. We’d need to have manual steps for users to not only enable WSL, but pick out a distro and set that up, and then install dependencies specific to that distribution. So we felt that was too much of a hurdle
- The other option we looked at was Cygwin. Cygwin has been around for a long-time and is pretty mature. Today, if you want to use OCaml packages on Windows - Cygwin is the happy path. It’s the blessed way to get the ocaml package manager working on Windows. It provides a POSIX environment, with all the utilities we need - and the MingW cross-compilers to output windows-native binaries. And importantly for esy, to make setup ‘easy’ - we could deliver it in a silent, unattended install. I want to reiterate that binaries we produce are native windows executables - the executables that come as output have no dependency on Cygwin at all.
- To facilitate that, we created this NPM package called `esy-bash`. Which is really a thin wrapper around Cygwin and around bash that is bundled with Cygwin. It works from the CLI or via an API - and you can send it commands pretending to be in a POSIX environment. I’ll show you a quick demo of what that looks like.
- I’ll show you a demo of what I mean about `esy-bash` used as a CLI application. I’m going over to a Windows Powershell environment, where I’ve already npm installed esy-bash.
- Esy-bash can be used as a CLI application, or as a JavaScript API. I’ll show you the CLI version - it’s pretty simple, it just takes a command to run in bash as an argument. You can access to all the POSIX utilities necessary, like make, or ls - we can even ls the root and see what the Cygwin sandbox root looks like. This gives us a layer of abstraction - so that esy can forward commands to this esy-bash sandbox, and have them just work.
- Line endings cross-platform are a mess. And this is to blame. This is a teletype machine - it’s 60s era hardware, and its kind of how we computed before had these nice retina displays. Instead of a screen, there’d actually be a physical printout. Can you imagine?
- And going to the next line was actually a two-part manuever. There was the carriage return, which was a horizontal motion of the print head - moving the print head back to the far left of the sheet. Then there was the line feed, which moving the sheet up vertically. Pressing the enter key meant doing both - advancing the sheet, and moving the carriage back to the start of the line. On Multics, and Unix and beyond inherited this - they decided to simplify. It didn’t make sense to have just a line feed - so if you meant that, the driver just helped you out and assumed you meant both. On Dos, and Windows, there was no such assistance - you had to send both, and that lives on to this day. On some other machines, like Commodore, Apple II - they were different expected a CR. So we have all these standards. The problem though is Windows decided to handle this at the software level - there are two ways to read a binary stream in C - as a text mode which does automatic conversion, or a binary mode which doesn’t. And on other platforms - this is a no-op - and that makes it easy to pick the wrong one by default.
- Windows uses the backslash as its path separator. This also happens to be the escape character. So if you’re not careful - you can get this sort of unexpected escaping popping up!
- There were a few other bonus challenges - like path length limits, environment size limits, issues with symlinks - but esy works on Windows now. With esy - we have a one-click install, and the ability to setup and run projects on all platforms. This is really powerful, because it is the first step to developing cross-platform with confidence - like we do with Node, Electron, and NPM.
- Esy + rely + ci
- Having that tooling is really the beginning of the story, not the end - I want to share with you a few projects myself and collaborators are working on
- The first project is Revery
- Bullet points Platform-agnostic UI is a trade-off. It means that we render the same UI the same everywhere. This is great for that idea of cross-platform with confidence, because you know it will look the same everywhere. But - sometimes that’s not the right trade-off, and if you’re looking for something like Revery that uses Platform widgets - check out Brisk. The team that built Brisk created brisk-reconciler - which is our ‘native React’
- This is the simplest, smallest Revery app - simply opens a Window, and renders an empty view. The `App.start` is the entry point, and takes a function that gets passed an app instance. And with that app instance - you can create a window, and with that window, you can create a UI. `UI.start` is a lot like `ReactDOM.render` just a native rendition
- Courtesy of a project called brisk-reconciler, we actually have native, type-safe hooks as part of our library. And this concept of type-safe hooks really showcases the power of the Reason and OCaml type-system. Before I learned about Reason, coming from a C, C#, TypeScript background - I kind of thought all type systems were more less the same. But this idea of type-safe hooks is something that you can’t model in those sorts of type systems. So it was kind of a gateway drug for me to learn more about the type system. We would like to reduce this boilerplate, and still get the same compile-time benefits, so that’s an active area of exploration.
- I’ll show a quick demo of the native example app
- Just like before - we can use esy to install and build the project.
- I’ll show a quick demo of the native example app
- So I switched gears, I wanted to show a few demos on Mac to show that things actually work cross-platform.
- Revery isn’t done - there’s a lot we need to do! This is just a mental model of the next steps I see for Revery - there’s no dates or promises in terms of any of these deliverables. Just ideas. Component Model Simplification - pax’s to streamline component creation and usage of hooks. Less boilerplate! 2) Integrate Skia - a battle-tested, high-performance vector graphics library. Used by Sublime Text, Flutter, Chrome. 3) Custom Layout - today our only primitive is ‘flex’ - but we’d like to be able model other layout systems too, and let the user of the framework determine the layout strategy. 4) Native Integration - text input, accessibility 5) DX - performance traces, debugging, REPL support, live-reloading and time-travel 6) Mobile support - compile to iOS and Android Kind of a fun with Revery is that we get the opportunity to revisit our assumptions - we’re not shackled to the DOM or the performance characteristics of it.
- Revery was built from the ground-up for Onivim 2.
- I built a code editor with Electron. The goal was to be a hybrid of Vim and modern code editors like Visual Studio Code, Atom, Sublime Text. We got pretty far with it - it was downloaded ~150k times, and there was a clear niche that wanted this hybrid of best-of-both-worlds, works-out-of-the-box. However, we ran into several performance issues - and around the time I was figuring out what to do next, I learned about Reason - and realized Reason-Native would be the perfect technology platform to build a v2 and really deliver on that vision.
- So we got a new cool logo and went to work. We rearchitected - we decided to build our front-end in native Reason. We wanted to leverage all the great work that is happening in the VSCode ecosystem - things like language support of course, but also things like themes, source control integration, tasks, even debugging. So we are using the VSCode extension host as our ‘backend’. That work is in development right now.
- This is the latest build of Onivim 2. It’s definitely pre-alpha quality - we have so much work to do in terms of basic functionality - not to mention fit-and-finish and design. But we’re moving along and making progress. A major win for us is startup performance - startup is essentially instant. I’m going to start it up a couple times. With this Reason native tech stack - we’ve been able to deliver on performance for startup. Our v1 started in about 5s, whereas our v2 starts in less than half a second - 10x improvement. Memory usage is another major win. What’s crazy is that there is still a lot of headroom, we haven't even integrated the ocaml optimizing compiler. We're still missing a lot - but there's some non-trivial UI that demonstrates what we can do with Revery. Even though there is still a lot of work ahead - features to implement and bugs to fix- I think the foundation we have is really promising. We can build and ship a reason native application, and we’ve already seen tremendous benefit in our performance with this new stack. Now it’s all about delivering the features and functionality and stabilizing on top of it.
- We’re building Onivim 2 in the open - and we’re taking pre-orders. So if you’re interested in helping to support us building the editor, or supporting projects like Revery, you can do that by pre-order. —- We plan on raising the price as we go - as we stabilize and add features and meet our milestones. We’re also experimenting with a ‘time delay’ license - we’re building the app in the open, under a proprietary license, but 18 months after a commit hits master, we dual-license as MIT. So in reality, this funding model is helping us build better open source than we’d otherwise be able to. And it keeps a fire under us - if we’re not implementing new features or fixes - why not just the MIT licensed one? I hope it’s successful, my hope is that more people could use a functional language like this to build applications.
- Everything I’ve been building is standing on the shoulders of others - there is so much great work happening in the OCaml ecosystem, the Reason ecosystem, and I’ve been tremendously appreciative of all the help and support community and from the Reason association. I actually had trouble making this slide just because there were so many people, and so much tech, that has helped us get to where are now. So I’m really sorry if I missed anyone! It’s tough being an early adopter - but we’re all pioneers together, exploring this wild west, this new frontier.
- That’s all I have - thanks for listening! If you want more info on what I’m building - check out outrunlabs.com or onivim.io. Feel free to reach out to me on twitter or Discord Thank you!