I got involved with jsdom a couple years ago, mainly using it for testing. I was writing a Windows 8 app in HTML and JS, and it turns out that there’s no way to run automated unit tests in Windows 8. Crazy, right? So I found this project called jsdom, that provided a virtual environment in which I could run my unit tests. I submitted a few pull requests, then a few more, and pretty soon I was hooked. I’ve been helping to maintain the project ever since, with involvement from some 124 other contributors as well.
Here’s an example of perhaps the simplest use of jsdom: using it as a web scraper. You can see how we load a URL into the jsdom environment, then also load our own script---jQuery. One everything has loaded into the environment, we can manipulate the window object to do things like select elements using jQuery and count or manipulate them.
Here’s an example that’s a bit more complicated. It shows how you can use jsdom for testing.
I use jsdom for these kind of tests in many of the libraries I write. In this way it can replace costly out-of-process tools like PhantomJS or Selenium.
Finally, here’s an example of using jsdom in real-time as part of a server pipeline. This is a quick-and-dirty HTTP server that will take any URL you give it, and use jsdom to flip all the images, in real time. It does this by using jsdom’s canvas functionality to rotate the image, then get the result as a data URL, which it then modifies the DOM to point to. And indeed…
Zombie.js, which takes the idea of an in-memory window one step further to give you an in-memory browser, where you can submit forms, click links, and more.
Or Facebook’s Jest testing framework, which uses jsdom to run your tests lightning-fast against a mocked browser environment.
The actual “DOM” standard is just a single document, a living standard hosted at dom.spec.whatwg.org. It defines only the basics, really: events, node trees, attributes, mutation observers, and documents.
You might think creating jsdom was just a matter of translating this spec into code.
It turns out that to create a useful document and window object, you need a lot more than just the DOM Standard and its node trees. You also need HTML, for the definition of all the elements that will appear in that DOM. You need the spec for parsing the DOM, in order to construct the node tree from a string. You need the spec for serializing, in order to make innerHTML work. And of course you need things like XHR and URL parsing. What’s more, even though jsdom doesn’t do any actual layout calculations (yet?), you do need some CSS implemented. CSS selectors, of course, so that querySelector works, but also the CSS object model, so that when things animate or hide or show, you can reflect that in the .style property.
jsdom was coded before the modern DOM and HTML standards existed. Instead, we have this code structure that tries to build itself up in steps: first DOM level 1, then… well… (Slides)It turns out browsers never actually implemented a lot of this stuff. Much of it was crazy XML things; others were just very obscure; etc. The modern DOM and HTML standards codify what browsers actually implement, in a single spec, and they have a note about how a bunch of stuff is obsolete and should be removed from implementations that still have it. So jsdom is actually built in this same layered way, reflecting the historical way specs grew up, and needs a good clean-up effort.
That’s just a small taste of the unfortunate truth about the standards underlying the web platform, but we’ll stop there for today.
Anyway, the good news is we’re fixing that; in jsdom 2.0, we’re starting the process of squashing everything down into a single implementation, instead of the layer cake.
As part of this effort, we’re starting to run the web-platform-tests suite against jsdom. Web-platform-tests is a cross-vendor test suite meant to cover, essentially, all of the web platform: DOM, HTML, CSS, Shadow DOM, Service Worker … it’s all there.
It’s not perfect by any means. Coverage is pretty spotty … there’s only two tests for all of <select>, for example. It’s generally understaffed, and it’s sometimes hard to get vendors to contribute their tests to the suite. But it’s still a really cool project, and I’m excited for jsdom to start participating in it---hopefully we can become one of the vendors contributing our tests back, so that they’re run in all the “real” web browsers out there. Right now we’re just running a tiny subset, but our policy is that new features should be implemented by pulling in the appropriate web-platform-test.
Finally, I want to give a shout out to the other projects that make jsdom possible. We depend on these guys for some of the trickiest and most important parts of jsdom. They’re all maintained by separate people, outside of the jsdom team, who have taken the time to produce a faithful implementation of the relevant part of the web platform. I’ve been consistently impressed by how professional the maintainers of these projects are, and the extent to which they’re willing to work with jsdom.
For example, we recently had a great collaboration with the author of parse5, our HTML parser, to get support for the <template> tag into jsdom. And when we wanted to make jsdom browserifyable, so that you could run it in a web worker, we discussed some of the tricks we needed with the author of cssstyle, and were able to get that up and running pretty quickly. It’s been really great.
I want to finish up by talking about the future of jsdom, but I’m going to do it in a bit of a roundabout way. I promise it’ll all connect up.
First I want to share with you this interesting issue that was opened on jsdom pretty recently. (read the slide)
I was pretty puzzled by this, as I’d never heard of querySelectorAll taking an array before. So I went to look at the spec:
But what’s important to realize about the spec is that when it says something takes a DOMString, it means that any argument you pass it gets converted to a string. So from this perspective, the behavior makes sense:
When you convert an array to a string, it does a join with commas. So that’s why you can pass an array to querySelectorAll, and it’ll still work.
It turns out that this is a pretty general problem, and it occurs because we’ve been mostly ignoring the way specs are written. Specs are written in this horrible language called “WebIDL,” with weird concepts like “interfaces” and “readonly attributes.”What browsers actually do, given this, is they write code generation tools to take WebIDL and turn it into C++ implementations of the DOM APIs, with all the type conversions and such baked in. This is the kind of step we’re missing from jsdom: something that takes the machine-readable WebIDL language written in the specs, and auto-generates the correct type conversions and so on.
So that’s what I’ve been doing.
WHAT IS JSDOM?
for use with Node.js**
* Actually much more than just the DOM
** Actually it runs in more places than just Node.js
“In the browser, you can do:
This doesn't work in jsdom.”