Or, how I learned to stop worrying and love npm. Presented at HTML5DevConf Spring 2013.

1. Client-Side Packages
Or, how I learned to stop worrying and love
@DOMENIC

2. Domenic Denicola
› http://domenic.me
› https://github.com/domenic
› https://npmjs.org/~domenic
› http://slideshare.net/domenicdenicola
Things I’m doing:
› @esdiscuss on Twitter
› The Promises/A+ spec
› Creating truly RESTful APIs
@DOMENIC

3. Modules
@DOMENIC

4. Module systems express code dependencies
› 2000s state of the art:
– One big file with carefully ordered functions, or
– Many small files, and carefully ordered <script> tags
› Other languages solve this problem easily; in JS, until
ES6, we have to solve it ourselves.
› We have two hacky solutions:
– CommonJS modules
– AMD modules
› CommonJS is better for code reuse.
@DOMENIC

5. CommonJS modules
› AMD:
define(['a', './b'], function (a, b) {
return { c: 'd' };
});
› CommonJS:
var a = require('a');
var b = require('./b');
module.exports = { c: 'd' };
@DOMENIC

6. The AMD configuration problem
› In require('a'), what does 'a' refer to?
– AMD module called 'a' (per baseUrl config)
– Some other AMD module (per map config)
– Non-AMD code (per shim config)
– AMD plugin (per plugin config)
– A package’s main module (per packages config)
› In a CommonJS + npm system, the answer is always the
main module for package 'a'.
@DOMENIC

7. The AMD dependency problem
› If Ember depends on RSVP.js, how does it express this in
AMD?
– require('rsvp') doesn’t cut it.
– I know! We’ll use more config!
› Now your Ember-using project needs to know about
Ember’s internal implementation details in order to make
Ember’s require('rsvp') work, via configuration.
› This severely limits the extent to which library authors can
bring in third-party code.
› Where did this 'rsvp' string come from anyway?
These are all problems to be solved by packages, but AMD
tries—and fails—to solve them at the module level.
@DOMENIC

8. AMD is not an advantage
› AMD tries to solve too many problems that are way
beyond the level of a module system.
› CommonJS can solve all AMD use cases except cross-
domain single-module loading.
› Using AMD cuts you off from consuming most module
code out there.
› Your code is more universal with CommonJS. More tools
consume it, and it’s easier to test.
› AMD users can use r.js to convert from CommonJS.
@DOMENIC

9. Enter Packages
@DOMENIC

10. Packages
› Modules let us reuse our own code.
› Packages let us reuse other people’s code.
› Examples of packages:
– Backbone
– Underscore
– Sinon.JS
– jQuery
– jsdom
– Express
› Packages have dependencies
– E.g. Backbone depends on Underscore and jQuery
@DOMENIC

11. Packages as a unit of code reuse
› Packages are small, often one module.
› They can shuffle work off to their dependencies, allowing
greater code reuse.
› If jQuery UI were composed of packages:
– jqueryui-position depends on nothing
– jqueryui-mouse depends on jqueryui-core, jqueryui-widget
– jqueryui-autocomplete depends on core, widget, menu, position
– etc.
› ―Download builders‖ are hacky and un-encapsulated ways
of solving package management.
› Multi-thousand line modules belong in the 2000s.
@DOMENIC

12. Packages can be versioned independently
› When you separate functionality into small packages, you
aren’t tied to monolithic, coordinated release cycles.
› By depending on small packages, you can get bug fixes
and new features as soon as their authors publish them.
› Then users of your package get bug fixes, without you
doing anything!
› Conventionally, we try to adhere to semantic versioning:
– 0.x.y releases: unstable; upgrade at your peril.
– 1.x.y and beyond:
› Increasing ―patch version‖ y means bug fixes.
› Increasing ―minor version‖ x means new features.
› Increasing ―major version‖ means new API, breaking backward-compat.
– So we can express dependencies as 0.1.2 or 2.x or 2.3.x to get
the benefits of independent versioning.
@DOMENIC

13. Packages can be app-specific
› Each ―widget‖ can be a package.
– Sidebar, header, currency table, news ticker, …
› Or you could have packages for business logic.
– Financial calculations, market predictions, discount rules, …
› Or you can isolate hard problems in packages.
– Talking to SMTP servers, traversing a RESTful API, rendering
PDFs with JavaScript, sandboxing user input, …
› They can separate out cross-cutting concerns.
– Number formatting, CSS theming, injecting logging and error
handling, …
› They can even help with domain-driven design.
– Encapsulating bounded contexts, exposing services, isolating a
shared kernel, building an anticorruption layer, …
@DOMENIC

14. Packages with npm
@DOMENIC

15. What problems do we need to solve?
› Get code onto our computers
› Automatically install expressed dependencies
› Have a way for one package’s code to ―require‖ another
package’s code.
› Make other peoples’ code available to you
@DOMENIC

16. npm stands for JavaScript package manager
› 26,000 packages and growing
› The emphasis on small packages means many work in the
browser already.
› The npm registry is anarchic.
– This makes package discovery tricky, but in balance is a big win.
– Server code, browser code, even AMD code: it’s all welcome.
– A single namespace means no enforced prefixing.
› npm packages follow a set of conventions that make your
life easy.
@DOMENIC

17. What’s in an npm package?
› lib/
– index.js
– AuxiliaryClass.js
– helpers.js
› test/
› LICENSE.txt
› README.md
› package.json
› .jshintrc, .gitignore, .npmignore, .travis.yml, …
@DOMENIC

18. package.json
{
"name": "sinon-chai",
"description": "Extends Chai with assertions for Sinon.JS.",
"keywords": ["sinon", "chai", "testing", "spies", "stubs", "mocks"],
"version": "2.3.1",
"author": "Domenic Denicola <domenic@domenicdenicola.com>",
"license": "WTFPL",
"repository": {
"type": "git",
"url": "git://github.com/domenic/sinon-chai.git"
},
"main": "./lib/sinon-chai.js",
⋮
@DOMENIC

19. package.json
⋮
"dependencies": {
"sinon": ">= 1.5 <2"
},
"peerDependencies": {
"chai": ">= 1.3.0 <2"
},
"devDependencies": {
"coffee-script": "~1.6",
"jshint": "~1.1",
"mocha": "~1.7",
},
"scripts": {
"test": "mocha",
"lint": "jshint ./lib"
}
} @DOMENIC

20. Packages are encapsulation boundaries
› Single point of entry API via the main module:
– require("backbone")
– require("underscore")
– require("sinon-chai")
– require("express")
› You don’t need to know about a package’s dependencies
– Unlike AMD!
› You don’t need to know about a package’s strategy
– Client-side MVC framework choice, templating engine, CSS
precompiler, …
@DOMENIC

21. Packages are concern boundaries
› Their own license
› Their own readme
› Their own coding style and linting rules
› Their own compile-to-JS authoring language
› Their own tests and test style
› Their own author/owner/maintainer
@DOMENIC

22. Dependencies
@DOMENIC

23. npm dependency basics
› npm uses a global registry by default: https://npmjs.org
– A CouchDB server: http://isaacs.ic.ht/_utils/
› It uses package.json to install dependencies
@DOMENIC

24. Dependencies are hierarchical
› Unlike some other package managers, there is no global shared
―DLL hell‖ of packages on your system.
› They get installed in a hierarchy of node_modules folders:
request@2.12.0
├─┬ form-data@0.0.3
│ ├── async@0.1.9
│ └─┬ combined-stream@0.0.3
│ └── delayed-stream@0.0.5
└── mime@1.2.7
request/node_modules/form-data
request/node_modules/form-data/node_modules/async
request/node_modules/form-
data/node_modules/async/node_modules/combined-stream
⋮
@DOMENIC

25. Git dependencies
› Dependencies can be Git URLs
› Useful for pointing to forked-and-patched versions of a
library while you wait for your pull request to land.
› Useful for private Git servers.
› For GitHub, use "package": "user/repo#version" syntax.
› This allows an escape hatch from the central registry and
its single namespace, if you’re into that.
@DOMENIC

26. Other important npm features
› npm dedupe: creates the minimal dependency tree that
satisfies everyone’s versions
› npm link: symlinks, for local development
› The "scripts" field in package.json can give you arbitrary
scripts which have access to your devDependencies.
– npm run <scriptName>
– npm test (shortcut for npm run test)
› Packages can have arbitrary metadata: it’s just JSON!
@DOMENIC

27. Private Code
@DOMENIC

28. How can we use this without publishing?
› We can’t put proprietary code in the npm registry.
› But we’d still like to use the package concept, and npm’s
features.
› There are hacks:
– npm link
– Using Git URLs entirely (this actually works pretty well).
› But, really we need our own registry!
@DOMENIC

29. Your own registry
› CouchDB is really good at replicating. So just do that!
› Then:
npm install lab49-ip –reg http://npm.lab49.org
› You can use the publishConfig settings in package.json to
make a given package automatically publish to your
internal registry, instead of the public one.
› We’re working on ―fallback registries,‖ to obviate the
replication step.
@DOMENIC

30. Packages, Client-Side
@DOMENIC

31. A note on CSS
› Just include it in your package!
› Don’t reference it from your JS, e.g. automatically inject it
into the page.
› It shouldn’t matter whether the CSS is in
node_modules/mywidget or in app/widgets.
› The application’s build tool will consume it, if the
application author needs it; otherwise it sits inert.
@DOMENIC

32. Browserify!
› The best solution for bringing CommonJS modules and
npm packages to the browser.
› Shims common (and not-so-common) Node APIs to give
you access to many npm packages.
– events, path, url, querystring, vm, http, crypto, zlib (!) and more…
› Understands npm’s node_modules structure.
› Has a flexible pipeline and vibrant ecosystem, for e.g.
source map support or just-in-time bundle compilation
when embedded in simple HTTP servers.
› Allows packages to declare source transforms and browser
overrides in their package.json metadata.
@DOMENIC

33. Discovering client-side packages
› We trust packages most if they are validated by continuous
integration, e.g. by Travis CI:
› We can trust client-side packages if they are validated by
testling-ci:
@DOMENIC

34. Testling-ci setup
In package.json:
"testling": {
"browsers": [
"ie/7..10",
"firefox/10..13", "firefox/nightly",
"chrome/14..16", "chrome/canary",
"safari/5.0.5..latest",
"opera/10.6", ―opera/11.0", "opera/11.6",
"iphone/6",
"ipad/6",
"android/4.2"
},
"harness": "mocha",
"files": "test/tests.js―
}
@DOMENIC

35. browserify.org/search
@DOMENIC

36. Demo
https://github.com/domenic/client-side-packages-demo
@DOMENIC

37. › Make sure all your code is in
WHAT’S NEXT CommonJS module format.
› Then organize that code into
packages.
› Consume other peoples packages.
› Start publishing to npm—or to a
private registry.
› Use browserify in your build process
(or in your server).
› Use testling-ci to encourage trust
and discovery.
@DOMENIC