The document categorizes common tools used in modern web development into external management tools, internal language conveniences, and subordinate language conveniences. It provides examples of version control systems, dependency isolation tools, package managers, deployment tools, and more that are commonly used for languages like Ruby, Python, PHP, and JavaScript. The document also discusses enhanced REPLs/shells, web server abstraction layers, asset managers, and preprocessing tools for HTML, CSS and JavaScript.

1. Dynamic Language Dev Process at LTS:
An attempt at a taxonomy of modern web development tooling.
Dave Mayo
Thursday, June 4, 2015 (2015-06-04)

2. Précis
This presentation is aimed at getting across two things:
A set of categories of tools/components that are common
to current "reasonable practices" in dynamic language
development. Not necessarily best practices, but some
collection of stuff that looks like the hammer and wrench
of "modern" development practice.
Examples, mostly Ruby/Python, of tools in each of these
categories, and some attempts at explaining their use.

3. "Reasonable," not "best?"
Best Practices - Optimum way to do things, basically
hygiene. If you’re not doing them, you’re wrong.
"Reasonable Practices" - Ways to do things, which
reasonable people have selected, across multiple
languages and environments.

4. Some reasonable practices are kind of terrible over some
problems.

5. I’ll be covering some general overviews, and comparing the
tools in the field available in the Ruby and Python ecosystems,
with some mention of PHP/JS/Clojure.

6. This isn’t supposed to be utterly unstructured - but where
departing from structure has let me go on for a minute or so
about useful tools/practices, I have done so. Because tools!

7. Basic anatomy of the stacks we target
Most deployments of systems in dynamic languages involve the
more or less the following setup, from the outside in:
Web Server
Application Server
Database

8. Web Server
This handles static traffic, and then proxies requests for
dynamic stuff to an application server.

9. Application server
This is what actually runs your Ruby/Python/PHP/whatever
processes.
Can be managed directly by the web server (think mod_php,
passenger, mod_wsgi) or external (unicorn, gunicorn,
candycorn1
, php-fpm)
1
Not real, just checking if you’re all awake

10. Database
Or databases. A common pattern is, SQL by default, mixed
SQL/NoSQL if you have a combination of mostly static
structured data and fast-churning unstructured data, and
NoSQL if you’re under thirty and wear t-shirts to work. I’m
semi-arbitrarily throwing in-memory caches like Redis or
memcached in here, we’re not interested in them, anyway.

11. Our Taxonomy
External management tools
version control (as first class citizen)
Dependency Isolation (This is NOT virtualization)
Package management
Deployment/Remote operations tool
Project Generators
Internal language conveniences
Enhanced REPL/Shell/Console
Web server abstraction layer
Standalone development server
Own-language task runner
Asset management
Subordinate language conveniences
HTML/CSS/JS preprocessing

12. External Management Tools

13. Version Control
Using version control is a best practice. Do it or be ashamed.
What I mean by version control here is really integration of
other tools with your version control, i.e. your other tools
interact with your version control system.

14. Examples
Capistrano pulls software (in most common cases) from
distributed VCs.
Code review tools pull lists of commits from VCS to
attach reviews to.
Continuous Integration stuff runs integration tests
per-commit (or per commit to master repository)

15. Dependency Isolation
Tools to isolate your project from the OS-installed version of
your language and its libraries.
Enables complex versioning of language and libraries on a
per-project level, really handy when developing many projects
with different deps on same dev system.
Examples include:
RVM, rbenv, Bundler (Ruby)
virtualenv (Python)
leiningen (Clojure)
also exists for PHP, according to Google

16. Dependency Isolation
For Ruby projects at LTS, we use:
RVM to manage the ruby interpreter, and the Bundler
gem
Bundler to manage all per-project dependencies
For Python:
Virtualenv and wrappers around virtualenv are basically
the only game in town. . .

17. Package Management
Specifically, repository-based project management. Throw out
a name and a version/version spec, get package (and
dependencies).
Upgrade path as you get more serious basically involves either
support for external/local packages, or support for running the
repo software.
At this point, for anything nontrivial, you’re probably dealing
with multiple layers of package management.

18. Package Management
Bundler in Ruby processes a Gemfile
Virtualenv runs pip in the virtual environment
npm for node manages JS packages on the server
Bower manages client side packages for JS. Also
sometimes server-side.
This is ALL on top of OS package manager (e.g.
apt/RPM) and third party OS-level package managers
(e.g. Homebrew on OSX)

19. Deployment/Remote Operations
Some piece of software that gets you to single-step deploys
and single step running of admin tasks on remote systems.
There’s a non-zero value in having this be a tool in the
language you’re deploying, but it’s not really important.

20. Deployment/Remote Operations Examples
Capistrano (Ruby) This is what we’re using across our
Rails/Django projects at LTS. It’s fairly mature, it has a lot of
stuff built in for a standard deploy flow that looks like this:
1 Clone software from git-archive to host
2 Symlink configuration files
3 Run system update tasks (DB migrations, asset
compilation and preprocessing, etc)
4 Restart system.

21. Deployment/Remote Operations Examples
Fabric (Python) We don’t use this at LTS, but it seems fine.
Lower level, doesn’t have prebuilt workflow for deploy

22. Deployment/Remote Operations Examples
The main thing is, you should be able to enter a command on
your local dev box, and deploy to/run command on X target

23. Project Generators
This is an idea that’s already pervasive in frameworks.
A command that generates a somewhat customized project
skeleton is not news.
Yeoman tries to generalize this, provides essentially a generic
project skeleton generator.

24. Internal Language Conveniences

25. Enhanced REPL/Shell/Console
Interactive shells with much of the functionality of the editor,
and which are capable of maintaining state over a session.
The built-in shells for some langs suffice, but mostly this will
be a third party extension for reasons. Some reasons:
Python’s interactive shell doesn’t default to having
Readline support, or tab completion, or highlighting, or
anything really
Ruby’s irb doesn’t highlight code
Neither of them have good built in introspection tools.

26. Enhanced REPL/Shell/Console
Fortunately:
Ruby has ‘pry‘ and its various plugins
Python has ‘ipython‘, ‘bpython‘, etc.

27. Web Server Abstraction Layer
A lot of stuff these days is sitting on top of a simplified HTTP
layer that can plug into multiple servers and stack on top of
middleware.
Examples:
Rack (Ruby)
WSGI (Python)
Ring (Clojure)

28. Standalone Server Mode
"Just for development" HTTP server mode, so you don’t have
to set up Apache on your dev laptop.
Webrick (Ruby)
There are a bunch of different answers for Python, mostly
framework level.

29. Side note
It’s good to have this, but sometimes it’s better to put in a
little effort.
For LTS projects, my local dev environment is nginx, with a
server block (virtual host) per project, and an entry in my
hosts file to simulate DNS lookup.

30. Own Language Task Runner
Because who likes makefiles?

31. Own Language Task Runner
Own language is important, because introspection of
application
Examples:
Rake, Thor (Ruby)
Django has manage.py tasks, fabric can be used more
generally in Python
Grunt, Gulp, a bunch of other contenders for JS

32. Asset Management
For frameworks, this is usually baked in.
Rails asset pipeline
Django does via manage.py

33. Asset Management
Outside of frameworks, usually handled via the task runner
Preprocessing, moving assets into directories, setting e-Tags,
etc.

34. Subordinate Language Conveniences

35. HTML Preprocessing
Templates. Stuff better have em’

36. CSS Preprocessing
HUGE uptake, and with good reason.
Done even moderately well, MASSIVE deduplication of
repeated work.
Examples:
Less
Sass/SCSS
Stylus

37. JS Preprocessing
TBH, I don’t really do this.
JS is a high level language.
That being said, CoffeeScript and similar have sane iteration
primitives, do some evening out of scope stuff.

38. Miscellanea
If you work on the command line, run out and grab this right
now:
ack - http://beyondgrep.com
A grep that understands programming languages and VCS
Returns order of magnitude faster than recursive fgrep or most
other search methods, despite being written in Perl instead of
C

39. Miscellanea
If you use an editor or IDE and write markup
(HTML/XMLish), go grab this:
http://emmet.io
It’s an in-editor template language. Write a CSS-selector
looking thing, hit expand, markup!