Saturday, July 25, 2015

JavaScript anno 2015 – Node and npm

Node.js is server-side JavaScript; a “runtime environment for server-side and networking applications” [Wikipedia]. It’s open source as just about everything in the JavaScript world of frameworks and tools. Node is an amazing runtime and in just about 5 lines of code you can have a simple web server up and running:

var http = require('http');
http.createServer(function (req, res) {
res.writeHead(200, {'Content-Type': 'text/plain'});
res.end('Hello World\n');
}).listen(1337, '');

We’re not using Node per se in our project yet. We’re running our ASP.NET web application on IIS, but it might be something we’ll be looking more at for specific tasks later on. But for now we’re using it implicit through the Node Package Manager.

Node Package Manager (NPM)

A package manager is responsible for installing, upgrading, configuring and uninstalling software packages for a given platform. NPM is a package manager for JavaScript, just like NuGet is a package manager for .NET, CPAN for Perl, Maven and Ivy for Java, RubyGems for Ruby, and so on.

NPM is bundled with Node so the way to get NPM on your machine is to install Node. With Node installed you can run npm commands from PowerShell or the command prompt.

NPM differentiates between modules and executables. When you install a module it will be placed in a node_modules folder, whereas executables are placed in a sub-folder called .bin. You can run the npm root and npm bin commands respectively to see where those folders exists on your local machine.

npm install                                        

NPM also differentiates between installing globally or locally. Locally means local to the folder you run the npm install command from (typically your project root folder). A good rule is to never install modules globally. Only executables that need to be available across many project should be installed globally, but to avoid versioning dependencies one should strive to install most packages locally. To install a package globally you just run the install command with the --global (or -g) option.


npm install browserify => Downloads and unpack the Browserify package to a local node_modules folder

npm install grunt-cli –g => Downloads and unpack the Grunt command line interface to the central .bin folder, and adds grunt.exe to your PATH.


Often a package uses other packages and these dependencies are expressed in the package.json file inside the package. So when npm installs a package it will look in the package.json file and install any package that they rely on. Therefore an install of for instance Browserify will install no less than 49 direct dependencies, which again install their dependencies.

Unlike NuGet the dependencies of a package will not be installed at the same directory level as their dependent. Instead they will be installed in a node_modules folder inside the Browserify folder. Inside each dependent module there might be other dependencies. In fact, installing Browserify will create no less than 87 (!) node_modules folder underneath the Browserify folder.

Placing all dependencies inside each module is a great way to prevent versioning issues between dependencies of different modules. In .NET and NuGet this wouldn’t work since there’s no way to reference 2 different versions of the same assembly into the same project. You can however reference assemblies that are dependent on different versions of the same assembly. The conflict between them is solved by assembly binding redirects in web/app.config, but it can lead to problems if a dependent assembly has breaking changes between versions. In the JavaScript world there is no concept of binary assemblies. The modules are just one or more js-files and as long as they are loaded within different scopes, they will not cause any conflicts.

There’s a lot more to say about dependencies in npm, but the one thing you need to be aware of is the difference between dependencies and devDependencies. The first one is all dependencies required to run, while the latter is additional dependencies needed for development (there’s also peerDependencies and bundledDependencies, but I won’t go into that here). Typically devDependencies will include unit tests, test harnesses, minification, transpilers, etc.


If you run npm install without any package name, npm will look for a package.json in the directory where you run the install command. If it finds it npm will install all dependencies listed (including devDependencies). The package.json is similar to packages.config in NuGet, but I dare to say that npm seems a lot more sophisticated and solid than NuGet.

Typically you will have a package.json file in the root directory of your project and you will place all your dependencies there. A simple package file might look like this:

“name”: “my-app”,
“version”: “0.0.1”
“dependencies”: {
“browserify”: “11.0.x”

Note the ‘x’ in the Browserify version. This means that any 11.0-versions will do, so when you do a npm install (or update) 11.0.0, 11.0.1, etc, is OK, but not 11.1.0 or 12.0.0. Npm follows semantic versioning (semver) and the numbers means ‘major.minor.patch’. If you’re happy to trust that Browserify will be backward compatible across minor versions (which it should be if they follow semver), you can specify ‘11.x’ instead of ‘11.0.x’. If you just want the newest version – regardless of breaking changes (not recommended!), then you can just put an ‘x’ instead of ‘11.0.x’.

You could create the package.json file manually, but a better way is to use the init command:

npm init

Note; if you create the file manually be sure that the file is ASCII encoded. Unicode will result in a parsing error in npm and UTF-8 will result in the file not been updated (e.g. when running with the --save flag below).

You could also edit your package.json file manually and add all dependencies by hand, but again; it’s better to let npm handle this. You do this by appending a --save flag (-S for short) to the install command;

npm install browserify --save

If the package is only meant for development purpose and not applicable for the production environment, for instance testing frameworks, you can use the --save-dev instead;

npm install jasmine --save-dev

When you have a lot of packages installed, it’s a great chance that some of them share some dependencies. Because of npm’s hierarchical structure, it’s possible to optimize these shared dependencies by moving them further up the tree and thereby get rid of duplicated modules. The command for that is dedupe:

npm dedupe 

If you want to remove any packages that is not in your package.json, you can run the prune command:

npm prune

If you run the prune command with the --production flag, all devDependencies will be removed (nice when deploying to production).

If you want to see all installed packages there’s a ls command for that:

npm ls

Note that this will list all top level packages as well as all their dependencies. If you’re only interested in the top level packages you can add the –depth 0 parameter to ls.

If you want to search for available packages, there’s a search:

npm search

…which also can do regular expressions. But for the most part it’s just easier to browse and search on npm’s home page.

npm update

Installing is just one side of the story. Once you’ve added any dependencies to your project you would like to keep them updated as well:

npm update

If you run it without specifying a particular package to update, npm will go through the package.json file and see if any newer versions are available. It will off course respect the versioning you’ve applied, so if won’t upgrade to a newer major version if you have specified that only minor versions are acceptable.

You can update a specific package by providing the name of the package. If the package is installed in the global scope you need to add the –g flag.

npm update will also download any missing packages, but you need to add the --dev flag to get all devDependencies. One thing to be aware of is that npm will not do any recursive update of all package dependencies. It will only update the top level packages, but you can force recursion with the --depth flag.

As with the install command you can let npm update your package.json file with the updated package versions;

npm update --save

If you want to check whether any new packages exists without updating them, you can run the outdated command:

npm outdated

npm uninstall

Removing packages is as easy as installing. Just run the uninstall command with the name of the package to remove and the package is gone;

npm uninstall browserify

As with install and update you can let npm update package.json:

npm uninstall browserify --save

Wrap up

The three major take-aways from this post should be that

1. Npm packages comes in two flavors; executables and modules. Executables are typically command line tools, while modules are libraries that you want to use in your code.

2. Npm has two modus operandi; global and local. In general you should install executables in the global scope and modules in the local.

3. Put a package.json file in the root of your project and add all of your project dependencies there.

What I haven’t talked about is configuring npm. There’s a lot to say about this, but I’m just going to keep it short and say that npm is highly configurable and I’ll just point you to the resources below.

Resources – The home page for npm where you can search and browse for available packages – The documentation for npm is darn good if I may say so. I really recommend taking a look at it as I promise you’ll learn a lot from it.
As for configuring npm, here is a starting point for you.
For a great explanation of the difference between the various dependencies in npm, take a look at the top-voted answer to this question on StackOverflow.
To get some insight into the history of npm and why it is as it is, read through the answers from Isaac Schlueter (the main developer on npm) in this thread. – The home page for Node.

No comments: