Save the day with node.js domains

Laurent Perrin

Let's talk about exceptions

1. Runtime error


/opt/git/front-prod/server/pumps/message_template.js:71
    name: this.peer.name,
                   ^
TypeError: Cannot read property 'name' of undefined
    at MessageTemplate.toContact (/opt/git/front-prod/server/pumps/message_template.js:71:20)
    at /opt/git/front-prod/server/pumps/message_template.js:48:26
    at fn (/opt/git/front-prod/node_modules/async/lib/async.js:579:34)
    at Object._onImmediate (/opt/git/front-prod/node_modules/async/lib/async.js:495:34)
    at processImmediate (timers.js:330:15)

The stack points to the source of the error. Scary, but easy to fix.

Pain level: 3 /10

2. Runtime error without context


/Users/laurent/dev/talk_domains/test.js:4
    done();
    ^
TypeError: undefined is not a function
    at null._onTimeout (/Users/laurent/dev/talk_domains/test.js:4:5)
    at Timer.listOnTimeout [as ontimeout] (timers.js:110:15)

function doSomething(done) {
  setTimeout(function () {
    console.log('did something');
    done();
  }, 1000);
}

doSomething(/* missing */);

The stack looks incomplete and doesn't show the caller of doSomething (where the error is located). We'll talk about that in a minute.

Pain level: 7 /10

3. EventEmitter error


/opt/git/front-prod/node_modules/longjohn/dist/longjohn.js:181
        throw e;
              ^
Error: read ECONNRESET
    at errnoException (net.js:901:11)
    at onread (net.js:556:19)

When you don't listen to 'error'  events, they turn into exceptions.


Pain level: 9 /10

4. Nothing at all!


Your app forgets about a connection and the browser loads forever:


function doSomethingElse(done) {
  someBackend.load(function (err, data) {
    if (err) {
      console.log('Something happened', err);
      // you forgot to call the callback
      return;
    }

    done(null, data);
  })
}

Pain level: 10/10
This is a real problem that can be a real headache in real-world projects. Unfortunately, domains won't with that.

My solution is to assert the type of parameters in important functions.

Wait, where's my stack ?


function doSomething(done) {
  console.log('about to do something');
  setTimeout(function () {
    console.log('did something');
    done();
  }, 1000);
}

doSomething(/* missing */);

function doSomething(done) {
  console.log('about to do something');
  setTimeout(function () {
    console.log('did something');
    done();
  }, 1000);
}

doSomething(/* missing */);

function doSomething(done) {
  console.log('about to do something');
  setTimeout(function () {
    console.log('did something');
    done();
  }, 1000);
}

doSomething(/* missing */);

When node.js executes setTimeout, the callback is just a parameter and is not executed. After that, the function returns and unwinds the stack.
function doSomething(done) {
  console.log('about to do something');
  setTimeout(function () {
    console.log('did something');
    done();
  }, 1000);
}

doSomething(/* missing */);

When node.js finally executes the callback, setTimeout starts a new stack.

Check longjohn.

Countermeasures


try/catch


function doSomething(done) {
  try {
    setTimeout(function () {
      console.log('did something');
        done();
      }, 1000);
  } catch(e) {
    console.log('not executed');
  }
}

doSomething(/* missing */);

As before, the surrounding function no longer exists when setTimeout executes. The try/catch will not catch the exception.

proc.on('uncaughtException')


Will prevent your app from crashing, but you still have no idea where the error came from.

You probably leaked objects, and your app is now in a brittle state so you should terminate the process quickly.

Domains!


Added in node.js 0.8.0 (June 2012)


"a way to join multiple different IO actions, so that you can have some context when an error occurs."


The way I see it: the active domain is passed as a hidden parameter in function calls.


(at this point, I demoed the code)

What's wrong with MySQL ?


After calling MySQL, we end up in a different domain.

We are not even able to send a 500 error to client.

MySQL is a sequential protocol

Other requests wait in a queue

The driver runs in the domain that created the connection.

Solution: preserveDomain 

function preserveDomain(fun) {
  var d = domain.active;

  return function () {
    var args = arguments;

    d.run(function () {
      fun.apply(null, args);
    });
  };
}

Make sure you protect your domains when calling 3rd party modules.

Other Stuff


Catching error in usafe modules without patching them


domain.create().on('error', function (err) {
  // handle error
}).run(unsafeModule);

Logging with elapsed time

var d = domain.create();d.start = Date.now();d.log = function (msg) {  console.log(Date.now() - d.start, msg);};

Attach objects to domain


Find out about current request, current user with domain.active. 

 exports.wrapFetch = function (Resource, fetcher) {
  var type = Resource.prototype.className;

  return function (id, done) {
    var d = domain.active,
        key = type + '/' + id;

    d.ensureNotGarbaged();

    var cached = d.reqcache[key],
        pending = d.pendings[key];

    if (cached)
      return done(null, cached);

    if (pending)
      return pending.push(done);

    pending = [done];
    d.pendings[key] = pending;

    fetcher(id, function (err, resource) {
      d.ensureNotGarbaged();

      if (!err)
        d.reqcache[key] = resource;

      // destroy the pending list now in case the domain is garbaged by a callback
      delete d.pendings[key];

      // finally, notify all pending requests
      _(pending).each(function (cb) {
        cb(err, resource);
      });
    });
  };
};

Works for me in production, but domain.active if undocumented.

Thank you!



Laurent Perrin

github.com/lperrin, @l_perrin


frontapp.com

(we're hiring)