How I added awesome multi-threaded features to Express JS

TL;DR

I wrote express-http-context which is a ridiculously simple npm package that provides access to a request-scoped context that can be used anywhere in your codebase. It helps make awesome things easy, like adding correlation IDs to your logs.

A quick refresher on the Event Loop

Allow me to give a short and totally-not-rigorous refresher about the difference between Node’s Event Loop and “real” multi-threaded languages.

The Event Loop is one of the coolest features of Node and its underlying V8 engine (I guess you could say that it IS node … but I digress). It allows Node to run an application in a single-threaded context, yet also handle multiple concurrent asynchronous operations without blocking application flow. Nifty.

When asynchronous operations are kicked off (for example calling an API or writing to a database), Node holds onto that callback and then keeps running through the current call stack. When the stack is emptied, the current “frame” is done and Node either starts another frame or exits if there is nothing else to do. (I’ll get back to this in just a second.)

As asynchronous operations are completed, Node places the results on an internal message queue. When a frame ends, Node checks the queue to see if there are any completed operations. If there are, the next message is used to start a new frame; the code in the callback becomes the next call stack and execution continues.

In this way, Node simply plods through all of its synchronous work until it’s done. Then, it checks to see if any new work came in while it was busy. If there’s nothing, the application exits.

In a true multi-threaded environment, each thread would have its own call stack and the thread would be suspended during an async call. When the async call is completed, the thread picks up where it left off in its own call stack. When each of the threads reach the end of their respective call stacks, the application exits.

The one of the differences between the single-threaded Event Loop and a truly concurrent multi-threaded environment is the concept of Thread Local Storage. This storage how the CPU can keep track of data relevant to the thread while asynchronous stuff is happening. Again, super high-level and hand-wavy, but the general concept is useful for us here.

Threads and APIs

Building a multi-threaded application: Photo by sarathy selvamani on Unsplash

If you were to write an API in .Net and run it in IIS on Microsoft Windows (and live to tell the tale 😃), each request to the server would be handled by a different thread. For APIs that make async calls (for example to a database or other APIs) then each thread would be able to maintain its own context and pick up where it left off after those async calls are completed.

.Net uses the HttpContext class to expose the current thread which in turn represents the state of the request. You can read and write cookies and headers, look at errors which have occurred, and many other things just by looking at this context. This means that regardless of where you are in the code, you can access data scoped to the current request.

ExpressJS (my current Javascript API framework of choice) does a fantastic job of handling multiple concurrent requests, but doesn’t do a good job of providing access to request- and response-scoped data. This is because the single-threaded event-driven execution model doesn’t lend itself well to maintaining context across each of the messages coming off of the queue. As a result, if you want access to data that is relevant to the request such as an auth header or correlation ID, you have to pass the request or at least the data from the request throughout the the entire codebase.

For smaller API this isn’t a problem. A lot of times, all of the code lives in only a handful of files anyway. But as code size grows and as healthy code separation becomes critical, the lack of request-scoped storage in Express becomes palpable.

How to make this work in Express

Express-http-context adds extremely basic HttpContext-style functionally to Express by using the relatively obscure async_hooks module. Per the docs “The async_hooks module provides an API to register callbacks tracking the lifetime of asynchronous resources created inside a Node.js application.”

Like most developers, I didn’t want to “register callbacks tracking the lifetime of asynchronous resources” for every web API I built, so I put together express-http-context which exposes all of this as an Express middleware.

Here is the most basic of examples that shows how to setup a project with an http context:

Line 6 runs a middleware that creates a new context for each request. Any values added to this context during a request will be accessible only to the same request. Let’s add another middleware that generates a unique request ID for each request and then adds it to the context:

As you can see, this middleware creates a new ID which then gets added to both the context AND as a header to the response. Now we can get the current request ID anywhere in the project, even if we don’t have access to the original res simply by running const reqId = httpContext.get('requestId');.

Show me something cool!

I actually wrote this package specifically for managing correlation IDs for logging purposes. As a refresher, a correlation ID is a value that is passed throughout the many parts of a distributed system so that otherwise disjointed calls can be correlated.

Here is the middleware that I typically use for that purpose:

As you can see, we actually create two IDs. First we check if we have been passed an existing correlation ID by looking at the request headers. If we don’t find one, we create one and move on. Next we create a request ID. Both IDs are added to the context and then added as headers to the response.

I like to create distinct correlation and request IDs to ensure that we can always maintain a guaranteed unique request ID. There are cases where consumers will call your API multiple times with the same correlation ID. This typically happens in more foundational services. If consumers need to make multiple calls to your API during a single operation, they will likely be using the same correlation ID. Thus, the correlation ID represents a single unique operation distributed over multiple applications whereas the request ID represents a single unique call to a single application.

By including both the correlation and request IDs in your logs, it greatly reduces the amount of effort needed to track down any errors or performance issues across your ecosystem.

Lastly, make sure to also pass the correlation ID on to any APIs that you may be consuming. I generally create a helper for the request package that automatically adds the X-Correlation-ID header. There may be a better way of doing this, but here’s my solution:

Note that this helper module can be completely separated from the Express request handlers and yet it still has access to the current context and thus the correlation IDs.

In summary, express-http-context adds request-scoped context feature to Express APIs similar to what you would find in heavier, multi-threaded platforms.

Like what you see?

Hopefully you find the project interesting! If so, here’s how you can help:

1. Weird behavior? Confusing Readme? Github issues are super helpful! 😃
2. Give it a Star: https://github.com/skonves/express-http-context ⭐
3. Give this post a few 👏

How I added awesome multi-threaded features to Express JS was originally published in Hacker Noon on Medium, where people are continuing the conversation by highlighting and responding to this story.