For those of you who don't familiar with expressjs, it's a nodejs web framework, that allows you to build web applications and API's. I would suggest reading
Build a Complete MVC Website With ExpressJS and Building simple CRUD with expressjs.

This post is aimed for people who are familiar with express concepts, you can still follow along if you are new to express, but I suggest to read some getting started tutorials before.

When started working with nodejs about 2 years ago, our product was a small web app for sending automatic reports to customers, since then, it grew into a fully functional CRM application for the local financial and insurance industry experts.

Expressjs allows you to maintain large code bases quite easily, but I wish I had some of the tips I will share with you when started, it would defiantly save me a lot of time and headaches :)

#1 - Folder Structure

When writing server side rendered applications, maintaining your folder structure will help you navigate between your code, and assist new programmers to grasp your code base much faster.

Personally I prefer to make my folder structure coupled to the business logic and features, instead of structuring by the file type. Very soon your controllers folder will contain hundreds of files. It is not manageable at all. I still have a top folder named by the file type, but I keep all internal files inside feature directories.

├── config
├── modules
├── controllers
│   ├── api
│   │   ├── dashboard
│   │   └── user
│   ├── auth
│   └── public
├── crons
├── models
├── public
│   ├── app
│   │   ├── css
│   │   ├── fonts
│   │   ├── images
│   │   ├── js
│   │   │   ├── modules
│   │   │   ├── pages
│   │   │   └── utilities
│   │   └── scss
│   └── vendor
├── services
├── utilities
├── tests
│    └── integration
├── views
│    ├── layout
│    ├── pages
│    └── partials
└── server.js
Config

Contains all of our application configurations, it can be express config middlewares, or passport.js config files or other files related. When having one place to configure your app it is much easier to refactor or fix config related bugs later.

Controllers

Will make the interaction with out Models, it will handle all the necessary logic for our routes requests.

Modules

When dealing with reusable controller logic, I suggest creating a JS class , or file containing the reusable logic.

Crons

Will hold all of our cronjobs (agenda-js is a great library for those).

Services

Usually used to contain all of the external services used: Email, sms and other api's our app contacts with.

Models

This folder contains all of the mongoose models, file for each model.

Utilities

Will contain helper and reusable libraries. This can include some functions we can use across our platform.

Public

All of the public assets, this is the only folder that will be exposed to our the web. It will hold css, images and js files.

Views

Contains the views for the rendering engine, in our case .handlebars files. We separate the pages, from the reusable partials and widgets files.

Tests

The tests folder will contain your integration and e2e tests, I prefer placing the unit tests along the code files with a .spec.js suffix, our test runner can look for all files finishing with this suffix and run them one by one. I find it much easier to work and modify the tests when they are close to each other.

#2 - Use Express.Router()

When building your routes it is a good practice to create an express router module for it, it creates a "mini" application for each model, you can have middlewares for that particular route namespace or other validation needed.

For example let's examine the next api end point: GET /api/v1/posts/{postID}/media

When using express router we can break it down to 3 different routers:

  • /api - is the first name space, we can have different routers for auth and public routes.
  • /v1 - this is our api versioning. we can wrap the entire version in a router
  • /posts - this router encapsulating our posts "mini-app" we can have middlewares for checking the post id, or the authentication it requires based on the user role.
Router in practice

server.js

const express = require('express');
const app     = express();
const apiRouter = require('./controllers/api');

app.user('/api', apiRouter);

app.listen(1337);

./controllers/api/index.js

const express     = require('express');
const api         = new express.Router();
const v1Router    = require('./v1');

api.use('/v1', v1Router);

module.exports = api;

./controllers/api/v1/index.js

const express  = require('express');
const v1       = new express.Router();
const posts    = require('./v1');

v1.use('/posts/:postid', posts);

module.exports = v1;

./controllers/api/v1/posts/index.js

const express   = require('express');
const posts     = new express.Router();
const mediaCtrl = require('./media.ctrl);

posts.use((req, res, next) => {
    // here we can access the req.params object and make auth checks
    next();
});

posts.get('/media', mediaCtrl.getMedia);

module.exports = posts;

Writing your app this way makes modular parts you can embed or reuse in other parts of your application.

#3 - Use res.locals for your public routes

res.locals is an object used by your view engine. You can add properties to this object and access them in your view. You can use it for storing the user object and other global information you want to have access in your app view.

In conjunction with express.Router() you can apply your locals depending on the route namespace. For example, you can populate post data for post pages using locals.

./routes.js

pubRouter.use((req, res, next) => {
    res.locals.globalItemsList = ['item1', 'item2'];
    
    // You can fetch the user from db, desirialize from token or fetch it from other source here.
    res.locals.user = {...};
    next();
});

#4 - Avoid callback hell

When we first started working with nodejs, our knowledge of js was mainly based on some basic jquery DOM manipulation. So as you might guess, the async side of js was not used that much... I still remember when we wrote a very complicated XML parser, and it was entirely written using for loops, we actually had async db calls inside the for loop :))
After realising how stupid we are, we started writing async functions using callbacks. Very soon the good was looking something like:

someFunctionCallA(function (cb) {
    someFunctionCallB(function (cb) {
        someFunctionCallC(1, function () {
            someFunctionCallD(function () {
                someFunctionCallE(function () {
                    var FML = true;
                });
            });
        });
    });
});

You don't have to be a genius to realise that it is a really bad written piece of code.

Async.js to the rescue

When looking for a solution I came across asyncjs

"Async is a utility module which provides straight-forward, powerful functions for working with asynchronous JavaScript."(async's github page)

Async allows you to write async code in a more concise code:

async.series([
    function(callback){ ... },
    function(callback){ ... },
    function(callback){ ... }
], function(err, results) {
    // all tasks are finished!
};

Async.js exposes some useful flow control functions to deal with async code: parallel, waterfall, series, and others...

Promises

While writing code with async is great, it still had some downsides:

  • Writing if(err) return handleError(err) really sucks within every callback. You don't do those in promises.
  • Callback errors - When a callback fails due to some unhandled error(we're all humans after all...) it crashes the entire proccess. Promises will catch the error inside the catch step.
  • In general I believe that working with promises is a better way to structure your async code.
var promise = new Promise((resolve, reject) => {
 // this is the function that will make some activity
// it will then resolve or reject depending on its output
 resolve(5) 
});  

// we can then catch the action value 
promise.then((val) => console.log(val)) // 5
promise.then((val) => console.log(val + 1)) // 6
promise.then((val) => asyncFunction(val))
promise.catch(() => console.log('Reject occured here')); // you can call here another function that returns a promise, and make async calls inside the promise flow.

Promises are now part of ES6, but I suggest to use some promise library like bluebird.

#5 - Use .env files to store your app secrets

When deploying your apps it is recommended to store all of the secret keys and passwords inside a .env file. It is considered a good practice not to commit this file to your version control platform. This advice comes mainly for security reasons, but having a different .env file for production and development systems can really help with some weird bugs :)

You can use dotenv library, it will automaticly load your .env file contents to the process.env object. You can access it within your app easily.

If you are not submitting the file to the SVN, have an .env.example file committed, it will help developers understand how the .env file should look like. Also, make sure to throw an error when the .env file is missing.

#6 - Testing

One of the main aspects of deploying solid code is deploying tested code. It really helps developers to ship better code with more confidence.

Writing unit tests for express can be a little tricky, but here are some tips:

Separate your controller files

It is a common practice in online tutorials and getting started guides to have the route function as an anonymous function passed to the route:

app.get('/somepage', function(req, res){...});

While its easier to write, we cant really easily test this piece of code. I suggest using the approach from tip No.1 and separate your functions as follows:

app.get('/somepage', ctrl.somePage);

// in other imported file:
function somePage(req, res){
// we can test this function easily by mocking req and res
}

When writing large and more complicated controllers I suggest splitting the logic into smaller functions, each function we can later use and unit-test. Not only it will help writing tests, it will also allow you to reuse some of these functions in other places.

There are quite a few libraries to help you with expressjs testing:

  • sinon-mongoose - Is a helper module to mock your mongoose calls inside the unit tests.
  • supertest - Awesome library for integration testing of your express app. It will help you test your entire app(auth, middlewares, db and etc...)

#7 - Use process manager

One of the most important parts of working with node servers is process manager. Since the single threaded nature of node, unhandeled errors will cause a process crash for your entire app. So when going to production it is highly advised to make sure your app will restore in case of failure.

pm2 module

pm2, is a recomended tool for the job. You can install it easily, and run your application. When your server will fail, pm2 will restart it quickly.

Besides the auto process respawn pm2 have some extra features:

  • Logging - it allows you easily manage your app logs. Very useful feature when you want to debug your applications.
  • Clustering - pm2 allows you to spawn multiple instances of your app easily. So that the users will be transfered between the clustered processes in case one of them is not responding or crushed. It also allows you to perform 0 down time deploys.
  • Startup - pm2 offers easily create startup scripts for your node apps.
  • Deployment - You can configure your ecosystem.json file containing your application state, then, you can easily deploy your build using the integrated pm2 deploy module.
  • Monitoring - pm2 also allows easy integration with keymetrics.io. Which is a free tool for monitoring your apps.

Summary

Expressjs is a great framework for writing nodejs API's and webapps. It contains all the necessary tools to build extendable and maintainable code right out of the box. I hope that some of those tips can help you with your existing apps, if you have any suggestions or another tips I would love to hear about them!