Dirk Harriman Banner Image

 

Notes Node JS

Projects:

 

Node JS Terms & Definitions


 
Middleware

Software that acts as a bridge between an operating system or database and applications, especially on a network. Middleware is a type of computer software that provides services to software applications beyond those available from the operating system. It can be described as "software glue".

With Regard to Express

Express is a routing and middleware web framework that has minimal functionality of its own: An Express application is essentially a series of middleware function calls.

Middleware functions are functions that have access to the request object (req), the response object (res), and the next middleware function in the application's request-response cycle. The next middleware function is commonly denoted by a variable named next.

Types of Middleware

An Express application can use the following types of middleware:

You can load application-level and router-level middleware with an optional mount path. You can also load a series of middleware functions together, which creates a sub-stack of the middleware system at a mount point.


 

Application-Level Middleware

Bind application-level middleware to an instance of the app object by using the app.use() and app.METHOD() functions, where METHOD is the HTTP method of the request that the middleware function handles (such as GET, PUT, or POST) in lowercase.

This example shows a middleware function with no mount path. The function is executed every time the app receives a request.

const express = require('express') const app = express() app.use((req, res, next) => { console.log('Time:', Date.now()) next() })

This example shows a middleware function mounted on the /user/:id path. The function is executed for any type of HTTP request on the /user/:id path.

app.use('/user/:id', (req, res, next) => { console.log('Request Type:', req.method) next() })

This example shows a route and its handler function (middleware system). The function handles GET requests to the /user/:id path.

app.get('/user/:id', (req, res, next) => { res.send('USER') })

Here is an example of loading a series of middleware functions at a mount point, with a mount path.
It illustrates a middleware sub-stack that prints request info for any type of HTTP request to the /user/:id path.

app.use('/user/:id', (req, res, next) => { console.log('Request URL:', req.originalUrl) next() }, (req, res, next) => { console.log('Request Type:', req.method) next() })

Route handlers enable you to define multiple routes for a path. The example below defines two routes for GET requests to the /user/:id path. The second route will not cause any problems, but it will never get called because the first route ends the request-response cycle.

This example shows a middleware sub-stack that handles GET requests to the /user/:id path.

app.get('/user/:id', (req, res, next) => { console.log('ID:', req.params.id) next() }, (req, res, next) => { res.send('User Info') }) // handler for the /user/:id path, which prints the user ID app.get('/user/:id', (req, res, next) => { res.send(req.params.id) })

Middleware can also be declared in an array for reusability.

This example shows an array with a middleware sub-stack that handles GET requests to the /user/:id path

function logOriginalUrl (req, res, next) { console.log('Request URL:', req.originalUrl) next() } function logMethod (req, res, next) { console.log('Request Type:', req.method) next() } const logStuff = [logOriginalUrl, logMethod] app.get('/user/:id', logStuff, (req, res, next) => { res.send('User Info') })


 

Router-Level Middleware

Router-level middleware works in the same way as application-level middleware, except it is bound to an instance of express.Router().

const router = express.Router();

Load router-level middleware by using the router.use() and router.METHOD() functions.

The following example code replicates the middleware system that is shown above for application-level middleware, by using router-level middleware:

const express = require('express') const app = express() const router = express.Router() // a middleware function with no mount path. This code is executed for every request to the router router.use((req, res, next) => { console.log('Time:', Date.now()) next() }) // a middleware sub-stack shows request info for any type of HTTP request to the /user/:id path router.use('/user/:id', (req, res, next) => { console.log('Request URL:', req.originalUrl) next() }, (req, res, next) => { console.log('Request Type:', req.method) next() }) // a middleware sub-stack that handles GET requests to the /user/:id path router.get('/user/:id', (req, res, next) => { // if the user ID is 0, skip to the next router if (req.params.id === '0') next('route') // otherwise pass control to the next middleware function in this stack else next() }, (req, res, next) => { // render a regular page res.render('regular') }) // handler for the /user/:id path, which renders a special page router.get('/user/:id', (req, res, next) => { console.log(req.params.id) res.render('special') }) // mount the router on the app app.use('/', router)

To skip the rest of the router's middleware functions, call next('router') to pass control back out of the router instance.

This example shows a middleware sub-stack that handles GET requests to the /user/:id path.

const express = require('express') const app = express() const router = express.Router() // predicate the router with a check and bail out when needed router.use((req, res, next) => { if (!req.headers['x-auth']) return next('router') next() }) router.get('/user/:id', (req, res) => { res.send('hello, user!') }) // use the router and 401 anything falling through app.use('/admin', router, (req, res) => { res.sendStatus(401) })


 

Error-Handling Middleware

Note: Error-handling middleware always takes four arguments.
You must provide four arguments to identify it as an error-handling middleware function.
Even if you don't need to use the next object, you must specify it to maintain the signature.
Otherwise, the next object will be interpreted as regular middleware and will fail to handle errors.

Define error-handling middleware functions in the same way as other middleware functions, except with four arguments instead of three,
specifically with the signature (err, req, res, next)

app.use((err, req, res, next) => { console.error(err.stack) res.status(500).send('Something broke!') })

Catching Errors

It's important to ensure that Express catches all errors that occur while running route handlers and middleware.

Errors that occur in synchronous code inside route handlers and middleware require no extra work. If synchronous code throws an error, then Express will catch and process it. For example:

app.get('/', (req, res) => { throw new Error('BROKEN') // Express will catch this on its own. })

For errors returned from asynchronous functions invoked by route handlers and middleware, you must pass them to the next() function, where Express will catch and process them. For example:

app.get('/', (req, res, next) => { fs.readFile('/file-does-not-exist', (err, data) => { if (err) { next(err) // Pass errors to Express. } else { res.send(data) } }) })

Starting with Express 5, route handlers and middleware that return a Promise will call next(value) automatically when they reject or throw an error. For example:

app.get('/user/:id', async (req, res, next) => { const user = await getUserById(req.params.id) res.send(user) })

If getUserById throws an error or rejects, next will be called with either the thrown error or the rejected value. If no rejected value is provided, next will be called with a default Error object provided by the Express router.

If you pass anything to the next() function (except the string 'route'), Express regards the current request as being an error and will skip any remaining non-error handling routing and middleware functions.

If the callback in a sequence provides no data, only errors, you can simplify this code as follows:

app.get('/', [ function (req, res, next) { fs.writeFile('/inaccessible-path', 'data', next) }, function (req, res) { res.send('OK') } ])

In the above example next is provided as the callback for fs.writeFile, which is called with or without errors. If there is no error the second handler is executed, otherwise Express catches and processes the error.

You must catch errors that occur in asynchronous code invoked by route handlers or middleware and pass them to Express for processing. For example:

app.get('/', (req, res, next) => { setTimeout(() => { try { throw new Error('BROKEN') } catch (err) { next(err) } }, 100) })

The above example uses a try...catch block to catch errors in the asynchronous code and pass them to Express. If the try...catch block were omitted, Express would not catch the error since it is not part of the synchronous handler code.

Use promises to avoid the overhead of the try...catch block or when using functions that return promises. For example:

app.get('/', (req, res, next) => { Promise.resolve().then(() => { throw new Error('BROKEN') }).catch(next) // Errors will be passed to Express. })

Since promises automatically catch both synchronous errors and rejected promises, you can simply provide next as the final catch handler and Express will catch errors, because the catch handler is given the error as the first argument.

You could also use a chain of handlers to rely on synchronous error catching, by reducing the asynchronous code to something trivial. For example:

app.get('/', [ function (req, res, next) { fs.readFile('/maybe-valid-file', 'utf-8', (err, data) => { res.locals.data = data next(err) }) }, function (req, res) { res.locals.data = res.locals.data.split(',')[1] res.send(res.locals.data) } ])

The above example has a couple of trivial statements from the readFile call. If readFile causes an error, then it passes the error to Express, otherwise you quickly return to the world of synchronous error handling in the next handler in the chain. Then, the example above tries to process the data. If this fails then the synchronous error handler will catch it. If you had done this processing inside the readFile callback then the application might exit and the Express error handlers would not run.

Whichever method you use, if you want Express error handlers to be called in and the application to survive, you must ensure that Express receives the error.

The Default Error Handler

Express comes with a built-in error handler that takes care of any errors that might be encountered in the app. This default error-handling middleware function is added at the end of the middleware function stack.

If you pass an error to next() and you do not handle it in a custom error handler, it will be handled by the built-in error handler; the error will be written to the client with the stack trace. The stack trace is not included in the production environment.

Set the environment variable NODE_ENV to production, to run the app in production mode.

When an error is written, the following information is added to the response:

So when you add a custom error handler, you must delegate to the default Express error handler, when the headers have already been sent to the client:

function errorHandler (err, req, res, next) { if (res.headersSent) { return next(err) } res.status(500) res.render('error', { error: err }) }

Note that the default error handler can get triggered if you call next() with an error in your code more than once, even if custom error handling middleware is in place.

Writing Error Handlers

Define error-handling middleware functions in the same way as other middleware functions, except error-handling functions have four arguments instead of three: (err, req, res, next). For example:

app.use((err, req, res, next) => { console.error(err.stack) res.status(500).send('Something broke!') })

You define error-handling middleware last, after other app.use() and routes calls; for example:

const bodyParser = require('body-parser') const methodOverride = require('method-override') app.use(bodyParser.urlencoded({ extended: true })) app.use(bodyParser.json()) app.use(methodOverride()) app.use((err, req, res, next) => { // logic })

Responses from within a middleware function can be in any format, such as an HTML error page, a simple message, or a JSON string.

For organizational (and higher-level framework) purposes, you can define several error-handling middleware functions, much as you would with regular middleware functions. For example, to define an error-handler for requests made by using XHR and those without:

const bodyParser = require('body-parser') const methodOverride = require('method-override') app.use(bodyParser.urlencoded({ extended: true })) app.use(bodyParser.json()) app.use(methodOverride()) app.use(logErrors) app.use(clientErrorHandler) app.use(errorHandler)

In this example, the generic logErrors might write request and error information to stderr, for example:

function logErrors (err, req, res, next) { console.error(err.stack) next(err) }

Also in this example, clientErrorHandler is defined as follows; in this case, the error is explicitly passed along to the next one.

Notice that when not calling “next” in an error-handling function, you are responsible for writing (and ending) the response. Otherwise those requests will “hang” and will not be eligible for garbage collection.

function clientErrorHandler (err, req, res, next) { if (req.xhr) { res.status(500).send({ error: 'Something failed!' }) } else { next(err) } }

Implement the “catch-all” errorHandler function as follows (for example):

function errorHandler (err, req, res, next) { res.status(500) res.render('error', { error: err }) }

If you have a route handler with multiple callback functions you can use the route parameter to skip to the next route handler. For example:

app.get('/a_route_behind_paywall', (req, res, next) => { if (!req.user.hasPaid) { // continue handling this request next('route') } else { next() } }, (req, res, next) => { PaidContent.find((err, doc) => { if (err) return next(err) res.json(doc) }) })

In this example, the getPaidContent handler will be skipped but any remaining handlers in app for /a_route_behind_paywall would continue to be executed.

Calls to next() and next(err) indicate that the current handler is complete and in what state. next(err) will skip all remaining handlers in the chain except for those that are set up to handle errors as described above.

 
 

Routing

What is routing?

To start with routing in Node.js, one needs to know what is routing and its purpose. The route is a section of Express code that associates an HTTP verb (GET, POST, PUT, DELETE, etc.), an URL path/pattern, and a function that is called to handle that pattern.

Basics to Route Creation

A route method is derived from one of the following HTTP methods, and is attached to an instance of the express class as below :

var express = require('express'); var router = express.Router();

Route Definition

It takes the following structure:

app.METHOD(PATH, HANDLER)

Where:

There are two ways of using the router methods. Either we can define route (for example /users) and attach the methods to it, or we can create a new method for each route.

// Method 1 router.route('/users') .get(function (req, res, next) { ... }) .post(function (req, res, next) { ... }); // Method 2 router.get('/users', function (req, res) { ... }); router.post('/users', function (req, res) { ... });

Note: We can use the same ('/users') route for multiple methods like GET, POST within a Node application.

Route Methods

A method which is derived from any one of the following HTTP methods, and it has to be attached to an instance of the express class. Here are the express routing methods corresponding to the HTTP methods of the same names.

  • checkout
  • copy
  • delete
  • get
  • head
  • lock
  • merge
  • mkactivity
  • mkcol
  • move
  • m-search
  • notify
  • options
  • patch
  • post
  • purge
  • put
  • report
  • search
  • subscribe
  • trace
  • unlock
  • unsubscribe

app.all() is a special routing method used to load middleware functions at a path for all HTTP request methods. When this method is routed, the attached handler will be executed for requests to the specific route whether using GET, POST, PUT, DELETE, or any other HTTP request method supported in the HTTP module.

Route Paths

The route paths in combination with a request method define the endpoints at which requests can be made. It can be strings, string patterns, or regular expressions.

The characters ?, +, *, and () are subsets of their regular expression counterparts.
The hyphen (-) and the dot (.) are interpreted literally by string-based paths.
If we need to use the dollar character ($) in a path string, enclose it escaped within ([ and ]).
Additionally, Express uses path-to-regexp for matching the route paths.
Query strings are not part of the route path.

Examples of route paths based on strings:

This route path will match requests to the root route, /.

app.get('/', function (req, res) { res.send('root') })

And this route path will match requests to /home.

app.get('/home', function (req, res) { res.send('home') })

While this route path will match requests to /profile.text.

app.get('/profile.text', function (req, res) { res.send('profile.text') })

Examples of route paths based on string patterns:

This route path will match acd and abcd.

app.get('/ab?cd', function (req, res) { res.send('ab?cd') })

And, this route path will match abcd, abbcd, abbbcd, and so on.

app.get('/ab+cd', function (req, res) { res.send('ab+cd') })

While, this route path will match abcd, abxcd, abRANDOMcd, ab123cd, and so on.

app.get('/ab*cd', function (req, res) { res.send('ab*cd') })

Furthermore, this route path will match /abe and /abcde.

app.get('/ab(cd)?e', function (req, res) { res.send('ab(cd)?e') })

Examples of route paths based on regular expressions:

This route path will match anything with an “a” in it.

app.get(/a/, function (req, res) { res.send('/a/') })

And, this route path will match mybook and handbook, however not mybookshelf, bookshop, and so on.

app.get(/.*book$/, function (req, res) { res.send('/.*book$/') })

Route Parameters

These are named URL segments that are used to capture the values specified at their position in the URL. The captured values are populated through the req.params object, with the respective key name defined in the path of request method.

app.get('/users/:userId/orders/:orderId', function (req, res) { res.send(req.params) })

Here, for instance, let us imagine that we need order information for a particular user. So for this purpose, we have defined the above route with the route parameters (/:userId) and (/:orderId). This definition makes the 'user Id' and corresponding 'order Id' available through req.params.userId and req.params.orderId, without any complex coding.

Route Handlers

It can be in the form of a function, an array of functions, or even combinations of both.

We can provide multiple callback functions that behave like middleware to handle a request. But, the only exception is that these callbacks might invoke next('route') to bypass the remaining route callbacks. Also, we can use this mechanism to impose pre-conditions on a route, and then pass control to subsequent routes if there's no reason to proceed with the current route.

app.get('/getDetails', function (req, res, next) { console.log('invoke response using next callback function ...') next() }, function (req, res) { res.send('Here are the details!....') })

To conclude, we can generate a better responsive Node application with Express framework by using the above routing methodologies. Thus, routing in Node.js is no big deal!

MySQL Application

A simple database application that fetches some values from a MySQL database and displays them in a HTML list.

Directory Structure

This is the directory structure for thsi application. 


  AppRoot/....................Application root folder
    package.json..............Application config file
    AppName.js................Main application file
    node_modules/.............Folder for module code
      ( Included Modules )
    public/...................Folder for client side includes
      js/.....................Folder for client side javascript
        main.js
      css/....................Folder for client side css like bootstrap
        bootstrap.min.css
        site.css
    views/....................Folder for primary html files
      index.ejs
      partials/...............Folder for partial html files
        footer.ejs
        header.ejs

Files

There are a minimum of two files neccessary for a node.js application to work.

package.json

This is more or less the application configuration file.

This file can be created manually or using npm. To create the file using npn, type the following and follow the prompts: 


  $ npm init

At some point you may need to add some modules to your colde and the package.json file. To do this, type the following, exchanging whatever component you need: 


  $ npm install express --save

To install a module globally: 


  $ npm install -g nodemon

Sample package.json File


{
  "name": "AppName",
  "version": "1.0.0",
  "description": "Simple MySQL database example. Uses Express.",
  "main": "AppName.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "dependencies": {
    "body-parser": "*",
    "ejs": "^2.6.1",
    "express": "*",
    "express-validator": "^5.3.0",
    "mysql": "^2.16.0"
  },
  "author": "Dirk Harriman",
  "license": "ISC"
}

AppName.js

This is the primary code file for the application.


  /* AppName - AppName.js MySQL DEVELOPMENT TESTING */
  const express = require('express');           // WEB APPLICATION FRAMEWORK
  const bodyParser = require('body-parser');    // BODY PARSER
  const path = require('path');                 // APPLICATION PATH
  const mysql = require('mysql');               // MySQL DATABASE

  const app = express();                  // SET WEB APP FRAMEWORK

  const db = mysql.createConnection({     // SET DATABASE CONNECTION PARAMETERS
    host: 'localhost',
    user: 'Dirk',
    password: 'Banyans25',
    database: 'dirksitedb'
  });
  db.connect((err) => {   // CONNECT TO DATABASE
    if (err) {  // CHECK FOR DB CONNECTION ERROR
      console.error('Error connecting: ' + err.stack);
      return;
    }
    console.log('Connected as id ' + db.threadId);  // LOG CONNECTION
  });
  app.set('view engine', 'ejs');                    // SET WEB VIEW ENGINE TO EJS
  app.set('views', path.join(__dirname, 'views'));  // SET views PATH

  /* BODY PARSER MIDDLEWARE */
  app.use(bodyParser.json());                         // PARSE application/json
  app.use(bodyParser.urlencoded({extended: false}));  // PARSE application/x-www-form-urlencoded

  /* SET STATIC PATH - USED TO HOLD LIBRARY FILES LIKE jQuery etc... */
  app.use(express.static(path.join(__dirname, 'public')));

  /* ROUTING */
  app.get('/', (req, res) => {
    let sql = 'SELECT id, ListName FROM tablist';
    let query = db.query(sql, (err, result) => {
      if(err) throw err;
      res.render('index', {
        title : 'My List',
        users : result
      });
    });
  });

  /* PORT LISTENER */
  app.listen(3000, function() {
    console.log('Server started on port 3000');
  });

views/index.ejs


  <% include partials/header %>
  <h1><%= title %></h1>
  <ul>
  <% users.forEach(function(user){ %>
    <li>
      <%= user.id %>
      <%= user.ListName %> - <a href="#" class="deleteUser" data-id="<%= user.id %>">Delete</a>
    </li>
  <% }) %>
  </ul>
  <% include partials/footer %>

views/partials/header.ejs


  <!DOCTYPE html>
  <html>
  <head>
  <title>Node JS - Express Test Page</title>
  <meta charset="utf-8"/>
  <meta http-equiv="X-UA-Compatible" content="IE=edge"/>
  <meta name="viewport" content="width=device-width, initial-scale=1"/>
  <meta name="description" content=""/>
  <meta name="author" content=""/>
  <script src="http://code.jquery.com/jquery-3.3.1.min.js" integrity="sha256-FgpCb/KJQlLNfOu91ta32o/NMZxltwRo8QtmkMRdAu8=" crossorigin="anonymous"></script>
  </head>
  <body>
    <h4>Header here</h4>
    <hr/>

views/partials/footer.ejs


    <hr/>
    <h4>Footer Here</h4>
  </body>
  </html>

Node JS Basics

As an asynchronous event driven JavaScript runtime, Node is designed to build scalable network applications. In the following "hello world" example, many connections can be handled concurrently. Upon each connection the callback is fired, but if there is no work to be done, Node will sleep. 


    const http = require('http');
    
    const hostname = '127.0.0.1';
    const port = 3000;
    
    const server = http.createServer((req, res) => {
      res.statusCode = 200;
      res.setHeader('Content-Type', 'text/plain');
      res.end('Hello World\n');
    });
    
    server.listen(port, hostname, () => {
      console.log(`Server running at http://${hostname}:${port}/`);
    });
 

Node.js

 

Event Loop

Event Emitters -> Events Event Loop Event Handlers
 

What can be built with Node.js?

 

NPM - Node Package Manager

Popular Node.js Packages

Express
Web development framework
Connect
Extensible HTTP server framework. A component of Express.
Socket.io
Server-side component for websockets
Pug / Jade
Template engine inspired by HAML. Default for Express.
Mongo / Mongoose
Wrappers to interact with MongoDB
Coffee-Script
CoffeeScript compiler
Redis
Redis client library. NoSql DB. Cacheing systems

Examples:


    $ npm install express

OR (with the global install option)

  
    $ npm install -g express

Note: Packages can also be installed using the package.json file.

 

Package.Json File:

Located in the root of the package / application. Tells NPM how your package is structured and what to do to install it.

The file can be created in a text editor or you can run "npm init" to have it created from the prompts. 


    $ npm init  (Answer prompts)

Example of a package.json file:


    {
      'name' : 'MyAppName',
      'version' : '1.0.0',
      'description' : 'Simple App',
      'main' : 'app.js',
      'author' : 'Dirk Harriman',
      'license' : 'ISC',
      'dependencies' : {
        'body-parser' : '^1.15.2',
        'express' : '^4.14.0',
        'mongojs' : '^2.4.0',
      }
    }
 

Command Line

If you go to the Node.js install folder, there's an executable called node.exe This will open the Node.js command line interface. At the prompt you can enter javascript or you can call on a javascript file to be run. 


    $ node myFile.js

OR 


    $ node myFile
 

Basic Web Server

File and folder structure:


    myApp/
      myApp.js
      package.json
      index.html

package.json :


    {
      'name' : 'myApp',
      'version' : '1.0.0',
      'description' : 'Simple App',
      'main' : 'myApp.js',
      'author' : 'Dirk Harriman',
      'license' : 'ISC',
      'dependencies' : {
        'body-parser' : '^1.15.2',
        'express' : '^4.14.0',
        'mongojs' : '^2.4.0',
      }
    }

myApp.js


    const http = require('http');    /* A core node.js package module */
    const fs = require('fs');
    const hostname = '127.0.0.1';
    const port = 3000;
    fs.readfile('index.html', (err, html) => {
    
      if(err) {
        throw err;
      }
      const server = http.createServer((req,res) => {
        res.statusCode = 200;
        res.setHeader('Content-type','text/html');
        res.write(html);
        res.end();
      });
      server.listen(port, hostname, () => {
        console.log('Server Started on Port: ' + port);
      });

    });

index.html


    <!DOCTYPE html>
    <html>
    <head>
    <title>Node JS Index Page</title>
    <meta charset="utf-8"/>
    <meta http-equiv="X-UA-Compatible" content="IE=edge"/>
    <meta name="viewport" content="width=device-width, initial-scale=1"/>
    <meta name="description" content=""/>
    <meta name="author" content=""/>
    </head>
    <body>
      <form>
        <div class="container">
          <h1>Node JS Index Page</h1>
        </div>
      </form>
    </body>
    </html>