Node Js (Jonas - Udemy)

Notes of Node

What is Node?

Node js is a JavaScript runtime built on google's V8 engine.

1. Traditionally, JS was only run on the browser (Browser was the only JS runtime), but now it can also be run outside of the browser. It can now be run on another runtime environment and that env is called Node

2. There are now two JS runtimes - Browser and Node

3. With node, we can now use JS to access the file system, and do other things with JS which wasn't possible before. With this, we can use Node Js as a web server and use JS for server-side web development

Pros of using Node Js

1. Single-threaded, based on non-blocking i-o modal makes it efficient

2. Perfect for building fast and scalable data-intensive apps

3. There is a huge library of open-source packages for free - NPM - We call them modules or packages

4. An active community of developers

Cons of using Node Js (when should we not use Node)

When building heavy server-side processing apps that require more CPU power like heavy image manipulation, video conversion, file compression, and so on (we'll see why later). We can use python, PHP, or ruby-on-rails for these kinds of apps

Why node is single threaded architecture?

Because V8 engine is single threaded, node is also single threaded. Over the time single threaded architecture of Node has provided much better performance compared to multithreaded (thread per request) architecture.

What are the different types of APIs avaialable in Node JS?

• Synchronous API - Blocks the code

• Asynchronous API - Dont block the code

Watch this for better understanding

Node, in the terminal

// to run node in cmd, type node and enter. 
// It opens repl (read eval print loop) and can write JS code in cmd line (including ES6)
> node 
> const name = "sandeep"
> console.log(name) // sandeep
> 6+6 // 12
> _+2 // 14 -> _ gives the previous result 
>.exit // to exit or ctrl+D

// to check version
> node -v

// to get all the variables available in node
> <hit tab>

// to execute a file
> node index.js

Make use of a module

// make use of module like file-system
const fs = require('fs') // returns an object  

Node documentation

Click on your node version on the left panel, once clicked we get all kinds of modules on the left

Documentation | Node.jsNode.js

Reading from and writing to a file synchronously

index.js

// read file sync'ly
const fs = require('fs');
const textIn = fs.readFileSync('./txt/input.txt', 'utf-8');
console.log(textIn);

// write file sync'ly
const textOut = `This is what we know about avacado : ${textIn}\nCreated on ${Date.now()}`
fs.writeFileSync('./txt/output.txt', textOut); // writes into the file

Run the code using the command>node index.js and you see the output in the console.

Synchronous vs Asynchronous behavior

Sync (Blocking)	Async (Non-blocking)
Each statement is processed one after another (each line waits for the previous line to complete)	The async task happens in the background and doesn't block the normal flow of the code.
This can become a problem with slow operations (blocking)	We register a call-back function that executes once the result is available

Sync (Will read the file by blocking other code)

Notice below in async code inside call-back function, the error parameter comes first and not data. Also, the encoding like utf-8 is important otherwise you will get a Buffer output like this <Buffer 72 65 61 64 2d 74 68 69 73>

Async (will read the file in the background)

There is only a single thread in the Node JS process. Meaning, for each application, there is only one thread. Thread is a set of instructions/code executed in the machine's processor. What does it mean for the node js to be single-threaded?

That means all the users who are accessing the application are all using the same thread.

All users use the same thread

This also means, when one user reads the file in a sync way, then all other users need to wait until the file read is done.

Sync'ly reading file will make other users wait

So we need to use async file read. This will make the file read in the background and doesn't block the other users' requests. PHP and other languages are multithreaded but the founder of the node found this single-threaded as an efficient way.

Callback doesn't mean asynchronous always. For some functions, it is and not for some others.

Callback functions when depends on other callbacks soon cause callback-hell and would be difficult to manage.

2nd file read depends on first one; 3rd file read depends on 2nd and so on causing CB hell

The Solution to CB hell would be to use ES6 Promises or ES8 async-await.

// Example of reading and writing file asynchronously
// Here we are using multiple call-backs one dependent on the other 
// (inner most callback depends on outer callback)


// reading file asyncly
fs.readFile("./txt/start.txt", 'utf-8', (err, data1) => {
    console.log(data1)
    
    // reading file asyncly
    fs.readFile(`./txt/${data1}.txt`, 'utf-8', (err, data2) => {
        console.log(data2)
        
        
        // reading file asyncly
        fs.readFile('./txt/append.txt', 'utf-8', (err, data3) => {
            console.log(`${data2}\n${data3}`)
            
            
            // writing file asyncly
            fs.writeFile('./txt/final.txt', `${data2}\n${data3}`, 'utf-8', (err) => {
                console.log("The file has been written")
            })
            
        })
        
    })
    
})

// notice that, in the call-back functions, the first param is error and second one is data

Creating a simple webserver

• We require http module for creating a web server so we can get the networking capabilities

• We first need to create the server and need to listen to incoming requests

// When we have a new request everytime, the call-back will get called
// Let's now send a response. We can use request.end for sending the response

const http = require('http')
http.createServer((request, response) => {
    response.end("Hello from the server")
})

• Let's now make use of this server created above by sending a request to this server. For this, we need to store the result of createServer inside a variable.

• We can now make our server listen to the incoming requests.

index.js

const http = require('http')
const server = http.createServer((request, response) => {
    response.end("Hello from the server")
})

// Server listens to requests. The CB func is called when server starts listening
server.listen(8000, '127.0.0.1', () => {
    console.log("Server started listening on 8000 port")
})

// 8000 is the port and 127.0.0,1 (local-host) is the IP address on which the server is listening

// Even If we don't specify IP address, it defaults to this local host itself

Test your server

• Run the command node index.js and you see the output in the console : Server started listening on 8000 port

• Meaning your server is listening now. You can now send a request to the server by typing 127.0.0.1:8000 in chrome's URL space and you see the output on your browser

When you make some changes to your server (modify your code inside const server), then ctrl+c and then run node index.js again in the console for the changes to take effect.

Chome output from your server

Routing

Now we are on localhost:8000. If we type localhost:8000/anything, it doesn't take us to any page but remains on the same page. Here's where routing is necessary. Routing can get complicated soon hence we use modules like Express. But for now, let's see how to implement routing in the node itself.

const server = http.createServer((request, response) => {
    console.log(request.url) 
    response.end("Hello from the server")
})

In line 2 we have now included the request.url. If we type localhost:3000/test, we get the /test and /favicon.ico in the console. Upon every request to the specific route, it also makes a request to favicon which can be ignored now.

browser

console

We can use, request.url to do a simple routing. response.writeHead() can take response header object that can be displayed in the network tab.

basic routing with response header

header can be see in network tab in browser

NPM

Node Package Manager is a command-line app that comes included with node js used to install and manage open source packages. This comes pre-installed with node js so that we can download any package we want from npm.

npm

How to initialize npm and download packages?

In the command line, type to initialize npm

npm init

This creates the package.json file which is the file where project configuration is present.

To install any package locally,

npm install packagename

No need of --save which was required before

To install any package globally. Once you install it globally, then it can be used in any other project folder,

npm install packagename --global

To install dev dependency, that might make the developer life easy but not actually needed for the project to work,

npm install packagename --save-dev

What is package-lock.json?

While package.json holds the dependency information of the packages/modules we installed, package-lock.json will have the information of all the dependencies used by the modules we installed. It can be therefore called dependencies of dependencies.

All the downloads will get saved in node modules folder. You don't need to put this into git because you can download it later if required. To download all the node modules specified in package.json and package-lock.json you need the command

npm install

This downloads all the dependencies you specified as well as the dependencies used by the modules you downloaded looking at package.json and package-lock.json.

Package versioning and updating

Most of the packages/modules follow this convention of numbering the version of their package. Consider this example package,

package.json

"dependencies" : {
  "slugify" : "^1.3.4" // can also be "~1.3.4" or "*1.3.4"
}

In version 1.3.4, 1 means major version, 3 means minor version, and 4 means patch version. If there are any bug fixes then the next release will be 1.3.5. If there are any features added but no breaking changes then the next version will be 1.4.0. In case of a major update where there might be breaking changes and our current code might no longer work upon update, then the next version would be 2.0.0.

To know which packages are outdated in our code, we can use

npm outdated

The ^ symbol means, we accept minor changes and bug fixes. Let's say the new release is 1.3.8 or 1.4.2 or 2.0.1 and we use the below command to update the package,

npm update

Then, since we have^symbol that indicates we accept minor changes and bug fixes, the package gets updated to 1.4.2.

In case we use ~symbol that indicates we accept only bug fixes then the package gets updated to 1.3.8

In case we use* symbol that indicates we accept major changes (not recommended as it might break our current code) then the package gets updated to 2.0.1.

Back-end and Front-end Terminologies

MVC is a server-side rendered website

API powered website is called client-side rendered as front-end rendered seperately

Node behind the scenes

Node high-level diagram

Node runtime has several dependencies. It depends most importantly on libraries like V8 Js Engine and Lib-UV. V8 in the node understands JS and converts it to machine code but that alone is not enough to give Node the server-side functionality. That's when Lib-UV comes into the picture. Lib-UV is an open-source library with a strong focus on Asynchronous IO. Lib-UV is what gives node the access to the computer's underlying operation system, file-system, networking, and more.

Lib-UV also implements two important features into the Node which are Event-Loop and Thread-Pool. Event-loop is for performing easy tasks such as executing call-backs whereas Thread-pool performs heavy tasks such as file-access or compression etc. V8 is written in both JS and C++ while Lib-UV is written in C++ alone.

Node not only relies on V8 and Lib-UV but also on other libraries like

• http-parser - for parsing http

• c-ares - for DNS requests

• openSSL - for cryptography

• zLib - for compression

Node Processes and Threads

Processes and Threads

Two fundamental parts of the Node are Event-loop and Thread-pool. Node runs on a computer/server which is a C++ program that will start the process while running.

We have a variable called process in the node which we are going to use

In that process, the node runs on a single thread. Thread is basically a sequence of instructions of a program. Note that multi-thread means multiple sequences/parts of the same program. Since Node is single-threaded, it runs on 1 thread for 10 users or 10 million users.

When the program is initialized,

• all the top-level code gets executed (code that is not inside any call-back function)

• all the modules are required

• all the call-backs are registered

• after all this, the event-loop finally starts running. Event-loop is where most of the work is done in the app. They are the heart of the node app. But some tasks in the event-loop are too expensive to be executed in the event-loop as they may block the single thread. That's when the thread-pool comes to the rescue.

• thread-pool is provided by the Lib-UV library. It gives 4 different threads that are completely separate from the main single thread. We can configure this up to 128 threads but usually, these 4 are enough. These 4 threads form the thread-pool and the event-loop automatically off-loads or delegates heavy tasks to the thread-pool. All this happens behind the scenes. The heavy tasks that might be delegated from the event-loop to the thread-pool might be File-System APIs, cryptography, and more. This way the event loop is not blocked.

Event Loop

Event-loop

Event-loop runs all the code that is inside the call-back function (not top-level code). Node is built around call-back functions which are called events that are executed at later times like timer expiring, HTTP-requests, file reading, and so on. So we call Node the Event-driven architecture. Event-loop is the one that calls all the call-backs associated with the events that are registered.

Simply put, Event-loop

• receives events

• calls their call-back functions

• and offloads the more expensive tasks out of them to the thread pool

The order in which the events are called by event loop

There are 4 phases/ 4 callback queues of the event loop and each phase will be its own callbacks to process by the event loop

Event loop Phase 1 - Expired Timer Callbacks

Expired timer callbacks - If there are any timer events like setTimout whose timer gets expired, then these callbacks are first executed. If the event loop is in some other phase and the timer expires then the callback of that expired timer will be executed as soon as the event loop comes back to this phase. Callbacks are executed one by one and until all the call-backs are not completed the event loop will not go to the next phase

2.

I/O Polling and callbacks - Eventloop looks for new Input/Output events that are ready to be processed and puts them into the callback queue. I/O generally refers to tasks like networking and file access. It is in this phase where 99% of our code gets executed.

3.

SetImmediate callbacks anc closed callbacks

Set Immediate callbacks - We can use this if we need our callbacks to be executed immediately after I/O polling callbacks.

4. Close callbacks - All close events are processed for example, shutting down a web socket or web server.

Besides these 4 callback queues there are other 2 queues

• NEXTTICK() Queue

• Microtask Queue

Callback queues in each phase of event loop

If there are any callbacks in one of these 2 queues to be processed, they will be executed right after the current phase of event loop instead of waiting for all 4 phases to be finished. In other words, after each of the 4 phases, it checks if there are any callback exists in one of these two queues and execute them if any.

Multiple event loop ticks

Till now the 4 phases we saw was one event loop tick. Node checks whether there are any timers or I/O tasks running such as HTTP requests or file reads and so on and if there aren't any then it will exit the application. If there are then it will continue to the next tick.

How not to block the event loop since it's single-threaded?

Donts - not to block event loop