# Node Js (Jonas - Udemy) ## What is Node? Node js is a **JavaScript runtime** built on google's **V8 engine**. 1. Traditionally, JS was only run on the browser (Browser was the only JS runtime), but now it can also be run outside of the browser. It can now be run on another runtime environment and that env is called Node 2. There are now two JS runtimes - **Browser and Node** ![](https://1944679227-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVEiPUp08kYt33g51v7%2F-MVJlxDSn5Ll8DGW4ddp%2F-MVKCWTRUmZSAc2s0q4K%2Fimage.png?alt=media\&token=15a78d6a-6966-405a-97a4-1331e01eee5a) 3\. With node, we can now use JS to access the file system, and do other things with JS which wasn't possible before. With this, we can use Node Js as a web server and use JS for server-side web development #### Pros of using Node Js 1. **Single-threaded**, based on **non-blocking i-o** modal makes it efficient 2. Perfect for building **fast and scalable data-intensive apps** 3. There is a huge library of open-source packages for free - **NPM -** We call them modules or packages 4. An active community of developers #### Cons of using Node Js (when should we not use Node) When building heavy server-side processing apps that require **more CPU power** like heavy image manipulation, video conversion, file compression, and so on (we'll see why later). We can use python, PHP, or ruby-on-rails for these kinds of apps {% hint style="success" %} **Why node is single threaded architecture?** Because V8 engine is single threaded, node is also single threaded. Over the time single threaded architecture of Node has provided much better performance compared to multithreaded (thread per request) architecture. **What are the different types of APIs avaialable in Node JS?** * Synchronous API - Blocks the code * Asynchronous API - Dont block the code {% endhint %} {% embed url="" %} Watch this for better understanding {% endembed %} #### Node, in the terminal ```javascript // to run node in cmd, type node and enter. // It opens repl (read eval print loop) and can write JS code in cmd line (including ES6) > node > const name = "sandeep" > console.log(name) // sandeep > 6+6 // 12 > _+2 // 14 -> _ gives the previous result >.exit // to exit or ctrl+D // to check version > node -v // to get all the variables available in node > // to execute a file > node index.js ``` #### Make use of a module ```javascript // make use of module like file-system const fs = require('fs') // returns an object ``` #### Node documentation Click on your node version on the left panel, once clicked we get all kinds of modules on the left {% embed url="" %} #### Reading from and writing to a file synchronously {% tabs %} {% tab title="index.js" %} ```javascript // read file sync'ly const fs = require('fs'); const textIn = fs.readFileSync('./txt/input.txt', 'utf-8'); console.log(textIn); // write file sync'ly const textOut = `This is what we know about avacado : ${textIn}\nCreated on ${Date.now()}` fs.writeFileSync('./txt/output.txt', textOut); // writes into the file ``` {% endtab %} {% endtabs %} Run the code using the command`>node index.js` and you see the output in the console. #### Synchronous vs Asynchronous behavior | **Sync (Blocking)** | Async (Non-blocking) | | --------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- | |

Each statement is processed one after another

(each line waits for the previous line to complete)

| The async task happens in the background and doesn't block the normal flow of the code. | | This can become a problem with slow operations (blocking) | We register a call-back function that executes once the result is available |
Sync (Will read the file by blocking other code)
{% hint style="info" %} Notice below in async code inside call-back function, the error parameter comes first and not data. Also, the encoding like `utf-8` is important otherwise you will get a Buffer output like this `` {% endhint %} ![Async (will read the file in the background)](https://1944679227-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVEiPUp08kYt33g51v7%2F-MVe735bJKDDfWIDXRHz%2F-MVeDZHTnGcWmlXTTrYN%2Fimage.png?alt=media\&token=b1432bb4-aa37-4d29-a853-1cbb4cedde78) There is only a single thread in the Node JS process. Meaning, for each application, there is only one thread. Thread is a set of instructions/code executed in the machine's processor. What does it mean for the node js to be single-threaded? That means all the users who are accessing the application are all using the same thread. ![All users use the same thread](https://1944679227-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVEiPUp08kYt33g51v7%2F-MVe735bJKDDfWIDXRHz%2F-MVeFfiAgnkM1pQpDqzE%2Fimage.png?alt=media\&token=c4d5806e-c5ac-4914-848c-9eb00c090538) This also means, when one user reads the file in a sync way, then all other users need to wait until the file read is done. ![Sync'ly reading file will make other users wait](https://1944679227-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVEiPUp08kYt33g51v7%2F-MVeFl4ia28cm9xRZSrq%2F-MVeGQrxZQFPlqJmFqkh%2Fimage.png?alt=media\&token=32427240-e439-4980-97de-38670392c329) So we need to use async file read. This will make the file read in the background and doesn't block the other users' requests. PHP and other languages are multithreaded but the founder of the node found this single-threaded as an efficient way. {% hint style="warning" %} Callback doesn't mean asynchronous always. For some functions, it is and not for some others. {% endhint %} Callback functions when depends on other callbacks soon cause callback-hell and would be difficult to manage. ![2nd file read depends on first one; 3rd file read depends on 2nd and so on causing CB hell](https://1944679227-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVEiPUp08kYt33g51v7%2F-MVeHLq7DciifuodF9Mo%2F-MVeI8jxIqUvsskysZVx%2Fimage.png?alt=media\&token=17bb91cd-ef45-42d9-8153-71c66612e85f) {% hint style="success" %} The Solution to CB hell would be to use **ES6 Promises or ES8 async-await**. {% endhint %} ```javascript // Example of reading and writing file asynchronously // Here we are using multiple call-backs one dependent on the other // (inner most callback depends on outer callback) // reading file asyncly fs.readFile("./txt/start.txt", 'utf-8', (err, data1) => { console.log(data1) // reading file asyncly fs.readFile(`./txt/${data1}.txt`, 'utf-8', (err, data2) => { console.log(data2) // reading file asyncly fs.readFile('./txt/append.txt', 'utf-8', (err, data3) => { console.log(`${data2}\n${data3}`) // writing file asyncly fs.writeFile('./txt/final.txt', `${data2}\n${data3}`, 'utf-8', (err) => { console.log("The file has been written") }) }) }) }) // notice that, in the call-back functions, the first param is error and second one is data ``` ## Creating a simple webserver * We require `http` module for creating a web server so we can get the networking capabilities * We first need to create the server and need to listen to incoming requests ```javascript // When we have a new request everytime, the call-back will get called // Let's now send a response. We can use request.end for sending the response const http = require('http') http.createServer((request, response) => { response.end("Hello from the server") }) ``` * Let's now make use of this server created above by sending a request to this server. For this, we need to store the result of createServer inside a variable. * We can now make our server listen to the incoming requests. {% tabs %} {% tab title="index.js" %} ```javascript const http = require('http') const server = http.createServer((request, response) => { response.end("Hello from the server") }) // Server listens to requests. The CB func is called when server starts listening server.listen(8000, '127.0.0.1', () => { console.log("Server started listening on 8000 port") }) // 8000 is the port and 127.0.0,1 (local-host) is the IP address on which the server is listening // Even If we don't specify IP address, it defaults to this local host itself ``` {% endtab %} {% endtabs %} {% hint style="success" %} **Test your server** * Run the command `node index.js` and you see the output in the console : `Server started listening on 8000 port` * Meaning your server is listening now. You can now send a request to the server by typing `127.0.0.1:8000` in chrome's URL space and you see the output on your browser When you make some changes to your server (modify your code inside const server), then `ctrl+c` and then run `node index.js` again in the console for the changes to take effect. {% endhint %} ![Chome output from your server](https://1944679227-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVEiPUp08kYt33g51v7%2F-MVjOgnEO7nYCDq39ndX%2F-MVjOv0zk2PbriHSwm-T%2Fimage.png?alt=media\&token=7f5448e3-3b17-4ea3-a43e-599cd1228cd2) #### Routing Now we are on localhost:8000. If we type localhost:8000/anything, it doesn't take us to any page but remains on the same page. Here's where routing is necessary. Routing can get complicated soon hence we use modules like Express. But for now, let's see how to implement routing in the node itself. ```javascript const server = http.createServer((request, response) => { console.log(request.url) response.end("Hello from the server") }) ``` In line 2 we have now included the `request.url`. If we type `localhost:3000/test`, we get the /test and /favicon.ico in the console. Upon every request to the specific route, it also makes a request to favicon which can be ignored now. ![browser](https://1944679227-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVEiPUp08kYt33g51v7%2F-MWMatgqR2zOhkdpfsIF%2F-MWMdM8WX2KSpHOy2-x5%2Fimage.png?alt=media\&token=88d967a3-8647-49d5-9397-7b80b8e4e3fa) ![console](https://1944679227-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVEiPUp08kYt33g51v7%2F-MWMatgqR2zOhkdpfsIF%2F-MWMdD0cGlNKN0AUuGMp%2Fimage.png?alt=media\&token=75d7fb4e-b0ca-490b-971c-2a45f570c5fd) We can use, `request.url` to do a simple routing. `response.writeHead()` can take response header object that can be displayed in the network tab. ![basic routing with response header](https://1944679227-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVEiPUp08kYt33g51v7%2F-MWN0jMLpBXzNOrlGRNg%2F-MWN1coeVm96eqy0pGO7%2Fimage.png?alt=media\&token=9dd4a2f2-8edc-4a5b-9eb9-884be1a25422) ![header can be see in network tab in browser](https://1944679227-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVEiPUp08kYt33g51v7%2F-MWN-3MM8eRGSmayDYEY%2F-MWN0QG_lxgIVeRscgjX%2Fimage.png?alt=media\&token=e259fa8f-301d-43f7-b5f7-831bc2d6d232) ## NPM **N**ode **P**ackage **M**anager is a command-line app that comes included with node js used to install and manage open source packages. This comes pre-installed with node js so that we can download any package we want from npm. {% embed url="" %} #### How to initialize npm and download packages? In the command line, type to initialize npm > npm init This creates the package.json file which is the file where project configuration is present. To install any package locally, > npm install packagename {% hint style="info" %} No need of `--save` which was required before {% endhint %} To install any package globally. Once you install it globally, then it can be used in any other project folder, > npm install packagename --global To install dev dependency, that might make the developer life easy but not actually needed for the project to work, > npm install packagename --save-dev **What is package-lock.json?** While package.json holds the dependency information of the packages/modules we installed, package-lock.json will have the information of all the dependencies used by the modules we installed. It can be therefore called dependencies of dependencies. All the downloads will get saved in `node modules` folder. You don't need to put this into git because you can download it later if required. To download all the node modules specified in package.json and package-lock.json you need the command > npm install This downloads all the dependencies you specified as well as the dependencies used by the modules you downloaded looking at package.json and package-lock.json. #### Package versioning and updating Most of the packages/modules follow this convention of numbering the version of their package. Consider this example package, {% code title="package.json" %} ```javascript "dependencies" : { "slugify" : "^1.3.4" // can also be "~1.3.4" or "*1.3.4" } ``` {% endcode %} In version 1.3.4, 1 means major version, 3 means minor version, and 4 means patch version. If there are any bug fixes then the next release will be 1.3.5. If there are any features added but no breaking changes then the next version will be 1.4.0. In case of a major update where there might be breaking changes and our current code might no longer work upon update, then the next version would be 2.0.0. To know which packages are outdated in our code, we can use > npm outdated The ^ symbol means, we accept minor changes and bug fixes. Let's say the new release is 1.3.8 or 1.4.2 or 2.0.1 and we use the below command to update the package, > npm update Then, since we have**`^`**symbol that indicates we accept minor changes and bug fixes, the package gets updated to 1.4.2. In case we use **`~`**symbol that indicates we accept only bug fixes then the package gets updated to 1.3.8 In case we use**`*`** symbol that indicates we accept major changes (not recommended as it might break our current code) then the package gets updated to 2.0.1. ## Back-end and Front-end Terminologies ![MVC is a server-side rendered website](https://1944679227-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVEiPUp08kYt33g51v7%2F-MX-KFZ_WqPj6diiMcrB%2F-MX-TK0DVW8V6ipDvzUx%2Fimage.png?alt=media\&token=12b5ed1b-1a8f-4190-8db0-2333481b22b0) ![API powered website is called client-side rendered as front-end rendered seperately](https://1944679227-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVEiPUp08kYt33g51v7%2F-MX-KFZ_WqPj6diiMcrB%2F-MX-TeXMXFU1QPMpQhSP%2Fimage.png?alt=media\&token=d26d720b-5b77-4c51-957c-020bc000ee46) ## Node behind the scenes ![Node high-level diagram](https://1944679227-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVEiPUp08kYt33g51v7%2F-MX-XSAvkH-6XIYtZN4c%2F-MX-Xgpl8Ko_7PWBSx5f%2Fimage.png?alt=media\&token=02468a67-6e3e-48ff-9270-a618f5e7ca7e) Node runtime has several dependencies. It depends most importantly on libraries like V8 Js Engine and Lib-UV. V8 in the node understands JS and converts it to machine code but that alone is not enough to give Node the server-side functionality. That's when Lib-UV comes into the picture. Lib-UV is an open-source library with a strong focus on Asynchronous IO. Lib-UV is what gives node the access to the computer's underlying operation system, file-system, networking, and more. **Lib-UV** also implements two important features into the Node which are **Event-Loop and Thread-Pool.** Event-loop is for performing easy tasks such as executing call-backs whereas Thread-pool performs heavy tasks such as file-access or compression etc. V8 is written in both JS and C++ while Lib-UV is written in C++ alone. Node not only relies on V8 and Lib-UV but also on other libraries like * http-parser - for parsing http * c-ares - for DNS requests * openSSL - for cryptography * zLib - for compression ### Node Processes and Threads ![Processes and Threads](https://1944679227-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVEiPUp08kYt33g51v7%2F-MX-XvVi5m1PIQ_YEKkM%2F-MX-ZA1a9YEqDXc147IX%2Fimage.png?alt=media\&token=86eff0b9-77ed-43b6-8d86-37dd96461731) Two fundamental parts of the Node are Event-loop and Thread-pool. Node runs on a computer/server which is a C++ program that will start the **process** while running. {% hint style="info" %} We have a variable called **`process`** in the node which we are going to use {% endhint %} In that process, the **node runs on a single thread.** Thread is basically a sequence of instructions of a program. Note that multi-thread means multiple sequences/parts of the same program. Since Node is single-threaded, it runs on 1 thread for 10 users or 10 million users. When the program is **initialized**, * all the top-level code gets executed (code that is not inside any call-back function) * all the modules are required * all the call-backs are registered * after all this, the event-loop finally starts running. Event-loop is where most of the work is done in the app. They are the heart of the node app. But some tasks in the event-loop are too expensive to be executed in the event-loop as they may block the single thread. That's when the thread-pool comes to the rescue. * thread-pool is provided by the Lib-UV library. It gives 4 different threads that are completely separate from the main single thread. We can configure this up to 128 threads but usually, these 4 are enough. These 4 threads form the thread-pool and the **event-loop automatically off-loads or delegates heavy tasks to the thread-pool**. All this happens behind the scenes. The heavy tasks that might be delegated from the event-loop to the thread-pool might be File-System APIs, cryptography, and more. This way the event loop is not blocked. ### Event Loop ![Event-loop](https://1944679227-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVEiPUp08kYt33g51v7%2F-MX-fFZv4-DQy8jGusvf%2F-MX-fNS6Y3pCDWtW_d6s%2Fimage.png?alt=media\&token=0f9e817c-fc05-4370-b1db-30efa94ea57a) Event-loop runs all the code that is inside the call-back function (not top-level code). Node is built around call-back functions which are called events that are executed at later times like timer expiring, HTTP-requests, file reading, and so on. So we call Node the Event-driven architecture. Event-loop is the one that calls all the call-backs associated with the events that are registered. Simply put, Event-loop * receives events * calls their call-back functions * and offloads the more expensive tasks out of them to the thread pool **The order in which the events are called by event loop** There are 4 phases/ 4 callback queues of the event loop and each phase will be its own callbacks to process by the event loop 1. ![Event loop Phase 1 - Expired Timer Callbacks](https://1944679227-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVEiPUp08kYt33g51v7%2F-MX-fQ2Lu4VwFjVa8A-i%2F-MX0Anl37K5NjWfBMCl-%2Fimage.png?alt=media\&token=38b224cd-260d-4bb3-8348-b299ef67810b) **Expired timer callbacks -** If there are any timer events like `setTimout` whose timer gets expired, then these callbacks are first executed. If the event loop is in some other phase and the timer expires then the callback of that expired timer will be executed as soon as the event loop comes back to this phase. Callbacks are executed one by one and until all the call-backs are not completed the event loop will not go to the next phase 2\. ![](https://1944679227-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVEiPUp08kYt33g51v7%2F-MX0C9w5z9VMlpe4jngW%2F-MX0DIrqYd3qUjvX34Zr%2Fimage.png?alt=media\&token=4af3c108-ec69-4b18-881c-5061c44dfc8c) **I/O Polling and callbacks -** Eventloop looks for new Input/Output events that are ready to be processed and puts them into the callback queue. I/O generally refers to tasks like networking and file access. It is in this phase where 99% of our code gets executed. 3\. ![SetImmediate callbacks anc closed callbacks](https://1944679227-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVEiPUp08kYt33g51v7%2F-MX0Dd4jOAVRFaK95IZ9%2F-MX0E3SpsW7K57dGL9A4%2Fimage.png?alt=media\&token=53b6ce67-fc1e-4c12-80bb-39e5463237b3) **Set Immediate callbacks** - We can use this if we need our callbacks to be executed immediately after I/O polling callbacks. 4\. **Close callbacks** - All close events are processed for example, shutting down a web socket or web server. Besides these 4 callback queues there are other 2 queues * NEXTTICK() Queue * Microtask Queue ![Callback queues in each phase of event loop](https://1944679227-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVEiPUp08kYt33g51v7%2F-MX0FCd99hCFtTbM3h2r%2F-MX0FKa_M1mgcGitzhkf%2Fimage.png?alt=media\&token=39df26b7-a7a0-4d61-bc6a-d570b0631266) If there are any callbacks in one of these 2 queues to be processed, they will be executed right after the current phase of event loop instead of waiting for all 4 phases to be finished. In other words, after each of the 4 phases, it checks if there are any callback exists in one of these two queues and execute them if any. ![Multiple event loop ticks](https://1944679227-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVEiPUp08kYt33g51v7%2F-MX0GrApTJ-maKUK5sXC%2F-MX0HXorhwg1RVdyq2cd%2Fimage.png?alt=media\&token=9a4e8f93-0c59-4b31-a362-e4d685cbe0e1) Till now the 4 phases we saw was one event loop tick. Node checks whether there are any timers or I/O tasks running such as HTTP requests or file reads and so on and if there aren't any then it will exit the application. If there are then it will continue to the next tick. **How not to block the event loop since it's single-threaded?** ![Donts - not to block event loop ](https://1944679227-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVEiPUp08kYt33g51v7%2F-MX0I74hmjWnNpV7toFa%2F-MX0JJKe2LRc6gInB6Cw%2Fimage.png?alt=media\&token=78bc4ec8-80f6-4540-8c12-c9e6ef765dff) --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://sandeepamaranath.gitbook.io/notes/languages-and-frameworks/javascript/node-js.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.