Stupid question. Does this mean that a web server written in node is running single-threaded? I know that there are callbacks and promises to hand over control between different execution flows. But doesn't running on a single thread put an upper limit on the amount of work a server can handle?
And, if the solution is to spin up more servers with access to the same database, doesn't that mean that we are now having multiple threads accessing the database concurrently? Much like, say, Python Django?
> Does this mean that a web server written in node is running single-threaded?
Yes.
> But doesn't running on a single thread put an upper limit on the amount of work a server can handle?
Is not about how much work it can handle, it's about how much it can offload. Async servers can handle a much greater volume of I/O bounded tasks. So it can handle more connections. When the task is CPU bounded you can either create a thread (which does not really scale well) or offload it to some other servers that can scale horizontally (i.e doing micro services)
> And, if the solution is to spin up more servers with access to the same database, doesn't that mean that we are now having multiple threads accessing the database concurrently? Much like, say, Python Django?
Yes, to take advantage of all CPU cores you have to create more server instances. But why would they talk to the same database instance? It could be a replica or a shard. You can even have a pool of shards connections per server instance.
The problem with offloading to a thread is that you can't structurally share data with another thread. So any copying of e.g. arguments can be very expensive, and awkward/inconvenient as well.
If the other thread/process is running on a different machine (or you want to keep that option open) that's what you're going to have to do anyway though.
>Sometimes you want to use a thread to keep the CPU from blocking (e.g. long database operations).
On the contrary. That's the prototypical use case for non-blocking event based IO. No threads needed.
>Also, sometimes you want to use the CPU to its full capabilities.
You can use all CPUs by using multiple processes. That's not an issue.
Threads are useful when you want to run multiple algorithms in parallel on the same bigish in-memory data structure, especially one that has a lot of pointers.
Something like an in-memory graph database or a desktop application that lets you work with huge files in-memory, or even complex user interfaces. For instance, I'm not convinced that the cross process bridging that React Native has to do is a great idea.
So yes there are use cases where threads are very beneficial. But on the server side it's essentially the database/analytics systems themselves, not the code that accesses them.
Database access, likely the most common I/O operation from a web server, is certainly running in another process (or on another machine) - I'm not sure that "if" is as big as you suggest.
> > Does this mean that a web server written in node is running single-threaded?
> Yes.
Just to expand, this (like most things) is a simplification, as I'm sure nitely is just being too brief to explain. It's true for Hello World, and a little further, but real-world web servers in non-trivial contexts typically utilise techniques like clustering, workers, and other ways of delegating tasks to external processes.
>Yes, to take advantage of all CPU cores you have to create more server instances. But why would they talk to the same database instance?
Why not? Unless you've exceeded the capacity of a single DB and have a real use for sharding etc, it would not make sense to have difference DBs (+ replication overhead) for different Node processes.
The parent is asking how comes async I/O can handle higher volumes of requests given it's single threaded and even if it does how comes the database is still not the bottleneck. I answered both of these questions.
Yes, and I had an objection with the answer to the second question, that it might leave the impression to the parent that sharding or replication and pooling is required to have good DB IO performance with multiple Node processes -- when in practice it might or might not be an issue.
You can have a 12-processes node cluster and still not need a second db.
>Does this mean that a web server written in node is running single-threaded?
Yes -- node is by default a single threaded, single process server.
>But doesn't running on a single thread put an upper limit on the amount of work a server can handle?
Not any more than this is the case with Python, PHP, Rails, etc -- which also don't do multi-threaded (or don't do it well and not by default), and which on top of that don't have asynchronous capabilities (again, not by default) and are even less performant than a single Node app.
Which is why a simple Node running with its single process and single threaded execution can e.g. beat a Python server with two dozens of workers (e.g. gunicorn) in handling simultaneous connections (assuming Node code is properly async in the most part).
>And, if the solution is to spin up more servers with access to the same database, doesn't that mean that we are now having multiple threads accessing the database concurrently?
Databases take care of serialization of multiple queries for you -- and for more complex cases (with or without transactions for fuller control).
>Not any more than this is the case with Python, PHP, Rails, etc -- which also don't do multi-threaded (or don't do it well and not by default), and which on top of that don't have asynchronous capabilities (again, not by default) and are even less performant than a single Node app.
uWSGI makes running a threaded or multi-proc python webapp trivially easy (and as of Python v3.6 async comes as standard)...
>Which is why a simple Node running with its single process and single threaded execution can e.g. beat a Python server with two dozens of workers (e.g. gunicorn) in handling simultaneous connections (assuming Node code is properly async in the most part).
Node can beat a threaded python app for sheer volume of concurrent connections to clients, yes. But for a lot of traditional backend work (e.g. talking to a DB) async is no faster (indeed it's often slower) than a threaded approach.
Node (or async in general) is great for terminating inbound client connections; talking to local, in-memory caches or making backend calls to remote, non-local REST services.
For making local DB connections or doing any CPU work (e.g. parsing XML docs returned from an API) single-threaded async rarely yields better performance over threads/multi-proc. A good illustration of this is pgbouncer (async on the client facing end; threaded - i think? - on the db facing side).
Basically, all node really does is reduce the number of front end app servers you need to serve X incoming client connections. Just because node can handle a high concurrent connection count doesn't mean the rest of your backend services can. Regardless of whether connections originate from a single node instance or a large fleet of php/python/rails servers; you still need reverse proxies like pgbouncer/haproxy/twemproxy/squid to manage and shape those connections before they get to things like your DB or internal micro-service APIs.
Because node is single-threaded you also need to keep a very close eye on any CPU bound activity to avoid blocking all your connections. This is not always obvious and can crop up in unexpected ways (see: https://news.ycombinator.com/item?id=15477419)
>I thought the difference was in fact that PHP fires up new threads for each connection.
PHP doesn't get to decide what happens for each new connection. That's determined upstream, and there's a lot of different pieces of software people choose to do that. Could be directly a webserver like apache's mod_php, or something like fastcgi, php-fpm, etc.
All of those front-ends chose to implement a model of a php process pool. Multiple processes, each with a single php interpreter running. Incoming connections are sent to a process in the pool. So, if the pool is 5 processes, and you get a 6th concurrent connection, that one waits.
Node is written using libev, which under the hood uses system calls like select, poll, epoll etc (whatever is fastest for the combination of the IO task at hand and the current kernel) to provide an abstraction called an event loop.
It acts as an intermediary between your application code and the kernel, notifying you as soon as some IO action was completed by the kernel via an event callback.
This notification is provided to you as a single queue of events; the event loop is hence single threaded.
The important thing this facilitates is making it easier to reason about your application code, since you can be assured that only one of hundreds of callbacks in your application can be running at any one time.
Does it put an upper limit on the amount of work a single server process can handle? Depending on your use case, possibly yes. NodeJS shines when most of the work each call to your server involves mostly IO, i.e. are IO bound tasks.
If on the other hand, if any of the calls are CPU bound (some complex mathematical calculations say), you're probably going to hit this limit much sooner.
Even in cases where you have to run CPU bound tasks, it is far 'simpler' to offload these to an entirely different process that uses some sort of IPC to run the calculations and communicate the completed results back to your main server process, rather than spinning up a new thread in your server process to handle those CPU-bound tasks.
Does it now not possibly involve multiple threads accessing the database concurrently? Well, yes. Databases though are rather good at handling races. Most mainstream databases provide some sort of locking mechanism to make sure that some shared record cannot be erroneously modified by two processes at the same time.
If database locks are not for you, there are other solutions possible for these kinds of issues as well. By implementing a proper message queue, you can filter out calls that access this shared record into a separate synchronous queue, while all the other calls can be made to the DB simultaneously.
Why bother with mutex locks and races in your application code when other people (authors of libev/databases) are willing to do it for you?
"In our tests, however, we found that React’s renderToString() takes quite a while to execute — and since renderToString() is synchronous, the server is blocked while it runs. Every server side render executes renderToString() to build the HTML that the application server will be sending to the browser."
"The average renderToString()call with this configuration took 153.80 ms."
> Does this mean that a web server written in node is running single-threaded?
JavaScript is single threaded. Node.js is not.
Node executes the entire JavaScript code in a single thread. However, the I/O requests dispatched by the JavaScript code (file I/O, network I/O, db I/O) can be executed in separate threads.
That is why Node is efficient for applications with lot of I/O (typical web apps), and can still provide a default thread safety for global objects.
But I think, it will not work efficiently for number-crunching CPU intensive applications.
NodeJS userland is effectively single-threaded, but NodeJS is not single-threaded. NodeJS outsources some event loop scheduling to libuv, and libuv is multi-threaded (by default, 4 threads, but this can be configured. See:
I'm a noob to concurrency, but my current understanding is that a "thread" is "virtual". Fundamentally, your CPU has finite cores so no matter how many threads you spawn your (individual) CPU cores will resolve the threads into a synchronous set of instructions to execute.
Having a single threaded (asynchronous) server simply displaces the abstraction of threads.
Please feel free to correct if I'm wrong about this...
No, async & single-threaded is not an abstract replacement for threads. Single-threaded async code can only use 1 CPU core at a time. For example, JS running in the browser without web workers. Multi-threaded uses multiple cores simultaneously.
Whether a thread is "virtual" - I don't know what that means to you, but threads are a primitive the OS provides. Some CPUs have hardware support for threads, and some don't. So, I suspect they are less virtual than you're thinking.
Yes, but you need at least as many thread or processes as you have cores to take advantage of them.
A multicore cpu is wasted on single threaded program unless you are lucky enough that your problem is so embarassingly parallel that can be handled by N indipendent the processes
Node doesn't eliminate parallelism, it just pushes it outside of JavaScript. A node-based system is parallel at multiple levels: within the process itself library calls can run in parallel, and like you suggest, you have many parallel processes running instances of the same code.
And, if the solution is to spin up more servers with access to the same database, doesn't that mean that we are now having multiple threads accessing the database concurrently? Much like, say, Python Django?