What I'm missing in these articles is a performance comparison. All WASMed tools I've tried were really cool proofs of concept, but the performance was always lacking at the very least.
I see several languages moving towards more and more WASM but on a technical level I don't see the benefit of WASM over something like Firecracker. Docker and other sandboxes have to deal with shared kernels and all the risks associated with that, but leveraging virtual machines instead solves that issue. There are already proof of concept implementations to replace Docker with VMs as a virtualisation layer, so I wonder if it wouldn't be better to invest time in getting those wrappers completely up and running rather than coming up with essentially "Java but we also emulate the OS".
Until WASM advocates start including benchmarks in their blogs, I'll keep watching this stuff from a distance.
No cold-starts means no overhead of starting a process to answer a request like most container-based serverless environments (without having to keep pre-warmed instances which kind of defeats the purpose) A couple references regarding cold-starts and performance in serverless environments.
Are we so what's-old-is-new-again as to be re-inventing fast-cgi at this point? And why are we pretending this has anything to do with WASM instead of just how your API/service is designed?
FastCGI reuses the same process for multiple requests. As I understand it wasmtime now supports very fast startup so you can use a new instance per request (avoiding the risk of inter-request bugs) with very low overhead (5 microseconds on their benchmark https://bytecodealliance.org/articles/wasmtime-10-performanc....)
With Firecracker I believe snapshot restore time is around 2-3ms. In my tests wasmtime ran about 50% the speed of native so depending on your workload it might still be faster for short running jobs where the startup time dominates. (Wasmer was maybe 80-90% of native speed but I don't know their startup times.)
> As I understand it wasmtime now supports very fast startup
WASM startup isn't going to be any faster than native code startup. It's going to be strictly worse if anything thanks to the JIT, although you can AOT that to native and then just restore parity with native code.
Which just gets back to the speed of your startup depends on what your startup does.
From what I understand, wasmtime's fast startup is conceptually similar to forking a process per instantiation, but much faster since it uses lazy initialization and has fewer operating system resources to setup.
But I think wasmtime can always be faster to instantiate since the guarantees provided by the runtime allow it to safely reset and reuse instantiations:
"We implemented an “instance allocator” in Wasmtime that makes use of this copy-on-write (CoW) technique for very fast instantiations. It also uses a Linux syscall known as madvise to quickly “reset” the page mappings back to the original read-only heap image, so we can reuse the same mappings over and over when the same Wasm program is re-instantiated many times. (One might imagine this would be the case in a server serving many requests, for example!)"
What approach would be faster that provides similar isolation? It certainly seems a lot faster than fork and seemingly faster than the on-demand-fork I mentioned elsewhere.
That’s basically the history of computing! There are some intrinsic advantages to using Wasm vs VMs or containers for certain scenarios, like serverless. That’s very similar to what’s going on with cloudflare workers and V8 isolates. It’s not one size fits all by any means, but it is certainly really good for many scenarios where containers are not
Some applications can benefit a a lot from CPU specific instructions combinations which are not available to wasm
(with available I mean implicitly, i.e. your wasm code gets compiled to them).
Luckily for a lot of use-cases this doesn't matter much(1) and some degree of SIMD support is often(2) available.
(1): Without micro-optimizations which most times aren't done as due to their maintenance/development cost.
(2): I'm not quite up to date. I think 128bit SIMD is available in most (all?) relevant WASI runtimes and at least some browsers.
yeah I have known about this for a while, but no one I've spoken to personally believes me. to them WASM is pure win and there are no downsides.
when I mention performance, they kinda waffle a bit, saying "CPU is cheap" or something similar, and they start to show a hint of understanding when I say that cloud resources are billed by unit of CPU time, and by amount of RAM used. then I say that our mutual employer invokes lambdas hundreds of trillions of times per year and I think they briefly understand before being caught up in "new stuff is awesome" technology fetishism again.
it's exhausting.
everyone should live overseas for a couple years because it changes how you view the world... everyone should be a game developer for a couple years as well, because you will quickly notice just how unbelievably slow modern software is. more people need to see that.
security is important! portability is important! other things are important, always, and when you gain a sense of just how slow software is today in comparison to how unbelievably fast modern hardware is, it becomes very hard for me to think positively of anything that lowers performance further for almost any reason.
I wouldn't call 50% loss "pretty bad". I mean sure, it's not great, but if you were to go from Rust to C# or Java you would most likely see a similar loss.
That would depend on the type of code you write. Heavily allocating code can be very fast in case of the JVM, and you can’t always avoid dynamic allocations/arenas are not always a solution.
The biggest missing thing (for production) is observability.
Look at old-good JVM. It has tons of tools to analyze and understand behavior of your production system. You could have thread dumps (stack traces of all existing threads) at any moment with negligible performance impact, you could dump heap and analyze it off-site, you could have tons of metrics, about each dark corner of mutexes, GC process, about JIT, including, if you need it, generated native code!
Many of these thing you could get on production, not in sand-box.
If you system behaves strangely, live-locks, consume more memory than you think it should, tharsh GC, you name it, you have all tools to understand what is wrong, find bugs or mis-configurations, etc.
With all these new-and-shiny WASM and not-so-shiny JS VMs you mostly in the dark now. Service become unresponsive? latency goes to the roof? Only thing you could do - restart.
It is not property of WASM per se, but this infrastructure is too immature now, comparing to 25+ year old technology.
This a 1000 times. People like to hate on Java but when there are problems to diagnose on production systems it is second to none.
But from my experience most people don't know these tools even exist so the only thing they do is restarting and guessing where the problem might be if it persists.
Flight recorder is a godsend and I've not seen it's equal in any other language/ecosystem.
Any JVM anywhere can answer the question "why am I running slow" with a quick run of flight recorder. Memory, CPU, socket time, GC impact, TLB, thread dumps, etc. It's all there in one file that imposes something like a 1->2% performance impact if you run it constantly.
Completely agree. Observability and debugging are some areas where the ecosystem is quite immature or inexistent. My take is that wasm is more or less where the web was in 97-98 Lots of excitement and possibilities but also lots of technical challenges and experimentation
Threading support is already implemented in some browsers and well on its way to standardization https://webassembly.org/roadmap/
Should address that concern. However there is another way of looking at performance and is in the context of serverless where typically single threaded performance is inportant, as well as cold start time etc and that’s why Wasm is popular in that scenario
“Threading support” in this sense is a bit of a misnomer, it’s really support for thread-safe memory constructs. Creating threads is left up to the runtime. On the web, this is done with WebWorkers, but on the server side I don’t think there is yet a standard way to do it supported by major runtimes.
> Creating threads is left up to the runtime. On the web, this is done with WebWorkers
WebWorkers don't give you multi-threading behaviors (heaps/address spaces are not shared). WebWorkers would be how you launch a new process, but there's still otherwise no way to make a thread (nor even a fork() equivalent for that matter).
That only shares one allocation (like shared memory does in regular multi-process scenarios), but you still can't share the address space or even any object heaps at all. Like it's not possible to allocate javascript objects out of a SharedArrayBuffer such that you could pretend you had a shared address space by sticking everything in that.
As in, SharedArrayBuffer is equivalent to shm_open. Which means it's not even that good as a shared memory construct as it's missing all the protection enforcement of memfd (or Android's ashmem)
WASM stores everything in an array buffer, it doesn't use JavaScript objects because it's not JavaScript (though there are starting to be features that allow it to interoperate with JS objects). If it didn't store everything in a big memory array then it wouldn't really work because C assumes that
There is a SharedMemoryBuffer, but it’s a web platform thing, not available in out-of-browser runtimes like wasmtime or wasmer or wasmedge (which Docker uses).
but you can have threaded wasm code in any evergreen browser since a more then a year as far as I'm aware
Basically the trick is that you use multiple web-workers with the same WAS program and the same shared buffer. Then you also add some JS glue code to coordinate which thread is the main thread and which threads you use as thread pool (e.g. in rust/wasm with rayon you can set it up as worker pool).
Now there are some drawbacks (last time when I used it, might have gotten better):
- threads are started/managed from outside (so don't expect any kind of "spawn" function to work, generally spawning new threads is non-trivial and so is (properly) cleaning up old threads, through if you need a fixed worker pool it's all fine)
- there where some limitations wrt. threading/synchronization which made certain usages of concurrency rather slow (through many where fine)
- no "synchronized" operations mustn't be called from WASM code called by the main JS thread. This means in most situations you need to pass data to web workers and then to WASM (instead of e.g. passing it to WASM and then using in wasm a mpmc-channel to pass it to the worker pool). There are some optimizations around passing pointers as numbers to/from the web-workers but it's limited and not nice. Or at least wasn't ~a year ago.
- bugs in Safari leading to strange crashed for code running in all other browsers nicely under unclear and non-debuggable circumstances (probably fixed, I hope)
Anyway all in all using rust->wasm with rayon and a thread pool was already surprisingly viable ~1 year ago.
Those are really more like separate processes. There's no ability to do a shared heap in browser JS, meaning it functionally doesn't have threads.
Whether or not workers are actually implemented as threads or processes in the runtime is irrelevant. As far as the JS code itself is concerned & what you can do with it, browser JS is lacking multi-threading. There's just no way to do a shared heap, and that is the biggest defining difference between a process and a thread.
The JavaScript heaps are distinct, that's true, but there is a single shared wasm heap which is used from multiple threads. That is enough to implement the pthreads API.
Applications like Photoshop and Google Earth use ptheads on the Web so their compiled C++ is multithreaded, very similar to how it would run natively, and with similar responsiveness and throughput speedups. Though there are some limitations too, see
That's not really threading by the definition normally used in other languages. You can't allocate JS objects or structures and read/write them from multiple threads at once. JS is an inherently thread unsafe language and likely always will be.
Not exactly what you've asked for, but I did a "dummy" benchmark of Rust vs WASM on my blog[0]. WASM is impressive technology indeed, and it will be interesting to see whether it will get into the mainstream of software engineering.
> but the performance was always lacking at the very least.
This is an easy engineering problem which will be solved when there's enough motivation and engineers working on it.
Increasing adoption is more of a business problem though, and it is unclear if performance is the bottleneck here.
If you have a wasm product that has to be as fast as native code, the solution is to find compiler engineers (or a company that specializes in this) who will solve this for your situation.
I see several languages moving towards more and more WASM but on a technical level I don't see the benefit of WASM over something like Firecracker. Docker and other sandboxes have to deal with shared kernels and all the risks associated with that, but leveraging virtual machines instead solves that issue. There are already proof of concept implementations to replace Docker with VMs as a virtualisation layer, so I wonder if it wouldn't be better to invest time in getting those wrappers completely up and running rather than coming up with essentially "Java but we also emulate the OS".
Until WASM advocates start including benchmarks in their blogs, I'll keep watching this stuff from a distance.