Title is click bait-y. This has next to nothing to do with kernel interfaces and is all about network tuning and encapsulation. Not sure why the authors went with the title as networking is interesting enough.
Also, the "slow" things about kernel interfaces (if you aren't doing IO which is nearly always the slowest thing) usually isn't a given syscall, it's the transition from user to kernel space and back. Lots of stuff going on such as flushing cache and buffers due to security concerns these days.
We certainly didn't try to make it click-baity. The point of the title is that people assume that Tailscale was slower than kernel wireguard because the kernel must be intrinsically faster somehow. The point of the blog post is to say, "no, code can run fast on either side... you just have to cross the boundary less." The blog post is all about how we then cross that boundary less, using a less obvious kernel interface.
I would have titled it something like "Userspace is slow if you do lots of context switches/userspace transitions" (if I understand the point of the post)... but that's sort of been known for a while now. I think the novelty of this comes mainly from the inversion of control in your design, and the post explicitly points out that it's likely every single performance improvement in userspace could be equalled by kernelspace, and likely, exceeded by kernelspace.
What's really crazy is that we're talking about userspace and 10Gbit, which shows that CPUs, busses, and the interface protocols have all been scaling well with interface speeds!
Personally I don't ever increase MTU, even if there's a significant performance win, since I prefer to not place our oncall in a situation where they have to debug an outage due to MTU incompability.
Just some feedback, that's not what I expected from the title and I would agree with the previous poster that the title is a little (quite minor though) clickbaity.
The purpose of the title is to summarize, enabling the reader to decide if the article is relevant and interesting to him. If the title presents a situation that seems more dire, urgent, or relevant then the article, then it is written to entice click through rates, even if it makes sense after having read the article.
Thanks for the clarifying reply. I thought most folks who cared knew it was about context switches and not speed on one side vs the other. Now I'm really interested to read the full article.
I disagree. Two main points of the article are "nothing is inherently slow about doing stuff in userland (as shown by the fact that we made a fast implementation)" and "kennel interfaces, e.g. particular methods of boundary crossing, can be (as shown by the fact that the way they made it faster was in large part by doing the boundary crossings better)".
The title gave me a reasonably decent idea of what to expect, and the article delivered.
One of the authors here: What I was going for with the title is that singular read/write switching (the before case) is very slow (for packet sized work), and batching (~ >=64kb) is much faster - it's about amortizing the cost of the transition, as you rightly point out. That's the point the title is making - some interfaces do not provide the ability to amortize that cost, others do!
I mostly just find the title fun. But I guess my main issue with clickbait is more the bait-and-switch when the result doesn’t live up to expectations. I felt like I had a reasonable idea that what was inside wouldn’t be a waste of time from the words in the title. There are plenty of articles posted on this site with dry titles that don’t match the body, which seems worse to me.
Also, the "slow" things about kernel interfaces (if you aren't doing IO which is nearly always the slowest thing) usually isn't a given syscall, it's the transition from user to kernel space and back. Lots of stuff going on such as flushing cache and buffers due to security concerns these days.