Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

ZMQ presents its own set of challenges, because it's typically implemented as a linked-in native library, and this presents some interesting safety problems. In a previous life, we tried out ZMQ as a replacement to RabbitMQ for a use-case that we didn't actually need long-term persistence for (real-time notifications), but we found Java, and Python ZMQ to provide exciting, hard-to-debug memory leaks, as well as plain segfaults. But, for some time I've been thinking about this, because having a smart client library does allow you to do really neat things efficiently (See: Fast Paxos: http://msr-waypoint.com/pubs/64624/tr-2005-112.pdf, Chubby: http://static.googleusercontent.com/media/research.google.co...). I think the ideal model is to split up responsibilities in a distributed system between nodes that are smart clients, and the rest of the system, and in this, linked-in drivers can continue to utilize simple protocols like RPC + Protocol Buffers, and the actual system can use more complex, higher-level semantics, like dual-dispatch to smooth over latency. -- For whatever reason, people are more comfortable with running a totally foreign, binary library in their same process / memory space, as opposed to running a binary and depending on it. -- I realize that operationally, it's somewhat more complex to do the later, but that's a question about maturity of methods. We've seen the Go community adopt this approach somewhat with things like the Packer plugin architecture (https://www.packer.io/docs/extend/plugins.html).


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: