To understand what happened to RethinkDB, you want to contrast it with MongoDB.
RethinkDB is amazing software and a real breakthrough. And the audience of deep thinking people who can really leverage the event model is ... limited. Today.
MongoDB is no breakthrough, but there are legions of people who just want to plug something together with PHP or Node. Given what they're doing, it works fine.
I sincerely hope Rethink can turn into a vibrant open source project and I'm willing to donate money to that. Given sufficient time, the world will learn to use it. And until then, it will grant superpowers to people willing to invest a little effort to understand and leverage it.
FWIW I found Rethink easier than Mongo, especially the query language. ReQL is actual JavaScript—Mongo's query language always seemed weird to me.
Mongo was "there" first and generated a lot of hype through sales and marketing. For a while and probably still now, a lot of Hackathons at college campuses are sponsored by Mongo and you can usually get a prize for using Mongo in your project.
Rethink never marketed themselves too much—it was one of those things you found out about in HN or from another dev.
I'm clueless about business stuff so I don't mean to say Rethink should've marketed heavily like Mongo did, but that's definitely a factor. I also think Rethink should've done Horizon earlier and offered paid hosting/deployment for it and Rethink users.
I admit I don't have a lot of experience with MongoDB, but RethinkDB was really easy to plug into Node. While we didn't support PHP officially, the PHP RethinkDB driver was maintained by RethinkDB's engineering manager, Daniel Mewes, and it looked pretty solid.
When I started at RethinkDB, it was before changefeeds were a thing. I always thought the "killer feature" was, well, ReQL. It had the qualities of an ORM/DB Builder in your native language of choice. There was a poster on the wall that said "Databases should be indistinguishable from magic"; I always thought that was a perfect pitch to the RethinkDB audience. I understand why they canted toward pitching the realtime aspects first and foremost after changefeeds came about, but I thought it had great power even if you weren't focused on realtime/event-driven aspects, and perhaps that should have been emphasized more.
In my opinion rethinkdb is super easy to get going and use. The low touch maintenance is what drew me originally.
However, learning to live without transactions takes some rethinking of the approaches taken. It's totally doable but it may not come easy if you're used to transactions in an RDBMS.
Of course in today's distributed multi service multi database world, learning to live without transactions is becoming more and more important anyway.
You can get transactions, but at a cost. Distributed transactions exist, they're just slow (and network-chatty). RethinkDB decided not to go that direction, I guess.
Exactly, they gave it a good go as a company but their burn rate was likely too high/too soon and differentiation insufficient (whether lack of actual novel features and/or communicating advantages in marketing). For example, a polar opposite of something like Couchbase which is used by giant enterprise shops to move away from Memcache and/or CouchDB.
It was also competing in a way Peter Thiel and Paul Graham warn about. Reinventing something without a vital quantum of improvement is like trying to juggle while climbing Half Dome.
I hope Rethink lives on as open source and adds MongoDB support. If they can hustle to encourage more people to try it by selling its obvious advantages, they have a slim but nonzero chance of rebooting.
Next time, gotta find more paying, alpha customers first and then start building. Then, try VC only after being capital-constrained when hiring/expansion/development is a blocker to realizing imminent profit. New category brands and segment-optimized products are riskier but more likely to be able to set up moats with defensible, competitive value.
For those not "in the know," what capability is that? Everything I hear implies a focus on ease of use: Easy replication/distribution, JS API, single number type (float64), usefulness when teaching novices, etc.
RethinkDB's niche is allowing a service to listen for changes to a dataset. As in, your Node application could listen for published events by Rethink, rather than your Node application long-polling the dataset for changes. "Real Time" data analytics can then be performed.
I've never done it, but I think it's possible to do this in Postgres as well using LISTEN and NOTIFY. You may want to do all of your database mutations through functions that ultimately call NOTIFY for this to work though. But uh, that's how you'd do it in postgraphql or postgrest anyway.
You can definitely use LISTEN and NOTIFY to watch for changes in Postgres. Here's a presentation I did on that last year, though oriented more towards web apps than analytics. https://www.youtube.com/watch?v=PsorlkAF83s
It is true. It was NOT true a year ago, but it is now. I pushed it hard, Aphyr pushed it hard, and the RethinkDB team worked very hard addressing every issue we found. That's why RethinkDB failing is so tragic.
Well it's easy to shard when you don't support joins or foreign key constraints.
Btw, as a person who works on Firebase, perhaps you can answer this question that has been bothering me about it. There's no way to model a foreign key relationship that's traversable in both ways without trusting the client application to update both directions, right? Isn't that a bit of a dealbreaker for anything other than a toy app? I would assume if you wanted to make something that's actually reliable, you'd have to run all your mutations through your own personal server instead of letting clients mutate Firebase directly, right?
You can use it as the mobile device connector as clientside auth, push, latency compensation, presense and offline persistence are big features on their own. For sensitive joins, have a server side joiner with administrative privileges. Still, it's work that should be easier I agree. Firebase's strength is not what other DBs can do, but what they cannot.
I'm curious what Firebase users do in real life. Do people realize this is a problem?
I can imagine making an app like that where all mutations go through my server and reads go through Firebase. How would this interact with the offline persistence feature though? I suppose you could allow the client to do some simple mutations like editing text fields. For the mutations that must go through the server, though, perhaps you could do optimistic local updates?
The latency compensation through the sdk is optimistic writes. If you shortcut to server, you have to reimplement that so that's not a good solution for many cases. It would confuse the offline persistence too.
Instead, your server should listen to the Firebase, rather than have writes go a separate path (which loses lots of useful features). I think the term now is a client-database-server architecture.
it's not actually a problem for 90% of apps. It's usually in the users best interest not to break their relationships in the db. Think chat. Think info services like Pokemon. Cross table transactions are relative minority in the space of useful apps that's can make money. For those more complex domains, you need technical know how, and Firebase is a useful system component in a larger architecture.
As soon as you want to view something through a different direction you run into this problem. Sure, you can embed chat messages into a conversation, but what if you want to look up every message by a particular user?
I would be surprised to find that 90% of apps don't have this kind of requirement. Pokemon is easy because that data is static.
Viewing all posts by a user is solved with classic denormalization. You maintain a separate index of post by user and do two writes per chat msg. It's a case where a user has no incentive not do the second write so it be fine to implement clientside. If however it was business critical, you would have a server side process listen to all chat messages and write the user to message index with the admin account. That index would essentially be eventually consistent.
[edit: this is an irrelevant remark] RethinkDB supports joins: https://www.rethinkdb.com/docs/table-joins/
"Like many traditional database systems, RethinkDB supports JOIN commands to combine data from multiple tables. In RethinkDB joins are automatically distributed—a join command is automatically sent to the appropriate nodes across the cluster, the relevant data is combined, and the final result is presented to the user."
Rethink joins aren't consistent (in fact, no Rethink multikey transaction guarantees any degree of consistency), so even though the original conversation was about Firebase IMO it is just as applicable to Rethink.
I think Firebase is a closed source program that can only be used via "the cloud", i.e., via a web application. It's not something you can run yourself. In contrast, RethinkDB+Horizon would have soon provided similar functionality to Firebase, but with the option of self hosting.
Just going out on a limb here, and I know neither MongoDB or RethinkDB (nor the companies behind them) very well.
But, from what you're saying it almost sounds like that RethinkDB would be a great companion to MongoDB. And if so, that the company behind MongoDB might be a great custodian for RethinkDB.
All these comments are starting to make me feel stupid. I picked mongo for a side project, and it's been really nice so far. I have a hard time imagining how one would use it for a complicated and/or large schema though. I assume that is just my newness to it.
Don't feel bad for not understanding how to make it work for a big project, it just doesn't. You can try make it work for a big project, but at that point you are just implementing database level code in your application.
Better to just use something like PostgreSQL. Contrary to popular belief PostgreSQL is actually a better document store than MongoDB via it's jsonb type, operators and GIST/GIN indexing.
I'm not entirely to sure if I understand your comment, however, what I mean is;
The company behind MongoDB is a database company so they probably have some sort of affinity and appreciation for a product like RethinkDB, and, they are commercially successful (or at least, it appears to be that way).
Just because, hypothetically, MongoDB could be a custodian for RethinkDB don't mean that all of a sudden RethinkDB becomes MongoDB.
As Paul pointed out, both solutions seem to be two different products covering two different use cases and audiences and from what I can imagine, they might actually complement each other.
MongoDB was much more breakthrough than this little thing called rethinkdb. Hell, people even create the term NoSQL because of it. Maybe the failure of this project came from this dellusional thinking I keep reading in all posts about their failure.
MongoDB coined the term maybe, along with webscale and a whole bunch of meme.
But "NoSQL" atleast as it'd defined has been around forever, it predates SQL because everything before then was trying to work out how to store bits on disk as efficiently as possible before relational theory really found it's way into storage systems and revolutionized things by focussing on modelling.
Even putting aside that fact it was a terrible database then, it's a slightly less terrible database now and everything it can do is done better by other databases.
RethinkDB is amazing software and a real breakthrough. And the audience of deep thinking people who can really leverage the event model is ... limited. Today.
MongoDB is no breakthrough, but there are legions of people who just want to plug something together with PHP or Node. Given what they're doing, it works fine.
I sincerely hope Rethink can turn into a vibrant open source project and I'm willing to donate money to that. Given sufficient time, the world will learn to use it. And until then, it will grant superpowers to people willing to invest a little effort to understand and leverage it.