Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I would store the reservation date and truck length as separate data objects in the batch layer. You would tie them together with the reservation id. It isn't a big deal to do those joins since the batch system is good at that kind of large-scale computation and you've already accepted high latency in the batch layer.

I wrote an article about graph schemas on the batch layer here: http://nathanmarz.com/blog/thrift-graphs-strong-flexible-sch...

The realtime db only accounts for the last few hours of data. Unless you're doing a query that only needs the last few hours of data, you need to do a merge between the batch view and the realtime view. We constantly flush old data out of our realtime db so as to keep it small and fast.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: