My matrix server is like a hungry beast. I feed it CPU and RAM and it just keeps asking for more. Then again it is handling thousands of chat rooms and accounts at this point and needs to calculate a huge diff to send to my client every time I load. :P
Heads up: Matrix like all append only log systems sucks when you're not always online and loading data.
@mauve I'm going to need to know more about this append-only-log problem for Matrix. In other words, your server wants to send you the 100% update diff all at once? This is instead of X versions for each room based on usage or something? :x
I'm now understanding the impact that offline-only systems get.
@Filene So the main issue is with applications that pull oplogs from remote systems in order to calculate local state in a local index. In the case of matrix your client pulls all your new messages at boot and queries the state of all your rooms (and I think users in those rooms?). This is fine when your client is always online and is just listening for updates. It's also fine when you're in a small number of rooms. I am in hundreds of rooms with thousands of new messages.
@Filene The solution is to expose a query layer to pull data for the specific view you're displaying and to sync any extra data on an as-needed or background basis. This is why I think p2p apps should be building on indexed databases as a building block and performing queries on remote datasets instead of "catching up" with all the remote data before showing it to the user.
Long read: https://blog.mauve.moe/posts/peer-to-peer-databases