I think what I want instead of apps is a standard datamodel in a graph database with live queries I can tie to table views.

@yala @mauve ^^^ logseq is great! I do like the datomic sorta data model.

@pry The Datalog query language is just so far beyond everything I had ever used. But advanced queries are very flexible and I'm always learning a new thing when trying to write one.

@mauve

@yala @pry I looove datalog. Have you seen any good impls out there other than datomic? I also don't think I've seen any "live query" options out there yet.

I was thinking wrapping stuff in HTTP with an EvenSource endpoint would get pretty far towards easy to use interfaces.

@mauve @yala github.com/tonsky/datascript

this has always looked pretty cool to me! but rlly this and datomic are all I've seen. I'm a big datalog fan.

lemme know if u know of any others!

I also kinda want to implement a datomic like db myself tbh

@pry @yala I think the work @quinn was doing on Rhizomatic seems relevant. Unsure what the state of it is.

youtube.com/watch?v=vkMXbk7Pn_

@mauve @yala @quinn oh that's interesting. I'd be interested in learning what it has to do with decentralized queries and stuff. I suppose I'm a little more interested in implementing the datalog data model efficiently on a single node

@pry @yala @quinn IIRC the datamodel and execution is similar to datalog? I'm mostly interested in "local-first" daabases so you can merge data from multiple sources as you network connectivity grows with local data being available.

Datalog over might also be cool to do some day.

@mauve @yala @quinn ooh I actually really became interested in distributed systems because of learning about local first software and CRDTs. I really want to do work making CRDTs more practical for more use cases. I think it's rlly cool to combine the eventually consistency of CRDTs with maybe slightly stronger snapshot consistency of some other sort of protocol. CRDT truncation also seems important?

@pry Oh! Have you seen my blog post on peer to peer databases by any chance? blog.mauve.moe/posts/peer-to-p

I'm less into realtime document sync and more into optimizing for sparse replication of views. CRDTs are cool but they often share the same issues as oplogs. I think the latest version of automerge is pretty speedy though.

@mauve ok this is a really really cool blog post. I definitely am going to read up more on prolly trees.

I think I don't fully understand how these can help in the world of p2p data management yet

@pry Right now most p2p data things use some variation on an "oplog". As your data grows it gets slower and slower for a peer to get the state of the data and it makes it hard to do "sparse replication" of just the data you want.

Using an indexed database as a building block means you can replicate just the data you need for a particular db query. This means you applications get more speedy and you use less data overall. You can still do full replication in the background.

@pry The other cool part is that compared to other DBs like hyperbee from holepunch.to Prolly Tree based databases can be determenistically merged without much hassle so you can have communities merging datasets into larger and larger indexes without having to fully replicate all the data, just the parts at boundries.

@mauve ooh interesting. do you know any good resources for learning the merging semantics of a prolly tree? wasn't able to find too much about them. I've also been pretty interested in Merkle Search Trees and once I understand what a prolly tree is, I'll be able to compare them

Follow

@pry Merkle Search Trees are way more complicated to deal with and implement in my experience. I wrote up a spec for the prolly tree impl I use here: github.com/ipld/ipld/blob/776b

It talks about how merging works.

The tl;dr is you traverse the merkle tree and where you see a diff, you add in keys from there, you can quickly skip places where the data is the same. You could treat it like an eventually consistent system in that way. Also could merge just subsets of the tree according to the keyspace

· · Web · 1 · 0 · 2

@mauve ah thank you!! really super cool work you are doing! this is exactly the sort of work I'd one day like to be involved with

Sign in to participate in the conversation
Mauvestodon

Escape ship from centralized social media run by Mauve.