heh heh our p2p graph processing lib is now shitting events across the address space. we shall have our distributed data and we shall have our distributed compute at human scales that do not require a career's worth of sysadmin knowledge or an amazon's worth of infrastructure to use.
@mauve most of the work has gone into "how do you allow arbitrary graphs but also make that tractable to think about and use at the low end as if it was just a normal function but at the high end like a big bazooka of interconnected graphs"
@mauve the test case pipeline definitions are probably the best entrypoint since we are writing this on deadline lol
@mauve yes, it's not very impressive yet, but we're just moving from the baseline design of 'what is an ergonomic way to describe processing graphs' with sync execution to distributed execution, starting just having a local pool of nodes, but it immediately extends to allowing the nodes to be on multiple machines (just want to have a bit of auth before doing something that someone might run on an open tcp port)
https://github.com/miniscope/noob/pull/54
it's written to be composable graphs, so rather than having one gigantic graph that has to have all contingencies, you can switch between a bunch of reusable things that can nest tidily/deduplicate/have separation of concerns