Anyone know of tools kind of like but instead of keeping the AST in memory they do streaming parse / search on the fly? I'm not a huge fan of masssive memory use and it feels like we're leaving performance on the table by parsing entire files/folders instead of just enough to get to what you want.

@mauve probably unrelated, but I'd love something similar for local LLM's too..something which does not require to keep it all in memory..restore ollama state-snapshot via swapfile etc..

Follow

@lvk ooo yeah. I wonder how hard it'd be. Was most of the state just the context and kv cache? Sounds like not a lot. For some reason I though Ollama had some sort of "continue from the last call" feature already but I might be hallucinatinf

Sign in to participate in the conversation
Mauvestodon

Escape ship from centralized social media run by Mauve.