Having tested a bunch of #OpenSource #LLM projects, I gotta say that OpenHermes 2.5 is the most helpful out of the ones I can run locally.
I recently wasted a bunch of time getting Phi-2 to do some summarization work, and it just couldn't stay focused for more than a sentence or two.
@mauve I was tinkering with ollama for a bit, but my local hardware just isn't fast enough to make it useful.
@skryking What have you been using to run the models? I find LM Studio really nice for tinkering. https://lmstudio.ai/
I find Q4 quantized models work pretty well on my steam deck.
@mauve have you heard of a community somewhere (lemmy? matrix?) where we could share our experiences?
We (@codelutin) may want to start to play with things but "it's dangerous to go alone". There is @stablehorde lead by @Db0 which is a great start (but I won't join a Discord 😔)
@lutindiscret I'm putting together a Matrix one here: https://matrix.to/#/#userless-agents:mauve.moe
