Mauve 👁💜 @mauve@mastodon.mauve.moe

@jonny Are you gonna play the pokemon with guns game?

feels kinda satisfying to prolapse this function to return an iterator instead of its heavy for loop.

@skryking it has less innate knowledge of facts but it is pretty good at "reasoning". I'm gonna teach it to make function calls and traverse datasets + summarize stuff. 😁

@skryking Nice. I only do CPU workloads. Try running phi 2 some time! It's super low in resurce usage. Particually the Q4 quantized models.

@laskov Oh yeah, I read their release but haven't used it yet. Was there anything specific they excelled at?

@staltz I've already got the programming socks. :P

@skryking What have you been using to run the models? I find LM Studio really nice for tinkering. https://lmstudio.ai/

I find Q4 quantized models work pretty well on my steam deck.

Hmm, after testing the raw Phi 2 within LM Studio instead of the examples provided by HuggingFace candle, I think it's actually pretty decent after all.

Specifically I got phi 2 Q4_K_S gguf working from TheBloke.

Can't get that model running with candle since it can't seem to load the model weight.

Having tested a bunch of #OpenSource #LLM projects, I gotta say that OpenHermes 2.5 is the most helpful out of the ones I can run locally.

I recently wasted a bunch of time getting Phi-2 to do some summarization work, and it just couldn't stay focused for more than a sentence or two.

Woot, I have finally written enough #rust code to be unable to avoid the lifetime specifications / borrow checker stuff.

@beka_valentine RIP. What software are you using? I generally find OBS does everything I want.

K. I swallowed my pride and gave up on trying to run all my build tools as "close to the metal" as possible. I'm now going to pretend I'm in an Ubuntu environment via distrobox :P

Ugh C++ dependencies bring me so much pain. I'd take a hundred weird react-native version mismatches over rustc being unable to find OpenSSL for a library I didn't think would even need rustc in the first place. This is not an invitation to help because my setup is far far away from the happy path of debugging.

I think I'd enjoy living in a wizard tower.

The dowbside with lrimarily interacting with folks I regularly see in meatspace is that they are in my tinezone but aren't night owls

I love it when I get to talk to folks that have more knowledge than me on a subject I am interested in. It's tough because it doesn't happen as often as I'd like due to being the domain expert for the subject in a lot of groups I interact with.