Mauve 👁💜 @mauve@mastodon.mauve.moe

@fredy_pferdi Cool TY, I found the HSA_OVERRIDE online and it ended up working great in my ubuntu container. 😁 Wish I had this for last night's demo! Also I don't have nearly enough RAM on this thing with 16 GB. TT_TT

@fredy_pferdi Spoke too soon, ollama dies when I try to load the model. Will need to mess with it another day :) TY again for the tip.

@fredy_pferdi Sweet just followed this guide to install it in my ubuntu distrobox container and it's working great :o

https://www.reddit.com/r/steamdeck_linux/comments/102hzav/guide_how_to_install_rocm_for_gpu_julia/

@fredy_pferdi Interesting I may be able to get it running without a container too. https://github.com/rocm-arch/rocm-arch

@fredy_pferdi Oh that's great to know TY. I'll look into it. Is this going to use Vulkan for the GPU acceleration? I wasn't sure what my options would be since Ollama seems to only support Cuda and Metal

@fredy_pferdi Yeah exactly! I'm running #ChimeraOS on it in desktop mode. Lately been thinking of just installing Manjaro on it instead since the steam bits are a bit janky for me.

Video of my talk about making an #OpenSource #LLM perform function calling on my machine.

https://www.youtube.com/watch?v=GztPt4Yhl8M

#ai

@fleeky Oh yeah! My mind goblin demo uses JavaScript so you could use any javascript function that takes json and outputs json

@fleeky It was a freeform demo going through my code so there's no slides, but I think it was recorded so I'll ping you when I get a copy

@andia I didn't ask *that many* people so maybe there was more but also maybe AI people in Ottawa just aren't into alternative social media 🤷

Went to a tech related meetup today and I think nobody knows what the fediverse is.

Woof I haven't interacted with that many straight people in a row inba while. Low key kinda draining.

Source for my demo code: https://github.com/RangerMauve/mind-goblin

@fleeky I actually used Vosk for my talk today with my wikipedia enabled agent and it's way faster and more accurate than whisper (for the small models)

@fleeky Stuff like searching github, running shell scripts, checking my emails. Generally individual function calls that take in and output text

@fleeky the code they generate is also often subtly or glaringly wrong

@fleeky Not yet! It can generate text based on some initial text. You can tell it to generate somw code. You can also tell it that if it generates some JSON in a specific format that you'll call the function for it and put the result back in for it to generate more text after. It could be possible for it to call a function thar saves some code and another function to invoke it, but it's harder to keep it focused long enough with small models