Thinking about how to add memory to an #LLM
Might be good to have two models running in tandem. One is trained to use a memory API. The other is just the personality. Before the personality gets data, memory decides if it wants to do some RAG queries. Then the result is fed to personality to respond with without it needing to understand memory. Then memory asks itself if it needs to remember any of personality's outputs.
@fleeky I don't know the folklore but probably. Have you played Disco Elysium? Kinda like all the personalities talking to each other and the user. Maybe each gets "You only respond if you think this is relevant to you" in their system prompt and have an order for processing.
@mauve i dont either , but a jewish storyteller friend once told me about them and lots of other neat stuff , alas , good morning !
Unprompted exploration of your discussion regarding how to simulate limited language models for conversation and memories! :3
@mauve Wow, Mauve, this might be one of the closest description of how to make a language model, and I like how you're applying it. So I'm going to go a bit further from where you start in abstraction. Have you heard about Integrated Information Theory? :o Anyways..
The mind you describe reminds me of the amusing, "I'm being random!" joke about internet conversations. B) Let me try something.
What about the Memory model gets called with a series of memories it's told to call up from the Personality model using a memory cache it uses to keep running. Maybe then the Personality model chooses which lens of expertise model to describe the memory as.
The more I think about this, I can see the trigger behind the memory recall will affect whether or not the system decides to recall the output or response. I imagine it suppressing logs or verbosity then wondering why it's burn out and has memory consumed with a bunch of suppressed cache and suppressed memory blocks.. 🫠
"Why is my log filling with, 'An Error Has Occurred' just because I'm recalling the first Error-Causing event?" 😎
@mauve
like the crows memory and thought from folklore :)