Bouta skip the language model and autistically read all the training data into my own brain instead.
Follow
Seriously though what the hell are some of these outputs they're teaching these things "he was staring at the beautiful mexican girl" as an "answer" to a random rant.
https://huggingface.co/datasets/cognitivecomputations/dolphin?row=30
Maybe this is a side effect of using AI to generate datasets?
@mauve this looks like parsing out the "Answer the following question:" was the trigger to pick one of "-"-items at the end of the text
@mauve probably a result of getting to the long tail of the probabilities. When there aren't too many occurrences of the previous thread of words..
Note to self: Dolphin models are brainrot to LLMs. No wonder my dolphin-phi was unable to think for more than a few sentences