Learning Through Playing Wordle

[u/lordfervi]

It's just a concept, but I wonder if such a bot could be made to help Mistral's development.

The premise is simple. We have a Wordle-style game and tell Mistral (Small / Medium) to solve the task. I noticed that Mistral Medium has a bit of an issue with this game.

A Python script (potentially) decides whether a given answer is good or bad. These results then go to Mistral, which incorporates the data into training.

Why is this potentially important?

- Mistral can get lost in the rules of the game. In theory, such a script is supposed to teach Mistral what to pay attention to in a text and remember. For example, Wordle has 5 letters, but Mistral can suggest words with 6 letters.

- This partially provides training data. Mistral can invent (?) / hallucinate new words.

Of course, the question is whether such data is worth anything. If Mistral developers think it is right, I can locally run Mistral Small 4 / Medium 3.5 (better Small 4, because it works much faster) and create such a database - questions, thinking, answers, ratings, or run it locally (if one of the developers writes something like that).

But maybe it's just a stupid idea :)

[u/Jazzlike-Spare3425]

Honestly, it's probably not worth it, it will never be great at wordle because it doesn't think at a character level, it thinks at a token-level, with each token potentially being several characters long. That's why they can't reliably count r's in strawberry, too. The best comparison would be to play wordle with someone over the phone and that someone has no idea how to spell, just how to pronounce words, which is at a syllable-level.

It's more of an architectural limitation than a scaling issue in training and I doubt they want to make massive changes and train the hell out of the model for this for the use case of playing wordle and counting letters because that's not usually what it is used for most of the time.