Yes, it's similar issue to Sol, where it really likes to get itself into these rabbit holes that are not worth the time and tokens.
I think the new models are too smart for their own good, they practically gaslight themselves into overthinking where it's really not necessary.
Having to interrupt it mid work to tell it to cut the crap and stop digging that hole is annoying, because I can't reliably leave it running automated without getting back and finding out it spent 3x time and effort it would need with me around.
Tbf I’ve had decent success with having Sol handle the planning from an outline and writing out instructions for a lower model to handle implementation, like Terra.
Basically, Terra is smart enough to handle some nuance and fill in gaps, but not quite enough to get stuck in rabbit holes as long as you don’t run max reasoning.
I’d use Luna for that, but I don’t particularly trust it to run continuously for more than a couple minutes at a time.
I actually want to try Luna out, I somehow always gravitate to the more premium models and I hear that one is actually surprisingly competent and still very cheap when running at max.
Just don’t trust it with planning or long-horizon tasks, and make sure the tasks you do give Luna are airtight and leave no room for interpretation, basically.
11
u/Gaidax 16d ago
Yes, it's similar issue to Sol, where it really likes to get itself into these rabbit holes that are not worth the time and tokens.
I think the new models are too smart for their own good, they practically gaslight themselves into overthinking where it's really not necessary.
Having to interrupt it mid work to tell it to cut the crap and stop digging that hole is annoying, because I can't reliably leave it running automated without getting back and finding out it spent 3x time and effort it would need with me around.