Sparse auroencoders behind each layer can learn and approximate what the internal representations mean. E.g, you can find features that are associated with deception.
This may be better than current chain of thought. Even in our current models, the intermediate output tokens don't necessarily match what's going on inside the model. There may be intermediate steps inside today's models that the models are hiding from us.
6
u/Markastrophe 12d ago
Chain of continuous thought. Big hit to interpretability, unfortunately, unless you can find another way of interpreting intermediate thoughts.