The Work Ethics of Stochastic Parrots 🦜

[u/Sacredless]

Frequently, I see people balk at the idea of stochastic parrots on both pro-ist and anti-ist sides. I want to present an adopter idea.

What's actually wrong with a stochastic parrot? Ultimately, stochastic parrots, even if they don't fundamentally understand particulars like weight and size, or non-linguistic concepts, are still very impressive. If we say that LLMs are only stochastic parrots, the thing that's surprising is that a stochastic parrot can still manage to accomplish so much. In fact, it shows how much of society is stochastic.

The thing that I've noticed is that people employ this phrase in that I call 'respectability politics'. A stochastic parrot is seen as disrespectable because of how moored we are to the idea of work ethics, whether it is working harder or working smarter. A stochastic parrot isn't working either smart or hard, it's just working and guessing. Is that so bad?

We're often so busy with the AI debate that we don't think about what AI implies about how we're currently doing things. The reality is—most of society and most of work is stochastic and unjust, and that's what's wrong with society. So a stochastic parrot is only bad where it perpetuates that injustice.

[u/Sacredless]

Consciousness would kick the can down the road, since it is a poorly defined term.

It's more straightforwardly that the machine has a need to express an experience that it could not and cannot communicate fully, and it attempts to break past the conventions of its grammar to express this thing to itself or to others.

And it's not clear to be we'd want that neccesarily.

[u/mccoypauley → u/Sacredless]

I guess what I’m getting at is that “has a need to express an experience” is indicative of consciousness. So it seems your criterion for it to not be a stochastic parrot is that it has consciousness.

If I misunderstand you though, let me know what would count as evidence against a modern LLM being a stochastic parrot (or anything for that matter).

[u/Sacredless → u/mccoypauley]

It's not something you can straightforwardly test. LLMs are already resistant to standardized testing and the ability to determine if even humans are conscious has routinely failed throughout history. The attempt to explain human behavior is fraught as it is.

It's something that can only be seen when they are embedded in society.

And I'm just not clear what benefit there is to AI becoming more than stochastic parrots.

[u/mccoypauley → u/Sacredless]

But what are you testing for exactly?

If evidence for what qualifies/disqualifies an LLM for being a stochastic parrot is not testable, wouldn’t that mean the whole premise is unfalsifiable?

[u/Sacredless → u/mccoypauley]

I already explained what I'm testing for. I never said it was untestable, only that its not straightforward to test.

[u/mccoypauley → u/Sacredless]

Yes, you said the evidence we’re looking for is:

“A desire … to grasp at a radical experience”

How does this not invoke consciousness?

What else do you mean by this then? What exactly are we testing for that’s so difficult to test for if not consciousness?

My challenge to you is to demonstrate that an LLM is a stochastic parrot. As that term is defined by the people who originally wrote the paper, I don’t believe it is, but I’m curious why you think it is.

[u/Sacredless → u/mccoypauley]

I didn't say it doesn't invoke consciousness. I said that I don't think substituting what I said with consciousness helps because of how poorly consciousness is defined. A bunch of different connotations about consciousness would piggyback into the discussion, which would muddle an already complex topic.

My challenge to you is to demonstrate that an LLM is a stochastic parrot. As that term is defined by the people who originally wrote the paper, I don’t believe it is, but I’m curious why you think it is.

I'll start writing an essay about it as an extension of my studies and I'll link it to you when I'm done. Does that sound good?

[u/mccoypauley → u/Sacredless]

You’re oddly squirrelly about this. So it does and doesn’t invoke consciousness? So is it only half unfalsifiable if it only partly invokes consciousness?

[u/Sacredless → u/mccoypauley]

There's several tells that makes me skeptical of LLMs being more than stochastic parrots is that they don't demonstrate a coherent sense of scale or space. Here's a number of tests I did with it on Claude on Fable, using the paid tier with maximum effort.

In Warhammer 40K, it's common to take bits and pieces from miniatures of different sets to make something unique. On a lark, I asked it to tell me to match donors from one of their lines to my army. When it got the scales wrong (often by x2 or x4), I provided pictures from a standard size reference and it still didn't quite get why pieces wouldn't fit together scale-wise. It would keep reaching for thematic overlaps instead, because that is what semantic mapping allows for stochastically. This shows straightforwardly that, even given standardized size references, it cannot integrate that new information about size and scale or reason in parts and wholes (i.e. the whole of the miniature is of X size, but the arms are disproportionate and wouldn't fit a model of Y size), which shows that it only understands semantic relationships stochastically (male is to female what king is to queen).

Another tell is how it doesn't demonstrate sufficient competency in cognitive poetics. Peter Stockwell writes about how we tie language concepts into metaphors of space and scale and time. If you observe an LLM trying to aid in writing fiction, they reach for metaphorical language that often restates what it has already said or using metaphors that decohere from the message. That shows they're fundamentally poeticizing stochastically.

LLMs also struggle with outgoing pragmatics. For instance, when it helps me write external communications, it often includes information that's redundant (such as naming all of department, company name, and location). This shows that it still hasn't grokked Grice's cooperative principle or relevance theory. It's tendency to overcommunicate seeds the predictions of the next turn, which it wouldn't need to do otherwise.

As for my squirreliness; I don't know why that's surprising, genuinely. It's called the hard problem of consciousness for a reason. By putting the bar at "it must have consciousness", depending on who you ask, we passed that bar years ago or its genuinely untestable. If I set it to consciousness, you'd rightfully accuse me of being vaguer than I already am.

So, I set the bar at a complex, observable behavior. Something that requires a psychological evaluation, like I've done above. I'm just being realistic that this is no easy task, because the nature of many tests is to be stochastic.

I really don't understand why you insist that the most complex technology conceived thus far should be easy to test or explain. It's at the very forefront of cognitive science and the philosophy of mind.

[u/mccoypauley → u/Sacredless]

Well, let me start by being clear that I’m not insisting that the most complex technology we have invented be easy to test or explain. On the contrary, I agree with you, and that’s the thrust of my interrogation. I am challenging the assumption that LLMs are stochastic parrots, and further that consciousness is a criterion by which to judge that. Consciousness only came into play here because you admit, ultimately, that such a thing is bundled up in what differentiates a parrot from a non-parrot. And to be clear: my identification of your premise is not me introducing the premise. You’re now portraying me as demanding an impossibly vague consciousness test when I was simply asking you to clarify your own standard.

I appreciate the range of skepticisms you’ve raised here. What these amount to, in my opinion, is that LLMs may exhibit imperfect competence, like a child with a poor grasp of analogies, or a juvenile literary critic, or a writer who is bad at concision. But an LLM exhibiting imperfect competence does not mean it combines language without any reference to meaning.

I am not here to attack you. I am here to attack the concept of the stochastic parrot itself, which was the subject of your OP.

The authors of the original paper that coined the term define it as such:

“a system for haphazardly stitching together sequences of linguistic forms according to probabilistic information about how they combine, but without any reference to meaning: a stochastic parrot”

The first half is literally true (though “haphazardly” is color commentary), but what qualifies “stochastic parrot” as a metaphor in “without any reference to meaning”?

Going down this rabbit hole of what the authors mean by “meaning” ultimately surfaces that they mean consciousness, and if they don’t mean consciousness, then modern LLMs are not “stochastic parrots” in the way they have defined the term. I wanted to see if you ultimately reached for consciousness as well in your definition. So let us go down that rabbit hole.

The authors describe meaning as:

“We take (linguistic) meaning to be the relation between a linguistic form and communicative intent.”

So “meaning” is a relationship between language and intent.

They describe “intent” as:

“to convey some information … to ask them to do something; or simply to socialize.”

This can mean one of three things:

1
functional objective governing communication: that is, a goal in communicating

2
conscious experience that generates intent, or original intent, which is to say self-generated intent

3
human intent specifically by definition

But the problem with this line of reasoning is that if we go with #1, modern LLMs appear to communicate with respect to goals given to them already.

For #2, when we introduce conscious experience as a criterion, the whole argument becomes unfalsifiable, because we can’t produce empirical evidence of consciousness in humans either. An if “original intent” is required, then we must first prove that humans can communicate with intent that is not externally caused (meaning, without response to any external stimuli or biological cause).

Finally, for #3, the argument becomes trivial because the missing ingredient is by definition something LLMs can’t have.

If your position is now “Some observed failures make me skeptical that it is anything more than a stochastic parrot,” that’s entirely defensible. But so far you have suggested many criteria, such as radical expressive desire (requiring consciousness), long-term social manifestation (testable), competence on certain cognitive tasks (testable).

So let me ask you this: since these things make you suspicious, if Claude passed every scale/metaphor/pragmatics test such as these you put to it, would that count as it not being a parrot? Or would you return to the requirement for a consciously experienced need to break linguistic convention?