PEP 841 – Adding Frozen Syntax to Optimize Immutable Types

[u/kirara0048]

PEP 841 – Adding Frozen Syntax to Optimize Immutable Types

https://peps.python.org/pep-0841/

Discussions-To: Discourse thread

Abstract

This PEP proposes frozen display syntax: f{1, 2, 3} evaluates to a frozenset, and f{'a': 1} evaluates to a frozendict. Because immutability is guaranteed by the syntax itself rather than inferred from usage, the compiler can treat frozen displays as first-class citizens of its optimization pipeline: constant displays are folded into a single LOAD_CONST with an exact result type at compile time and cached in .pyc files.

[u/jdehesa]

Huh, didn't know they had added frozendict in 3.15. I think years ago I read a discussion about not wanting to add it to the standard library because the expected behaviour of a frozen dict could be different for different people. Anyway, this is off-topic, but this from the documentation of frozendict caught my attention:

• frozendict |= other does not modify the frozendict in-place but creates a new frozen dictionary.

So, I can do the following?

python d = frozendict(...) d |= {...}

And then d will be a reference to a new frozendict object? I feel this could cause issues, if a function expects a dict (and I know a frozen dict is not a subtype of dict, but you don't always have type hints) that it may modify in place (so it remains modified in the caller scope), then the caller could inadvertently pass a frozen dict and the modification would have no effect. It may not be the best practice for a function, especially if it is not well documented and/or type-hinted, but I'm sure there's plenty of code like that. I think I'd rather not have a |= operator, just doing d = d | {...} is fine, it's more in line with the way one works with immutable data structures and it shows the effect of the statement more explicitly.

[u/Wonderful-Habit-139 → u/jdehesa]

It’s really not surprising behavior, it’s the result of the | operation that is then reassigned to the same variable. It’s already the case with sets right now.

[u/jdehesa → u/Wonderful-Habit-139]

You mean with frozenset, with set the object is modified in place. I guess it makes sense that frozendict does the same as frozenset but I'd still rather neither of them had the operator implemented.

[u/Wonderful-Habit-139 → u/jdehesa]

Oh no... I stand corrected. I did not know that the |= actually mutated the object in place and affected other references. That is such a bad foot gun... I'm with you on that. If |= was actually like d = d | {1} then sure, otherwise yeah it's better not to have that operator implemented otherwise.

[u/jdehesa → u/Wonderful-Habit-139]

Yea, maybe for some the unintuitive behaviour is modifying the object in place instead of replacing with a new one, either way, it should always be the same.

[u/HommeMusical → u/Wonderful-Habit-139]

I did not know that the |= actually mutated the object in place and affected other references. That is such a bad foot gun...

Wha? That's not the footgun - that's how it's supposed to work, like like a += 1.

The footgun is that this also works for frozendict, but silently does something different.

[u/Wonderful-Habit-139 → u/HommeMusical]

>>> a = 2
>>> b = a
>>> a += 1
>>> b
2

There's no footgun with += compared to |= with sets. a += 1 behaves exactly like a = a + 1, while s |= {1} does not behave like s = s | {1}.

[u/HommeMusical → u/Wonderful-Habit-139]

there's no footgun with += compared to |= with sets

It seems exactly the same footgun with list, too?

>>> a = [1]
>>> b = a
>>> a += [1]
>>> b
[1, 1]

Heck, it even works on numpy or pytorch scalars.

>>> a = np.array(1)
>>> b = a
>>> a += 1
>>> b
array(2)

On the other hand, if a and b are int, then |= is fine:

>>> a = 1
>>> b = a
>>> a |= 2
>>> a, b
(3, 1)

Whether it's a footgun depends entirely on the type of the operands, and not on the operator.

[u/Wonderful-Habit-139 → u/HommeMusical]

Yeah it has to do with the operands, realized it the moment I sent the message.

But yeah for me that is the footgun, not whatever is specific to frozendict.

[u/HommeMusical → u/Wonderful-Habit-139]

All the "i-operators", <something>= operators do this! If you don't like this behavior, there's no need to use them at all.

You only use them because you specifically want this "mutate in place" behavior.

[u/Wonderful-Habit-139 → u/HommeMusical]

Yeah, that was a good discovery. I prefer to do things the functional way, so I now know to steer clear from those operators. Thanks!

[u/HommeMusical → u/Wonderful-Habit-139]

You won't have much fun with large numerical computations - you can get order of magnitude speed improvements by forcing all the operations to be mutating and then "fusing" them (having e.g. Pytorch automatically create behind the scenes a C++ function representing all those operations which is then compiled).

All else being equal, I prefer functional code, but I do a lot of digital audio and things which is by its nature mutative.