diff --git a/peps/pep-0767.rst b/peps/pep-0767.rst index 88be22ce078..2c7b3acec34 100644 --- a/peps/pep-0767.rst +++ b/peps/pep-0767.rst @@ -1,6 +1,6 @@ PEP: 767 Title: Annotating Read-Only Attributes -Author: Eneg +Author: Łukasz Modzelewski Sponsor: Carl Meyer Discussions-To: https://discuss.python.org/t/pep-767-annotating-read-only-attributes/73408 Status: Draft @@ -9,6 +9,7 @@ Topic: Typing Created: 18-Nov-2024 Python-Version: 3.15 Post-History: `09-Oct-2024 `__ + `05-Dec-2024 `__ Abstract @@ -24,15 +25,26 @@ Akin to :pep:`705`, it makes no changes to setting attributes at runtime. Correc usage of read-only attributes is intended to be enforced only by static type checkers. +Terminology +=========== + +This PEP uses "attribute mutation" as a reference to assignment to, +or ``del``\ etion of the attribute. +This is distinct from :term:`mutability ` of the attribute's value. + +"Read-only" is used to describe attributes which may be read, but not assigned to +or deleted. + + Motivation ========== The Python type system lacks a single concise way to mark an attribute read-only. This feature is present in other statically and gradually typed languages -(such as `C# `_ -or `TypeScript `_), -and is useful for removing the ability to reassign or ``del``\ ete an attribute -at a type checker level, as well as defining a broad interface for structural subtyping. +(such as `C# `__ +or `TypeScript `__), +and is useful for removing the ability to mutate an attribute at a type checker level, +as well as defining a broad interface for structural subtyping. .. _classes: @@ -58,7 +70,7 @@ Today, there are three major ways of achieving read-only attributes, honored by - Overriding ``number`` is not possible - the specification of ``Final`` imposes that the name cannot be overridden in subclasses. -* read-only proxy via ``@property``:: +* marking the attribute "_internal", and exposing it via read-only ``@property``:: class Foo: _number: int @@ -70,7 +82,7 @@ Today, there are three major ways of achieving read-only attributes, honored by def number(self) -> int: return self._number - - Overriding ``number`` is possible. *Type checkers disagree about the specific rules*. [#overriding_property]_ + - Overriding ``number`` is possible, but limited to using ``@property``. [#overriding_property]_ - Read-only at runtime. [#runtime]_ - Requires extra boilerplate. - Supported by :mod:`dataclasses`, but does not compose well - the synthesized @@ -90,8 +102,7 @@ Today, there are three major ways of achieving read-only attributes, honored by - Read-only at runtime. [#runtime]_ - No per-attribute control - these mechanisms apply to the whole class. - Frozen dataclasses incur some runtime overhead. - - ``NamedTuple`` is still a ``tuple``. Most classes do not need to inherit - indexing, iteration, or concatenation. + - Most classes do not need indexing, iteration, or concatenation, inherited from ``NamedTuple``. .. _protocols: @@ -123,8 +134,10 @@ This syntax has several drawbacks: * It is somewhat verbose. * It is not obvious that the quality conveyed here is the read-only character of a property. * It is not composable with :external+typing:term:`type qualifiers `. -* Not all type checkers agree [#property_in_protocol]_ that all of the above five - objects are assignable to this structural type. +* Currently, Pyright disagrees that some of the above five objects + are assignable to this structural type. + `[Pyright] `_ + `[mypy] `_ Rationale ========= @@ -137,7 +150,6 @@ A class with a read-only instance attribute can now be defined as:: from typing import ReadOnly - class Member: def __init__(self, id: int) -> None: self.id: ReadOnly[int] = id @@ -146,28 +158,24 @@ A class with a read-only instance attribute can now be defined as:: from typing import Protocol, ReadOnly - class HasName(Protocol): name: ReadOnly[str] - - def greet(obj: HasName, /) -> str: - return f"Hello, {obj.name}!" - * A subclass of ``Member`` can redefine ``.id`` as a writable attribute or a - :term:`descriptor`. It can also :external+typing:term:`narrow` the type. -* The ``HasName`` protocol has a more succinct definition, and is agnostic - to the writability of the attribute. -* The ``greet`` function can now accept a wide variety of compatible objects, - while being explicit about no modifications being done to the input. + :term:`descriptor`. It can also :external+typing:term:`narrow` its type. +* The ``HasName`` protocol has a more succinct definition, + and can be implemented with custom descriptors or class variables. Specification ============= +Usage +----- + The :external+py3.13:data:`typing.ReadOnly` :external+typing:term:`type qualifier` becomes a valid annotation for :term:`attributes ` of classes and protocols. -It can be used at class-level or within ``__init__`` to mark individual attributes read-only:: +It can be used at class-level and within ``__init__`` to mark individual attributes read-only:: class Book: id: ReadOnly[int] @@ -176,19 +184,20 @@ It can be used at class-level or within ``__init__`` to mark individual attribut self.id = id self.name: ReadOnly[str] = name -Type checkers should error on any attempt to reassign or ``del``\ ete an attribute -annotated with ``ReadOnly``. -Type checkers should also error on any attempt to delete an attribute annotated as ``Final``. +Use of bare ``ReadOnly`` (without ``[]``) is not allowed. +Type checkers should error on any attempt to mutate an attribute annotated with ``ReadOnly``. + +It should also be an error to delete an attribute annotated as ``Final``. (This is not currently specified.) Use of ``ReadOnly`` in annotations at other sites where it currently has no meaning (such as local/global variables or function parameters) is considered out of scope for this PEP. -Akin to ``Final`` [#final_mutability]_, ``ReadOnly`` does not influence how -type checkers perceive the mutability of the assigned object. Immutable :term:`ABCs ` -and :mod:`containers ` may be used in combination with ``ReadOnly`` -to forbid mutation of such values at a type checker level: +``ReadOnly`` does not influence the mutability of the attribute's value. Immutable +protocols and :term:`ABCs ` (such as those in :mod:`collections.abc`) +may be used in combination with ``ReadOnly`` to forbid mutation of those values +at a type checker level: .. code-block:: python @@ -214,12 +223,12 @@ to forbid mutation of such values at a type checker level: def read_games(shelf: HasGames[abc.Sequence[Game]]) -> None: - shelf.games.append(...) # error: "Sequence" has no attribute "append" + # shelf.games.append(...) # error: "Sequence" has no attribute "append" shelf.games[0].name = "Blue Shift" # ok: "name" is not read-only shelf.games = [] # error: "games" is read-only -All instance attributes of frozen dataclasses and ``NamedTuple`` should be +All instance attributes of frozen dataclasses and named tuples should be implied to be read-only. Type checkers may inform that annotating such attributes with ``ReadOnly`` is redundant, but it should not be seen as an error: @@ -243,24 +252,25 @@ with ``ReadOnly`` is redundant, but it should not be seen as an error: x: ReadOnly[uint] # ok, redundant; narrower type y: Final[uint] # not redundant, Final imposes extra restrictions; narrower type -.. _init: Initialization -------------- -Assignment to a read-only attribute can only occur in the class declaring the attribute. +Assignment to a read-only attribute can only occur in the class declaring the attribute +and its nominal subclasses, at sites described below. There is no restriction to how many times the attribute can be assigned to. -Depending on the kind of the attribute, they can be assigned to at different sites: Instance Attributes ''''''''''''''''''' -Assignment to an instance attribute must be allowed in the following contexts: +Assignment to a read-only instance attribute must be allowed in the following contexts: -* In ``__init__``, on the instance received as the first parameter (likely, ``self``). +* In ``__init__``, on the instance received as the first parameter (usually, ``self``). * In ``__new__``, on instances of the declaring class created via a call to a super-class' ``__new__`` method. -* At declaration in the body of the class. +* In ``@classmethod``\ s, on instances of the declaring class created via a call + to a super-class' ``__new__`` method. +* At declaration in the class scope. Additionally, a type checker may choose to allow the assignment: @@ -298,11 +308,6 @@ Additionally, a type checker may choose to allow the assignment: band.songs = [] # error: "songs" is read-only band.songs.append("Twilight") # ok: list is mutable - - class SubBand(Band): - def __init__(self) -> None: - self.songs = [] # error: cannot assign to a read-only attribute of a base class - .. code-block:: python # a simplified immutable Fraction class @@ -339,8 +344,8 @@ Class Attributes Read-only class attributes are attributes annotated as both ``ReadOnly`` and ``ClassVar``. Assignment to such attributes must be allowed in the following contexts: -* At declaration in the body of the class. -* In ``__init_subclass__``, on the class object received as the first parameter (likely, ``cls``). +* At declaration in the class scope. +* In ``__init_subclass__``, on the class object received as the first parameter (usually, ``cls``). .. code-block:: python @@ -365,8 +370,8 @@ default for instances: self.number = number .. note:: - This feature conflicts with :data:`~object.__slots__`. An attribute with - a class-level value cannot be included in slots, effectively making it a class variable. + This is possible only in classes without :data:`~object.__slots__`. + An attribute included in slots cannot have a class-level default. Type checkers may choose to warn on read-only attributes which could be left uninitialized after an instance is created (except in :external+typing:term:`stubs `, @@ -387,7 +392,7 @@ protocols or ABCs):: Subtyping --------- -The inability to reassign read-only attributes makes them covariant. +The inability to mutate read-only attributes makes them covariant. This has a few subtyping implications. Borrowing from :pep:`705#inheritance`: * Read-only attributes can be redeclared as writable attributes, descriptors @@ -406,7 +411,7 @@ This has a few subtyping implications. Borrowing from :pep:`705#inheritance`: game = Game(title="DOOM", year=1993) game.year = 1994 - game.title = "DOOM II" # ok: attribute is not read-only + game.title = "DOOM II" # ok: attribute is no longer read-only class TitleProxy(HasTitle): @@ -423,8 +428,8 @@ This has a few subtyping implications. Borrowing from :pep:`705#inheritance`: year: int def __init__(self, title: str, year: int) -> None: - super().__init__(title) - self.title = title # error: cannot assign to a read-only attribute of base class + super().__init__(title) # preferred + self.title = title # ok self.year = year @@ -433,6 +438,8 @@ This has a few subtyping implications. Borrowing from :pep:`705#inheritance`: * Subtypes can :external+typing:term:`narrow` the type of read-only attributes:: + from collections import abc + class GameCollection(Protocol): games: ReadOnly[abc.Collection[Game]] @@ -445,6 +452,8 @@ This has a few subtyping implications. Borrowing from :pep:`705#inheritance`: * Nominal subclasses of protocols and ABCs should redeclare read-only attributes in order to implement them, unless the base class initializes them in some way:: + import abc + class MyBase(abc.ABC): foo: ReadOnly[int] bar: ReadOnly[str] = "abc" @@ -464,8 +473,9 @@ This has a few subtyping implications. Borrowing from :pep:`705#inheritance`: def pprint(self) -> None: print(self.foo, self.bar, self.baz) -* In a protocol attribute declaration, ``name: ReadOnly[T]`` indicates that a structural - subtype must support ``.name`` access, and the returned value is assignable to ``T``:: +* In a protocol attribute declaration, ``name: ReadOnly[T]`` indicates that values + that inhabit the protocol must support ``.name`` access, and the returned value + is assignable to ``T``:: class HasName(Protocol): name: ReadOnly[str] @@ -493,6 +503,15 @@ This has a few subtyping implications. Borrowing from :pep:`705#inheritance`: has_name = NamedClassVar() has_name = NamedDescriptor() + Type checkers should not assume that access to a protocol's read-only attributes + is supported by the protocol's type (``type[HasName]``). Even if an attribute + exists on the protocol's type, no assumptions should be made about its type. + + Accurately modeling the behavior and type of ``type[HasName].name`` is difficult, + therefore it was left out from this PEP to reduce its complexity; + future enhancements to the typing specification may refine this behavior. + + Interaction with Other Type Qualifiers -------------------------------------- @@ -513,10 +532,13 @@ Interaction with Other Type Qualifiers This is consistent with the interaction of ``ReadOnly`` and :class:`typing.TypedDict` defined in :pep:`705`. -An attribute cannot be annotated as both ``ReadOnly`` and ``Final``, as the two -qualifiers differ in semantics, and ``Final`` is generally more restrictive. -``Final`` remains allowed as an annotation of attributes that are only implied -to be read-only. It can be also used to redeclare a ``ReadOnly`` attribute of a base class. +``Final`` can be used to (re)declare an attribute which is already read-only, +whether due to mechanisms such as ``NamedTuple``, or because a parent class +declared it as ``ReadOnly``. + +Semantics of ``Final`` take precedence over the semantics of read-only attributes; +combining ``ReadOnly`` and ``Final`` is redundant, +and type checkers may choose to warn or error on the redundancy. Backwards Compatibility @@ -557,7 +579,7 @@ following the footsteps of :pep:`705#how-to-teach-this`: `type qualifiers `_ section: The ``ReadOnly`` type qualifier in class attribute annotations indicates - that the attribute of the class may be read, but not reassigned or ``del``\ eted. + that the attribute of the class may be read, but not assigned to or ``del``\ eted. For usage in ``TypedDict``, see `ReadOnly `_. @@ -576,73 +598,46 @@ This PEP makes ``ReadOnly`` a better alternative for defining read-only attribut in protocols, superseding the use of properties for this purpose. -Assignment Only in ``__init__`` and Class Body ----------------------------------------------- +Assignment Only in ``__init__`` and Class Scope +----------------------------------------------- -An earlier version of this PEP proposed that read-only attributes could only be -assigned to in ``__init__`` and the class' body. A later discussion revealed that -this restriction would severely limit the usability of ``ReadOnly`` within -immutable classes, which typically do not define ``__init__``. +An earlier version of this PEP specified that read-only attributes could only be +assigned to in ``__init__`` and the class' body. This decision was based on +the specification of C#'s `readonly `__. -:class:`fractions.Fraction` is one example of an immutable class, where the -initialization of its attributes happens within ``__new__`` and classmethods. -However, unlike in ``__init__``, the assignment in ``__new__`` and classmethods -is potentially unsound, as the instance they work on can be sourced from -an arbitrary place, including an already finalized instance. +Later revision of this PEP loosened the restriction to also include ``__new__``, +``__init_subclass__`` and ``@classmethod``\ s, as it was revealed that the initial +version would severely limit the usability of ``ReadOnly`` within immutable classes, +which typically do not define ``__init__``. -We find it imperative that this type checking feature is useful to the foremost -use site of read-only attributes - immutable classes. Thus, the PEP has changed -since to allow assignment in ``__new__`` and classmethods under a set of rules -described in the :ref:`init` section. +Allowing Bare ``ReadOnly`` With Initializing Value +-------------------------------------------------- +An earlier version of this PEP allowed the use of bare ``ReadOnly`` when the attribute +being annotated had an initializing value. The type of the attribute was supposed +to be determined by type checkers using their usual type inference rules. -Open Issues -=========== - -Extending Initialization ------------------------- - -Mechanisms such as :func:`dataclasses.__post_init__` or attrs' `initialization hooks `_ -augment object creation by providing a set of special hooks which are called -during initialization. - -The current initialization rules defined in this PEP disallow assignment to -read-only attributes in such methods. It is unclear whether the rules could be -satisfyingly shaped in a way that is inclusive of those 3rd party hooks, while -upkeeping the invariants associated with the read-only-ness of those attributes. +`This thread `_ +surfaced a few non-trivial issues with this feature, like undesirable inference +of ``Literal[...]`` from literal values, differences in type checker inference rules, +or complexity of implementation due to class-level and ``__init__``-level assignments. +We decided to always require a type for ``ReadOnly[...]``, as *explicit is better than implicit*. -The Python type system has a long and detailed `specification `_ -regarding the behavior of ``__new__`` and ``__init__``. It is rather unfeasible -to expect the same level of detail from 3rd party hooks. - -A potential solution would involve type checkers providing configuration in this -regard, requiring end users to manually specify a set of methods they wish -to allow initialization in. This however could easily result in users mistakenly -or purposefully breaking the aforementioned invariants. It is also a fairly -big ask for a relatively niche feature. Footnotes ========= .. [#overriding_property] Pyright in strict mode disallows non-property overrides. - Mypy does not impose this restriction and allows an override with a plain attribute. + Mypy permits an override with a plain attribute. + Non-property overrides are technically unsafe, as they may break class-level ``Foo.number`` access. `[Pyright playground] `_ `[mypy playground] `_ .. [#runtime] - This PEP focuses solely on the type-checking behavior. Nevertheless, it should + This PEP focuses solely on type-checking behavior. Nevertheless, it should be desirable the name is read-only at runtime. -.. [#property_in_protocol] - Pyright disallows class variable and non-property descriptor overrides. - `[Pyright] `_ - `[mypy] `_ - `[Pyre] `_ - -.. [#final_mutability] - As noted above the second-to-last code example of https://typing.python.org/en/latest/spec/qualifiers.html#semantics-and-examples - Copyright =========