Python Data Model: Operator Overloading and NotImplemented

How does operator overloading work in Python?

Operator overloading in Python works through special methods: a + b calls a.__add__(b), and if that returns NotImplemented, Python tries the reflected b.__radd__(a) before raising TypeError. a += b tries __iadd__ first and falls back to a = a + b. When the right operand's type is a subclass that overrides the reflected method, that method is tried first.

Lesson

Module 8 covered the dunders a value type needs first. This lesson completes the map: __call__, __format__, __index__, __bool__, __contains__, the in-place operators (__iadd__), unary and reflected arithmetic, __matmul__, __missing__ for dict subclasses, __reversed__, __del__ and why it cannot be relied on, and the dispatch rules — NotImplemented, reflected fallbacks, subclass priority — that make mixed-type operations work. It ends with a Vector class that exercises most of them, as the checkpoint will.

Calling, formatting, converting

class Multiplier:
    def __init__(self, k):
        self.k = k
    def __call__(self, x):                  # obj(x)
        return self.k * x

class Money:
    def __format__(self, spec):             # f"{m:>10}" — spec is what follows the colon
        text = f"{self.cents / 100:.2f}"
        return format(text, spec)
    def __bool__(self):                     # truthiness; without it, __len__; without that, always True
        return self.cents != 0
    def __int__(self): return self.cents // 100
    def __float__(self): return self.cents / 100
    def __index__(self): return self.cents  # lets the object be used as a slice index or in bin(); implies __int__

__call__ makes instances callable — stateful callables, decorator classes, strategy objects. __format__ receives the format spec and returns the formatted text; the default delegates to __str__ and ignores the spec. __index__ is the "this is really an integer" hook: range, slicing, bin and hex accept anything that defines it.

Arithmetic: forward, reflected, in-place

For a + b, Python tries a.__add__(b); if that returns NotImplemented, it tries b.__radd__(a); if both fail, TypeError. One exception: if type(b) is a subclass of type(a) and overrides __radd__, the reflected method is tried first, so a subclass can specialise mixed operations. a += b tries a.__iadd__(b) and falls back to a = a + b; an __iadd__ mutates and must return self.

Return NotImplemented (the singleton, not the exception) for operands you do not handle; never raise TypeError yourself, or the reflected attempt is skipped.

Containers

class Grid:
    def __getitem__(self, key):             # key may be an int, a slice, or a tuple for g[r, c]
        if isinstance(key, tuple):
            r, c = key
            return self._cells[r][c]
        return self._cells[key]
    def __setitem__(self, key, value): ...
    def __delitem__(self, key): ...
    def __contains__(self, value): ...       # in; falls back on iteration
    def __reversed__(self): ...              # reversed(); falls back on __len__ + __getitem__
    def __len__(self): ...
    def __iter__(self): ...

class Defaults(dict):
    def __missing__(self, key):             # called by dict.__getitem__ for an absent key
        return f"<{key}>"

g[1, 2] passes the tuple (1, 2) as one key — the syntax NumPy uses. __missing__ is the hook defaultdict is built on: dict[key] calls it instead of raising; get, in and setdefault do not.

Object lifetime

__del__ runs when the reference count reaches zero — usually promptly in CPython, never guaranteed (cycles, interpreter shutdown, other implementations), and exceptions in it are printed and ignored. It is not a destructor to rely on: resources are released by with and close(), not by __del__. weakref.finalize is the reliable way to attach cleanup to an object's collection when you must.

A Vector

import math

class Vector:
    __slots__ = ("_xs",)

    def __init__(self, *xs):
        self._xs = tuple(float(x) for x in xs)

    def __repr__(self):
        return f"Vector({', '.join(f'{x:g}' for x in self._xs)})"

    def __len__(self):
        return len(self._xs)

    def __iter__(self):
        return iter(self._xs)

    def __getitem__(self, i):
        result = self._xs[i]
        return Vector(*result) if isinstance(i, slice) else result

    def __eq__(self, other):
        return isinstance(other, Vector) and self._xs == other._xs

    def __hash__(self):
        return hash(self._xs)

    def __bool__(self):
        return any(self._xs)

    def __abs__(self):
        return math.hypot(*self._xs)

    def __neg__(self):
        return Vector(*(-x for x in self._xs))

    def __add__(self, other):
        if not isinstance(other, Vector) or len(other) != len(self):
            return NotImplemented
        return Vector(*(a + b for a, b in zip(self, other)))

    def __mul__(self, k):                     # scalar
        if not isinstance(k, (int, float)):
            return NotImplemented
        return Vector(*(x * k for x in self._xs))

    __rmul__ = __mul__                        # 3 * v

    def __matmul__(self, other):              # dot product
        if not isinstance(other, Vector) or len(other) != len(self):
            return NotImplemented
        return sum(a * b for a, b in zip(self, other))

    def __format__(self, spec):
        return f"({', '.join(format(x, spec) for x in self._xs)})"

Vector(1, 2) + Vector(3, 4), 3 * v, v @ w, abs(v), -v, v[0], v[1:], len(v), list(v), bool(Vector(0, 0)), {v}, f"{v:.1f}" — each is one dunder, and together they make the class feel built in. __slots__ keeps instances small; __eq__ with __hash__ makes them set members; every binary operator returns NotImplemented for foreign types.

Pitfalls

• Raising TypeError instead of returning NotImplemented.
• __iadd__ that mutates but forgets return self.
• __format__ ignoring the spec, or __str__ when __format__ was needed for f"{x:>8}".
• Relying on __del__ to release a resource.
• __getitem__ that handles ints but not slices (or the reverse).
• __hash__ on a mutable object.

Key takeaways

• __call__, __format__, __bool__, __index__, __int__/__float__ connect an object to calling, f-strings, truth and integer contexts.
• Binary operators try the forward method, then the reflected one; return NotImplemented, not an exception; __iadd__ returns self.
• __getitem__ receives ints, slices or tuples; __missing__ customises dict misses; __contains__ and __reversed__ have iteration fallbacks.
• __del__ is not a reliable destructor; use with, close or weakref.finalize.
• A complete value type is a dozen small methods; the checkpoint's Vector is the template.

Common questions

Should a dunder method return NotImplemented or raise TypeError?

Return NotImplemented, the singleton, for an operand you do not handle. That tells Python to try the other operand's reflected method; raising TypeError yourself skips that attempt and breaks mixed-type operations that could have worked.

What is __radd__ for in Python?

It is reflected addition, called for a + b when a.__add__(b) is missing or returns NotImplemented. The reflected methods are what make 3 * v work for a vector class, often simply __rmul__ = __mul__ when the operation is commutative.

Why can't I rely on __del__ in Python?

__del__ runs when an object's reference count reaches zero, which is usually prompt in CPython but never guaranteed: not for cycles, not at interpreter shutdown, not on other implementations, and exceptions in it are printed and ignored. Release resources with with or close(), or use weakref.finalize.

What does __missing__ do on a dict subclass?

dict.__getitem__ calls it for an absent key and returns its result instead of raising KeyError; defaultdict is built on the same hook. get, in and setdefault do not call it.

What is the difference between __index__ and __int__?

__int__ converts an object for int(x); __index__ declares that the object really is an integer, so range, slicing, bin and hex accept it. A float has __int__ but not __index__, which is why a float cannot be used as a list index.

Exercises

1 2
3 4
add
dot
rscale 3
fmt .1f
bad

Vector(4, 6)
11
Vector(3, 6)
(1.0, 2.0)
TypeError

a=1
2 2
set 0 1 7
get 0 1
get 5 5
has 7
lookup a
lookup z
size

7
out of range
True
1
<z>
1

In this module: Decorators, descriptors and the data model

• Decorators — functions that wrap functions
• Closures and late binding — cells, factories and stateful callables
• Descriptors — how properties, methods and validated attributes work
• Attribute access — __getattr__, __getattribute__, __setattr__, __dict__ and __slots__
• Classes as objects — type, __new__, __init_subclass__ and metaclasses in outline
• The data model — the rest of the dunders, and a Vector that uses them (this lesson)
• Checkpoint — Decorators, descriptors and the data model