Types of Descriptors

Functions are non-data descriptors

This is how methods work in Python: function objects, are non-data descriptors that implement __get__().

If we think about it, ac­cord­ing to ob­jec­t-ori­ent­ed soft­ware the­o­ry, an ob­ject is a com­pu­ta­tion­al ab­strac­tion that rep­re­sents an en­ti­ty of the do­main prob­lem. An ob­ject has a set of meth­ods that can work with, which de­ter­mines its in­ter­face (what the ob­ject is and can do) 2.

How­ev­er, in more tech­ni­cal terms, ob­jects are just im­ple­ment­ed with a da­ta struc­ture (that in Python are dic­tio­nar­ies), and it’s be­haviour, de­ter­mined by their meth­od­s, are just func­tion­s. Again, meth­ods are just func­tion­s. Let’s prove it 3.

If we have a class like this and in­spect its dic­tio­nary we’ll see that what­ev­er we de­fined as meth­od­s, are ac­tu­al­ly func­tions stored in­ter­nal­ly in the dic­tio­nary of the class.

class Person:
    def __init__(self, name):
        self.name = name

    def greet(self, other_person):
        print(f"Hi {other_person.name}, I'm {self.name}!")

We can see that among all the things de­fined in the class, it’s dic­tio­nary con­tains an en­try for ‘greet’, whose val­ue is a func­tion.

>>> type(Person.greet)
<class 'function'>

>>> Person.__dict__
mappingproxy({'__dict__': ...
              'greet': <function ...Person.greet>})

This means that in fact, it’s the same as having a function defined outside the class, that knows how to work with an instance of that same class, which by convention in Python is called self. Therefore inside the class, we’re just creating functions that know how to work with an instance of that class, and Python will provide this object, as a first parameter, under the name that we usually call self. This is basically what the __get__ method does for functions: it returns a bound instance of the function to that object.

In CPython, this logic is implemented in C, but let’s see if we can create an equivalent example, just to get a clear picture. Imagine we have a custom function, and we want to apply it to a class, as an instance method.

First we have an isolated function, that computes the mean time between failures for an object that collects metrics on systems that monitors. Then we have a class called SystemMonitor, that represents all sort of objects that collect metrics on monitored systems.

def mtbf(system_monitor):
    """Mean Time Between Failures
    https://en.wikipedia.org/wiki/Mean_time_between_failures
    """
    operational_intervals = zip(
        system_monitor.downtimes,
        system_monitor.uptimes)

    operational_time = sum(
        (start_downtime - start_uptime)
        for start_downtime, start_uptime in operational_intervals)
    try:
        return operational_time / len(system_monitor.downtimes)
    except ZeroDivisionError:
        return 0


class SystemMonitor:
    """Collect metrics on software & hardware components."""
    def __init__(self, name):
        self.name = name
        self.uptimes = []
        self.downtimes = []

    def up(self, when):
        self.uptimes.append(when)

    def down(self, when):
        self.downtimes.append(when)

For now we just test the function, but soon we’ll want this as a method of the class. We can easily apply the function to work with a SystemMonitor instance:

>>> monitor = SystemMonitor('prod')
>>> monitor.uptimes = [0,7, 12]
>>> monitor.downtimes = [5, 12]

>>> mtbf(monitor)
>>> 5.0

But now we want it to be part of the class, so that I can use it as a in­stance method. If we try to as­sign the func­tion as a method, it will just fail, be­cause it’s not bound:

>>> monitor.mtbf = mtbf
>>> monitor.mtbf()
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-7-...> in <module>()
----> 1 monitor.mtbf()

TypeError: mtbf() missing 1 required positional argument: 'system_monitor'

In this case the system_monitor positional argument that requires, is the instance, which in methods is referred to as self.

Now, if the function is bound to the object, the scenario changes. We can do that the same way Python does: __get__.

>>> monitor.mtbf = mtbf.__get__(monitor)
>>> monitor.mtbf()
5.0

Now, we want to be able to define this function inside the class, the same way we do with methods, like def mtbf(self):.... In this case, for simplicity, I’ll just use a callable object, that represents the actual object function (the body of __call__ would represent what we put on the body of the function after it’s definition). And we’ll declare it as an attribute of the class, much like all methods:

class SystemMonitor:
    ...
    mtbf = MTBF()

Provided that MTBF is a callable object (again, representing our “function”), is equivalent to doing def mtbf(self): ... inside the class.

In the body of the callable, we can just reuse the original function, for simplicity. What’s really interesting is the __get__ method, on which we return the callable object, exposed as a method.

class MTBF:
    """Compute Mean Time Between Failures"""
    def __call__(self, instance):
        return mtbf(instance)

    def __get__(self, instance, owner=None):
        return types.MethodType(self, instance)

To explain: the attribute mtbf is a “function” (callable actually), defined in the class. When we call it as a method, Python will see it has a __get__, and when this is called, it will return another object which is the function bound to the instance, passing self as first parameter, which in turn is what’s going to be executed.

This does the trick of making functions work as methods, which is a very elegant solution of CPython.

We can now ap­pre­ci­ate the el­e­gance of the de­sign be­hind meth­od­s: in­stead of cre­at­ing a whole new ob­jec­t, re­use func­tions un­der the as­sump­tion that the first pa­ram­e­ter will be an in­stance of that class, that is go­ing to be used in­ter­nal­ly, and by con­ven­tion called self (although, it can be called oth­er­wise).

Following a similar logic, classmethod, and staticmethod decorators, are also descriptors. The former, passes the class as the first argument (which is why class methods start with cls as a first argument), and the latter, simply returns the function as it is.