Descriptors & Decorators

De­scrip­tors are an amaz­ing tool to have in our tool­box, as they come in handy in many op­por­tu­ni­ties.

Prob­a­bly the best thing about de­scrip­tors, is that they can im­prove oth­er ­so­lu­tion­s. Let’s see how we can write bet­ter dec­o­ra­tors, by us­ing de­scrip­tors.

Decorate a class method

Imag­ine we have a very sim­ple dec­o­ra­tor, that does noth­ing but re­turn­ing a ­tex­t, with what the orig­i­nal func­tion re­turn­s:

class decorator:
    def __init__(self, func):
        self.func = func
        wraps(func)(self)

    def __call__(self, *args, **kwargs):
        result = self.func(*args, **kwargs)
        return f"decorated {result}"

class Object:
    @decorator
    @classmethod
    def class_method(cls):
        return 'class method'

If we ap­ply the dec­o­ra­tor to a sim­ple func­tion, it’ll work, as ex­pect­ed. How­ev­er, when it’s ap­plied to a class method, we can see an er­ror:

>>> Object.class_method()
Traceback (most recent call last):
...
TypeError: 'classmethod' object is not callable

The ex­cep­tion is telling us that we tried to call some­thing that is not ac­tu­al­ly a callable. But if that’s the case then, how do class meth­ods run?

The fact is that, this is true, class methods are indeed not callable objects, but we rarely notice this, because when we access a class method, it’s usually in the form of <class>.<class_method> (or maybe also from an instance doing self.<class_method>). For both cases the answer is the same: by calling the method like this, the descriptor mechanism is triggered, and will call the __get__ inside the class method. As we already know from the analysis of the Types of Descriptors, @classmethod is actually a descriptor, and the definition of its __get__ method is the one that returns a callable [1], but @classmethod is not itself a callable.

Hint

@class­method is not a callable ob­jec­t. It’s a de­scrip­tor whose __get__ method re­turns a callable.

Now, when the dec­o­ra­tor is ap­plied to the class method, this is equiv­a­len­t of do­ing:

class Object:
    ...
    class_method = decorator(class_method)

Which does­n’t trig­ger the de­scrip­tor pro­to­col, so the __get__ in @class­method is nev­er called, there­fore what the dec­o­ra­tor re­ceives, is not a callable, hence the ex­cep­tion.

By now, it be­comes clear that if the rea­son why it fails is be­cause @class­method is a non-­callable de­scrip­tor, then the so­lu­tion must be re­lat­ed to de­scrip­tors. And in­deed, this can be fixed by just im­ple­ment­ing __get__.

class decorator:
    ...
    def __get__(self, instance, owner):
        mapped = self.func.__get__(instance, owner)
        return self.__class__(mapped)

This links the func­tion of the de­scrip­tor to the ob­ject that is go­ing to use it (in this case, the class), and re­turns a new in­stance of the dec­o­ra­tor for this ­func­tion, which does the trick.

It’s im­por­tant to no­tice that this er­ror was due to the or­der on which de­scrip­tors where ap­plied, be­cause @dec­o­ra­tor was dec­o­rat­ing @class­method and not the oth­er way around. This prob­lem would­n’t have oc­curred if we swapped the or­der of the dec­o­ra­tors. So it’s a fair ques­tion to ask, why was­n’t this just ap­plied like this to be­gin with? Af­ter al­l, a class method­-­like func­tion­al­i­ty is or­thog­o­nal from ev­ery oth­er sort of dec­o­ra­tion we might want to ap­ply, so it makes sense to be it the last one be­ing ap­plied. True, but the fix is rather sim­ple, and more im­por­tant­ly, it makes the dec­o­ra­tor more gener­ic and ap­pli­ca­ble, as it’s shown on the next sec­tion.

Note

Keep in mind the or­der of the dec­o­ra­tors, and make sure @class­method is the last one be­ing used, in or­der to avoid is­sues. Even de­spite this ­con­sid­er­a­tion, is bet­ter to have dec­o­ra­tors that will work in many pos­si­ble s­ce­nar­ios, re­gard­less of their or­der.

The com­plete code for this ex­am­ple can be found here

Decorators that change the signature

Sce­nar­i­o: Sev­er­al parts of the code have callable ob­jects that in­ter­act with­ their pa­ram­e­ters in the same way, re­sult­ing in code rep­e­ti­tion. As a re­sult of that, a dec­o­ra­tor is de­vised in or­der to ab­stract that log­ic in a sin­gle place.

For ex­am­ple, we have a func­tion that re­solves some at­tributes based on it­s ­pa­ram­e­ter­s, but it does so, by us­ing a helper ob­jec­t, cre­at­ed from the ­pa­ram­e­ter­s, like this:

def resolver_function(root, args, context, info):
    helper = DomainObject(root, args, context, info)
    ...
    helper.process()
    helper.task1()
    helper.task2()
    return helper.task1()

If there are more func­tions with this sig­na­ture do­ing the same as in the first ­li­nes, it’ll be bet­ter to ab­stract this away, and sim­ply re­ceive the helper ob­ject di­rect­ly. A dec­o­ra­tor like this one should work:

class DomainArgs:
    def __init__(self, func):
        self.func = func
        wraps(func)(self)

    def __call__(self, root, args, context, info):
        helper = DomainObject(root, args, context, info)
        return self.func(helper)

This dec­o­ra­tor changes the sig­na­ture of the orig­i­nal func­tion. There­fore, we dec­o­rate a func­tion that will re­ceive a sin­gle ar­gu­men­t, when in fact (thanks ­to the dec­o­ra­tor), the re­sult­ing one will end up re­ceiv­ing the same old four ar­gu­ments, main­tain­ing com­pat­i­bil­i­ty. By ap­ply­ing the dec­o­ra­tor, we could hap­pi­ly as­sume that the re­quired ob­ject will be passed by:

@DomainArgs
def resolver_function2(helper):
    helper.task1()
    helper.task2()
    ...
    return helper.process()

How­ev­er, there are al­so ob­jects whose meth­ods have this log­ic, and we want to ap­ply the same dec­o­ra­tor to them:

class ViewResolver:
    @DomainArgs
    def resolve_method(self, helper):
        response = helper.process()
        return f"Method: {response}"

But with this im­ple­men­ta­tion, it won’t work:

>>> vr = ViewResolver()
>>> vr.resolve_method('root', 'args', 'context', 'info')
Traceback (most recent call last)
...
     39     def __call__(self, root, args, context, info):
     40         helper = DomainObject(root, args, context, info)
---> 41         return self.func(helper)
TypeError: resolve_method() missing 1 required positional argument: 'helper'

The prob­lem is that in­stance meth­ods are func­tion­s, that take an ex­tra first ­pa­ram­e­ter, name­ly self, which is the in­stance it­self. In this case, the er­ror shown in line 41, means that the dec­o­ra­tor is com­pos­ing the ob­ject as usu­al­ly, and pass­es it was the first pa­ram­e­ter, in the place where self would go for the method, and there is noth­ing be­ing passed for helper (the pa­ram­e­ters are “shift­ed” on place to the left­), hence the er­ror.

In or­der to fix this, we need to dis­tin­guish when the wrapped func­tion is be­ing ­called from an in­stance or a class. And de­scrip­tors do just that, so the fix is rather sim­ple as in the pre­vi­ous case:

def __get__(self, instance, owner):
    mapped = self.func.__get__(instance, owner)
    return self.__class__(mapped)

The same method works here as well. When the wrapped func­tion is a reg­u­lar one, the __get__ method does­n’t take place at al­l, so adding it, does­n’t af­fect the dec­o­ra­tor. Where­as, when is called from a class, the __get__ method is en­abled, re­turn­ing a bound in­stance, which will pass self as the ­first pa­ram­e­ter (what Python does in­ter­nal­ly).

Hint

De­scrip­tors can help writ­ing bet­ter dec­o­ra­tors, by fix­ing com­mon prob­lem­s in a very el­e­gant fash­ion.

Here is the list­ing for this ex­am­ple.

[1] An equivalent Python implementation of classmethod and others can be found at https://docs.python.org/3.6/howto/descriptor.html#descriptor-protocol