Descriptors & Decorators

De­scrip­tors are an amaz­ing tool to have in our tool­box, as they come in handy in many op­por­tu­ni­ties.

Prob­a­bly the best thing about de­scrip­tors, is that they can im­prove oth­er ­so­lu­tion­s. Let’s see how we can write bet­ter dec­o­ra­tors, by us­ing de­scrip­tors.

Decorate a class method

Imag­ine we have a very sim­ple dec­o­ra­tor, that does noth­ing but re­turn­ing a ­tex­t, with what the orig­i­nal func­tion re­turn­s:

class decorator:
    def __init__(self, func):
        self.func = func
        wraps(func)(self)

    def __call__(self, *args, **kwargs):
        result = self.func(*args, **kwargs)
        return f"decorated {result}"

class Object:
    @decorator
    @classmethod
    def class_method(cls):
        return 'class method'

If we ap­ply the dec­o­ra­tor to a sim­ple func­tion, it’ll work, as ex­pect­ed. How­ev­er, when it’s ap­plied to a class method, we can see an er­ror:

>>> Object.class_method()
Traceback (most recent call last):
...
TypeError: 'classmethod' object is not callable

The ex­cep­tion is telling us that we tried to call some­thing that is not ac­tu­al­ly a callable. But if that’s the case then, how do class meth­ods run?

The fact is that, this is true, class methods are indeed not callable objects, but we rarely notice this, because when we access a class method, it’s usually in the form of <class>.<class_method> (or maybe also from an instance doing self.<class_method>). For both cases the answer is the same: by calling the method like this, the descriptor mechanism is triggered, and will call the __get__ inside the class method. As we already know from the analysis of the Types of Descriptors, @classmethod is actually a descriptor, and the definition of its __get__ method is the one that returns a callable 1, but @classmethod is not itself a callable.

Hint

@classmethod is not a callable object. It’s a descriptor whose __get__ method returns a callable.

Now, when the dec­o­ra­tor is ap­plied to the class method, this is equiv­a­len­t of do­ing:

class Object:
    ...
    class_method = decorator(class_method)

Which doesn’t trigger the descriptor protocol, so the __get__ in @classmethod is never called, therefore what the decorator receives, is not a callable, hence the exception.

By now, it becomes clear that if the reason why it fails is because @classmethod is a non-callable descriptor, then the solution must be related to descriptors. And indeed, this can be fixed by just implementing __get__.

class decorator:
    ...
    def __get__(self, instance, owner):
        mapped = self.func.__get__(instance, owner)
        return self.__class__(mapped)

This links the func­tion of the de­scrip­tor to the ob­ject that is go­ing to use it (in this case, the class), and re­turns a new in­stance of the dec­o­ra­tor for this ­func­tion, which does the trick.

It’s important to notice that this error was due to the order on which descriptors where applied, because @decorator was decorating @classmethod and not the other way around. This problem wouldn’t have occurred if we swapped the order of the decorators. So it’s a fair question to ask, why wasn’t this just applied like this to begin with? After all, a class method-like functionality is orthogonal from every other sort of decoration we might want to apply, so it makes sense to be it the last one being applied. True, but the fix is rather simple, and more importantly, it makes the decorator more generic and applicable, as it’s shown on the next section.

Note

Keep in mind the order of the decorators, and make sure @classmethod is the last one being used, in order to avoid issues. Even despite this consideration, is better to have decorators that will work in many possible scenarios, regardless of their order.

The com­plete code for this ex­am­ple can be found here

Decorators that change the signature

Sce­nar­i­o: Sev­er­al parts of the code have callable ob­jects that in­ter­act with­ their pa­ram­e­ters in the same way, re­sult­ing in code rep­e­ti­tion. As a re­sult of that, a dec­o­ra­tor is de­vised in or­der to ab­stract that log­ic in a sin­gle place.

For ex­am­ple, we have a func­tion that re­solves some at­tributes based on it­s ­pa­ram­e­ter­s, but it does so, by us­ing a helper ob­jec­t, cre­at­ed from the ­pa­ram­e­ter­s, like this:

def resolver_function(root, args, context, info):
    helper = DomainObject(root, args, context, info)
    ...
    helper.process()
    helper.task1()
    helper.task2()
    return helper.task1()

If there are more func­tions with this sig­na­ture do­ing the same as in the first ­li­nes, it’ll be bet­ter to ab­stract this away, and sim­ply re­ceive the helper ob­ject di­rect­ly. A dec­o­ra­tor like this one should work:

class DomainArgs:
    def __init__(self, func):
        self.func = func
        wraps(func)(self)

    def __call__(self, root, args, context, info):
        helper = DomainObject(root, args, context, info)
        return self.func(helper)

This dec­o­ra­tor changes the sig­na­ture of the orig­i­nal func­tion. There­fore, we dec­o­rate a func­tion that will re­ceive a sin­gle ar­gu­men­t, when in fact (thanks ­to the dec­o­ra­tor), the re­sult­ing one will end up re­ceiv­ing the same old four ar­gu­ments, main­tain­ing com­pat­i­bil­i­ty. By ap­ply­ing the dec­o­ra­tor, we could hap­pi­ly as­sume that the re­quired ob­ject will be passed by:

@DomainArgs
def resolver_function2(helper):
    helper.task1()
    helper.task2()
    ...
    return helper.process()

How­ev­er, there are al­so ob­jects whose meth­ods have this log­ic, and we want to ap­ply the same dec­o­ra­tor to them:

class ViewResolver:
    @DomainArgs
    def resolve_method(self, helper):
        response = helper.process()
        return f"Method: {response}"

But with this im­ple­men­ta­tion, it won’t work:

>>> vr = ViewResolver()
>>> vr.resolve_method('root', 'args', 'context', 'info')
Traceback (most recent call last)
...
     39     def __call__(self, root, args, context, info):
     40         helper = DomainObject(root, args, context, info)
---> 41         return self.func(helper)
TypeError: resolve_method() missing 1 required positional argument: 'helper'

The prob­lem is that in­stance meth­ods are func­tion­s, that take an ex­tra first ­pa­ram­e­ter, name­ly self, which is the in­stance it­self. In this case, the er­ror shown in line 41, means that the dec­o­ra­tor is com­pos­ing the ob­ject as usu­al­ly, and pass­es it was the first pa­ram­e­ter, in the place where self would go for the method, and there is noth­ing be­ing passed for helper (the pa­ram­e­ters are “shift­ed” on place to the left­), hence the er­ror.

In or­der to fix this, we need to dis­tin­guish when the wrapped func­tion is be­ing ­called from an in­stance or a class. And de­scrip­tors do just that, so the fix is rather sim­ple as in the pre­vi­ous case:

def __get__(self, instance, owner):
    mapped = self.func.__get__(instance, owner)
    return self.__class__(mapped)

The same method works here as well. When the wrapped function is a regular one, the __get__ method doesn’t take place at all, so adding it, doesn’t affect the decorator. Whereas, when is called from a class, the __get__ method is enabled, returning a bound instance, which will pass self as the first parameter (what Python does internally).

Hint

De­scrip­tors can help writ­ing bet­ter dec­o­ra­tors, by fix­ing com­mon prob­lem­s in a very el­e­gant fash­ion.

Here is the list­ing for this ex­am­ple.

1

An equiv­a­lent Python im­ple­men­ta­tion of class­method and oth­ers can be ­found at http­s://­doc­s.python.org/3.6/how­to/de­scrip­tor.htm­l#de­scrip­tor-pro­to­col