Descriptors & Decorators

Des­crip­tors are an ama­zing tool to ha­ve in our tool­bo­x, as they co­me in hand­y in many oppor­tu­ni­tie­s.

Pro­ba­bly the best thing about des­crip­tor­s, is that they can im­pro­ve othe­r ­so­lu­tion­s. Le­t’s see how we can wri­te be­tter de­co­ra­tor­s, by using des­crip­tor­s.

Decorate a class method

Ima­gi­ne we ha­ve a ve­ry sim­ple de­co­ra­to­r, that does no­thing but re­tur­ning a ­tex­t, wi­th what the ori­gi­nal func­tion re­turn­s:

class decorator:
    def __init__(self, func):
        self.func = func
        wraps(func)(self)

    def __call__(self, *args, **kwargs):
        result = self.func(*args, **kwargs)
        return f"decorated {result}"

class Object:
    @decorator
    @classmethod
    def class_method(cls):
        return 'class method'

If we apply the de­co­ra­tor to a sim­ple func­tio­n, it’­ll wo­rk, as ex­pec­te­d. Ho­we­ve­r, when it’s applied to a cla­ss me­tho­d, we can see an erro­r:

>>> Object.class_method()
Traceback (most recent call last):
...
TypeError: 'classmethod' object is not callable

The ex­cep­tion is te­lling us that we tried to ca­ll so­me­thing that is no­t ac­tua­lly a ca­lla­ble. But if tha­t’s the ca­se then, how do cla­ss me­tho­ds run?

The fact is that, this is true, class methods are indeed not callable objects, but we rarely notice this, because when we access a class method, it’s usually in the form of <class>.<class_method> (or maybe also from an instance doing self.<class_method>). For both cases the answer is the same: by calling the method like this, the descriptor mechanism is triggered, and will call the __get__ inside the class method. As we already know from the analysis of the Types of Descriptors, @classmethod is actually a descriptor, and the definition of its __get__ method is the one that returns a callable [1], but @classmethod is not itself a callable.

Hint

@cla­ss­me­thod is not a ca­lla­ble ob­jec­t. It’s a des­crip­tor who­se __­ge­t__ me­thod re­turns a ca­lla­ble.

No­w, when the de­co­ra­tor is applied to the cla­ss me­tho­d, this is equi­va­len­t of doin­g:

class Object:
    ...
    class_method = decorator(class_method)

Whi­ch does­n’t tri­gger the des­crip­tor pro­to­col, so the __­ge­t__ in @cla­ss­me­thod is ne­ver ca­lle­d, the­re­fo­re what the de­co­ra­tor re­cei­ve­s, is not a ca­lla­ble, hen­ce the ex­cep­tio­n.

By no­w, it be­co­mes clear that if the rea­son why it fails is be­cau­se @cla­ss­me­thod is a no­n-­ca­lla­ble des­crip­to­r, then the so­lu­tion must be­ ­re­lated to des­crip­tor­s. And in­dee­d, this can be fixed by just im­ple­men­tin­g __­ge­t__.

class decorator:
    ...
    def __get__(self, instance, owner):
        mapped = self.func.__get__(instance, owner)
        return self.__class__(mapped)

This li­nks the func­tion of the des­crip­tor to the ob­ject that is going to use it (in this ca­se, the cla­ss), and re­turns a new ins­tan­ce of the de­co­ra­tor for this ­func­tio­n, whi­ch does the tri­ck.

It’s im­por­tant to no­ti­ce that this error was due to the or­der on whi­ch ­des­crip­tors whe­re applie­d, be­cau­se @de­co­ra­tor was de­co­ra­tin­g @cla­ss­me­thod and not the other way aroun­d. This pro­blem would­n’t ha­ve oc­cu­rred if we swa­pped the or­der of the de­co­ra­tor­s. So it’s a fair ques­tion to­ a­sk, why was­n’t this just applied like this to be­gin wi­th? After all, a cla­ss ­me­tho­d-­like func­tio­na­li­ty is or­tho­go­nal from eve­ry other sort of de­co­ra­tion we ­mi­ght want to appl­y, so it makes sen­se to be it the last one being applie­d. ­True, but the fix is ra­ther sim­ple, and mo­re im­por­tan­tl­y, it makes the ­de­co­ra­tor mo­re ge­ne­ric and appli­ca­ble, as it’s sho­wn on the next sec­tio­n.

No­te

Keep in mind the or­der of the de­co­ra­tor­s, and make su­re @cla­ss­me­thod is ­the last one being us­e­d, in or­der to avoid is­sues. Even des­pi­te this ­con­si­de­ra­tio­n, is be­tter to ha­ve de­co­ra­tors that wi­ll wo­rk in many po­s­si­ble s­ce­na­rio­s, re­gard­le­ss of their or­de­r.

The com­ple­te co­de for this exam­ple can be found he­re

Decorators that change the signature

Sce­na­rio: Se­ve­ral par­ts of the co­de ha­ve ca­lla­ble ob­jec­ts that in­te­ract wi­th ­their pa­ra­me­ters in the sa­me wa­y, re­sul­ting in co­de re­pe­ti­tio­n. As a re­sult of ­tha­t, a de­co­ra­tor is de­vi­s­ed in or­der to abs­tract that lo­gic in a sin­gle pla­ce.

For exam­ple, we ha­ve a func­tion that re­sol­ves so­me attri­bu­tes ba­sed on its ­pa­ra­me­ter­s, but it does so, by using a hel­per ob­jec­t, created from the ­pa­ra­me­ter­s, like this:

def resolver_function(root, args, context, info):
    helper = DomainObject(root, args, context, info)
    ...
    helper.process()
    helper.task1()
    helper.task2()
    return helper.task1()

If the­re are mo­re func­tions wi­th this sig­na­tu­re doing the sa­me as in the firs­t ­li­nes, it’­ll be be­tter to abs­tract this awa­y, and sim­ply re­cei­ve the hel­pe­r ob­ject di­rec­tl­y. A de­co­ra­tor like this one should wo­rk:

class DomainArgs:
    def __init__(self, func):
        self.func = func
        wraps(func)(self)

    def __call__(self, root, args, context, info):
        helper = DomainObject(root, args, context, info)
        return self.func(helper)

This de­co­ra­tor chan­ges the sig­na­tu­re of the ori­gi­nal func­tio­n. The­re­fo­re, we ­de­co­ra­te a func­tion that wi­ll re­cei­ve a sin­gle ar­gu­men­t, when in fact (thanks ­to the de­co­ra­to­r), the re­sul­ting one wi­ll end up re­cei­ving the sa­me old fou­r ar­gu­men­ts, main­tai­ning com­pa­ti­bi­li­ty. By appl­ying the de­co­ra­to­r, we coul­d ha­ppi­ly as­su­me that the re­qui­red ob­ject wi­ll be pa­ss­ed by:

@DomainArgs
def resolver_function2(helper):
    helper.task1()
    helper.task2()
    ...
    return helper.process()

Ho­we­ve­r, the­re are al­so ob­jec­ts who­se me­tho­ds ha­ve this lo­gi­c, and we want to­ a­pply the sa­me de­co­ra­tor to the­m:

class ViewResolver:
    @DomainArgs
    def resolve_method(self, helper):
        response = helper.process()
        return f"Method: {response}"

But wi­th this im­ple­men­ta­tio­n, it wo­n’t wo­rk:

>>> vr = ViewResolver()
>>> vr.resolve_method('root', 'args', 'context', 'info')
Traceback (most recent call last)
...
     39     def __call__(self, root, args, context, info):
     40         helper = DomainObject(root, args, context, info)
---> 41         return self.func(helper)
TypeError: resolve_method() missing 1 required positional argument: 'helper'

The pro­blem is that ins­tan­ce me­tho­ds are func­tion­s, that take an ex­tra firs­t ­pa­ra­me­te­r, na­me­ly se­lf, whi­ch is the ins­tan­ce itsel­f. In this ca­se, the erro­r s­ho­wn in li­ne 41, means that the de­co­ra­tor is com­po­sing the ob­ject as usua­ll­y, and pa­s­ses it was the first pa­ra­me­te­r, in the pla­ce whe­re se­lf would go fo­r ­the me­tho­d, and the­re is no­thing being pa­ss­ed for hel­per (the pa­ra­me­ters are “s­hi­fte­d” on pla­ce to the le­ft), hen­ce the erro­r.

In or­der to fix this, we need to dis­tin­guish when the wra­pped func­tion is bein­g ­ca­lled from an ins­tan­ce or a cla­ss. And des­crip­tors do just tha­t, so the fix is ­ra­ther sim­ple as in the pre­vious ca­se:

def __get__(self, instance, owner):
    mapped = self.func.__get__(instance, owner)
    return self.__class__(mapped)

The sa­me me­thod wo­rks he­re as we­ll. When the wra­pped func­tion is a re­gu­la­r o­ne, the __­ge­t__ me­thod does­n’t take pla­ce at all, so adding it, does­n’­t a­ffect the de­co­ra­to­r. Whe­rea­s, when is ca­lled from a cla­ss, the __­ge­t__ me­thod is ena­ble­d, re­tur­ning a bound ins­tan­ce, whi­ch wi­ll pa­ss se­lf as the ­first pa­ra­me­ter (what Py­thon does in­ter­na­ll­y).

Hint

Des­crip­tors can help wri­ting be­tter de­co­ra­tor­s, by fi­xing co­m­mon pro­ble­ms in a ve­ry ele­gant fas­hio­n.

He­re is the lis­ting for this e­xam­ple.

[1] An equivalent Python implementation of classmethod and others can be found at https://docs.python.org/3.6/howto/descriptor.html#descriptor-protocol