Descriptors & Decorators

Des­crip­tors are an ama­zing tool to ha­ve in our tool­bo­x, as they co­me in hand­y in many oppor­tu­ni­tie­s.

Pro­ba­bly the best thing about des­crip­tor­s, is that they can im­pro­ve othe­r ­so­lu­tion­s. Le­t’s see how we can wri­te be­tter de­co­ra­tor­s, by using des­crip­tor­s.

Decorate a class method

Ima­gi­ne we ha­ve a ve­ry sim­ple de­co­ra­to­r, that does no­thing but re­tur­ning a ­tex­t, wi­th what the ori­gi­nal func­tion re­turn­s:

class decorator:
    def __init__(self, func):
        self.func = func
        wraps(func)(self)

    def __call__(self, *args, **kwargs):
        result = self.func(*args, **kwargs)
        return f"decorated {result}"

class Object:
    @decorator
    @classmethod
    def class_method(cls):
        return 'class method'

If we apply the de­co­ra­tor to a sim­ple func­tio­n, it’­ll wo­rk, as ex­pec­te­d. Ho­we­ve­r, when it’s applied to a cla­ss me­tho­d, we can see an erro­r:

>>> Object.class_method()
Traceback (most recent call last):
...
TypeError: 'classmethod' object is not callable

The ex­cep­tion is te­lling us that we tried to ca­ll so­me­thing that is no­t ac­tua­lly a ca­lla­ble. But if tha­t’s the ca­se then, how do cla­ss me­tho­ds run?

The fact is that, this is true, class methods are indeed not callable objects, but we rarely notice this, because when we access a class method, it’s usually in the form of <class>.<class_method> (or maybe also from an instance doing self.<class_method>). For both cases the answer is the same: by calling the method like this, the descriptor mechanism is triggered, and will call the __get__ inside the class method. As we already know from the analysis of the Types of Descriptors, @classmethod is actually a descriptor, and the definition of its __get__ method is the one that returns a callable 1, but @classmethod is not itself a callable.

Hint

@classmethod is not a callable object. It’s a descriptor whose __get__ method returns a callable.

No­w, when the de­co­ra­tor is applied to the cla­ss me­tho­d, this is equi­va­len­t of doin­g:

class Object:
    ...
    class_method = decorator(class_method)

Which doesn’t trigger the descriptor protocol, so the __get__ in @classmethod is never called, therefore what the decorator receives, is not a callable, hence the exception.

By now, it becomes clear that if the reason why it fails is because @classmethod is a non-callable descriptor, then the solution must be related to descriptors. And indeed, this can be fixed by just implementing __get__.

class decorator:
    ...
    def __get__(self, instance, owner):
        mapped = self.func.__get__(instance, owner)
        return self.__class__(mapped)

This li­nks the func­tion of the des­crip­tor to the ob­ject that is going to use it (in this ca­se, the cla­ss), and re­turns a new ins­tan­ce of the de­co­ra­tor for this ­func­tio­n, whi­ch does the tri­ck.

It’s important to notice that this error was due to the order on which descriptors where applied, because @decorator was decorating @classmethod and not the other way around. This problem wouldn’t have occurred if we swapped the order of the decorators. So it’s a fair question to ask, why wasn’t this just applied like this to begin with? After all, a class method-like functionality is orthogonal from every other sort of decoration we might want to apply, so it makes sense to be it the last one being applied. True, but the fix is rather simple, and more importantly, it makes the decorator more generic and applicable, as it’s shown on the next section.

No­te

Keep in mind the order of the decorators, and make sure @classmethod is the last one being used, in order to avoid issues. Even despite this consideration, is better to have decorators that will work in many possible scenarios, regardless of their order.

The com­ple­te co­de for this exam­ple can be found he­re

Decorators that change the signature

Sce­na­rio: Se­ve­ral par­ts of the co­de ha­ve ca­lla­ble ob­jec­ts that in­te­ract wi­th ­their pa­ra­me­ters in the sa­me wa­y, re­sul­ting in co­de re­pe­ti­tio­n. As a re­sult of ­tha­t, a de­co­ra­tor is de­vi­s­ed in or­der to abs­tract that lo­gic in a sin­gle pla­ce.

For exam­ple, we ha­ve a func­tion that re­sol­ves so­me attri­bu­tes ba­sed on its ­pa­ra­me­ter­s, but it does so, by using a hel­per ob­jec­t, created from the ­pa­ra­me­ter­s, like this:

def resolver_function(root, args, context, info):
    helper = DomainObject(root, args, context, info)
    ...
    helper.process()
    helper.task1()
    helper.task2()
    return helper.task1()

If the­re are mo­re func­tions wi­th this sig­na­tu­re doing the sa­me as in the firs­t ­li­nes, it’­ll be be­tter to abs­tract this awa­y, and sim­ply re­cei­ve the hel­pe­r ob­ject di­rec­tl­y. A de­co­ra­tor like this one should wo­rk:

class DomainArgs:
    def __init__(self, func):
        self.func = func
        wraps(func)(self)

    def __call__(self, root, args, context, info):
        helper = DomainObject(root, args, context, info)
        return self.func(helper)

This de­co­ra­tor chan­ges the sig­na­tu­re of the ori­gi­nal func­tio­n. The­re­fo­re, we ­de­co­ra­te a func­tion that wi­ll re­cei­ve a sin­gle ar­gu­men­t, when in fact (thanks ­to the de­co­ra­to­r), the re­sul­ting one wi­ll end up re­cei­ving the sa­me old fou­r ar­gu­men­ts, main­tai­ning com­pa­ti­bi­li­ty. By appl­ying the de­co­ra­to­r, we coul­d ha­ppi­ly as­su­me that the re­qui­red ob­ject wi­ll be pa­ss­ed by:

@DomainArgs
def resolver_function2(helper):
    helper.task1()
    helper.task2()
    ...
    return helper.process()

Ho­we­ve­r, the­re are al­so ob­jec­ts who­se me­tho­ds ha­ve this lo­gi­c, and we want to­ a­pply the sa­me de­co­ra­tor to the­m:

class ViewResolver:
    @DomainArgs
    def resolve_method(self, helper):
        response = helper.process()
        return f"Method: {response}"

But wi­th this im­ple­men­ta­tio­n, it wo­n’t wo­rk:

>>> vr = ViewResolver()
>>> vr.resolve_method('root', 'args', 'context', 'info')
Traceback (most recent call last)
...
     39     def __call__(self, root, args, context, info):
     40         helper = DomainObject(root, args, context, info)
---> 41         return self.func(helper)
TypeError: resolve_method() missing 1 required positional argument: 'helper'

The pro­blem is that ins­tan­ce me­tho­ds are func­tion­s, that take an ex­tra firs­t ­pa­ra­me­te­r, na­me­ly se­lf, whi­ch is the ins­tan­ce itsel­f. In this ca­se, the erro­r s­ho­wn in li­ne 41, means that the de­co­ra­tor is com­po­sing the ob­ject as usua­ll­y, and pa­s­ses it was the first pa­ra­me­te­r, in the pla­ce whe­re se­lf would go fo­r ­the me­tho­d, and the­re is no­thing being pa­ss­ed for hel­per (the pa­ra­me­ters are “s­hi­fte­d” on pla­ce to the le­ft), hen­ce the erro­r.

In or­der to fix this, we need to dis­tin­guish when the wra­pped func­tion is bein­g ­ca­lled from an ins­tan­ce or a cla­ss. And des­crip­tors do just tha­t, so the fix is ­ra­ther sim­ple as in the pre­vious ca­se:

def __get__(self, instance, owner):
    mapped = self.func.__get__(instance, owner)
    return self.__class__(mapped)

The same method works here as well. When the wrapped function is a regular one, the __get__ method doesn’t take place at all, so adding it, doesn’t affect the decorator. Whereas, when is called from a class, the __get__ method is enabled, returning a bound instance, which will pass self as the first parameter (what Python does internally).

Hint

Des­crip­tors can help wri­ting be­tter de­co­ra­tor­s, by fi­xing co­m­mon pro­ble­ms in a ve­ry ele­gant fas­hio­n.

He­re is the lis­ting for this e­xam­ple.

1

An equi­va­lent Py­thon im­ple­men­ta­tion of cla­ss­me­thod and others can be­ ­found at http­s://­do­cs.­p­y­tho­n.or­g/3.6/ho­w­to­/­des­crip­to­r.ht­m­l#­des­crip­to­r-­pro­to­col