The __set_name__ method for descriptors

Descriptors generally have to interact with attributes of the managed object, and this is done by inspecting __dict__ on that object (or calling getattr/setattr, but the problem is the same), and finding the key under the specific name.

For this rea­so­n, the des­crip­tor wi­ll ha­ve to know the na­me of the key to look ­fo­r, whi­ch is re­lated to the na­me of the attri­bu­te is ma­na­gin­g.

On pre­vious ver­sions of Py­thon this had to be do­ne ex­pli­ci­tl­y. If we wanted to­ wo­rk around it, the­re we­re so­me mo­re ad­van­ced wa­ys to do so. Lu­cki­l­y, afte­r PE­P-487 (a­dded in Py­thon 3.6), ­the­re are so­me enhan­ce­men­ts re­gar­ding cla­ss crea­tio­n, whi­ch al­so affec­ts ­des­crip­tor­s.

Le­t’s re­view the pro­ble­m, the pre­vious appro­aches to ta­ck­le it, and the mo­der­n way of sol­ving it.

Configure the name of the descriptor

The des­crip­tor nee­ds to so­me­how know whi­ch attri­bu­te wi­ll be mo­di­fyin­g, and fo­r ­this, the most co­m­mon so­lu­tion is to sto­re the attri­bu­te na­me in­ter­na­ll­y. Fo­r e­xam­ple in:

class LoggedAttr:
    def __init__(self, name=None):
        self.name = name

    def __get__(self, instance, owner):
        if instance is None:
            return self
        return instance.__dict__[self.name]


class Managed:
    descriptor = LoggedAttr('descriptor')

What we re­qui­re is to che­ck that the na­me is pa­ss­ed to the des­crip­tor pro­per­l­y, ­ba­si­ca­ll­y:

assert Managed.descriptor.name == 'descriptor'

But we don’t want to pass the string 'descriptor' as a parameter when constructing it, because it’s repetitive. Instead, we want this to be configured automatically. Let’s see some options.

A class decorator

Wi­th a cla­ss de­co­ra­to­r, we could de­fi­ne all de­co­ra­tors for the cla­ss as ­pa­ra­me­ters of the de­co­ra­to­r, and make the as­sig­n­ment of the na­me in it as we­ll.

So­me­thing like this:

class configure_descriptors:
    def __init__(self, **kwargs):
        self.descs = {dname: dcls(dname) for dname, dcls in kwargs.items()}

    def __call__(self, class_):
        for dname, descriptor in self.descs.items():
            setattr(class_, dname, descriptor)
        return class_


@configure_descriptors(
    descriptor=LoggedAttr
)
class DecoratedManaged:
    """The descriptor is provided by the decorator"""

The con­di­tion is pre­ser­ve­d:

assert DecoratedManaged.descriptor.name == 'descriptor'

In this decorator, we provide the name and the class of the descriptor to be created, and the decorator instantiates the class with this name. We could also have created the instance directly in the descriptor, and then update the value with setattr(descriptor, 'name', dname), which is more general, in case you want to create descriptors that take multiple arguments on their __init__ method, but for this case it’s just fine.

Then we set the new des­crip­tor (the one that has the na­me al­ready up­dated on i­t), to the wra­pped cla­ss.

Ho­we­ve­r, it sti­ll see­ms a bit un­fa­mi­liar or coun­te­r-in­tui­ti­ve that we’­re ­de­fi­ning the des­crip­tor not in the body of the cla­ss, but as a pa­ra­me­ter of a ­de­co­ra­to­r.

The­re must be ano­ther wa­y.

A meta-class

Imagine we flag the class by adding a __set_name = True attribute on it, in order to hint the meta-class that this is going to be one of the attributes that need its name changed. Then the meta-class would look something like:

class MetaDescriptor(type):
    def __new__(cls, clsname, bases, cls_kwargs):
        for attrname, cls_attr in cls_kwargs.items():
            mangled_attr = "_{0}__set_name".format(cls_attr.__class__.__name__)
            if hasattr(cls_attr, mangled_attr):
                setattr(cls_attr, 'name', attrname)
        return super().__new__(cls, clsname, bases, cls_kwargs)


class MetaManaged(metaclass=MetaDescriptor):
    descriptor = LoggedAttr()

And agai­n:

assert MetaManaged.descriptor.name == 'descriptor'

One detail is that the __init__ of the descriptor accepts the name to be nullable so this works. Another option would have been defining only the descriptor assigned to the class, and then, re-mapping the attribute with the instance, passing the name when it’s being constructed on the meta-class. Both options are the same, and the example was made with simplicity in mind.

This wo­rks but it has a cou­ple of is­sues. First we ha­ve to so­me­how iden­ti­fy when the cla­ss attri­bu­te nee­ds to be up­dated (in this ca­se, a flag was added to­ i­t, but other al­ter­na­ti­ves are no be­tter at all). The se­cond pro­blem should be­ ­ra­ther ob­vious: it’s not a good use of me­ta-­cla­sses, and this is ove­rki­ll (to­ s­ay the leas­t) for what should be a sim­ple ta­sk.

The­re must be a be­tter wa­y.

__set_name__

And there is. At least for Python 3.6 and higher. The __set_name__ method was included, which is automatically called when the class is being created, and it receives two parameters: the class and the name of the attribute as it appears defined in the class.

Wi­th this, the pro­blem is re­du­ced to just sim­pl­y:

class LoggedAttr:
    ...
    def __set_name__(self, owner, name):
        self.name = name

And tha­t’s it, no other co­de is nee­de­d. The so­lu­tion is mu­ch sim­ple­r, and it en­tails le­ss pro­ble­ms.

Actually, I deliberately named the flag __set_name, to get an idea of what’s coming, and to hint that with __set_name__, Python must be doing something similar to the example, but in this case we shouldn’t worry about it.

Conclusion

Even thou­gh it’s fi­ne to just know about the last me­tho­d, and we could sim­pl­y u­se tha­t, it’s sti­ll im­por­tant to ha­ve fo­llo­wed this pa­th, thi­nking about ho­w ­things we­re do­ne pre­vious­l­y, be­cau­se it’s not fair to just as­su­me things we­re a­lwa­ys good, and take that for grante­d. Othe­rwi­se, we would miss the evo­lu­tio­n of the lan­gua­ge, and as­su­me the­re we­re ne­ver is­sues, pro­ble­ms or things tha­t ­nee­ded re­vi­sio­n.

And more importantly, there still are. Python still has lots of other areas for improvement. Just as in this example __set_name__ seems to solve a small, yet annoying problem, there are many other scenarios on which things are not crystal clear in Python, so the language still needs to evolve.