Effective Python (26) — Use multiple inheritance only for mix-in utility classes

Python is an object-oriented language with built-in facilities for making multiple inheritance tractable. However, it's better to avoid multiple inheritance altogether.

If you find you yourself desiring the convenience and encapsulation that comes with multiple inheritance, considering writing a mix-in instead. A mix-in is a small class that only defines a set of additional methods that a class should provide. Mix-in classes don't define their own instance attributes nor require their

1
__init__

constructor to be called.

Writing mix-ins is easy because Python makes it trivial to inspect the current state of any object regardless of its type. Dynamic inspection lets you write generic functionality a single time, in a mix-in, that can be applied to many other classes. Mix-ins can be composed and layered to minimize repetitive code and maximize reuse.

For example, say you want the ability to convert a Python object from its in-memory representation to a dictionary that's ready for serialization. Why not write this functionality generically so you can use it with all of your classes?


1
2
3
class ToDictMixin(object):
    def to_dict(self):
        return self._traverse_dict(self.__dict__)

The implementation details are straightforward and rely on dynamic attribute access using

1
hasattr

, dynamic type inspection with

1
isinstance

, and accessing the instance dictionary

1
__dict__

.


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
def _traverse_dict(self, instance_dict):
    output = {}
    for key, value in instance_dict.items():
        output[key] = self._traverse(key, value)
    return output

def _traverse(self, key, value):
    if isinstance(value, ToDictMixMin):
        return value.to_dict()
    elif isinstance(value, dict):
        return self.__traverse_dict(value)
    elif isinstance(value, list):
        return [self._traverse(key, i) for i in value]
    elif isinstance(value, '__dict__'):
        return self._traverse_dict(value.__dict__)
    else:
        return value

Here, I define an example class that uses the mix-in to make a dictionary representation of a binary tree:


1
2
3
4
5
class BinaryTree(ToDictMixin):
    def __init__(self, value, left=None, right=None):
        self.value = value
        self.left = left
        self.right = right

Translating a large number of related Python objects into a dictionary becomes easy.


1
2
3
4
5
6
7
8
9
10
11
12
13
tree = BinaryTree(10,
  left=BinaryTree(7, right=BinaryTree(9)),
  right=BinaryTree(13, left=BinaryTree(11)))
print(tree.to_dict())

>>>
{'left': {'left': None,
         'right': {'left':None, 'right':None, 'value':9},
         'value': 7},
 'right': {'left': {'left': None, 'right':None, 'value': 11},
          'right': None,
          'value': 13},
 'value': 10}

The best part about mix-ins is that you can make their generic functionality pluggable so behaviors can be overridden when required. For example, here I define a subclass of

1
BinaryTree

that holds a reference to its parent. This circular reference would cause the default implementation of

1
ToDictMixin.to_dict

to loop forever.


1
2
3
4
5
class BinaryTreeWithParent(BinaryTree):
    def __init__(self, value, left=None,
                right=None, parent=None):
        super().__init__(value, left=left, right=right)
        self.parent = parent

The solution is to override the

1
ToDictMixin._traverse

method in the

1
BinaryTreeWithParent

class to only process values that matter, preventing cycles encountered by the mix-in. Here, I override the

1
_traverse

method to not traverse the parent and just insert its numerical value:


1
2
3
4
5
6
def _traverse(self, key, value):
    if (isinstance(value, BinaryTreeParent) and
          key == 'parent'):
        return value.value # Prevent cycles
    else:
        return super()._traverse(key, value)

Calling

1
BinaryTreeWithParent.to_dit

will work without issue because the circular referencing properties aren't followed.


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
root = BinaryTreeWithParent(10)
root.left = BinaryTreeWithParent(7, parent=root)
root.left.right = BinaryWithParent(9, parent=root.left)
print(root.to_dict())

>>>
{'left': {'left': None,
         'parent': 10,
         'right': {'left': None,
                  'parent': 7,
                  'right': None,
                  'value': 9},
         'value': 7},
'parent': None,
'right': None,
'value': 10}

By defining

1
BinaryTreeWithParent._traverse

, I've also enabled any class that has an attribute of type

1
BinaryTreeWithParent

to automatically work with

1
ToDictMixin

.


1
2
3
4
5
6
7
8
9
10
11
12
13
14
class NamedSubTree(ToDictMixin):
    def __init__(self, name, tree_with_parent):
        self.name = name
        self.tree_with_parent = tree_with_parent
       
    my_tree = NamedSubTree('foobar', root.left.right)
    print(my_tree.to_dict()) # No inifinite loop
   
>>>
{'name': 'foobar',
'tree_with_parent': {'left': None,
                    'parent': 7,
                    'right': None,
                    'value': 9}}

Mix-ins can also be composed together. For example, say you want a mix-in provides generic JSON serialization for any class. You can do this by assuming that a class provides a

1
to_dict

method (which may or may not be provided by the

1
ToDictMixin

class).


1
2
3
4
5
6
7
8
class JsonMixin(object):
    @classmethod
    def from_json(cls, data):
        kwargs = json.loads(data)
        return cls(**kwargs)
   
    def to_json(self):
        return json.dumps(self.to_dict())

Note how the

1
JsonMixin

class defines both instance methods and class methods. Mix-ins let you add either kind of behavior. In this example, the only requirements of the

1
JsonMixin

are that the class has a

1
to_dict

method and its

1
__init__

method takes keyword arguments.

This mix-in makes it simple to create hierarchies of utility classes that can be serialized to and from JSON with little boilerplate. For example, here I have a hierarchy of data classes representing parts of a datacenter topology:


1
2
3
4
5
6
7
8
9
10
11
class DatacenterRack(ToDictMixin, JsonMixin):
    def __init__(self, switch=None, machines=None):
        self.switch = Switch(**switch)
        self.machines = [
          Machine(**kwargs) for kwargs in machines]

class Switch(ToDictMixin, JsonMixin):
    # ...
   
class Machine(ToDicMixin, JsonMixin):
    # ...

Serializing these classes to and from JSON is simple. Here, I verify that the data is able to be sent round-trip through serializing and deserializing:


1
2
3
4
5
6
7
8
9
10
11
12
serialized = """{
    "switch": {"ports": 5, "speed": 1e9},
    "machines": [
        {"cores": 8, "ram": 32e9, "disk": 5e12},
        {"cores": 4, "ram": 16e9, "disk": 1e12},
        {"cores": 2, "ram": 4e9, "disk": 500e9}
    ]
}"""

deserialized = DatacenterRack.from_json(serialized)
roundtrip = deserialized.to_json()
assert json.loads(serialized) == json.loads(roundtrip)

When you use mix-ins like this, it's also fine if the class already inherits from

1
JsonMixin

higher up in the object hierarchy. The resulting class will behave the same way.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.