Separating type-specific code with single­dispatch

Author 

Have you ever found yourself writing a litany of if-elif-else statements peppered with isinstance() calls? Notwithstanding error handling, they are often found where your code intersects with APIs; third-party libraries; and services. As it turns out, coalescing complex types – such as pathlib.Path to string, or decimal.Decimal to float or string – is a common occurrence.

But writing a wall of if-statements makes code reuse harder, and it can complicate testing:

# -*- coding: utf-8 -*-
from pathlib import Path
from decimal import Decimal, ROUND_HALF_UP


def convert(o, *, bankers_rounding: bool = True):
    if isinstance(o, (str, int, float)):
        return o
    elif isinstance(o, Path):
        return str(o)
    elif isinstance(o, Decimal):
        if bankers_rounding:
            return float(o.quantize(Decimal(".01"), rounding=ROUND_HALF_UP))
        return float(o)
    else:
        raise TypeError(f"Cannot convert {o}")


assert convert(Path("/tmp/hello.txt")) == "/tmp/hello.txt"
assert convert(Decimal("49.995"), bankers_rounding=True) == 50.0
assert convert(Decimal("49.995"), bankers_rounding=False) == 49.995

In this example I have a convert function that converts complex objects into their primitive types, and if it cannot resolve the given object type, it raises a TypeError. There’s also a keyword argument, bankers_rounding intended for the decimal converter.

Let’s quickly test the converter to make sure it works:

>>> json.dumps({"amount": Decimal('49.995')}, default=convert)
'{"amount": 50.0}'

Yep. It does. Remove the default= argument and the dumps function throws an exception because it does not understand how to serialize Decimal.

But now I’ve trapped a number of independent pieces of logic in one function: I can convert data, yes, but how can I easily test that each conversion function actually does what it’s supposed to? Ideally, there would be a clear separation of concerns. And the keyword argument bankers_rounding only applies to the Decimal routine and yet it’s passed to our shared convert function. In a real-world application there might be dozens of converters and keyword arguments.

But I think we can do better. One easy win is to separate the converter logic into distinct functions, one for each type. That has the advantage of letting me test – and use independently – each converter in isolation. That way I specify the keyword arguments I need for the converter functions that need them. The bankers_rounding keyword is not tangled up with converters where it does not apply.

The code for that will look something like this:

def convert_decimal(o, bankers_rounding: bool = False):
    if not bankers_rounding:
        return str(o)
    else:
        # ...

# ... etc ...

 def convert(o, **kwargs):
     if isinstance(o, Path):
         return convert_path(o, **kwargs)
     else:
         # ...

At this point I have built a dispatcher that delegates the act of converting the data to distinct functions. Now I can test the dispatcher and the converters separately. At this point I could call it quits, but I can get rid of the convert dispatcher almost entirely, by offloading the logic of checking the types to a little-known function hidden away in the functools module called singledispatch.

How to use @singledispatch

First, you need to import it.

>>> from functools import singledispatch

In Python 3.7 singledispatch gained the ability to dispatch based on type hints which is what this article uses.

Much like the dispatcher from before, the approach used by singledispatch is much the same.

@singledispatch
def convert(o, **kwargs):
    raise TypeError(f"Cannot convert {o}")

The singledispatch decorator works in a similar way to the home-grown approach above. You need a base function to act as a fallback for any unknown types it is given. If you compare the code with the earlier example, this is akin to the else portion of the code.

At this point the dispatcher cannot handle anything and will always throw a TypeError. Let’s add back the decimal converter:

1@convert.register
2def convert_decimal(o: Decimal, bankers_rounding: bool = True):
3    if bankers_rounding:
4        return float(o.quantize(Decimal(".01"), rounding=ROUND_HALF_UP))
5    return float(o)

Note the decorator. The singledispatch decorator transmutes the base function into a registry for future types you want to register against that base function. As I am using Python 3.7+ I have opted for type annotations, but if you do not wish to do this, you must replace the decorator with @convert.register(Decimal) instead.

The name of the function is convert_decimal and sure enough it works on its own:

>>> convert_decimal(Decimal('.555'))
0.56
>>> convert_decimal(Decimal('.555'), bankers_rounding=False)
0.555

Now I can write tests for each converter and leave the messy type checking to singledispatch.

Simultaneously, I can invoke convert with the self-same arguments and it works as you would expect: the arguments I give it are dispatched to the convert_decimal dispatcher function I registered earlier:

>>> convert(Decimal('.555'), bankers_rounding=True)
0.56

Dynamically querying and adding new dispatchers

One useful side-effect of singledispatch is the ability to dynamically register new dispatchers and even interrogate the existing registry of converters.

def convert_path(o: Path):
    return str(o)

If you wanted to add the convert_path function dynamically you can:

>>> convert.register(Path, convert_path)
<function __main__.convert_path(o: pathlib.Path)>

If you want a mapping of types to the underlying functions, the convert.registry will show you what it supports:

>>> convert.registry
mappingproxy({object: <function __main__.convert(o, **kwargs)>,
              pathlib.Path: <function __main__.convert_path(o: pathlib.Path)>,
              decimal.Decimal: <function __main__.convert_decimal(o: decimal.Decimal, bankers_rounding: bool = True)>})

You can also ask the dispatcher to tell you the best candidate function to dispatch to, given a type:

>>> fn = convert.dispatch(Path)
>>> assert callable(fn)
>>> fn(Path('/tmp/hello.txt'))
'/tmp/hello.txt'

Limitations of @singledispatch

The singledispatch function is useful, but it is not without limitations. Its main limitation is also apparent in its name: it can only dispatch based on a single function parameter, and then only the first. If you require multiple dispatching you will need a third-party library as Python does not come with that built in.

Another limitation is that singledispatch works only with functions. If you need it to work on methods in classes you must use singledispatchmethod.

Summary

singledispatch encourages separation of concerns

By separating the type checking from the converter code you can test each function in isolation and your code, as a result, is easier to maintain and reason about.

Converter-specific parameters are separate from the dispatcher

This ensures that, say, bankers_rounding is declared only on converters that understand it. That makes the function signature easier to parse for other developers; it greatly improves the self-documenting nature of your code; and it cuts down on bugs as you cannot pass invalid keyword arguments to functions that do not accept it.

singledispatch makes it easy to extend the central dispatcher

You can attach new dispatchers (and query the registry of existing ones) to a central hub of dispatchers in your code: a common library can expose common dispatchable functions, and each “spoke” that uses the dispatchers can add their own without modifying the original dispatcher code.

singledispatch works with custom classes and even abstract base classes

Dispatching based on custom classes – including subclasses – is possible, and even encouraged. If you’re using ABCs you can also use them to dispatch to your registered functions.

Liked the Article?

Why not follow us …
NewsletterIcon for TwitterTwitterIcon for LinkedInLinkedIn

Be Inspired Get Python tips sent to your inbox

We'll tell you about the latest courses and articles.

Absolutely no spam. We promise!