Mastering Structural Pattern Matching

The match statement

match <expression>:
    case <pattern 1> [<if guard>]:
        <handle pattern 1>
    case <pattern n> [<if guard>]:
        <handle pattern n>

OK, so it’s time to introduce a bit of terminology. The match statement is a soft keyword, and takes an expression (think: similar to the right-hand side of a variable assignment) and that becomes the subject of your case clauses.

The match statement is not a function, nor does it return anything. It simply demarcates the beginning of one or more case clauses, like a daisy-chain of if statements.

When Python executes the pattern matcher code it simply checks, in the order that you wrote the case clauses, for the first one that matches. You can have multiple case clauses that match, but only the first that it encounters that does match is used. Therefore, the ordering does matter.

The match statement picks the first match that evaluates to true

So try to order the case statements in the order you want them to match. In this sense, it is similar to how you might order a chain of if-elif-else statements.

The case clause

Each case clause represents one or more patterns that you want to match against the subject defined in the match statement.

The case clause takes one or more patterns. Each pattern can in turn have its own sub-patterns.

A case clause can optionally have a guard, which is an if statement that lets you apply boolean conditionals that must be truthy for the case clause to match. It’s a bit like the if statement in a list comprehension.

Each case clause takes a statement block of code that is executed if that case clause is the first one in the match block that matches the subject. If you want to return, yield or, say, talk to a database inside a case clause’s statement block, you can and should. This is where you put all the logic that you must invoke if the subject matches.

The match-case statements may well become the center of your code in some applications

Finite state machines; walking tree and tree-like structures with declarative patterns and recursion; an infinite loop that processes incoming requests in a microservice; the part of your ETL application that reads in raw data from a live system before cranking out JSON and putting into another. The sky is the limit.

And what’s a Capture Pattern?

The only confounding part so far is the bound names. Yes, they look an awful lot like variables. But they’re not variables, even though if it were a dictionary in any other part of your code, they would be. That’s because it’s a Capture Pattern and is part and parcel of the pattern matching engine.

When Python has to map the subject onto the patterns in case blocks it can, along the way, bind the value it finds to a name given by you. They are called name bindings or bound names because they are captured as part of the pattern matching process. Once they are bound, though, you can use them as though they were variables. Crucially, it is only when Python attempts to pattern match that they take on the ephemeral nature of not being a variable.

Indeed, if the case clause succeeds, we print the greeting, and at that point everything makes sense again.

You can use named bindings to match large swathes of the subject

So you are by no means limited to just the values of a dictionary. As you’ll soon see, we can do so much more than that.

But always remember that named bindings are not variables. There is also the awkward matter of what happens when a pattern is partially matched, but ultimately fails. But I’ll cover the gotchas in a later chapter as they deserve scrutiny too.

A named binding itself matches (or not!) parts of your pattern

Indeed I can capture the values of the dictionary but of course there’s an implicit assumption: that the keys exist in the first place and has some value, even if that value is None.

Therefore – and this is crucial – the named binding itself affects the pattern you want the subject to match against.

A pattern is declarative and not imperative

Recall that imperative programming is writing code that tells Python what to do. With a pattern you do not tell Python what to do; instead, you declare the result or outcome you want, and you expect Python to figure out the nitty-gritty details.

By the way …

Specifically PEP-634, PEP-635 and PEP-636

This is very important and remembering that patterns are declarative is critical if you want to truly understand how pattern matching works. Consider the example from before: how does Python do what it does? I mean, it’s documented in several PEP specifications , and there’s of course the pattern matcher’s source code as well.

But – gotchas and engine limitations aside – it is not something that matters here. To use the structural pattern matching engine you must define patterns that are meaningful to you and Python and trust that Python will work out how to get you the answer.

Now you know that a pattern is a way of articulating a desired structure a subject must have for the pattern to match. That structure can be almost anything. But you can also extract portions of the structure that you are most interested in. That is the critical piece of what makes Structural Pattern Matching useful.

Literal Patterns

In theory the simplest of all the pattern types, the Literal Pattern matches literals, like strings, bools, numbers and None.

def literal_pattern(p):
    match p:
        case 1:
            print("You said the number 1")
        case 42:
            print("You said the number 42")
        case "Hello":
            print("You said Hello")
        case True:
            print("You said True")
        case 3.14:
            print("You said Pi")
        case _:
            print("You said something else")

The literal pattern matcher has to make a number of assumptions to work the way most people’s intuition of Python works. That means making a number of explicit exceptions that most people would otherwise find confusing.

Literal pattern checks are made with equality checking (a == b) but there are a couple of special-cased exceptions and gotchas that you should know about.

>>> assert 1.0 == 1
>>> assert 1.1 != 1

Floating point and integers are compared with equality checks. So some floating point numbers will naturally equal their integer counterparts.

You can force Python to prefer one or the other using the type constraint int() or float(), like so:

case int(1):
    print("You said the integer 1")
# or
case float(1.0):
    print("You said the floating point number 1.0")

Booleans require some forethought if you mix them with 0 and 1 literals:

>>> assert True == 1
>>> assert isinstance(True, bool)
>>> assert isinstance(True, int)

True and False are both bool and int, thus True == 1 and, in the literal pattern example above, the case True clause would never run as case 1 matches it first!

The way to fix that is to ensure the case True statement is run before case 1. That will fix the problem: 1 will match case 1 and True will match case True.

The reason is that True, False and None are matched by identity (a is b), like so:

>>> assert True is not 1
>>> assert False is not 0

In most codebases this is not going to be a problem, but it is worth knowing about nonetheless. I recommend you read Truthy and Falsy Gotchas to understand why mixing up equality and identity checking can get you into hot water.

AS Patterns

When you write patterns you may want to make certain declarations in your pattern that Python must adhere to for the pattern to match. But if you also want to bind that declaration to a name that you can use later, you must use the as pattern.

def as_pattern(p):
    match p:
        case int() as number:
            print(f"You said a {number=}")
        case str() as string:
            print(f"Here is your {string=}")

Here’s two patterns. One makes a type declaration that must match strings, and the other integers. Note that unlike the example in Literal Patterns, I have not specified a particular string or integer, though I certainly could.

When I call the code it works as you would expect, because the as statement binds the matched value on the left-hand side to the name on the right.

>>> as_pattern("Inspired Python")
Here is your string='Inspired Python'
>>> as_pattern(42)
You said a number=42

AS Patterns make it possible to bind grouped declarations

Without AS patterns you could only bind generic data in a pattern and never constrain it to a type, for instance.

Guards

Strictly speaking, guards are not patterns. They are invoked after a pattern is matched but before the code inside the case block is executed.

def greet_person(p):
    """Let's greet a person"""
    match p:
        # ... etc ...
        case str() as person if person.isupper():
            print("No need to shout - I'm not deaf")
        case str() as person:
            print(f"Nice to meet you, {person}.")

The greet_person example features a guard. Like the optional if in a list comprehension, you can optionally attach a guard to a case block. They are important if you want to make decisions based on the values bound to the names in the pattern.

In this example the greet_person function checks if a person’s name is in uppercase and, if it is, politely asks them not to shout.

So even if the pattern matches, if the guard is not truthy the whole case clause fails and the match statement proceeds to the next one.

Guards let you evaluate the bound names from a pattern and apply additional checks

Unlike the declarative nature of a pattern, expressions in a guard can have side effects or other complex logic like this:

match json.loads(record):
    case {"user_id": user_id, "name": name} if not has_user(user_id):
        return create_user(user_id=user_id, name=name)
    case {"user_id": user_id}:
        return get_user(user_id)
    case _:
        raise ValueError('Record is invalid')

You can therefore construct patterns and apply constraints that make sense from a functional perspective in your application without concerning yourself with the nitty-gritty of pulling data out of data structures.

OR Patterns

Wanting to match two or more patterns in a single case statement is a common feature. Thanks to Python’s pattern recognition system you are not limited to a single pattern. You can combine multiple patterns at the case clause level, or inside individual patterns also. The latter, in particular, is especially powerful.

One important caveat is that even though the pattern style is formally named OR Patterns the actual syntax requires you to use | and not or.

1def or_pattern(p):
2    match p:
3        case ("Hello" | "Hi" | "Howdy") as greeting:
4            print(f"You said {greeting=}")
5        case {
6            "greeting": "Hi" | "Hello",
7            "name": ({"first_name": name} | {"name": name}),
8        }:
9            print(f"Salutations, {name}")

Note that each highlighted line uses | and never or. Aside from that syntactic quirk, everything behaves in much the same way as it does in other parts of Python. I have specifically added brackets around the OR patterns on line 3 to ensure the as statement to make the relationship clear, even though it is not strictly required.

The most powerful feature of OR patterns is the ability to nest them deep inside data structures that you wish to pattern match against.

Let’s analyze lines 5 & 6 a bit more closely.

The top-most pattern’s a dictionary that mandates that a key named "greeting" must exist. But unlike the first example I gave this one expects "Hi" | "Hello" as a sub-pattern against the value of "greeting". So either "Hi" or "Hello" is a valid greeting.

Line 6 is a bit more specific. There must be a key "name" and it must have as value a dictionary with either "first_name" or "name" as a key. The value of either is bound to the name name.

Sub-patterns are powerful and expressive

The benefit of declaratively describing what we want strikes true again. It’s not uncommon to have a nice and neat data structure (and the code to understand it) in your application but, like most things, it’ll evolve and change over time. As it does, you’ll still need to support the legacy format and the newer one at the same time. OR Patterns combined with the ability to embed sub-patterns inside an existing pattern makes it readable, expressive and trivial to extend and understand.

When you bind a name in an OR Pattern it must be present in all OR patterns

Observe that on line 6 I bind the value of the keys "first_name" and "name" to name. It is impossible to have a bound variable in one part of an OR pattern and not another. If it were possible, that would mean some bound variables would be undefined and hard to reason about.

There are no equivalent AND patterns or NOT patterns

You only get OR patterns. But that is usually okay; you can constrain the patterns you define to precisely match what you need, which should hopefully eliminate the need for NOT patterns and AND Patterns.

Wildcard Patterns

Frequently you want to match anything to indicate that you do not care about the actual value at all, just that there is something there. In Python that role is historically served by _:

_, name = ["Greetings", "Elaine"]

And so it is in a pattern. You may have seen this pattern at the end of some of the examples:

match p:
    # ... etc ...
    case _:
        # ... do something. ...

That is a wildcard symbol and it matches anything. As you can represent the entirety of the subject as _ it serves as a fallback that matches anything in the event none of the other case clauses do.

You can interrogate structures with them as well, disregarding elements in a list, for instance, that you do not care about:

def wildcardpattern(p):
    match p:
        case [_, middle, _]:
            print(middle)

The pattern [_, middle, _] extracts the penultimate element from a list of exactly three elements. You cannot refer to the wildcarded elements as they are unbound; they do not have a name, and cannot be used. Any attempt to use _ in the code block will instead look for the actual variable _, if such a variable is in scope.

You can however name a wildcard with as to bind it if you so desire:

def wildcardpattern(p):
    match p:
        case [_ as first, middle, _ as last]:
            print(middle)

But that seems rather obtuse so I recommend you avoid doing that, and instead just use a bound name of your own choosing.

You can also use the *rest syntax to represent arbitrary sequences of elements, or **kwargs to represent keyword arguments, like so:

def star_wildcard(p):
    match p:
        case [_, _, *rest]:
            print(rest)
        case {"name": _, **rest}:
            print(rest)

This pattern returns *rest, a sequence of an unknown number of elements, provided there are two anonymous (wildcard) elements ahead of it:

>>> star_wildcard([1,2,3,4,5])
[3, 4, 5]

It behaves as you would expect for dictionaries also:

>>> star_wildcard({"name": "Cosmo", "age": 42, "last_name": "Kramer"})
{'age': 42, 'last_name': 'Kramer'}

Although Python is reasonably clever in deducing the structure of a list or dictionary, you cannot have more than one *rest or **kwargs token at a time. So if you want complex Prolog-style finitary relations and backtracking you will need to do some of that legwork yourself.

Do not bind things you do not need

Although you can bind most things in a pattern, you should avoid doing so if you do not require the binding. Wildcards instruct Python to disregard the value so the pattern matcher can decide the most efficient way to return the bound names you do care about.

Prefer *_ and **_ to named variants if you do not care about the bound values.

You can use wildcards in guards

So this is perfectly legitimate and a useful way of constraining a pattern beyond what you can reasonably achieve with a pattern alone:

match p:
    case [_, _, *rest] if sum(rest) > 10:
        print(rest)

Value Patterns

This is perhaps the most contentious and debated part of Python’s pattern matching implementation.

So far everything I have written pertains to static patterns. Meaning, I typed them into a Python file and I did not, in any way, include values derived from constants, variables or function arguments in the pattern itself. The subject, yes, but not the pattern.

The problems begin when you write code like this:

PREFERRED_GREETING = "Hello"


def value_pattern(p):
    match p:
        case {"greeting": PREFERRED_GREETING, "name": name} as d:
            print(d)
        case _:
            print("No match!")

It looks fine and it works. But there is a problem. PREFERRED_GREETING is a bound name and it shadows the module constant with the same name.

So the result is:

>>> value_pattern({"greeting": "Salutations", "name": "Elaine"})
{'greeting': 'Salutations', 'name': 'Elaine'}

Not the answer we were looking for. Leaving out the "greeting" key and it won’t match at all:

>>> value_pattern({"name": "Elaine"})
No match!

And the reason for that is to do with an unsettled argument about syntax. In languages that typify the use of pattern matching, like LISP for instance, you can (simplifying a bit here) quote or unquote something to indicate that it is (or is not) a variable or a symbol.

Python does not have that. There were endless discussions and, I admit, it’s a hard one to resolve without complicating the syntax and the notation further with a concept that is limited to this one feature of the language. Essentially the problem you saw above could’ve been resolved if there was a way of marking PREFERRED_GREETING as being a value (Maybe .PREFERRED_GREETING or $PREFERRED_GREETING – it does not matter) or the other way around: that every capture pattern is clearly distinguished from values sourced from outside the pattern.

The only way to use value patterns is to put the values somewhere where Python can deduce that attribute access is required.

import constants


def value_pattern_working(p):
    match p:
        case {"greeting": constants.PREFERRED_GREETING, "name": name} as d:
            print(d)
        case _:
            print("No match!")

This works, because constants is a module and getattr(constants, 'PREFERRED_GREETING') is an example of attribute access. Another would be to put constants in an Enum which – if you can – is a much better way to do things anyway. Enums are symbolic and capture both a name and a value and is a marriage made in heaven when you combine it with pattern matching.

You cannot use plain variables, arguments or constants

Python gets them confused with Capture patterns and it’s a big old mess. Where possible you should avoid passing values into the pattern matching engine unless you gate them behind an attribute lookup (some_customer.user_id for instance instead of user_id)

This is likely to be a source of bugs

Tread carefully and decide on a standard way of presenting constants or variable values you want to share with the pattern matching engine:

1. A dunder class (namedtuple, dataclasses, etc.) that hosts the values you wish to use

2. A simple wrapper class that exposes a single property with the value you want to use in the pattern

3. Use an Enum, if that is possible

4. Store constants and other module-level stuff in a module and refer to it explicitly, like so: constants.MY_VALUE

Sequence Patterns

Sequences are lists and tuples, or anything that inherits from the Abstract Base Class collections.abc.Sequence. Note though that the pattern matching engine will not expand iterables of any kind.

And unlike other parts of Python where list("Hello") is a legitimate way of generating a list of a string’s characters, this use case does apply here. Strings and bytes are treated as Literal patterns and are not considered Sequence patterns.

As you have seen by now, lists and tuples behave the way you expect them to.

You cannot represent sets in a pattern

You can have them in the subject, but you cannot use pattern matching or set constructs in a case clause. I recommend you use guards to check for equality if that is what you are looking to do.

Mapping (“Dictionary”) Patterns

Mappings here implies dictionaries (or anything that uses collections.abc.Mapping), which you have also seen how to do by now. One caveat when you pattern match dictionaries is that the pattern you specify in the case clause implies a subset check against the subject:

match {"a": 1, "b": 2}:
    case {"a": 1} as d:
         print(d)

The case clause matches the full-length dictionary. If you do not want this, you should instead enforce it with a guard:

match {"a": 1, "b": 2}:
    case {"a": 1, **rest} as d if not rest:
         print(d)

The guard checks if the rest of the dictionary is empty and only allows the match if it is.

Dictionary entries must exist when the pattern matching takes place

Relying on defaultdict to create elements as a side effect of the pattern matching process will not work, and no elements are created as a result of a pattern matching attempt. The matcher uses the object’s get(k) method to match the subject’s keys and values against the mapping pattern.

Class Patterns

Matching elementary structures like dicts and lists is useful, but in larger applications you’ll often capture this knowledge in compound objects and rely on encapsulation to present a homogeneous view of your data.

Luckily, Python 3.10 can work with most object structures with either no work, or very little work required.

from collections import namedtuple

Customer = namedtuple("Customer", "name product")


def read_customer(p):
    match p:
        case Customer(name=name, product=product):
            print(f"{name}, you must really like {product}.")

Both namedtuple and dataclasses work out of the box with the pattern matching engine. As the example above shows, extracting attributes from an object is very simple indeed.

>>> read_customer(Customer(name="George", product="bosco"))
George, you must really like bosco.

Now let’s consider an anti-pattern that oh-so-many make. Namely, putting complex side-effect-causing code in the __init__ constructor on a custom class:

class Connection:
    def connect(self):
        print(f"Connecting to server {self.host}")
        # ... do something complicated ...

    def __init__(self, host, port):
        self.host = host
        self.port = port
        self.connect()


def parse_connection(p):
    match p:
        case Connection(host=host, port=port):
            print(f"This Connection object talks to {host}")

When you create an instance of Connection with a given host and port the connect() method is called and, as it’s a demo, prints a message saying it’s connecting to the host.

>>> connection = Connection(host="example.com", port="80")
Connecting to server example.com

>>> parse_connection(connection)
This Connection object talks to example.com

Note that Python’s clever enough to not create instance of Connection during the pattern matching step. (If it had tried that, we would have seen another “Connecting to server” message.)

So even if you have side effects in your __init__ method there are some safeguards made to avoid causing them directly.