Five Advanced Pytest Fixture Patterns

Factory Fixture: Fixtures with Arguments

Passing arguments to a fixture is the first thing people want to do with fixtures when they start using them.

But initializing hard-coded data is the precursor to that, and it’s what the @fixture decorator excels at:

import pytest


@pytest.fixture
def customer():
    customer = Customer(first_name="Cosmo", last_name="Kramer")
    return customer


def test_customer_sale(customer):
    assert customer.first_name == "Cosmo"
    assert customer.last_name == "Kramer"
    assert isinstance(customer, Customer)

But because pytest is just passing objects around – with a sprinkling of magic to make it all work properly behind-the-scenes – you can return just about anything — including a factory that lets you control how your test data is initialized by passing arguments with the values you want:

import pytest


@pytest.fixture
def make_customer():
    def make(
        first_name: str = "Cosmo",
        last_name: str = "Kramer",
        email: str = "test@example.com",
        **rest
    ):
        customer = Customer(
            first_name=first_name, last_name=last_name, email=email, **rest
        )
        return customer

    return make


def test_customer(make_customer):
    customer_1 = make_customer(first_name="Elaine", last_name="Benes")
    assert customer_1.first_name == "Elaine"
    customer_2 = make_customer()
    assert customer_2.first_name == "Cosmo"

This is a very powerful pattern; the new fixture, now named make_customer to make it obvious to someone that it’s a factory that makes something, lets you override the first_name and last_name parameters instead of hardcoding them. But if you don’t care what they are in a particular test, you can leave them out.

How it works is like this: instead of returning an instance of Customer (like the first example demonstrates), instead it returns a function (called make) that does all the heavy lifting for us. That function is the factory; it’s responsible for creating the ultimate object and return it.

Clearly name your factory fixtures

You can absolutely use generic names, like customer, for factory fixtures, but I recommend you do not. Instead make it clear the user of the fixture – it may well be someone other than you – should instantiate it first and not just assume it returns an instance of Customer.

Initializing a fixture with static values is easy, but passing arguments requires a function

You can create fixtures with arguments by returning a function — that function is called a factory.

Composing Fixtures

Consider two separate fixtures in a test suite for a fictional ecommerce store. Now let’s say you want to represent the concept of a transaction – i.e., a customer having completed a sale – you could do this by creating fixture like this:

@pytest.fixture
def make_transaction():
    def make(amount, sku, ...):
        customer = Customer(...)
        sale = Sale(amount=amount, sku=sku, customer=customer, ...)
        transaction = Transaction(sale=sale, ... )
        return transaction
    return make

But instead of repeating yourself unnecessarily, you can instead take advantage of the fact that pytest fixtures can, in turn, depend on other fixtures:

1import pytest
2
3
4@pytest.fixture
5def make_transaction(make_customer, make_sale):
6    def make(transaction_id, customer=None, sale=None):
7        if customer is None:
8            customer = make_customer()
9        if sale is None:
10            sale = make_sale()
11        transaction = Transaction(
12            transaction_id=transaction_id,
13            customer=customer,
14            sale=sale,
15        )
16        return transaction
17
18    return make

This fixture takes a mandatory transaction_id and an optional customer and sale. If the latter two aren’t specified, they’re created by the fixture automatically by calling their respective fixtures.

Two-way data binding with closures and monkeypatch

Occasionally, you need to mock, fake or stub out parts of your code base to ease the testing of other parts of your code. Usually to achieve a certain goal, like simulating rare error conditions, or scenarios that you cannot easily reproduce without such techniques.

The monkeypatch feature in pytest lets you do this with a test, but you can also do it in a fixture. When you patch out a piece of code you may want to check that it was at least called with the expected parameters. That is especially important if the patched function is nestled deep inside other pieces of logic that makes it hard or impossible to query directly.

That alone is why a lot of developers opt to put the monkeypatched code in a test instead of a fixture: you can directly control the patched code and interrogate its state, but there’s an easy way to do the same with a fixture:

1@pytest.fixture
2def mock_send_email(monkeypatch):
3    sent_emails = []
4
5    def _send_email(recipient, subject, body):
6        sent_emails.append((recipient, subject, body))
7        return True
8
9    monkeypatch.setattr("inspired.order.send_email", _send_email)
10    return sent_emails

Here I’m patching out a real send_email function somewhere in our fake ecommerce codebase. Let’s pretend it returns True if it sends an email. But if you want to check that it was called, you need to do a bit more leg work. So, I patch it out with a mocked version that captures the sent emails into a list sent_emails. But I also return that same list!

This approach works because:

The _send_email mock function lexically binds sent_emails to itself

That means the _send_email function holds on to the sent_emails object even though scope of the function changes when it’s patched into our “real” ecommerce codebase at inspired.order.send_email.

Everything is an object

Which, of course, includes both functions and lists. So when I return sent_emails I am in fact returning the selfsame list object that our test can then access.

The end result is that I can query the sent_emails list whenever I need to.

Let’s say I want to test that commit_order can take an instance of Transaction – which contains a reference to a customer and a sale along with a transaction_id – and send an email (and whatever housekeeping you’d expect it to do in a real ecommerce system):

1def test_send_email(make_transaction, mock_send_email):
2    assert mock_send_email == []
3    transaction = make_transaction(transaction_id="1234")
4    # Commit an order, which in turn sends a receipt via email with
5    # `send_email` to the customer.
6    commit_order(transaction=transaction)
7    assert mock_send_email == [
8        (
9            "test@example.com",
10            "Your order number 1234",
11            "Thank you for buying...",
12        )
13    ]

As you can see, the list object is available and its state changes whenever it’s mutated by _send_email, the mock function in the fixture. Now in this example the flow of data is one way: from mock function to the test caller, but you can easily reverse it, and modify the list in the test and have its changes reflected back into the mock function.

monkeypatch is itself a fixture

You can abstract frequently monkeypatched parts of your code into a fixture and use that instead. It’s a great abstraction tool and centralizes something that is very easy to mess up in ways that can be fantastically hard to debug.

You can combine this pattern with the factory pattern

And gain the benefits of simplifying complex or tedious instantiation patterns plus the two-way data binding.

Two-way data binding with closures is a useful tool in your toolbox

The example is trivial, of course, but for complex hierarchies of objects or state, the ability to track the changes made throughout the system is a useful pattern. And naturally this idea is generalizable to not just fixtures or test code.

The existing unittest.mock can do this also

If you prefer the classic unittest.mock approach you can do it with @patch, MagicMock.assert_called_once_with() and friends. However, I prefer this approach, because it’s explicit. The only magic is the patching; the rest is just Python. And there are no limits to what you can write in your patched function, nor what you return or how you choose to formalize the contract between fixture and test.

Tear down and Setup Fixtures with yield

If you yield inside a fixture you can return a scoped object or value at that point in time, and pytest will only resume the generator once the test is complete. You can use this pattern to create traditional setup and teardown patterns:

@pytest.fixture
def db_connection():
    connection = create_database_connection(host="localhost", port=1234)
    try:
        connection.open()
        yield connection
    finally:
        connection.close()

Here I create a connection object, open() it, and then yield it to the test. When the test exits – for any reason – the finally clause is run and then I close() the connection.

One unfortunate problem with this approach is that it does not work with the factory pattern. To work around that, you can combine them using pytest’s request.addfinalizer() function:

@pytest.fixture
def make_db_connection(request):
    def make(host: str = "localhost", port: int = 1234):
        connection = create_database_connection(host=host, port=port)
        connection.open()

        def cleanup():
            connection.close()

        request.addfinalizer(cleanup)
        return connection

    return make

Like the factory pattern from earlier, I return an instantiated connection with the host and port parameter values drawn from make. I open the connection – as before – but to ensure the cleanup happens when the test is finished, I add cleanup to pytest’s addfinalizer.

It’s a bit more complicated, but it ensures a clean separation between creation and destruction of a resource.

Now you might wonder why I couldn’t just yield from inside make? Well, you can, and it works OK… but you won’t get automatic cleanup when the test exits. The reason is that pytest only cleans up the make_db_connection fixture if it’s a generator function, but it isn’t… so it doesn’t.

Use yield to manage teardown and setup of application state, like database connections or files

It works well if you don’t need the factory pattern, and you can yield whatever you like: a tuple of objects, if that’s what you need.

I recommend you wrap the setup and teardown in try and finally even though there are some guarantees made by pytest that it’ll try and clean things up if pytest crashes.

You can use request.addfinalizer if you have especially complex requirements

It works with everything, and you can call it from tests, too, if you have to. It’s also the simplest way to pair factory patterns with other patterns I have demonstrated, like two-way binding.

Triggering Side-Effects and Errors with monkeypatch

Unlike the unittest.mock library, the monkeypatch tool is surprisingly simple and lightweight. Part of the reason is that you can freely use parts of the existing mock library as you like; but I am of the opinion that it’s simply a different approach to mocking and patching than what you get with monkeypatch.

def commit_order(transaction):
    # Checks the stock levels and returns the count
    # remaining and an error if it's out of stock.
    check_stock_levels(transaction.sale.product)

    # Saves the transaction the DB and raises an
    # error if its transaction ID already exists
    save_to_db(transaction)

    # ... do a bunch more stuff ...

    # Send a thank-you email
    send_email(
      first_name=transaction.customer.first_name,
      # .. etc ...
   )
   return True

Let’s go back to the ecommerce order system. Let’s say we want to test – from the perspective of the ecommerce’s frontend, say the UI or REST API – what happens if commit_order(transaction) triggers an error, somehow. Now let’s also pretend that this is the main entrypoint into the order system: it does all the database stuff; it sends emails; it checks and updates stock inventory — it’s got a lot of moving parts, and it’s an important cog in the machinery.

Let’s flesh out a couple of potential errors to simulate:

1. An item sold out in the time it took the user to click “order now” and before the system could reconcile its inventory. Or maybe there’s a human on the other end in the fulfillment center that updates a transaction with “out of stock”.

2. A duplicate transaction made its way through, somehow, due to an accidental double-buy. There’s a check in place to prevent duplicate entries in the database, like database-level constraints, for instance.

In that case we need a way to test that these scenarios are handled correctly. Ordinarily you’d write an exhaustive test suite to reproduce them, and that’s great and all, but maybe you’re testing other parts of the code, like we are here, and how they interact with error states. Or maybe you’re building a fixture that can trigger these cases arbitrarily, so other developers in your team can make use it for other test cases. Either way, it’s the same situation.

@pytest.fixture
def mock_fulfillment(monkeypatch):
    state = {"out_of_stock": False, "known_transactions": set()}

    def _check_stock_levels(product):
        if state["out_of_stock"]:
            raise OutOfStockError(product_id=product.sku)
        else:
            return product.stock

    def _save_to_db(transaction_id):
        if transaction_id in state["known_transactions"]:
            raise DuplicateTransactionError(transaction_id=transaction_id)
        return False

    monkeypatch.setattr("inspired.order.save_to_db", _save_to_db)

    monkeypatch.setattr("inspired.order.check_stock_levels", _check_stock_levels)

    return state

So what you’d want is a fixture specifically designed to patch key parts of commit_order – not the whole thing – to induce these two error scenarios. What I’d want is a feature switch to test before/after conditions and ensure the system correctly handles them all.

The example above accomplishes this by reusing several patterns from earlier. Instead of a list, it’s a dictionary with the state the system is supposed mirror. Substitute this for any number of situations in your own codebases.

Now a matching test:

def test_commit_order(mock_fulfillment, make_transaction):
    transaction = make_transaction(transaction_id=42)
    assert not mock_fulfillment["out_of_stock"]
    assert commit_order(transaction)
    # Now again, but this time we test an out of stock event:
    with pytest.raises(OutOfStockError):
        mock_fulfillment["out_of_stock"] = True
        commit_order(transaction)

I think it speaks for itself. By modifying state I can induce an error by flicking a switch. And, indeed, the mock function abides and raises OutOfStockError as we’d expect.

By selectively patching only the parts of the code that needs patching you minimize the likelihood that your patches are overreaching. An all too common occurrence in real life. You could easily do this with multiple fixtures, each with and without an error state, and multiple tests. But if you have a sufficiently complex set of interactions that may not be feasible or maintainable.

I like this approach because it hews closer to the idea that we swap out function-for-function and that the function has a minimal amount of logic that you can modify over time as your requirements grow. It’s all too easy to end up daisy-chaining Mock() objects only to have to refactor all of it if the structure changes slightly. Or, worse, due to how “flexible” Mock objects are, your code may have fundamentally changed and broken in some way, but your test stays green!

Summary

Fixtures are not just there to initialize simple objects

But, of course, if you don’t need more than that — good. Simplicity is a good thing. But, if you work on large or complex codebases, that may not be enough.

You, the developer, determine the relationship and contract you have with a fixture

I’ve shown that you can use two-way binding to effect changes inside a mocked function; but, it adds complexity. I find the complexity manageable when the alternative is a dozen tests, each just different enough to warrant a new test or new fixture to support it, but without the impetus for anyone to try and refactor how they test things to begin with.

Two-way binding and factory patterns go a long way towards solving some of the sprawling test suites you end up with in real codebases.

Don’t forget unittest.mock

The monkeypatch fixture is intentionally simple, possibly as a cautious overreaction to how confusing and complicated mock can be. But the silver lining is that it forces Python developers to trade magic for explicit code, even if lexical scoping and mutability adds its own complexities.