Koan 20: The Unreliable Messenger

Exploring traps in try/finally blocks, and the reasoning behind PEP601 and PEP765

How to Clean Up

When you work with external resources such as a database or temporary files, you often need to run some cleanup actions after you've done the work.

Python provides two options - the context manager and the try/finally block. Both are valid options, but the context manager is often lauded as being more Pythonic.

Despite this, try/finally block is still widely used. As we will discover, try/finally is simple to use and well-suited in some cases, but it does come with pitfalls, as the messenger tragically discovered.

Let us try and contact the messenger.

Part 1: The Assured Action

A variant of the try/finally pattern exists in most languages¹ and functions pretty much in the way you might imagine. Consider this trivial example:

def walk_path():
    try:
        print("Taking a step")
    finally:
        print("Leaving a footprint")

'Taking a step'
'Leaving a footprint'

The interpreter enters the first block and executes the statement. The interpreter then proceeds to the finally block. The final block always executes. It does not matter if the first block succeeds or fails.

If a failure is raised during execution as shown below, the error disrupts the normal flow and the interpreter stops executing the try block immediately. The interpreter then jumps directly to the finally block.

def walk_path():
    try:
        raise Exception("A fallen tree")
    finally:
        print("Leaving a footprint")

If you choose to handle the failure using an except block as shown below, the error is caught and handled by the except block before proceeding to the finally block. An error could also occur inside the except or else block. The interpreter would still execute the finally block before raising the new error.

def walk_path():
    try:
        raise Exception("A fallen tree")
    except Exception:
        print("Climbing over the trunk")
    else:
        print("Kicking the trunk away using superhuman strength")
    finally:
        print("Leaving a footprint")

Part 2: The Trapped Messenger

This brings us to the nature of returning values. A function can return a value from within the try block. When the interpreter encounters the return statement. It prepares to send this value back to the caller. However, it must still honor the finally block. It pauses the return process and executes the finally block first before returning the prepared value from the try block.

def walk_path():
    try:
        return "Reaching the destination"
    finally:
        print("Leaving a footprint")

However, the finally block can also contain its own return statement as shown below. When this happens, the return in the finally block wins and the return value from the try is effectively ignored.

def walk_path():
    try:
        return "Reaching the destination"
    finally:
        return "Returning home"

This behavior applies to exceptions as well. We can place a loop around our structure to observe break and continue statements.

def scout_path():
    for step in range(3):
        try:
            raise Exception("A hidden trap")
        finally:
            break
    return "The scout survives"
'The scout survives'

A break statement inside a finally block will swallow any unhandled exception from the try block. A continue statement will do the exact same thing. The exception disappears completely.

Part 3: The Trapped Voice

Lets examine a more complex example with nested try/finally statements. Do return statements break out of parent try/finally blocks?

def send_message():
    try:
        print("Everything is fine")
        return 0
    finally:
        try:
            try:
                print("Everything is still fine")
            finally:
                for x in range(2):
                    print(f"Scouting area {x}")
                return 1
        finally:
            for x in range(2):
                print(f"Covering tracks in area {x}")
                return 2
print(f"Return value: {send_message()}")

Everything is fine
Everything is still fine
Scouting area 0
Scouting area 1
Covering tracks in area 0
Covering tracks in area 1
Return value: 2

No, try/finally statements can be nested, and return statements from child blocks do not prevent parent finally blocks from running. However, the value from the last return statement trumps the rest and is still the one that is returned by the function.

Part 4: The Plot Thickens

As you can imagine, any language which allows you to write dead code that produces unintended outcomes is problematic. The Python language developers recognized this danger, and proposed that return/break/continue statements should be disallowed in finally blocks in PEP601².

However, it was voted down for the following reason:

Reading the references in the PEP it seems to me that most languages implement this kind of construct but have style guides and/or linters that reject it. I would support a proposal to add this to PEP 8 (if it isn’t already there).

I note that the toy examples are somewhat misleading – the functionality that may be useful is a conditional return (or break etc.) inside a finally block.

-Guido 2019 PEP601

Guido’s reasoning was that there may be valid scenarios where the user requires full control of exception handling in the finally block, and may wish to override the raising of exceptions. Preventing this behavior would effectively hamstring advanced users.

However, in 2024 the community tried again with PEP765³. This time they were armed with evidence. They analyzed the top 8000 PyPi packages and found that:

Most of the usages (of return in finally) are incorrect, and introduce unintended exception-swallowing bugs. - PEP765

This was enough for the proposal to get over the line, and from Python 3.14 onwards, using return, break or continue in a finally clause emits a SyntaxWarning.

Part 5: Why use try/finally at all?

A context manager is the pythonic choice most of the time when you’re working with resources that already expose acquire/release semantics that can easily slot into __enter__ and __exit__. It’s the natural choice for files, locks, database connections, temporary state changes, etc. It’s declarative and minimizes surface area for errors.

However, they don’t always make sense:

• Context manager code lives in a different location, and so introduces an extra layer of abstraction. Sometimes your code is so simple and localized that introducing an extra layer of abstraction would make the code less readable with little benefit.

• Context managers can only use variables passed in during initialization, whereas finally can reference variables mutated during execution in the try block.

• With try/finally, you can combine except and finally clauses to explicitly manage different failure modes for more granular control.

Closing the Circle

The novice believed the original message was secure. The master understood the final seal controls the truth.

The finally block always speaks last. You must ensure its final words do not obscure the truth of what came before.

1. https://en.wikipedia.org/wiki/Exception_handling_syntax

2. https://peps.python.org/pep-0601/

3. https://peps.python.org/pep-0765/