-1

I'm experimenting with Python’s asyncio to run multiple coroutines concurrently. However, in some cases asyncio.gather() seems to hang forever when one of the tasks raises an exception — even though I expect it to propagate and cancel the others

Expected behavior: As soon as task_b raises an exception, I expect task_a to be cancelled, asyncio.gather() to stop, and the exception to propagate.

Actual behavior: Sometimes it works as expected, but in other cases (especially when tasks have I/O or network calls) the program hangs indefinitely, and cancellation doesn’t seem to complete.

2
  • 1
    It'd be easier if you posted a code snippet showing how you define tasks and how you run gather. And please take a look at documentation - gather() doesn't cancel tasks when it encounters an exception Commented Oct 30 at 17:44
  • 1
    Your expected behaviour is not what asyncio.gather does, why did you expect that? You probably want to use asyncio.TaskGroup instead. Commented Oct 30 at 18:34

1 Answer 1

2

asyncio.gather will always keep waiting for every task it manages.
If one task raises an exception the others do not stop by themselves.
This becomes visible with IO bound work since those tasks remain in a
pending state forever if they never reach an await that reacts to
cancellation.

A reliable pattern is to catch the first failure and cancel the
remaining tasks explicitly. After cancellation we await them again
so the event loop can process the cancellation properly.
This prevents any hang and the original exception still propagates.

# Stop other tasks at first failure
import asyncio

async def taskA():
    try:
        while True:
            print("A running")
            await asyncio.sleep(1)
    except asyncio.CancelledError:
        print("A cancelled")
        raise

async def taskB():
    await asyncio.sleep(2)
    raise RuntimeError("Failure in B")

async def main():
    one = asyncio.create_task(taskA())
    two = asyncio.create_task(taskB())
    try:
        await asyncio.gather(one, two)
    except Exception as err:
        one.cancel()
        await one
        raise err

asyncio.run(main())

With this approach the moment taskB fails the entire group stops and the
program exits without getting stuck.

If you want you can even extend this pattern with network timeouts to make the
system more reliable

Sign up to request clarification or add additional context in comments.

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.