Skip to content

Run async commands when Fire is called from a running event loop - #717

Open
aashish254 wants to merge 1 commit into
google:masterfrom
aashish254:fix/async-from-running-loop
Open

aashish254 wants to merge 1 commit into
google:masterfrom
aashish254:fix/async-from-running-loop

Conversation

@aashish254

Copy link
Copy Markdown

Fixes #716

What

_CallAndUpdateTrace ran coroutine functions with loop.run_until_complete(...) whenever
asyncio.get_running_loop() succeeded. That branch can never succeed: get_running_loop()
only returns a loop that is already running in the current thread, and
run_until_complete refuses exactly that case. So calling Fire on an async command from
async code raised RuntimeError: This event loop is already running and left the coroutine
unawaited:

import asyncio, fire

async def target():
  return 42

async def main():
  fire.Fire(target, command=[])   # RuntimeError + "coroutine was never awaited"

asyncio.run(main())

docs/guide.md:608 says "Whenever fire encounters a coroutine function, it runs it, blocking
until it completes." This makes that sentence true for the running-loop case as well.

Why this shape

The new _RunCoroutine helper keeps the no-loop path exactly as it was (asyncio.run in the
calling thread) and only changes the case that was previously guaranteed to fail: the
coroutine is driven by a fresh loop on a one-worker thread, so Fire still blocks and still
returns the awaited value. The coroutine is created before the hand-off and awaited only
once, so no "coroutine was never awaited" warning can leak.

The issue suggests returning a clear error instead. I went with running it because the
calling code is synchronous anyway — a notebook, or any Fire call made from async code —
and it now produces the documented result rather than a message telling the user to rewrite
their program. ThreadPoolExecutor is stdlib, adds no dependency, and the path is only
taken where the previous behaviour was a hard failure, so nothing that worked before stops
working.

One honest behaviour note: the coroutine runs on a new loop, not the caller's. A coroutine
that awaits objects bound to the caller's loop only works today if the loop has been patched
for nesting (nest_asyncio); under this change it gets a fresh loop instead. The common
self-contained case is unaffected — I measured both shapes returning the same value with and
without nest_asyncio applied.

Verification

  • Reproduced the failure first, on master at 716bbc2, with
    warnings.simplefilter("error", RuntimeWarning): RuntimeError: This event loop is already running plus RuntimeWarning: coroutine 'target' was never awaited.
  • Two regression tests added next to the existing testFireAsyncio, which is untouched and
    still passes: one proves the value comes back from inside a running loop, one proves the
    command's own exception still propagates through the thread hand-off (it is not swallowed
    or re-wrapped) and that command args are consumed on that path.
  • Mutation check: with the old call site restored and only the tests kept, exactly those 2
    tests fail and testFireAsyncio passes; the unawaited-coroutine warning reappears in the
    pytest output. So the tests are load-bearing.
  • .github/scripts/build.sh run locally on the final tree:
    • python -m pytest → 275 passed, 0 failed (also with ipython installed).
    • pylint fire --ignore=test_components_py3.py,parser_fuzz_test.py,console → the single
      remaining message (E1120 in fire/formatting_windows.py:45) is present at base too;
      count unchanged at 1.
    • python -m ty check ... → 23 diagnostics, identical to the 23 measured at base.
  • Python version spread, since CI spans 3.7–3.14: the new tests pass on 3.12 and on 3.9, and
    every API used (asyncio.get_running_loop, asyncio.run,
    concurrent.futures.ThreadPoolExecutor) exists since 3.7.
  • Checked against open PRs: Catch TypeError from function calls and convert to FireError #654 and Await coroutines from functools.partial components #712 both edit this same block. Neither fixes this
    issue — Await coroutines from functools.partial components #712 keeps loop.run_until_complete for partials, Catch TypeError from function calls and convert to FireError #654 keeps it inside a
    TypeError wrapper — and my branch contains no asyncio-error-handling change, so the two
    ideas are independent. Whichever lands second will need a small manual resolution in
    fire/core.py, since both replace the lines right below if inspectutils.IsCoroutineFunction(fn):. feat: skip __init__ when calling classmethods and staticmethods #713 merges clean against this branch
    (git merge-tree --write-tree, both orders).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Calling an async command from an active event loop raises RuntimeError and leaves an unawaited coroutine

1 participant