Skip to content

gh-113433: Automatically Clean Up Subinterpreters in Py_Finalize() #121060

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ericsnowcurrently merged 8 commits into from
Jun 26, 2024
Merged

gh-113433: Automatically Clean Up Subinterpreters in Py_Finalize() #121060

Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
df63d15
Factor out resolve_final_tstate() and _Py_Finalize().
ericsnowcurrently Jun 25, 2024
f6df8be
Make sure the main interpreter is active in _Py_Finalize().
ericsnowcurrently Jun 25, 2024
071d270
Add a NEWS entry.
ericsnowcurrently Jun 26, 2024
ccb3aa2
Clean up all remaining subinterpreters.
ericsnowcurrently Jun 19, 2024
6b26e0a
Finalize subinterpreters *after* running pending calls and atexit hooks.
ericsnowcurrently Jun 26, 2024
74f4ed0
Make sure there's only one tstate left.
ericsnowcurrently Jun 26, 2024
02a969a
Add a NEWS entry.
ericsnowcurrently Jun 26, 2024
86cd3dd
Clean up subinterpreters only after modules (incl. __main__) have bee…
ericsnowcurrently Jun 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Make sure there's only one tstate left.
  • Loading branch information
@ericsnowcurrently
ericsnowcurrently committed Jun 26, 2024
commit 74f4ed0385efb921b888bfc5ad3fa45e8816c029
57 changes: 44 additions & 13 deletions Python/pylifecycle.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2422,33 +2422,61 @@ _Py_IsInterpreterFinalizing(PyInterpreterState *interp)
static void
finalize_subinterpreters(void)
{
PyThreadState *final_tstate = _PyThreadState_GET();
PyInterpreterState *main_interp = _PyInterpreterState_Main();
assert(_PyInterpreterState_GET() == main_interp);
assert(final_tstate->interp == main_interp);
_PyRuntimeState *runtime = main_interp->runtime;
struct pyinterpreters *interpreters = &runtime->interpreters;

/* Clean up all remaining subinterpreters. */
/* Get the first interpreter in the list. */
HEAD_LOCK(runtime);
PyInterpreterState *interp = interpreters->head;
if (interp == main_interp) {
interp = interp->next;
}
HEAD_UNLOCK(runtime);
if (interp != NULL) {
(void)PyErr_WarnEx(
PyExc_RuntimeWarning,
"remaining subinterpreters; "
"destroy them with _interpreters.destroy()",
0);

/* Bail out if there are no subinterpreters left. */
if (interp == NULL) {
return;
}

/* Warn the user if they forgot to clean up subinterpreters. */
(void)PyErr_WarnEx(
PyExc_RuntimeWarning,
"remaining subinterpreters; "
"destroy them with _interpreters.destroy()",
0);

/* Swap out the current tstate, which we know must belong
to the main interpreter. */
_PyThreadState_Detach(final_tstate);

/* Clean up all remaining subinterpreters. */
while (interp != NULL) {
/* Destroy the subinterpreter. */
assert(!_PyInterpreterState_IsRunningMain(interp));
PyThreadState *tstate =
_PyThreadState_NewBound(interp, _PyThreadState_WHENCE_FINI);
PyThreadState *save_tstate = PyThreadState_Swap(tstate);

/* Find the tstate to use for fini. We assume the interpreter
will have at most one tstate at this point. */
PyThreadState *tstate = interp->threads.head;
if (tstate != NULL) {
/* Ideally we would be able to use tstate as-is, and rely
on it being in a ready state: no exception set, not
running anything (tstate->current_frame), matching the
current thread ID (tstate->thread_id). To play it safe,
we always delete it and use a fresh tstate instead. */
assert(tstate != final_tstate);
_PyThreadState_Attach(tstate);
PyThreadState_Clear(tstate);
_PyThreadState_Detach(tstate);
PyThreadState_Delete(tstate);
}
tstate = _PyThreadState_NewBound(interp, _PyThreadState_WHENCE_FINI);

/* Destroy the subinterpreter. */
_PyThreadState_Attach(tstate);
Py_EndInterpreter(tstate);
(void)PyThreadState_Swap(save_tstate);
assert(_PyThreadState_GET() == NULL);

/* Advance to the next interpreter. */
HEAD_LOCK(runtime);
Expand All @@ -2458,6 +2486,9 @@ finalize_subinterpreters(void)
}
HEAD_UNLOCK(runtime);
}

/* Switch back to the main interpreter. */
_PyThreadState_Attach(final_tstate);
}


Expand Down