Skip to content

GH-126491: GC: Mark objects reachable from roots before doing cycle collection #127110

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
markshannon merged 18 commits into from
Dec 2, 2024
Merged

GH-126491: GC: Mark objects reachable from roots before doing cycle collection #127110

Changes from 1 commit
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
2b0a2c5
Change the young generation to 'not visited'. WIP
markshannon Nov 19, 2024
754bb68
Tidy up visited tracking and add consistency asserts
markshannon Nov 20, 2024
ac3ad26
Partially apply mark-first-gc patch
markshannon Nov 21, 2024
5c3a5ef
Rename get_heap_size
markshannon Nov 21, 2024
0ac1c07
Set visited on entry frame
markshannon Nov 21, 2024
bb51b19
Apply more diff, but comment out marking step
markshannon Nov 21, 2024
be636c9
Refactor work to do calculation
markshannon Nov 21, 2024
287f381
Turn on marking. Explain issue with tuple untracking
markshannon Nov 21, 2024
7e920d6
Add news
markshannon Nov 21, 2024
0b46494
Add stats for marking
markshannon Nov 21, 2024
698abb3
Do not touch permanent generation
markshannon Nov 21, 2024
6d8a0d4
Move informative failure message in test
markshannon Nov 22, 2024
d9632c6
Increase threshold a bit
markshannon Nov 22, 2024
0702959
Increase number of iterations
markshannon Nov 22, 2024
6063dc8
Return number of iterations to 20k
markshannon Nov 22, 2024
67b1e05
Fix compiler warning
markshannon Nov 22, 2024
eaea41e
Turn off extra debugging
markshannon Nov 22, 2024
79ab26c
Add missing macro
markshannon Nov 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Do not touch permanent generation
  • Loading branch information
@markshannon
markshannon committed Nov 21, 2024
commit 698abb384fd1d156ed78bbad723680ed03d2ca61
35 changes: 24 additions & 11 deletions Python/gc.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1400,15 +1400,28 @@ expand_region_transitively_reachable(PyGC_Head *container, PyGC_Head *gc, GCStat

/* Do bookkeeping for a completed GC cycle */
static void
completed_cycle(GCState *gcstate)
{
/* Flip spaces */
int not_visited = gcstate->visited_space;
int visited = other_space(not_visited);
gcstate->visited_space = visited;
/* Make sure all objects have visited bit set correctly */
gc_list_set_space(&gcstate->young.head, not_visited);
gc_list_set_space(&gcstate->permanent_generation.head, visited);
completed_scavenge(GCState *gcstate)
{
/* We must observe two invariants:
* 1. Members of the permanent generation must be marked visited.
* 2. We cannot touch members of the permanent generation. */
int visited;
if (gc_list_is_empty(&gcstate->permanent_generation.head)) {
/* Permanent generation is empty so we can flip spaces bit */
int not_visited = gcstate->visited_space;
visited = other_space(not_visited);
gcstate->visited_space = visited;
/* Make sure all objects have visited bit set correctly */
gc_list_set_space(&gcstate->young.head, not_visited);
}
else {
/* We must move the objects from visited to pending space. */
visited = gcstate->visited_space;
int not_visited = other_space(visited);
assert(gc_list_is_empty(&gcstate->old[not_visited].head));
gc_list_merge(&gcstate->old[visited].head, &gcstate->old[not_visited].head);
gc_list_set_space(&gcstate->old[not_visited].head, not_visited);
}
assert(gc_list_is_empty(&gcstate->old[visited].head));
gcstate->work_to_do = 0;
gcstate->phase = GC_PHASE_MARK;
Expand Down Expand Up @@ -1627,7 +1640,7 @@ gc_collect_increment(PyThreadState *tstate, struct gc_collection_stats *stats)

add_stats(gcstate, 1, stats);
if (gc_list_is_empty(not_visited)) {
completed_cycle(gcstate);
completed_scavenge(gcstate);
}
validate_spaces(gcstate);
}
Expand Down Expand Up @@ -1657,7 +1670,7 @@ gc_collect_full(PyThreadState *tstate,
gcstate->young.count = 0;
gcstate->old[0].count = 0;
gcstate->old[1].count = 0;
completed_cycle(gcstate);
completed_scavenge(gcstate);
_PyGC_ClearAllFreeLists(tstate->interp);
validate_spaces(gcstate);
add_stats(gcstate, 2, stats);
Expand Down