Reduce the memory footprint when -z option is used - #372
Merged
tylervick merged 2 commits intoMar 16, 2025
Merged
Conversation
Initialization of an Attachment is now inside an autoreleasepool because the memory footprint is huge when using the -z option. The memory usage jumps to 10 gigabytes under 10 seconds. Releasing the NSImage related objects at each iteration via autorelease pool reduces it to a few hundred megabytes.
aleksandar-angelov-smule
force-pushed
the
optimize-memory-footprint
branch
from
February 14, 2025 15:49
6679f03 to
a32fd9c
Compare
tylervick
approved these changes
Mar 16, 2025
tylervick
left a comment
Member
There was a problem hiding this comment.
Awesome improvement, thank you!
This was referenced Aug 11, 2026
tylervick
added a commit
that referenced
this pull request
Aug 16, 2026
* Pin the autoreleasepool that keeps -z from exhausting memory 183be4d (#372) wrapped per-attachment image work in an autoreleasepool, which is what stopped `-z` from being killed on a large report (#337). Nothing held that line down: deleting it left every test green while peak memory went back to scaling with attachment count. Downsizing decodes each screenshot into an NSImage, renders it at the new size, takes a TIFF representation of that, and encodes JPEG from it. Those are autoreleased Cocoa objects, so without a pool draining them per attachment they survive to the end of the process. Measured through this test, that is ~25-27 MB retained per screenshot: the reporter's 2,532 screenshots extrapolate to ~70 GB, which is the ">100 GBs of RAM" and status 137 they described, and it explains why the same flag worked on a small sample. The test asserts a shape rather than a number -- memory must not scale with attachment count -- because that is the property that actually failed. The margin is wide enough that resident-size noise cannot reach it: ~0.06 MB per attachment with the pool against ~25 MB without, and a 5 MB budget between them. Verified in both directions: it passes on main, and with the pool bypassed it fails at 27 MB per attachment naming the pool as the thing to check. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Close the two ways this guard could have passed while guarding nothing Both from review on #489, and both are the failure mode a regression test can least afford: reporting success without having measured anything. `residentBytes` returned 0 when `task_info` failed, which would have made `after - before` zero and the budget trivially satisfied. It throws now. More seriously, the test asserted only the attachment count, so it never established that downsizing had actually happened. Had the flag stopped being honoured, or `AttachmentType.isImage` stopped recognising the extension, no image would have been decoded, memory would have stayed flat, and the test would have passed while the autoreleasepool it exists to protect went unexercised. It now asserts the rendered attachment really is the downsized image -- a quarter of the source width, and smaller on disk than the source. Verified against both regressions independently: with the pool removed it fails at 32 MB per attachment, and with downsizing turned into a no-op it fails on the rendered width (1290 against the expected 322.5) rather than passing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Initialization of an attachment is now inside an autoreleasepool because the memory footprint is huge when using the -z option. The memory usage jumps to 10 gigabytes under 10 seconds. Releasing the NSImage related objects at each iteration via autorelease pool reduces it to a few hundred megabytes.