gh-114099 - Add iOS framework loading machinery.#116454
Merged
ned-deily merged 11 commits intopython:mainfrom Mar 19, 2024
Merged
gh-114099 - Add iOS framework loading machinery.#116454ned-deily merged 11 commits intopython:mainfrom
ned-deily merged 11 commits intopython:mainfrom
Conversation
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of the PEP 730 work to add iOS support.
This PR adds an extension module Finder and Loader to accommodate the unusual requirements of iOS app packaging.
The iOS App Store requires that all binary modules must be
.dylibobjects, contained in a framework, stored in theFrameworksfolder of the packaged app. There can be only a single binary object per framework, and there can be no executable binary material the app outside the Frameworks folder (other than the app executable itself). The testbed project includes a build step to do the post-processing required to move binary modules from thelib-dynloadfolder into the Frameworks folder, adding the appropriate metadata and signing the modules; this PR modifies the Python interpreter so that it can find the modules in the new location.With this PR, the testbed project added in #115930 will now run, with all but 4 tests passing (test_concurrent_futures.test_thread_pool, test_marshal, test_platform, and test_sysconfig); it takes about 12 minutes to run the test suite on a 2021 M1 MacBook Pro, with a couple more minutes required on the first run to prepare and boot the iOS simulator image. Fixing these last couple of tests will be the subject of the next PR.
There's one potentially controversial part of this PR that requires an decision to be made. In addition to the Finder looking for the .dylib file in the new location, the Loader also rewrites the
__file__attribute of the binary module so that it reports the original location of the module, not the location in the Frameworks folder. For example, if a third party has written afoo.bar._whizmodule,_whiz.abi3.dylibwill be moved to the Frameworks folder, but_whiz.__file__will report a location that is<somewhere on python path>/foo/bar/_whiz.abi3.dylib, rather than<sys.executable>/Frameworks/foo.bar._whiz.framework/_whiz.abi3.dylib, (which is where the file actually is).This has been done because there is an implied assumption in the broader Python ecosystem that binary modules will be in a directory hierarchy, and we've occasionally seen examples in the wild of modules that assume that the
__file__attribute of a binary module will be in the filesystem, and can be used to anchor requests for "adjacent" data files. This is somewhat analogous the the issues that are seen with the zipimporter and eggs - code breaks because it assumes__file__is a reliable way to find files adjacent to the module in question.There's a reasonable argument to be made that this should be considered a bug in user code, and part of adapting a project to iOS is to fix code that makes this assumption. If the decision is made to drop the
__file__rewriting, the custom Loader isn't required.The implementation of the loader also requires the creation of 2 additional files for each module - a
.fworkfile, which acts as a replacement of the extension module in the original location, and a.originfile that is put next to the extension module in it's new home so that you can find where it came from. Filesystem symlinks aren't an option because they're prohibited by Apple's guidelines; these are essentially "text based" symlinks.