尚未翻译
本页面尚未翻译成中文,因此显示英文原文。 帮助翻译
zipapp Module Complexity¶
The zipapp module packs a directory into a ZIP archive the interpreter can
run, and copies such archives with a new shebang line. Three costs are worth
separating.
Building from a directory copies every byte of every file under it and touches
every entry, directories included. Nothing is excluded by default, and the
filter callback decides inclusion one entry at a time without pruning: a
rejected directory is still walked and its contents are still offered.
Copying an existing archive streams its bytes after the shebang line and never parses the archive. Reading the interpreter back reads one line.
Running the result is a different module's cost. zipimport reads the central
directory, then compiles every .py member it imports on every start,
because nothing is written back into the archive. Packing .pyc members
beside the sources removes that compile.
Complexity Reference¶
Size variables: f = entries under the source directory, files and directories
both, before filter sees them; b = bytes in the files packed; a = bytes in
an existing archive; s = bytes in a shebang line, the one written or the one
replaced; c = the cost of one filter call.
Building an archive¶
| Operation | Time | Space | Notes |
|---|---|---|---|
create_archive(directory, target=None, ...) |
O(b + f) | O(f) | Walks every entry under the directory and writes each one as a member, directories included. Contents stream through a fixed buffer; the O(f) is the member list held for the central directory. 3.14 sorts the listing before writing, an extra O(f log f) |
filter=func |
O(f·c) | O(1) | Called once per entry with its path relative to the source. A rejected directory is not pruned: its contents are still walked and still offered |
compressed=True |
O(b) | O(1) | Deflates each member instead of storing it. The loader inflates a member each time it reads it |
main='mod:fn' |
O(1) | O(1) | Writes a three-line __main__.py. Refused when the directory already has one, and required when it does not |
interpreter=path |
O(s) | O(s) | Writes the shebang ahead of the archive and sets the executable bit on a path target |
ZipAppError |
— | — | A ValueError, raised before the target is opened, so a refused build writes nothing |
Up to 3.13 the directory is walked while the target is being written, so a
target inside the source directory becomes a member of itself. 3.14 lists the
entries before opening the target, and raises ZipAppError when a target that
already exists is among them.
Copying and inspecting an archive¶
| Operation | Time | Space | Notes |
|---|---|---|---|
create_archive(archive, target, interpreter=None) |
O(a + s) | O(s) | Skips the source's shebang line and streams the rest without parsing it. interpreter=None drops the shebang rather than keeping it. The executable bit is set for a str target only, not a pathlib.Path |
get_interpreter(archive) |
O(s) | O(s) | Reads two bytes, and the rest of the line only when they are #!. None when there is no shebang |
python -m zipapp |
as create_archive() |
as create_archive() |
--info is get_interpreter(). Copying an archive onto itself, or with -m, is refused |
Running an archive¶
| Operation | Time | Space | Notes |
|---|---|---|---|
python app.pyz |
O(m) + members imported | O(m) + members imported | m = members; reading the central directory is O(m). Each .py member imported is compiled at every start. A .pyc member beside it that still matches the source is used instead |
The filter prunes nothing¶
filter is asked about every entry under the source, so excluding a directory
means rejecting every path beneath it, not just the directory itself. Rejecting
the directory alone still packs its contents.
import tempfile
import zipapp
import zipfile
from pathlib import Path
with tempfile.TemporaryDirectory() as tmp:
source = Path(tmp) / "app"
(source / "pkg" / "__pycache__").mkdir(parents=True)
(source / "pkg" / "__init__.py").write_text("def main():\n print('hi')\n")
(source / "pkg" / "__pycache__" / "stale.pyc").write_bytes(b"")
seen = []
def keep(path):
seen.append(path.as_posix()) # called once per entry - O(f) calls
return "__pycache__" not in path.parts # reject the whole subtree
# Build - O(b + f)
zipapp.create_archive(source, Path(tmp) / "app.pyz", main="pkg:main", filter=keep)
# The directory was rejected, and its contents were still offered
offered = ["pkg", "pkg/__init__.py", "pkg/__pycache__", "pkg/__pycache__/stale.pyc"]
assert sorted(seen) == offered
names = zipfile.ZipFile(Path(tmp) / "app.pyz").namelist()
assert sorted(names) == ["__main__.py", "pkg/", "pkg/__init__.py"]
Copying an archive never parses it¶
Giving an existing archive as the source replaces its shebang line and copies
the rest byte for byte. The archive is not parsed, so the copy costs its size
and nothing more, and a file that is not a ZIP archive at all is copied just as
readily. Pass the target as a str when the copy should be executable.
import os
import tempfile
import zipapp
from pathlib import Path
with tempfile.TemporaryDirectory() as tmp:
source = Path(tmp) / "app"
source.mkdir()
(source / "__main__.py").write_text("print('hi')\n")
built = Path(tmp) / "app.pyz"
zipapp.create_archive(source, built) # no shebang
assert zipapp.get_interpreter(built) is None # reads two bytes - O(1)
# Copy with a shebang - O(a) in the archive's size
shipped = os.path.join(tmp, "app-shipped.pyz")
zipapp.create_archive(built, shipped, interpreter="/usr/bin/env python3")
assert zipapp.get_interpreter(shipped) == "/usr/bin/env python3" # one line - O(s)
assert built.stat().st_size + len(b"#!/usr/bin/env python3\n") == os.stat(shipped).st_size
# Copy again without an interpreter: the shebang is dropped, not kept
plain = Path(tmp) / "app-plain.pyz"
zipapp.create_archive(shipped, plain)
assert zipapp.get_interpreter(plain) is None
assert plain.read_bytes() == built.read_bytes()
Ship bytecode, or pay the compile at every start¶
zipimport never writes bytecode back into the archive, and ignores a
__pycache__ directory packed inside it, so a .py member is compiled each
time the application starts. What it does look for is a .pyc beside the
source, which compileall.compile_dir() writes with legacy=True, and it uses
that whenever it still matches the source. Compile once, then pack.
import compileall
import tempfile
import zipapp
import zipfile
from pathlib import Path
with tempfile.TemporaryDirectory() as tmp:
source = Path(tmp) / "app"
(source / "pkg").mkdir(parents=True)
(source / "pkg" / "__init__.py").write_text("def main():\n print('hi')\n")
# Compile once, at build time - O(source length)
compileall.compile_dir(source, legacy=True, quiet=1)
# Pack sources and bytecode together - O(b + f)
zipapp.create_archive(source, Path(tmp) / "app.pyz", main="pkg:main")
names = zipfile.ZipFile(Path(tmp) / "app.pyz").namelist()
assert "pkg/__init__.pyc" in names
# Running it: python app.pyz imports pkg from the .pyc, and compiles only
# the three-line __main__.py that main= generated
The command line does the same work as create_archive():
# Build from a directory - O(b + f)
python -m zipapp app -m pkg:main -o app.pyz
# Copy with a shebang - O(a); the copy is made executable
python -m zipapp app.pyz -p "/usr/bin/env python3" -o app-shipped.pyz
# Read the shebang back - O(s)
python -m zipapp app-shipped.pyz --info