c_mpos: jpegdec, a native JPEG decoder module (esp_new_jpeg) for video playback - #305
Open
bitcoin3us wants to merge 3 commits into
Open
bitcoin3us wants to merge 3 commits into
bitcoin3us wants to merge 3 commits into
Conversation
Adds a `jpegdec` MicroPython module wrapping Espressif's esp_new_jpeg
decoder (pulled in through the IDF component manager, like adc_mic):
jpegdec.info(jpeg) -> (width, height)
jpegdec.decode(jpeg, out, scale_w=0, scale_h=0) -> (width, height)
It decodes a JPEG held in any Python buffer straight into a caller-owned
RGB565 (little-endian) buffer, which an lv.image_dsc_t of cf=RGB565 can
show with no further decoding and no LVGL image-cache involvement.
Optional downscaling by up to 1/8 (multiples of 8).
Motivation: MJPEG video in ClipTV. LVGL's own TJpgDec route needs
LV_USE_FS_MEMFS, costs about 100 ms per 160x120 frame and 250 ms per
320x240 frame, and keeps every decoded frame in a pointer-keyed cache
that fills the heap. Measured on a Waveshare ESP32-S3-Touch-LCD-3.5 with
this module: 5.2 ms per 160x120 frame and 17.2 ms per 320x240 frame
(58 fps decode-only), giving 320x240 MJPEG playback at the clip's
declared rate up to at least 8 fps and 160x120 at 15 fps, decoder no
longer the bottleneck.
Cost: the prebuilt decoder library takes about 74 KB of the firmware
partition (headroom on the esp32s3 build drops from ~93 KB to ~19 KB).
ESP32-family only; the manifest rule limits the dependency to targets
the library ships for.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The native JPEG decoder (jpegdec) adds ~75 KB to every ESP32-family image and pushes both OTA app partitions past their size in CI (esp32 by 38.7 KB, esp32s3 by 51.1 KB). Rather than growing the partitions, which would break OTA updates for existing devices, trim what the firmware never exercises: - PPP (network.PPP): no board has a cellular modem or a serial PPP link; MicroPython drops the class cleanly when lwIP has no PPP. - DTLS: MicroPython's ssl module never speaks TLS over UDP. - TLS server side: nothing serves HTTPS; ssl keeps working as a client, client certificates included (wrap_socket(server_side=True) would now fail at runtime, which nothing calls). - BLE 5 controller features (extended/periodic advertising, 2M/coded PHY): MicroPython's bluetooth module only uses legacy advertising and scanning, so the code was dead weight in the NimBLE controller. Measured on a local esp32s3 build of this branch: 3,650,256 -> 3,585,408 bytes, about 63 KB smaller. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Kept separate from the other trims because it is the one users could notice: with CONFIG_LWIP_IPV6=n the socket module still exposes AF_INET6 but no IPv6 address can be bound or connected, and the Wi-Fi interface gets no link-local address. Every service the OS and its apps talk to (BadgeHub, updates.micropythonos.com, Nostr relays, Lightning wallets) is reachable over IPv4, and none of the supported boards is deployed on an IPv6-only network that we know of. It buys about 23 KB of flash on the esp32s3 image (and IRAM plus DRAM for the neighbour discovery tables); if that headroom is not needed after the other trims, this commit can simply be dropped. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A
jpegdecMicroPython module inc_mpos, wrapping Espressif's esp_new_jpeg decoder (pulled in through the IDF component manager inbuild_mpos.sh, the same wayadc_micis):The output buffer goes straight into an
lv.image_dsc_twithcf=RGB565, so LVGL blits it like a raw frame: no LVGL decoder, no image cache.Why
MJPEG video in ClipTV. LVGL's own TJpgDec path needs
LV_USE_FS_MEMFSto read from RAM at all (lvgl_micropython#14), costs about 100 ms per 160x120 frame and 250 ms per 320x240 frame, and keeps every decoded frame in a pointer-keyed cache that fills the heap.Measured (Waveshare ESP32-S3-Touch-LCD-3.5)
In-app MJPEG playback with this module: 160x120 at 15 fps plays at 13.9 fps and 320x240 at 8 fps plays at 7.8 fps, both paced to the clip with no frame skipping; the decoder is no longer the bottleneck (storage reads are).
Cost and scope
idf_component.ymlrule limits the dependency to those targets, so other builds are unaffected.ValueError(bad/truncated/unsupported JPEG),MemoryErrorandRuntimeError.Image size (update 2026-09-27)
The decoder adds ~75 KB to every ESP32-family image, which pushed both OTA app partitions past their size in CI (esp32 by 38.7 KB, esp32s3 by 51.1 KB). Growing the partitions would break OTA updates for devices in the field, so the two extra commits trim ESP-IDF features that no MicroPythonOS board or app uses instead:
build: drop ESP-IDF features no MicroPythonOS board or app uses— PPP (network.PPP; no board has a modem), DTLS (thesslmodule never speaks TLS over UDP), the TLS server side (nothing serves HTTPS;sslstays fully functional as a client, client certificates included) and the BLE 5 controller features (thebluetoothmodule only does legacy advertising/scanning).build: drop IPv6 from the ESP32-family images— kept separate because it is the one trim a user could notice (sockets are IPv4-only afterwards). Everything the OS and its apps talk to is reachable over IPv4. If you would rather keep IPv6, drop this commit: the first one alone already makes CI green, just with only 10 to 13 KB of headroom left.CI measurements on this PR (app partitions: esp32 3,737,600 B, esp32s3 3,670,016 B):
For scale, the image is dominated by the frozen Python (~917 KB), secp256k1 (~150 KB, already at the smallest precomputation), the LVGL binding and lodepng; the built-in app filesystem is already compressed and every enabled font is referenced, so there was nothing cheaper to cut.
Verified on a Waveshare ESP32-S3-Touch-LCD-3.5 running the image with both commits: boots normally, HTTPS fetches from badgehub.eu and updates.micropythonos.com succeed, a BLE scan sees 15 devices and legacy advertising works,
network.PPPis gone.🤖 Generated with Claude Code
With thanks to the scientists and engineers who did the hard, unglamorous work that got us here.