Skip to content

c_mpos: jpegdec, a native JPEG decoder module (esp_new_jpeg) for video playback - #305

Open
bitcoin3us wants to merge 3 commits into
MicroPythonOS:mainfrom
bitcoin3us:feat/jpegdec-native-module
Open

bitcoin3us wants to merge 3 commits into
MicroPythonOS:mainfrom
bitcoin3us:feat/jpegdec-native-module

Conversation

@bitcoin3us

@bitcoin3us bitcoin3us commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

What

A jpegdec MicroPython module in c_mpos, wrapping Espressif's esp_new_jpeg decoder (pulled in through the IDF component manager in build_mpos.sh, the same way adc_mic is):

import jpegdec
w, h = jpegdec.info(jpeg_bytes)
out = bytearray(w * h * 2)               # RGB565 little-endian, 16-byte aligned
jpegdec.decode(jpeg_bytes, out)          # optional: scale_w, scale_h (multiples of 8, up to 1/8)

The output buffer goes straight into an lv.image_dsc_t with cf=RGB565, so LVGL blits it like a raw frame: no LVGL decoder, no image cache.

Why

MJPEG video in ClipTV. LVGL's own TJpgDec path needs LV_USE_FS_MEMFS to read from RAM at all (lvgl_micropython#14), costs about 100 ms per 160x120 frame and 250 ms per 320x240 frame, and keeps every decoded frame in a pointer-keyed cache that fills the heap.

Measured (Waveshare ESP32-S3-Touch-LCD-3.5)

Frame Decode time Decode-only rate
160x120 (8 KB) 5.2 ms 191 fps
320x240 (24 KB) 17.2 ms 58 fps
320x240 scaled to 160x120 17.3 ms

In-app MJPEG playback with this module: 160x120 at 15 fps plays at 13.9 fps and 320x240 at 8 fps plays at 7.8 fps, both paced to the clip with no frame skipping; the decoder is no longer the bottleneck (storage reads are).

Cost and scope

  • The prebuilt decoder library takes about 74 KB of firmware: esp32s3 partition headroom drops from ~93 KB to ~19 KB. Worth knowing before merging.
  • ESP32 family only (esp32, s2, s3, p4); the idf_component.yml rule limits the dependency to those targets, so other builds are unaffected.
  • Errors map to ValueError (bad/truncated/unsupported JPEG), MemoryError and RuntimeError.

Image size (update 2026-09-27)

The decoder adds ~75 KB to every ESP32-family image, which pushed both OTA app partitions past their size in CI (esp32 by 38.7 KB, esp32s3 by 51.1 KB). Growing the partitions would break OTA updates for devices in the field, so the two extra commits trim ESP-IDF features that no MicroPythonOS board or app uses instead:

  1. build: drop ESP-IDF features no MicroPythonOS board or app uses — PPP (network.PPP; no board has a modem), DTLS (the ssl module never speaks TLS over UDP), the TLS server side (nothing serves HTTPS; ssl stays fully functional as a client, client certificates included) and the BLE 5 controller features (the bluetooth module only does legacy advertising/scanning).
  2. build: drop IPv6 from the ESP32-family images — kept separate because it is the one trim a user could notice (sockets are IPv4-only afterwards). Everything the OS and its apps talk to is reachable over IPv4. If you would rather keep IPv6, drop this commit: the first one alone already makes CI green, just with only 10 to 13 KB of headroom left.

CI measurements on this PR (app partitions: esp32 3,737,600 B, esp32s3 3,670,016 B):

Build esp32 image headroom esp32s3 image headroom
jpegdec alone (before) 3,778,368 −38,720 3,721,136 −51,120
+ commit 1 (PPP, DTLS, TLS server, BLE 5) 3,727,136 10,464 3,656,544 13,472
+ commit 2 (IPv6) 3,700,896 36,704 3,631,008 39,008

For scale, the image is dominated by the frozen Python (~917 KB), secp256k1 (~150 KB, already at the smallest precomputation), the LVGL binding and lodepng; the built-in app filesystem is already compressed and every enabled font is referenced, so there was nothing cheaper to cut.

Verified on a Waveshare ESP32-S3-Touch-LCD-3.5 running the image with both commits: boots normally, HTTPS fetches from badgehub.eu and updates.micropythonos.com succeed, a BLE scan sees 15 devices and legacy advertising works, network.PPP is gone.

🤖 Generated with Claude Code

With thanks to the scientists and engineers who did the hard, unglamorous work that got us here.

bitcoin3us and others added 3 commits September 21, 2026 19:34
Adds a `jpegdec` MicroPython module wrapping Espressif's esp_new_jpeg
decoder (pulled in through the IDF component manager, like adc_mic):

    jpegdec.info(jpeg) -> (width, height)
    jpegdec.decode(jpeg, out, scale_w=0, scale_h=0) -> (width, height)

It decodes a JPEG held in any Python buffer straight into a caller-owned
RGB565 (little-endian) buffer, which an lv.image_dsc_t of cf=RGB565 can
show with no further decoding and no LVGL image-cache involvement.
Optional downscaling by up to 1/8 (multiples of 8).

Motivation: MJPEG video in ClipTV. LVGL's own TJpgDec route needs
LV_USE_FS_MEMFS, costs about 100 ms per 160x120 frame and 250 ms per
320x240 frame, and keeps every decoded frame in a pointer-keyed cache
that fills the heap. Measured on a Waveshare ESP32-S3-Touch-LCD-3.5 with
this module: 5.2 ms per 160x120 frame and 17.2 ms per 320x240 frame
(58 fps decode-only), giving 320x240 MJPEG playback at the clip's
declared rate up to at least 8 fps and 160x120 at 15 fps, decoder no
longer the bottleneck.

Cost: the prebuilt decoder library takes about 74 KB of the firmware
partition (headroom on the esp32s3 build drops from ~93 KB to ~19 KB).
ESP32-family only; the manifest rule limits the dependency to targets
the library ships for.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The native JPEG decoder (jpegdec) adds ~75 KB to every ESP32-family
image and pushes both OTA app partitions past their size in CI
(esp32 by 38.7 KB, esp32s3 by 51.1 KB). Rather than growing the
partitions, which would break OTA updates for existing devices, trim
what the firmware never exercises:

- PPP (network.PPP): no board has a cellular modem or a serial PPP
  link; MicroPython drops the class cleanly when lwIP has no PPP.
- DTLS: MicroPython's ssl module never speaks TLS over UDP.
- TLS server side: nothing serves HTTPS; ssl keeps working as a client,
  client certificates included (wrap_socket(server_side=True) would now
  fail at runtime, which nothing calls).
- BLE 5 controller features (extended/periodic advertising, 2M/coded
  PHY): MicroPython's bluetooth module only uses legacy advertising and
  scanning, so the code was dead weight in the NimBLE controller.

Measured on a local esp32s3 build of this branch: 3,650,256 ->
3,585,408 bytes, about 63 KB smaller.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Kept separate from the other trims because it is the one users could
notice: with CONFIG_LWIP_IPV6=n the socket module still exposes
AF_INET6 but no IPv6 address can be bound or connected, and the
Wi-Fi interface gets no link-local address. Every service the OS and
its apps talk to (BadgeHub, updates.micropythonos.com, Nostr relays,
Lightning wallets) is reachable over IPv4, and none of the supported
boards is deployed on an IPv6-only network that we know of.

It buys about 23 KB of flash on the esp32s3 image (and IRAM plus
DRAM for the neighbour discovery tables); if that headroom is not
needed after the other trims, this commit can simply be dropped.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant