Skip to content

scripts/mpos_controller: give the serial port to mpremote - #226

Merged
ThomasFarstrike merged 2 commits into
MicroPythonOS:mainfrom
fdb:mpremote-serial-port
Aug 6, 2026
Merged

ThomasFarstrike merged 2 commits into
MicroPythonOS:mainfrom
fdb:mpremote-serial-port

Conversation

@fdb

@fdb fdb commented Aug 4, 2026 •

Copy link
Copy Markdown
Contributor

The problem

MPOSController accepts --serial-port for its own REPL connection. But it
started mpremote with no connect argument. mpremote then connects to the
first USB serial device that it finds:

# main.py
def ensure_connected(self):
    if self.transport is None:
        do_connect(self)          # no args, so dev becomes "auto"

# commands.py
elif dev == "auto":
    for p in sorted(serial.tools.list_ports.comports()):
        if p.vid is not None and p.pid is not None:
            state.transport = SerialTransport(p.device, baudrate=115200)
            return

The REPL connection used the correct port, because it uses pyserial directly.
Only the file transfers used the incorrect port.

Why this is difficult to find

If the host has one device, that device is always the correct device. The
problem stays hidden. If the host has two devices, mpremote sends the files
to the incorrect device.

mpremote sorts the devices by name. Because of this, it selects the same
incorrect device each time. A random selection is more safe: it fails only some
of the time, and the user then examines the connection quickly. This selection
fails all of the time. The user sees no error, and sees no change on the
correct device. The user then examines the application code, and does not
examine the connection.

How to see the problem

  1. Connect two devices to the host.
  2. Give this command:
    python3 scripts/mpos_controller.py --serial-port <the second device> installapp <path>
  3. Examine the two devices. The first device has the new files.

The change

Four functions started mpremote without the port:

Function Command Used by
main(), installapp action mkdir, fs cp -r app installation
SerialBackend._read_remote_file cp :path tmp screenshot()
SerialBackend.write_remote_file cp tmp :path write_file()
SerialBackend.get_widget_tree cp :/_mpos_tree.json tmp widget tree

Because screenshot() and get_widget_tree() also used the incorrect device,
a host with two devices showed the old screen after a correct installation.
This gave more incorrect data to the user.

All four functions now build the command with a new _mpremote_cmd() helper.
The helper puts connect <port> in the command when the port is known.

SerialBackend.run_test_file() is newer code, and it already gives
connect self.port to mpremote. This change makes the four older functions do
the same thing. I did not change run_test_file(), because it is correct.

The installapp action also stops and gives an error message if the user gives
no --serial-port and the host has more than one device. It is not possible to
select the correct device in that condition.

tests/cpython_mpos_controller.py had the same problem. The app management
test received a serial_port argument, but it did not use that argument.

Tests

A new mpremoteport section tests _mpremote_cmd(). It does not use a device,
so it runs with the desktop backend:

$ python3 tests/cpython_mpos_controller.py --only mpremoteport
  [mpremote port selection]
  + with a port: the command contains 'connect'
  + with a port: the port comes after 'connect'
  + with a port: the command keeps the other arguments
  + with no port: the command contains no 'connect'
  + with no port: the command keeps the other arguments

To show that the test is sufficient, I replaced _mpremote_cmd() with the old
behaviour. Two of the five checks then failed.

I also tested the change with one device. The files went to that device.
I did not test with two devices, because I have only one device.

Parts that this change does not correct

scripts/install.sh also starts mpremote with no connect argument. That
script has no port option, so it does not ignore a port that the user gave. It
always uses the first device. To correct it, it must get a new port option.
This is a change to its interface, so it is not in this pull request.

AGENTS.md also shows an mpremote command with no port. It has the same
condition.

Checklist

  • CHANGELOG.md: a new Tools: item is in the "Future release" section.
  • Tests: a new test section, as shown above.
  • make lint: all checks pass.
  • Formatting: the new code keeps the style of the code near it. I did not run
    ruff format on these files, because the files were not formatted with it
    before this change. make lint and the CI workflow use ruff check only.
  • MANIFEST.JSON and MAINTAINERS.md: no change, because this pull request does
    not change an app or a board.

MPOSController accepted --serial-port for its own REPL connection. But it
started mpremote with no `connect` argument. mpremote then connects to the
first USB serial device that it finds. It sorts the devices by name and
selects the first device in that list.

If the computer has one device, that device is always the correct device.
The problem stays hidden. If the computer has two devices, mpremote sends
the files to the incorrect device. mpremote selects the same incorrect
device each time, because it sorts the devices. The user sees no error, and
sees no change on the correct device. The user then examines the
application code, and does not examine the connection.

Four functions had this problem: the installapp action, _read_remote_file,
write_remote_file and get_widget_tree. Because of this, a host with two
devices also made screenshots of the incorrect device. All four functions
now build the command with _mpremote_cmd(). That function puts
`connect <port>` in the command.

The installapp action also stops and gives an error message if the user
gives no --serial-port and the computer has more than one device. It is not
possible to select the correct device in that condition.

The test for app management had the same problem. It received a serial_port
argument, but it did not use that argument.

The new mpremoteport test shows that the port goes into the command. This
test does not use a device.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012s21uqAdYgdHeyB4jnpiG4
@fdb
fdb force-pushed the mpremote-serial-port branch from 0e922b2 to 5aba5ef Compare August 4, 2026 08:51
@ThomasFarstrike

Copy link
Copy Markdown
Contributor

Oh, this is cool, and makes total sense, thanks!

@ThomasFarstrike

Copy link
Copy Markdown
Contributor

Note that it's possible some tests fail if you're not on the latest commit because I just reworked the CI/release workflows.

@ThomasFarstrike ThomasFarstrike left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice fix — the bug is real and the consolidation into _mpremote_cmd() is clean. Two concerns:

  1. run_test_file() left inconsistent. It correctly passes connect self.port but still inlines its own mpremote path resolution instead of using _mpremote_cmd(). The helper exists to centralize mpremote command construction — skipping one caller is divergence someone will copy-paste from and forget the connect argument again. Not blocking, but worth unifying while the change is fresh.

  2. _count_usb_serial_devices() returns 0 on ImportError. If pyserial is missing or has a broken serial.tools.list_ports, the multi-device guard silently bypasses. Practically: no pyserial = no serial = no problem. But a comment noting the assumption would help the next person debugging why the warning didn't fire.

@ThomasFarstrike

Copy link
Copy Markdown
Contributor

Note that these are minor concerns, of course, but if you wouldn't mind giving your view on them or applying a minor (probably one-line) fix, all the better!

…guish unknown device count

Review follow-up for MicroPythonOS#226:

- run_test_file() now builds its three mpremote commands with
  _mpremote_cmd(), like the other callers. No behaviour change:
  self.port always has a value in SerialBackend.

- _count_usb_serial_devices() gives None instead of 0 when pyserial
  is missing. mpremote runs with python3, which can be a different
  interpreter that does have pyserial. A 0 would make the installapp
  guard read 'no devices' and skip the multi-device check. The guard
  now warns on None and still errors on more than one device.

- New check in the mpremoteport test section: a missing pyserial
  gives None, not 0. Verified red with the old behaviour.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@fdb

fdb commented Aug 5, 2026

Copy link
Copy Markdown
Contributor Author

Hi, thanks for the review. I made two changes:

  1. run_test_file() now uses _mpremote_cmd(). It had three inline mpremote invocations, each repeating connect self.port. All three go through the helper now. This doesn't change th behaviour change: self.port always has a value in SerialBackend.

  2. The ImportError case now returns None instead of 0 — with a docstring, not just a comment. However, your assumption behind it doesn't hold. "No pyserial = no serial = no problem" conflates two interpreters: _mpremote_cmd() starts mpremote with python3, which is not necessarily the interpreter that runs mpos_controller.py (venvs, uv run). So, pyserial can be missing from the script's interpreter while mpremote works fine — and then a return of 0 silently bypasses the multi-device guard, which is exactly the bug this PR fixes. (this is now deep in the weeds territory 😁)

_count_usb_serial_devices() now returns None for "unknown". The installapp guard warns on None (and proceeds), still errors on >1. A new check in the mpremoteport test section pins this: with the old return 0 it fails, with return None it passes.

@ThomasFarstrike

Copy link
Copy Markdown
Contributor

Awesome, thanks for the replies, let's merge and see how it goes!

@ThomasFarstrike
ThomasFarstrike merged commit 9861eee into MicroPythonOS:main Aug 6, 2026
fdb added a commit to fdb/MicroPythonOS that referenced this pull request Aug 10, 2026
PR MicroPythonOS#226 moved mpremote command construction into _mpremote_cmd() and added
a test section for it. This PR adds a test section for read_until sentinel
matching. Both touched the same three places in
tests/cpython_mpos_controller.py, so all three conflicts are additive: each
side appended a new item where the other appended one.

Resolution keeps both sides in every case:
- the import list gains AIOREPLClient and END_MARKER next to
  _count_usb_serial_devices and _mpremote_cmd
- the sections registry gains both "readuntil" and "mpremoteport"
- both test functions are kept, ordered to match the registry

scripts/mpos_controller.py merged without conflict. The two changes do not
overlap: MicroPythonOS#226 rewrote the mpremote helpers, this PR changed read_until.
Verified that the merged file differs from upstream/main by the CRLF fix
and nothing else.

Also adds "mpremoteport" to the --only help string. MicroPythonOS#226 added the section
but not the help text, and the merge kept this branch's version of that
line, so the omission would otherwise have been invisible.

Both device-free sections pass in the merged tree (13 checks). Confirmed
test_read_until still fails against the pre-fix read_until: the CRLF case
waits out the full timeout, so the elapsed-time check catches it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pqs3t4Xx6zh1aHNsDSK31A
@fdb
fdb deleted the mpremote-serial-port branch August 10, 2026 19:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants