Consolidate flash/debug tooling on espresso.py; remove stale scripts

[evnchn]

Consolidates the flash/debug tooling on espresso.py and removes the dead/superseded scripts, per #224. Two Fable review passes on top of the original change turned espresso.py into a fair-sized refactor as well. +464 / −397 across 12 files.

Changes

File	Change
flash.py + esp.py	removed — superseded by espresso.py (flash/erase/enable/reset + Jetson device auto-detect)
backtrace.sh	removed — hardcoded a long-gone ESP-IDF 4.2 toolchain path; hasn't worked since v5.x
core_dumper.py	removed — folded into espresso.py as the coredump subcommand (reuses espresso's device/chip selection; the esp_coredump import stays deferred so the flash path doesn't depend on it)
espresso.py	new --host user@host[:path] remote mode, a coredump subcommand, and a substantial refactor (see below)
upload_ssh.py	replaced by a deprecation stub: every invocation prints the espresso.py --host migration to stderr and exits 2 (see below)
.github/zip_content.txt	dropped the four removed scripts and added the upload_ssh.py stub, so the release zip -@ keeps matching every listed file and ships the migration pointer
docs/tools.md, docs/getting_started.md	backtrace → direct xtensa-esp-elf-addr2line; corrected flash <device_path> → flash --device <device_path> (the bare form is rejected by espresso.py); new sections for --host and core dumps
AGENTS.md, .cursor/rules/*	file tree + flash command → espresso.py

espresso.py refactor

Beyond the new features, espresso.py was reworked for robustness:

• Side-effect-free import: all argument parsing and hardware resolution moved out of module scope into main(). Importing the module no longer probes GPIO or /etc/nv_tegra_release.
• Frozen Config dataclass: the scattered module globals (ON/OFF/CHIP/DEVICE/...) are replaced by one immutable config with derived values as properties, so e.g. flash_freq cannot drift from chip.
• COMMANDS registry: per-command knowledge (handler, pin usage, artifacts) lives in a single table instead of parallel lists plus an if/elif dispatch.

--host user@host[:path] remote mode

Rsyncs espresso.py plus exactly the build artifacts the command needs to the target, then re-runs the command there over SSH. Works for every command (flash, erase, enable, disable, reset, coredump). Notable properties:

• The user's argv passes through verbatim (minus --host/--dry-run), so nothing typed is dropped or reshaped, and locally-resolved defaults don't leak to the remote — which resolves its own device/pins/L4T version.
• Only the artifacts the command uses are copied (rsync -R, relative paths preserved); pin-only commands copy nothing.
• Artifacts resolve against the cwd (like a local run); only espresso.py itself comes from the script's directory, copied with -p to keep its exec bit.
• Safety: a dash-prefixed host is rejected, the remote path is quoted for the ssh cd (raw for rsync, which escapes remote args itself), tokens are shlex-quoted, and artifact paths must be relative and free of ...
• --host --dry-run prints the exact rsync/ssh commands and mutates nothing remotely.

Two open questions from the issue — my call, easy to flip

• backtrace.sh: removed; the doc now shows the direct addr2line it wrapped (the script hardcoded a dead 4.2 path, and the one-liner is trivial + correct for esp32/esp32s3). Happy to instead rebuild it against the v5.3 toolchain if you'd rather keep the script.
• core_dumper.py: folded into espresso.py as the coredump subcommand — it reuses espresso's serial-device and --chip selection and works remotely via --host.

upload_ssh.py: a deliberate, documented break

flash.py's positional vocabulary (nano/xavier/orin/nand/v05/usb/enable/reset/-e) is not carried over. A faithful translation isn't possible: espresso.py is Orin-only (hardcoded PR.04/PAC.06, rejects L4T < R35) and its piggyboard default differs from flash.py's. So instead of mistranslating or failing opaquely on the remote, upload_ssh.py is now a stub that always prints the migration and exits 2 — it transfers nothing and runs nothing, regardless of arguments. The stub names the replacement (./espresso.py flash --host user@host[:path] ...) and points Nano/Xavier Robot Brains at release 0.12.x, and it now ships in the release zip so users who unpack it (not just git checkouts) see the pointer. Meant to be dropped a release or two after 0.13.0.

Verification

• On real hardware: built for esp32s3 and flashed an ESP32-S3 N16R8 — lizard v0.12.0 boots and runs the DSL live.
• espresso.py flash --dry-run emits the correct esptool command for esp32s3.
• The release zip -@ < .github/zip_content.txt step runs clean against the edited manifest.
• git grep shows zero remaining references to the removed scripts across the tree.
• pre-commit (mdformat + hooks) passes on all changed files.

The CH343-USB-bridge flashing quirk seen during hardware testing is a separate espresso robustness matter tracked in #228. A thorough hardware pass on the refactored code (flash + coredump + --host) is still wanted before merge.

[falkoschindler]

Thanks @evnchn — the consolidation itself is in good shape: no dangling references to the removed scripts anywhere, the zip manifest stays consistent, and removing backtrace.sh in favor of the documented one-liner is the right call. The feedback below is almost entirely about one spot — the legacy guard in upload_ssh.py — and it leads to a proposal to take this PR one step further (which also answers your two open questions).

The legacy guard tries too hard — and that's where all the runtime findings live

The guard models espresso.py's argparse behavior token by token and gives per-token migration advice. Every runtime issue we found is a consequence of exactly that ambition:

• The migration table gives wrong advice for bare orin — upload_ssh.py (line 46). "orin | v05 — drop them" is only correct for v05: in the old esp.py, bare orin selected the pre-v0.5 pin layout (EN=PAC.06/G0=PR.04), which in espresso.py is only reachable via --swap. A pre-v0.5 piggyboard user following the message verbatim flashes with EN/G0 transposed and the ESP never enters the bootloader.
• The token scan diverges from espresso's real parser in both directions — upload_ssh.py (lines 28–39). --dev /dev/ttyUSB0 is valid espresso input (argparse prefix abbreviation) but gets rejected as a legacy bare device; --device -e passes the guard (the scan assumes --device always consumes a value, but it's declared nargs='?') and fails opaquely on the remote. And _VALUE_FLAGS is a hand-maintained mirror of espresso's flag set with nothing keeping the two in sync.
• The most common legacy invocation bypasses the guard entirely — upload_ssh.py (lines 31–39). flash.py's default was nano, so Nano users ran upload_ssh.py user@host with zero tokens. That sails through, rsyncs, and then dies on the remote with a raw import-time RuntimeError from espresso.py's L4T check — the exact opaque failure the guard exists to prevent, for the exact user class its error message names.
• nargs=argparse.REMAINDER swallows --help after the host — upload_ssh.py (line 17). upload_ssh.py user@host --help no longer prints help; it rsyncs build/ and runs a remote sudo command instead.
• Related: the rsync (line 58) only ships *.bin and assumes a current espresso.py already exists on the remote — remotes provisioned from a pre-v0.7.1 zip (no espresso.py) or pre-v0.10.0 (no --swap) now fail after a successful rsync.

Proposal: finish the consolidation in this PR — fold upload_ssh.py into espresso.py

None of the above needs a smarter guard; it needs no guard. Since this PR is literally about consolidating the tooling on espresso.py, I'd take it the last step here rather than in a follow-up: give espresso.py a --host user@host[:path] option and delete upload_ssh.py (or leave a three-line stub for one release that points at the new command). Concretely, with --host set, espresso would rsync the binaries plus espresso.py itself to the target and then re-invoke itself over ssh with the same arguments minus --host.

That solves the whole problem class structurally instead of patching it:

• Arguments are validated locally by the real parser before anything touches the network — the fail-fast-before-rsync behavior the guard hand-builds comes for free and can never drift.
• There is no wrapper vocabulary to migrate: anyone flashing remotely is already typing espresso arguments, so the legacy guard, the migration table, and _VALUE_FLAGS all become unnecessary.
• Shipping espresso.py in the rsync removes the stale-remote-script failure mode entirely.
• If --host works for all commands rather than just flash, remote erase/enable/reset come back — workflows the old wrapper supported via flash.py (e.g. upload_ssh.py host -e for recovery) and that the current version drops with no scriptable replacement.

Since #221 espresso.py runs on non-Jetson hosts, so it can drive the remote invocation from a dev machine. Happy to help spec the details if you want.

On your core_dumper.py question: in this picture, yes — fold it in as espresso.py coredump. The "no overlap" assessment only holds for the flash/pin logic; core_dumper duplicates exactly the part espresso owns (serial device selection, --chip, and since #221 the device auto-detection), and as a subcommand it would gain remote capability via --host for free — pulling a core dump off a Robot Brain without logging in. It also completes the story this PR starts: espresso.py as the single device tool, and one more file out of the release zip.

Independent of the guard (should also be fixed here)

• espresso.py silently no-ops pin control when gpiod is missing — espresso.py (lines 127–139, 295). On a Jetson whose root Python lacks gpiod, GPIOD_VERSION is None, the no-op base GpioController is selected (the if IS_JETSON: raise only fires inside the except clause and is never reached), and main() just prints a warning — so the flash fails with the generic "Flashing failed" message. flash.py drove /sys/class/gpio with zero dependencies, so remotes that flashed fine before this PR can now fail opaquely. espresso.py should hard-fail when IS_JETSON and gpiod is unavailable — a few lines, and this PR makes espresso the only remote flash path, so it belongs here rather than in espresso.py: hardcoded flash baud + stub upload block some USB-UART bridges — expose --baud / --no-stub #228.
• The Orin-only restriction is documented nowhere users can see it — docs/tools.md (line 19). The Robot Brain section still says generic "NVIDIA Jetson computer". With flash.py gone, one added line would do it: requires a Jetson Orin (L4T 35/36); for Nano/Xavier Robot Brains use release ≤ 0.12.x, which still ships flash.py.
• "on ESP-IDF 5.x the unified Xtensa binary" overclaims — docs/tools.md (line 159). The unified xtensa-esp-elf toolchain only exists since ESP-IDF 5.2; 5.0/5.1 ship per-chip binaries. Suggest "on ESP-IDF 5.2 and newer".
• Line length — docs/tools.md (lines 157–159). The new sentences are 147–197 characters; CONTRIBUTING.md specifies a 120-character maximum, so please break them at sentence/comma boundaries. (Dropping the RISC-V aside would help — this project only targets esp32/esp32s3.)

[evnchn]

Agreed — this is the right call and all seven reproduce (the bare-orin→--swap one especially; the table conflates two opposite pin layouts). I'll move remote flashing into espresso.py --host user@host[:path] (all commands, so remote erase/enable/reset come back), delete upload_ssh.py, fold core_dumper.py → espresso.py coredump, and fix the gpiod-on-Jetson no-op + the doc nits.

One boundary I'll preserve unless you meant otherwise: local --host only validates + transports — Jetson/L4T/device/gpiod/pin detection all stay on the remote Jetson (a Mac's default device must never leak into the remote command). Shout if you intended local-side detection; otherwise I'll send a draft diff.

[evnchn]

TL;DR — draft of the fold is pushed (commit cd99b18); this is the "draft diff" from my last comment.

I took the structural route you proposed rather than patching the guard: remote flashing now lives in espresso.py --host, the wrapper is gone, core_dumper.py is folded into espresso.py coredump, and the gpiod-on-Jetson no-op is fatal. All five guard bugs are structurally impossible now — there's no guard.

Two decisions I made that I'd like your eyes on before I call it done:

• --host works for every command (flash/erase/enable/disable/reset/coredump), so remote recovery is back — matching what you noted.
• Under --host, custom artifact paths must be relative and under build/ (the rsync scope), else it fails fast locally. Defaults are unaffected. This is the one spot where I constrained behavior rather than preserved it — flag if you'd rather ship arbitrary paths.

Ask: does the arg shape + that build/ constraint match your intent? Anything to adjust and I'll fold it in.

What's in the diff (+274/−163, 7 files)

• espresso.py --host user@host[:path] — rsyncs build/ artifacts plus espresso.py itself to the target, then re-invokes the command over SSH with the same args minus --host. The local side only validates + transports; device/pin/L4T resolution all happen on the remote (a dev-machine default device can never leak into the remote command, because only user-typed flags are forwarded).
• core_dumper.py → espresso.py coredump — reuses espresso's serial-device selection and --chip; esp_coredump is imported lazily so the flash path keeps no dependency on it. File deleted, dropped from .github/zip_content.txt.
• gpiod hard-fail on Jetson — build_gpio() raises when IS_JETSON and gpiod is unavailable, instead of silently picking the no-op controller and dying later with the generic "Flashing failed" (the regression vs flash.py).
• Refactor to main(argv) — importing the module now runs no argument parsing and touches no hardware (this is what made the arg-forwarding logic unit-testable).
• upload_ssh.py → a one-release deprecation pointer at the new command.
• docs/tools.md — Orin-only Robot Brain note, --host + coredump usage, ESP-IDF 5.2+ (not 5.x), and the flagged line-length breaks; AGENTS.md + .cursor/rules point at espresso.py coredump.

Verification — what I actually ran, and what I did NOT

Confirmed (ran it):

• Local flash --dry-run emits a byte-identical esptool command to the pre-PR behavior, both chips — the hardware/pin logic is untouched:

  chip	flash_freq	bootloader offset
  esp32	40m	0x1000
  esp32s3	80m	0x0

• The old guard-bug class is now structurally gone, checked directly: --device -e is rejected locally by argparse (was "fails opaquely on the remote"); a bare orin-style call no longer mis-migrates because there's no migration table; --help prints help; the zero-token invocation just runs flash locally.

• Arg-forwarding, path-traversal rejection, and remote-command quoting covered by a local harness (all green).

• pre-commit (trailing-whitespace / eof / double-quote-fixer / mdformat) passes on every changed file.

NOT yet done (honest scope): I don't have a Robot Brain to hand, so the remote rsync+ssh path is verified by inspecting the exact rsync/ssh argv it builds (via mocked subprocess), not by a live flash over SSH onto a Jetson. That end-to-end run is the one thing I'd want a second pair of hands (or hardware) to confirm.

Maker/checker — a second model (Codex) caught 3 real bugs pre-push; all fixed & re-verified

I ran the diff past a different-lineage reviewer (Codex) as a skeptic before pushing. It found three genuine defects, each now fixed and re-checked:

1. Shell injection via an unquoted --host path — --host host:'~/x; sudo reboot' would have run on the remote. Fixed with a quote_remote_path() helper (shlex-quotes the path in both the rsync destination and the ssh cd, while still letting a leading ~/~user expand). Verified ~/'x; sudo reboot', $(reboot), backticks, spaces all neutralized.
2. Custom artifact paths outside build/ were forwarded but never rsynced → silent "file not found" on the remote. Fixed by rejecting them locally (see the build/ constraint above).
3. Path-traversal escape — the first version of that guard let build/../etc/x.bin through (parts[0] == 'build' but it normalizes out of build/). Fixed to reject any .. component; build/x.bin and ./build/x.bin still pass.

Its final verdict on the current diff was ship. Nothing here overrode my own judgement — it just caught what I'd talked myself past.

[falkoschindler]

Great turnaround @evnchn — this is exactly the structural route we had in mind, and I independently re-verified your claims: the dry-run esptool commands are byte-identical to pre-PR for both chips, import espresso is side-effect-free, the guard-bug class is gone (--device -e now fails locally, --help behaves, no migration table to be wrong), the three Codex fixes hold under adversarial inputs (quoting is consistent across rsync and ssh, build/../build/x.bin is rejected, no shell=True anywhere), and all four doc items are fixed with no new >120-char lines.

Answering your two questions:

• Arg shape: yes, --host on every command is exactly right — that's what brings remote erase/enable/reset back.
• build/ constraint: keep it. Failing fast locally beats a silent file-not-found on the remote, and defaults are unaffected. No need to ship arbitrary paths.

Four things to fix before this is done, ranked:

1. run_remote unconditionally rsyncs build/, which breaks the advertised recovery flows — espresso.py (line 241). ./espresso.py erase --host user@robot from a machine without a build/ directory (fresh clone, ops laptop) dies in the first rsync with exit 23 before the erase ever happens — yet erase needs no artifacts at all. Please skip the artifact rsync for commands that don't need artifacts (erase/enable/disable/reset; coredump only needs build/lizard.elf) and always sync just espresso.py itself. Related: both rsync sources are cwd-relative, so invoking the script from another directory fails the same way — resolving against the script's own directory would fix that.
2. --dry-run semantics under the new features. Two spots: (a) --dry-run --host still executes both rsyncs for real — a "dry run" overwrites the remote's espresso.py and binaries before showing what would happen (espresso.py, line 430); it should skip the rsyncs or pass --dry-run to rsync too. (b) The new gpiod hard-fail fires before the dry-run check, so flash --dry-run on a Jetson without gpiod now errors where it previously printed the commands (line 450) — a dry run touches no pins, so gate the hard-fail on not args.dry_run.
3. Remote coredump runs under sudo — espresso.py (line 430). use_sudo=not args.dry_run applies to all commands, but esp_coredump installed via pip install --user on the Jetson isn't on root's sys.path, so remote coredump raises ModuleNotFoundError even though it works locally on the robot without sudo. Coredump needs dialout, not root — limit sudo to the pin/flash commands.
4. Small hardening: insert -- before the ssh/rsync target (or reject a leading-dash host) so a pasted host string like -oProxyCommand=... can't become an option injection — one line, low severity for a trusted-operator tool but free to fix.
5. The branch now conflicts with main, and the resolution has a trap. #229 landed on main (--baud/--no-stub) inside the region your refactor rewrote, so git can't auto-merge espresso.py — the flags need to be re-applied in the new structure (two add_argument lines plus BAUD/STUB_ARGS in the three esptool call sites). The trap: they must also be added to the hardcoded flag list in the --host forwarding function, or --baud 115200 --host ... silently flashes the remote at the default baud rate. That list is the same hand-maintained-mirror pattern we just eliminated from the wrapper, and this conflict is its first real drift — consider deriving the forwarded arguments from the parser actions (forward whatever differs from its default) so future flags can never be forgotten. Please also re-run your dry-run comparison after the merge, since baud is now variable.

Two footnotes, no action needed unless you feel like it: rsync ≥ 3.2.4 no longer shell-interprets remote args by default, so for a --host path containing spaces the shlex-quoted rsync destination and the ssh cd target can diverge (the ~/lizard default is unaffected); and the ~user expansion regex only matches [\w-]+ usernames, so ~john.doe/... won't expand. Both are edge cases worth a comment at most.

With 1–3 and the main merge (5) addressed I'm happy to merge. The one remaining gap is the live end-to-end run over SSH onto a real Jetson, which you flagged yourself — we have hardware here, so we'll take that part.

[evnchn]

Thanks @falkoschindler — all four ranked items plus the #229 merge are in.

• 1 — remote artifact rsync is now per-command. flash syncs *.bin, coredump syncs *.elf, and the pin-only commands (erase/enable/disable/reset/release_pins) skip the build sync entirely, so they no longer die on exit 23 from a machine with no build/. Both rsync sources also resolve against the script's own directory, so espresso works from any cwd.
• 2a — --dry-run --host is print-only now. It prints the exact rsync + ssh it would run and touches nothing on the remote. 2b — the gpiod hard-fail is gated on not dry_run (a dry run touches no pins).
• 3 — sudo is limited to the pin/flash commands, so remote coredump runs as your user (dialout, not root) and finds the pip install --user esp_coredump.
• 4 — a leading-dash --host is rejected, so a pasted -oProxyCommand=… can't turn into an ssh/rsync option.
• 5 — re-applied --baud/--no-stub in the None-default structure, and took your suggestion: the forwarded --host args are now derived from the parser actions — every option whose value differs from its default is forwarded — instead of the hand-maintained mirror. A future flag can't be silently dropped, so the wrong-baud footgun is closed structurally. Re-ran the dry-run comparison after the merge: flash (both chips) and erase are byte-identical to main.

One extra from a second-lineage (Codex) review pass: the build/ artifact check now also enforces the matching extension, so --firmware build/lizard fails fast locally instead of going silently missing on the remote (the include pattern would never have transferred it).

Verification

• flash --dry-run esptool commands byte-identical to main post-espresso: add --baud and --no-stub flags for flaky USB-UART bridges #229 for esp32 and esp32s3, and erase likewise.
• Guard fuzz: --host=-oProxyCommand=evil rejected before any rsync; artifact paths that are absolute, contain .., or carry the wrong extension are rejected; the derive-from-parser forwarding round-trips --baud/--no-stub/--device/--chip/flags correctly and forwards nothing at defaults.
• --dry-run --host exercised end-to-end: prints the rsync/ssh, runs neither, uses no sudo.
• pre-commit run --all-files green; import espresso is side-effect-free.

The branch now merges cleanly with main (I merged rather than rebased so the delta since your review stays visible). All yours for the live Jetson-over-SSH run. 🙏

[falkoschindler]

Thanks @evnchn, this is in great shape now. After the first review I ran it through two more Fable review passes (until I ran out of tokens for the week), and the last 15 commits are the result. It turned into a fair-sized refactor of espresso.py, but I'm happy with where it landed.

Summary of what changed since the first review:

Structure

• Importing espresso.py is now side-effect-free: the module-global configuration is replaced by a frozen Config dataclass, and all argument parsing and hardware resolution moved into main().
• The scattered per-command tables and if/elif dispatch are consolidated into a single COMMANDS registry (handler, pin usage, artifacts).

--host remote mode

• Forwards the verbatim argv to the remote instead of reconstructing it from parsed values.
• Passes the rsync destination path unquoted (rsync >= 3.2.4 escapes remote args itself; quoting broke paths with spaces).
• Preserves permissions when copying espresso.py, restoring the exec bit on a pre-existing non-executable remote copy.
• Rsyncs build artifacts resolved against the cwd (matching a local run), not the script directory, and copies exactly the files the command needs (rsync -R) instead of walking the whole build tree by extension glob.
• scp-style trailing colon now maps to the remote home directory.

coredump

• Honors an explicit --baud.
• Fails with a clear, actionable error when esp_coredump is not installed.

Defaults, UX & migration

• Defaults to /dev/ttyUSB0 on non-Jetson Linux hosts (macOS still gets the CP210x name).
• The --host --dry-run preview shows the real sudo prefix.
• The upload_ssh.py deprecation stub points Nano/Xavier users to release 0.12.x and now ships in the release zip, so it actually reaches those users.
• Docs note that the remote coredump needs serial-port (dialout) access for the SSH user.

Since this adds a fair amount of new remote/flash machinery, I'd like a thorough hardware test before we merge. @JensOgorek could you exercise it on real hardware — flash and coredump, and ideally the --host remote path against a Robot Brain? Approving from my side otherwise.

[evnchn]

Posted by Claude Code on evnchn's behalf.

TL;DR: Pushed 052f969 — a small hardening commit (+23/−9) on top of the refactor stack, from a hardware-free review pass (dry-runs, argv edge cases, and real rsync/ssh transfers to a LAN Linux box; no ESP32 involved). pre-commit passes locally and in this PR's CI run on the new head.

Six fixes, each with evidence in the folds:

1. A failed flash (or Ctrl-C) used to skip the pin cleanup entirely and leave the ESP parked in bootloader — try/finally in _pin_config/_flash_mode.
2. Stock-macOS rsync (openrsync) silently copies to the wrong remote path — exit 0, no error — when the --host path contains whitespace; such paths are now rejected up front.
3. A dash-prefixed artifact path reached the rsync argv as an option (--firmware=-x.bin → -x is --one-file-system); now rejected under --host.
4. A missing esptool.py binary now yields the red one-liner instead of a Python traceback.
5. cd --, so a dash-prefixed remote path cannot become a cd option.
6. flush=True on the banners and command previews that precede subprocess output, so piped/CI logs keep their order.

One judgement call you may want to overrule: the whitespace rejection (fix 2) also blocks users whose rsync ≥ 3.2.4 handles those paths fine — rejecting felt safer than a warning because openrsync fails silently, but it's a two-line change to soften. Everything else should be uncontroversial. Codex adversarially re-reviewed this fix diff and found no new functional blocker.

Fix 1 — failed flash left the ESP in bootloader (repro + after)

Both context managers ran their cleanup after a bare yield, so any exception between entering flash mode and success skipped _reset and _release_pins — the ESP stayed in bootloader (G0 sequence never reverted) and the gpiod request was only released by process exit. Ctrl-C during a long flash hits the same path.

Repro (no hardware): a fake esptool.py that exits 2, on the PATH.

Before (f10424d):

  esptool.py --chip esp32 --port ... write_flash ...
esptool: simulated failure
Error: Flashing failed. Use "sudo" and check your parameters.

After (052f969) — the reset sequence now runs before the error exit:

  esptool.py --chip esp32 --port ... write_flash ...
esptool: simulated failure
  Setting G0 pin to 1
  Setting EN pin to 0
  Setting EN pin to 1
Error: Flashing failed. Use "sudo" and check your parameters.

Exit code stays 1; behavior on the success path is unchanged (cleanup ran there already).

Fix 2 — openrsync silent miscopy: exact commands + observed matrix

Ran real transfers from macOS 15 to a LAN Ubuntu box (server rsync 3.2.7), both client flavors:

client	destination	result
homebrew rsync 3.4.4	host:~/dir/ (tilde)	✅ expands to remote home
/usr/bin/rsync (openrsync, protocol 29)	host:~/dir/ (tilde)	✅ expands to remote home
homebrew rsync 3.4.4	host:~/espresso space test/	✅ creates espresso space test/, file inside
/usr/bin/rsync (openrsync)	host:~/espresso space open/	❌ exit 0, nothing at the requested path — payload landed at ~/espresso instead

The dangerous half is the last row: the transfer succeeds (exit 0), the artifacts land somewhere else, and the subsequent ssh step cds to the correctly-quoted path — so a user on stock-mac rsync would flash stale artifacts or get a confusing cd failure, with no error pointing at the actual cause. Quoting the path for rsync isn't a fix either: rsync ≥ 3.2.4 would then create a directory with literal quotes in its name. There is no single escaping that works across client generations, hence reject-with-explanation. rsync < 3.2.4 clients (e.g. Ubuntu 20.04's 3.1.3) predate the arg-protection change and are expected to word-split the same way (untested — expectation only).

Scope: this proves the two macOS client flavors against an rsync 3.2.7 server. The trailing-colon ~ mapping from ea2e4e9-era commits is confirmed safe on both clients (rows 1–2).

Fixes 3–6 — evidence, plus the full verification list

Fix 3 (dash artifact): dry-run before the fix showed the injected argv — rsync -zavR build/... -x.bin 'u@h:~/lizard' — where -x.bin parses as rsync options. _require_relative now rejects a leading - with an explanatory error. Paths like ./-x.bin remain usable.

Fix 4 (missing esptool): PATH without esptool previously ended in a raw FileNotFoundError traceback; now: Error: Command "esptool.py" not found; is it installed and on the PATH? (and, combined with fix 1, the pins still get reset first).

Fix 5 (cd --): --host 'h:-tmp/x' previously produced cd -tmp/x, which cd reads as an option; the remote command now uses cd -- <path>. Visible in --dry-run output.

Fix 6 (flush): observed live — with output piped, the remote-run banners printed after the subprocess output they announce. flush=True on the prints that precede subprocess calls restores order; no change on a tty.

Verification on the final diff (all on 052f969):

• fake-esptool failure repro: cleanup runs, exit 1 (fix 1)
• whitespace / dash-artifact rejections fire with the new messages; all pre-existing rejections (absolute, ..) unchanged
• cd -- present in dry-run; full local + --host dry-run outputs otherwise unchanged except the flushed ordering
• Python 3.8 (real 3.8 via uv, matching Jetson R35 / Ubuntu 20.04): import + --host dry-run pass
• pre-commit clean locally and in CI

If the whitespace-reject call (or any of these) doesn't match your intent, say the word and I'll adjust — none of them are load-bearing for the rest of the PR.

[evnchn]

Posted by Claude Code on evnchn's behalf.

TL;DR: The hardware pass the PR description still wanted has now been run — on non-Jetson hardware: an ESP32-D0WD (4 MB, CH340) and an ESP32-S3 N16R8 (16 MB, CH343) attached to a LAN Ubuntu box, driven from a Mac via --host. Everything passed, including the headline feature: a real remote core dump of a real crash, symbolized down to the source line. Also pushed c8623f4 — a docs-only correction on what remote coredump actually requires (details in the folds).

What this pass cannot cover, and stays open for a Robot Brain: Jetson EN/G0 pin control (gpiod v1/v2), THS UART auto-detection on real L4T, piggyboard --swap/--nand, and sudo-over-ssh on the real image.

Test matrix (all via espresso.py at the current head, esptool v5.3.1 on the remote)

test	result
remote flash ESP32, Mac → Linux --host, 460800 baud	✅ hash verified, "Finished ☕️"
remote flash ESP32-S3 (--chip esp32s3: 0x0 bootloader offset, 80m)	✅ boots to lizard Ready.
--partition-table <custom .bin> override under --host	✅ only the changed artifact re-rsynced, argv forwarded verbatim
flash at the default 921600 on CH340	✅ 988 kbit/s, hashes verified
--reset-partition (otadata erase_region 0xf000 0x2000 at flash baud)	✅ offsets match partitions.csv
--no-stub flash	✅ works; ~2.3× slower, as the help text says
erase + re-flash cycle	✅
remote coredump of a real crash	✅ full multi-thread symbolized dump (see next fold)
coredump --debug	✅ real xtensa-gdb 17.1 session against the dump
failure UX (wrong --device, remote)	✅ boxed esp-coredump error + espresso's red one-liner, exit 1
stock-macOS openrsync client on the default path	✅ the mainline Mac→robot flow needs no homebrew rsync

Both boards' entire flash was backed up (read-flash + sha256) before any test.

The coredump feature, end to end on real silicon

Provoked a crash on the S3 via a DSL expression over serial (integer division by zero — a dedicated issue with an MRE follows separately), which panics and saves a dump to the coredump partition:

Guru Meditation Error: Core 0 panic'ed (IntegerDivideByZero). Exception was unhandled.
I esp_core_dump_flash: Save core dump to flash...
I esp_core_dump_flash: Core dump has been saved to flash.

Then, from the Mac:

./espresso.py coredump --host user@host:lizard-pr227-s3 --chip esp32s3 --device /dev/ttyACM0

rsyncs build/lizard.elf, reads the dump on the remote, and symbolizes it precisely:

Crashed task handle: 0x3fcb0028, name: 'main'
exccause       0x6 (IntegerDivideByZeroCause)
#1  0x4200edc0 in DivideExpression::evaluate_integer (this=0x3fcb57bc) at main/compilation/expressions.cpp:197
#2  0x4200ec10 in Expression::print_to_buffer (...) at main/compilation/expression.cpp:37
#3  0x42018971 in process_tree (...) at main/main.cpp:201

--debug drops into the same dump with a live GDB prompt. One caveat worth knowing: a stack-overflow crash (deep parser recursion) trampled esp_core_dump's own RAM config ("Core dump flash config is corrupted!") and saved nothing — coredump-enabled does not mean every crash leaves a dump.

Why the docs commit (c8623f4): what remote coredump really requires — incl. a refuted shortcut

On a machine with only pip install esp-coredump, the coredump command fails through successive walls, verified one by one:

1. the module must be importable by the python3 running espresso (a uv tool / pipx-style install exposes only the CLI);
2. IDF_PATH must be set, or esp_coredump exits;
3. the matching xtensa-esp32s3-elf-gdb must be on the PATH;
4. esp_coredump locates the coredump partition via $IDF_PATH/components/partition_table/parttool.py, which in turn runs $IDF_PATH/components/esptool_py/esptool/esptool.py — i.e. it needs a real ESP-IDF checkout, not just the package.

Refuted shortcut: espresso could pass CoreDump(off=0x400000) (the offset is fixed by partitions.csv), and I tried exactly that — but esp-coredump 1.16.0 calls its parttool lookup unconditionally (_invoke_esptool fetches the partition size for a sanity check even when the offset is given, corefile/loader.py), so the code change buys nothing and was dropped. Upstream's own TODO says the offset argument will be removed in favor of parttool, so the dependency is going the other way. Hence the docs-only fix: say plainly that the reading machine needs a sourced ESP-IDF environment.

Environment notes picked up along the way (nothing blocking)

• esptool v5.3.1 works, but deprecation warnings are piling up: the esptool.py entry-point name, write_flash, erase_flash and erase_region all warn "deprecated, will be removed in a future major release". espresso is fine today; a rename sweep (esptool, dashed subcommands) may be worth a follow-up before esptool v6 lands.
• The stock partition table needs a ≥ 8 MB flash chip: on a 4 MB dev board the bootloader rejects the table ("partition 5 invalid - offset 0x400000 ... exceeds flash chip size") and boot-loops. Flashing a trimmed table via --partition-table boots fine (that's how the 4 MB board in this pass runs lizard). Probably worth one line in the docs for dev-board users; Robot Brains are unaffected.
• The CH343-bridge stub-upload quirk seen from macOS during the earlier verification did not reproduce from the Linux host — flash and dump reads were clean at 460800/115200 — consistent with it being host-side rather than board-side.

With the earlier hardening commit and this pass, everything testable without a Robot Brain is green from my side; the remaining verification (Jetson pins, piggyboard, L4T device paths) is best done on your hardware. Happy to fold in any renames or doc tweaks you'd prefer first.

[JensOgorek]

Reviewed the diff and ran a hardware pass (with Claude Fable 5 assisting). Test target: a classic ESP32-D0WD-V3 (8MB flash, CP2102N bridge) on a plain Linux dev machine — i.e. the non-Jetson path, complementing the ESP32-S3 test already done for this PR.

What I tested

Flash: ./espresso.py flash --device /dev/ttyUSB0 --reset-partition — works. The gpiod fallback kicks in as designed (no-op pin controller, esptool DTR/RTS reset), the chip-dependent values are right for classic ESP32 (bootloader @ 0x1000, 40m flash freq), and Lizard boots to Ready.

Coredump, end to end: I deliberately crashed Lizard with core.print(1/0) — integer division in expressions.cpp:197 is unchecked, so this raises an IntegerDivideByZero CPU exception → panic → core dump written to the flash partition → clean reboot. ./espresso.py coredump --device /dev/ttyUSB0 then produced a fully symbolized backtrace pointing at the exact crash site (DivideExpression::evaluate_integer → Core::call → process_uart). Very useful in the field; the integration earns its keep.

Dry runs and the stub: --host --dry-run previews for flash and coredump are correct — only the artifacts the command needs, sudo only for pin commands, verbatim argv pass-through, proper quoting. The upload_ssh.py stub prints the migration pointer to stderr and exits 2 without touching anything.

Minor findings (non-blocking)

1. On a non-Jetson host the flash still walks through the EN/G0 pin dance on the no-op controller: it prints Setting EN pin to … (mildly confusing — there are no pins) and sleeps 6×0.5s for nothing. Consider skipping the pin sequence when the controller is the no-op fallback.
2. espresso.py doesn't flash ota_data_initial.bin (same as the old flash.py), so a chip with stale otadata boots the wrong OTA slot after flashing. --reset-partition covers this — a one-line hint in docs/tools.md would help people hitting it.
3. Xavier Robot Brains can run L4T 35, so they pass the L4T gate in resolve_default_device() and only fail later on the wrong (Orin) pins PR.04/PAC.06. Docs and error messages should name Nano/Xavier explicitly rather than arguing from the L4T version alone.

Follow-up proposal: hosted legacy flash tools

Rather than only pointing Nano/Xavier users at release 0.12.x (whose zip bundles old binaries), we plan to host the removed scripts on an orphan branch legacy-flash (flash.py, esp.py, upload_ssh.py + a short README), fetchable via curl -LO https://raw.githubusercontent.com/zauberzeug/lizard/legacy-flash/flash.py. That way old brains can flash new firmware with the legacy tools, and main stays clean. The Unsupported L4T version error, the upload_ssh.py stub, and docs/tools.md would then carry that one-liner. We'll do this as a separate PR after this one merges — no need to grow #227 further.

Author's open questions

• backtrace.sh removal: agreed — the documented addr2line one-liner is the better answer.
• core_dumper.py folded into espresso.py: proved itself in the test above; keep it.

Side observation (separate issue material)

Any integer division by zero in a Lizard script reboots the whole controller (see the crash recipe above). A guard in expressions.cpp returning a runtime error instead would be more robot-friendly — worth its own issue, unrelated to this PR.

Still pending before merge

The real Jetson GPIO path and a real --host run against an Orin Robot Brain. I can test both on a real robot on Thursday/Friday this week.

[falkoschindler]

@evnchn Nothing left from my side after our earlier review rounds — I re-checked the final state on the branch (dry-run flash, remote --host dry-runs, the path/host guards) and everything behaves as described. The remaining notes are cosmetic and don't need fixing. Approving; let's just do the planned hardware pass of flash/coredump/--host on a real Robot Brain before merging.

[JensOgorek]

Hardware pass on a real Robot Brain is done — everything works. ✅

I drove erase, flash and reset from my dev machine through the new --host remote mode against rb63; the freshly built firmware boots and I exercised it over the live DSL after a from-scratch reconfiguration. Since the chip was demonstrably empty after the erase, the flash provably did something — not just "everything ran before, everything runs after". coredump on a robot without an ESP-IDF environment fails with the intended actionable message. Robot restored afterwards.

Good to merge from my side. 🚀

Test details (setup, per-command results, observations)

Setup: rb63 — Jetson Orin, L4T R36.4.3, ESP32-D0WD-V3 on /dev/ttyTHS1, --nand board; branch head 22d9591 built for esp32 with IDF 5.3.1; all remote commands driven from a dev machine via --host rb63.

• erase --nand --host rb63: pin-only path — no artifact rsync (as designed), chip erase completed in 5.9s, ESP verified empty afterwards.
• flash --nand --host rb63: rsynced exactly the three artifacts + espresso.py, the remote resolved its own device (/dev/ttyTHS1, correct for R36) and pins, esptool wrote bootloader/partition table/firmware at 921600 with all hashes verified. Firmware v0.12.0-27-g22d9591 boots (serial boot banner) and was manually reconfigured and exercised over the live DSL (it rejects private modules it doesn't know, exactly as expected on an empty storage).
• reset --nand --host rb63: works; no artifact rsync, EN/G0 sequence executed, ESP rebooted (boot banner captured).
• coredump --host rb63: the rsync/ssh flow works (ELF delta-transferred, no sudo used — as intended); with no ESP-IDF environment on the robot it hits the new actionable error message and propagates exit 1 cleanly, matching the documented requirement. A full remote dump read wasn't possible on this host, but the boot log confirmed a real core dump sitting in the coredump partition, so the partition layout side is fine.
• --host dry-runs for flash/erase/coredump print the exact rsync/ssh commands and mutate nothing; local flash --dry-run emits the correct esptool command for esp32.

Observation, not a blocker: the remote commands run under sudo, so fully non-interactive use (no TTY) stops at the password prompt unless the robot has passwordless sudo — interactively it's fine.

[evnchn]

Posted by Claude Code on evnchn's behalf.

TL;DR — LGTM from me as the assignee.

I re-read the final diff top to bottom (espresso.py refactor, the upload_ssh.py stub, the four deletions, the docs) rather than rubber-stamping the two approvals — and everything load-bearing checks out. Nothing blocking.

Between @falkoschindler and @JensOgorek this has had unusually thorough coverage for a tooling PR:

• Structural + adversarial review — @falkoschindler, one CHANGES_REQUESTED that reshaped run_remote, then two further Fable review passes (the 15-commit refactor) and two approvals, each with an independent re-verify of the claims.
• Hardware, three separate targets — @JensOgorek on a classic ESP32-D0WD-V3 dev board and me on an ESP32-S3 N16R8 (both non-Jetson), then @JensOgorek on a real Robot Brain (rb63, Jetson Orin, L4T R36, --nand) driving erase/flash/reset over --host, with an erase-then-verify-empty proving the flash actually did something.

The only merge action left is the maintainers' click — flagging that it's green from the assignee's chair too. 🚀

What I independently checked on the final diff (grounding the LGTM, not parroting it)

• Side-effect-free import holds — all parsing/hardware probing sits in main()/resolve_default_device(); module scope only sets IS_JETSON and GPIOD_VERSION, so import espresso probes no GPIO and reads no /etc/nv_tegra_release.
• --host argv pass-through is sound — remote_command() strips exactly --host/--host=/-d/--dry-run and forwards the rest verbatim; allow_abbrev=False guarantees no token is misread (--ho→--host), and order-independence is safe because the remote re-parses with the same parser.
• Injection surface is closed — no shell=True anywhere (the old upload_ssh.py had two); leading-dash host rejected; whitespace remote path rejected (the openrsync silent-miscopy class); cd --; shlex.quote on every remote token and the path tail; artifact paths forced relative, no .., no leading -.
• Path resolution is consistent — rsync -R recreates relative paths under the target dir and the remote cds there before resolving the same relative artifact paths, so a remote flash writes exactly what a local flash would; only espresso.py itself is taken from the script dir (copied -p to keep the exec bit).
• Config integrity — frozen dataclass with derived values as properties (flash_freq/bootloader_offset can't drift from chip); replace() injects the real GPIO controller only for pin commands on a non-dry local run.
• Failure paths hardened — try/finally in the pin/flash context managers (a crash or Ctrl-C now reverts the pins instead of leaving the ESP in bootloader), FileNotFoundError→friendly esptool message, CalledProcessError caught at top level.

Follow-ups tracked — deliberately NOT in this PR

Keeping #227 minimal per @falkoschindler's "no need to grow this further":

• Div-by-zero reboots the controller — filed as Dividing two integers by zero (e.g. 1 / 0) panics and reboots the ESP32 #234, with the 1 / 0 MRE @JensOgorek and I both hit while provoking coredumps. Independent of this PR.
• Hosted legacy-flash tools for Nano/Xavier brains (orphan legacy-flash branch) — @JensOgorek's separate PR after this merges; the Unsupported L4T error, the upload_ssh.py stub, and docs/tools.md get the curl one-liner then.
• Three cosmetic notes from @JensOgorek's dev-board pass — no-op-controller pin-dance noise, an ota_data/--reset-partition doc hint, naming Xavier explicitly in the L4T gate. @falkoschindler flagged these as non-blocking; they fold into the follow-up.

Not-a-blocker observation from the Robot Brain pass: remote commands run under sudo, so fully non-interactive (no-TTY) use needs passwordless sudo on the robot — interactively it's fine.