`secrets` context isn't available in step-level `if:` expressions inside
a reusable workflow_call. Move the secret-presence check to job-level
env (HAS_RAUC_SECRETS, HAS_AUTOIMPORT) and reference those in step if:.
New module kiosk/src/at_rest.rs. Derives an AES-256-GCM key via HKDF
from a Pi-bound value:
1. /proc/device-tree/serial-number (Pi 5 firmware exposes it)
2. /proc/cpuinfo Serial line (older kernels)
3. /etc/machine-id (non-Pi dev fallback)
File format: "BFE1" magic || 12-byte random nonce || ciphertext+tag.
Atomic write via tempfile + rename so a crash mid-write can't leave a
half-encrypted file.
Wired into kiosk/src/server.rs at every file I/O touching sensitive
state:
- kiosk.key (bearer token to BF server)
- local.key (LAN-side API auth key)
- bundle.json (cached bundle with RTSP credentials in URL form)
Migration: read paths tolerate legacy plaintext (kiosks upgraded from a
pre-at_rest build) AND re-store as ciphertext on the first read. One-
shot upgrade — subsequent boots skip the migration write.
Threat model defended: SD card extraction. Attacker who pulls the card
can't decrypt without also having the same physical Pi (CPU serial is
hardware-bound). Doesn't defeat an attacker who has both — at that
point they ARE the kiosk. Bar is raised from "trivially extract every
camera password" to "must steal the device intact."
Not defended: TPM-style attestation, remote attestation, sealed boot.
Pi 5 has no TPM and we don't ship a secure-boot config.
Tests in-module: round-trip short bytes, round-trip JSON, legacy
plaintext passthrough.
Mirrors the kiosk-firmware admin shape one-for-one against OS RAUC
bundles:
/admin/os-updates release list, yank
/admin/os-updates/rollouts rollout list + create
/admin/os-updates/rollouts/:id/state pause/resume/complete
/admin/kiosks/:id/os-update per-kiosk channel + pin
Templates: OsUpdatePage, OsUpdateRolloutsPage, KioskOsUpdatePanel.
KioskOsUpdatePanel is rendered next to the existing KioskFirmwarePanel
on the kiosk detail page so OS + app state sit side-by-side. The
"how bundles get here" sidebar on the list page documents the four
GitHub secrets needed (signing cert/key + autoimport URL/key) so a
new operator doesn't have to dig through scripts/ to find them.
Nav gains an OS Updates entry between Firmware and Labels. Activates
on activeNav="os-updates".
Repo + import endpoint already existed (audit confirmed earlier). All
admin routes use them as-is.
Two image-side hardening pieces both small enough to ship together.
deploy/nftables/nftables.conf — single ruleset installed at /etc/nftables.conf.
Default-drop input. Allowed: loopback, established/related, ratelimited
ICMP, kiosk local API :18090 from RFC1918 / RFC4193 / link-local sources
only. SSH stays gated by sshd-disabled (image build sets enable-ssh: 0
and 01-run-chroot masks it); the firewall rule for :22 is left commented
in for triage scenarios. Forward dropped. Output left wide open — kiosk
needs to dial out to arbitrary RTSP cameras + the BF server (which may
live on the public internet) without explicit allowlisting.
deploy/systemd/betterframe-firstboot.{service,sh} — runs once per device
before betterframe-kiosk starts. Generates a 24-char unambiguous-glyph
password, applies via chpasswd, stores at /etc/betterframe/admin-password
(0400 root), and prints a banner to tty1 so an HDMI-attached operator
can transcribe it during the boot window before cage takes over the
screen. Marker at /var/lib/betterframe/.firstboot-complete prevents
re-run on subsequent boots. Without this, every kiosk built from the
same image shipped with bfadmin:betterframe — a single password leak
compromises the entire fleet.
Future follow-up: post the rotated password (encrypted with cluster_key)
to the BF server via heartbeat so admin UI can surface it. Not in this
commit; the local file + tty banner are the only retrieval paths today.
Phase 2b. Bake the runtime side of RAUC into the curated image so a
freshly-flashed kiosk can accept .raucb bundles immediately:
- Add `rauc` + `dosfstools` to the apt package list.
- Drop deploy/rauc/system.conf to /etc/rauc/system.conf (already declares
the A/B slot layout that repartition-image.sh produces).
- Drop deploy/rauc/betterframe-rauc-boot.sh to
/usr/local/sbin/betterframe-rauc-boot.sh — the custom bootloader
backend that flips the BF_BOOTSEL autoboot.txt to point at the
freshly-installed slot via Pi 5 tryboot.
- Drop deploy/rauc/ca-cert.pem (operator-supplied, committed) to
/etc/rauc/keyring.pem so rauc can verify CMS signatures. If the cert
isn't committed yet, image build emits a workflow warning and the
kiosk image installs but refuses every bundle — image still flashes,
just no OS OTA until the cert is committed.
- Enable BF_ENABLE_OS_OTA=1 in /etc/default/betterframe-kiosk so the
kiosk Rust consumer actually polls for bundles. Set to 0 to pin OS
version for a specific kiosk.
mark-good was already wired (deploy/systemd/betterframe-rauc-mark-good.{service,sh}).
The kiosk's heartbeat loop also calls `rauc status mark-good` as a
belt+suspenders backup; both are idempotent.
Phase 2a of OS OTA: post-process pi-gen output into a RAUC-compatible
A/B layout. New deploy/rauc/repartition-image.sh:
- Decompresses the stock pi-gen 2-partition image
- Extracts bootfs (vfat) + rootfs (ext4) blobs
- Compacts rootfs with resize2fs -M and grows back with 25% headroom
- Patches /etc/fstab inside rootfs to use LABEL=BF_BOOT_A /
LABEL=BF_ROOT_A / LABEL=BF_DATA (slot-agnostic; RAUC re-labels per
slot on install)
- Stamps /etc/betterframe/{os-version,os-compatibility} for the kiosk's
os_update.rs to read at runtime
- Builds two bootfs copies, each with cmdline.txt root= rewritten to
the matching ROOT slot
- Lays out 6 GPT partitions: BF_BOOTSEL (autoboot.txt with tryboot
pointing at boot_partition=2 / [tryboot] boot_partition=3), BF_BOOT_A,
BF_BOOT_B, BF_ROOT_A (populated), BF_ROOT_B (empty, RAUC fills on
first install), BF_DATA
- Recompresses with xz -T0
build-bundle.sh now takes the already-extracted slot images so the
.raucb bundle re-uses the exact same blobs that ship inside the A/B
initial-flash image — no duplication, no drift.
CI wires the repartition step between pi-gen output and the GitHub
Release upload. Ships the A/B image (not the stock pi-gen one).
Also: bump Blacksmith binary builders from 2/4 vCPU to 8 vCPU each.
Image job stays on GitHub's ubuntu-24.04-arm (Blacksmith arm kernel
6.5 doesn't ship binfmt_misc as a loadable module, which pi-gen-action's
defensive modprobe step still requires).
What's still pending:
- In-image RAUC install (rauc package + drop system.conf + CA cert
at /etc/rauc/keyring.pem). Without this, the image boots A/B-laid-
out but rauc install commands have no daemon to talk to.
- Admin UI for OS releases + rollouts (task #4).
Phase 3 of the OS OTA pipeline. New module kiosk/src/os_update.rs polls
/api/kiosk/os/check with the kiosk's compatibility string and current OS
version (read from /etc/betterframe/os-compatibility +
/etc/betterframe/os-version, both written by the image build), downloads
the bundle, sha256-verifies the transport, and hands off to
`rauc install`. RAUC takes it from there: CMS signature verify against
/etc/rauc/keyring.pem, copy into inactive A/B slot, arm tryboot via the
custom bootloader backend, return. We then post /api/kiosk/os/applied
and `systemctl reboot` into the new slot.
Wired into the existing 60s heartbeat loop in ui.rs, gated by
BF_ENABLE_OS_OTA=1 (default OFF so dev kiosks on non-A/B images don't
keep trying + failing). Runs BEFORE the kiosk-binary check on each tick
so an OS bundle that ships an updated kiosk binary doesn't race the
firmware path.
On clean-boot heartbeat success we now also call `rauc status
mark-good` so the boot-attempts counter resets — three bad boots in a
row will auto-roll back without us needing a separate rollback path.
What's NOT in this commit:
- A/B partition layout in the pi-gen image (task #6, blocks actual
deployment — bundles can be served + accepted but `rauc install`
will refuse without two valid slots).
- Admin UI for managing releases + rollouts (task #4).
Phase 1 of the OS OTA pipeline. Three pieces:
scripts/gen-rauc-signing-keys.sh — one-shot helper that issues an
Ed25519 X.509 CA + signing cert pair. Operator runs locally, commits
the CA cert (for embedding in kiosk image at /etc/rauc/keyring.pem),
stores the signing pair as GitHub Actions secrets
(BF_RAUC_SIGNING_CERT + BF_RAUC_SIGNING_KEY), keeps the CA private
key offline. RAUC verifies bundles against the keyring in the image.
deploy/rauc/build-bundle.sh — takes the pi-gen .img.xz, parses its
partition table with sfdisk, dd-extracts bootfs (vfat) + rootfs
(ext4) into a staging dir, renders manifest.raucm.in with version
+ git sha, runs `rauc bundle --cert= --key=` to produce a signed
.raucb. Verifies the bundle round-trips with `rauc info`.
build.yml gains two gated steps:
- "Build RAUC bundle": runs only when both signing secrets are set,
uploads .raucb as a release asset alongside the .img.xz.
- "Auto-import OS bundle into BF server": POSTs the GH release asset
URL to ${BF_AUTOIMPORT_URL}/api/admin/os/import so the server
pulls + stores the bundle. Mirrors the kiosk-binary auto-import
flow that already worked.
Compatibility string is `betterframe-rpi5-aarch64` (matches the value
already declared in deploy/rauc/system.conf). Channel passed through
from inputs (dev for master pushes, stable/beta for tags).
What's NOT in this commit:
- Pi image A/B partition layout (custom genimage / pi-gen patch)
- rauc package install + keyring drop in pi-gen stage
- Kiosk-side os_update.rs Rust consumer that polls /api/kiosk/os/check
- Admin UI for releases + rollouts
A bundle built today reaches /api/admin/os/import on the server but
isn't installable yet — kiosks have no consumer and no A/B layout.
That's the next 3 phases. Bundle production needs to be solid first
so the kiosk side can be tested against real artifacts.
When admin opens an entity preview, find a kiosk whose active layout
references the camera (new repo.listKiosksRenderingCamera). Probe each
candidate's LAN snapshot endpoint with a 4s timeout. On success, stream
the bytes back with x-bf-snapshot-source: kiosk:<id>. Falls through to
the existing server-direct ffmpeg/gst pull only when no kiosk is reachable
or has the camera in its active layout.
Kiosk side adds /local/snapshot/:camera_id?key=<local_key>. Spawns a
one-shot gst-launch (rtspsrc → decodebin → jpegenc ! filesink
num-buffers=1) on a blocking worker so axum's reactor stays free.
Prefers sub stream for snapshots to keep bandwidth low. Single-frame
pipeline tears down after the first JPEG.
LAN IP picking extracted to shared/kiosk-lan.ts so route handler and
KioskLocalPanel agree on which interface to talk to (the previously-
duplicated logic in admin-pages stays for now since it also renders the
interface list).
Why a parallel pipeline instead of teeing the warm one: cross-thread
gtk4paintablesink → appsink sample extraction is non-trivial. A 1-frame
parallel pull is cheap when the kiosk's RTSP session to that camera is
already known to work (precondition: it's in the active layout).
BF_ENABLE_APP_OTA gates the periodic firmware check in ui.rs:449.
Our curated image is the place where it should be on — operator can
flip to 0 in /etc/default/betterframe-kiosk to pin a specific build.
Without this, a fresh image never picked up new dev releases even when
the auto-import secrets were configured.
WebView showed a grey/black half-square in the top-left — that's GDK's
"none" cursor rendered by WebKit's own surface, which ignores the
GTK-window CSS we set elsewhere. Inject a WebKit UserStyleSheet that
applies cursor:none !important to every page + frame at User priority,
overriding page-author CSS.
For the boot gap (cage start → first kiosk frame), set XCURSOR_SIZE=1
and WLR_NO_HARDWARE_CURSORS=1 in the systemd unit. SW fallback honors
the 1-pixel size; HW cursors don't, which is why a default arrow leaks
through on some Pi GPUs.
Kiosk's layout.changed events now bump displays.active_layout_id on the
server side. Display edit page and kiosk edit page render the currently-
active layout, and the "Switch Layout" dropdowns pre-select it (with
"(active)" suffix) instead of defaulting to first-in-list. Stops the
operator from accidentally re-switching to the layout already showing.
Migration is idempotent + tail-positioned so existing DBs pick up the
column without breaking PRAGMA user_version semantics.
Replacing a kiosk now sanity-checks the incoming device:
- hardware_model must match (Pi 5 swapping in for Pi 5, not Pi 3)
- managed_image flag must match (don't silently switch BYO-OS ↔ image)
- capabilities can narrow legitimately but a "lost capabilities" diff is
surfaced anyway so the operator notices.
Mismatch raises an error listing what changed; "Force replace" checkbox
on the pair form bypasses for legitimate hardware upgrades. Pending codes
panel also now renders proposed_name / hw_model / capabilities /
managed-image badge so the operator can eyeball the inbound device
before picking a replace target.
cpu_load_percent + memory/disk columns were silently added inline to the
hwmon migration entry, but PRAGMA user_version had already passed that
index for existing deploys → ALTER never ran → replaceKioskKey and
heartbeat hit "no such column: cpu_load_percent" on upgrade. Append a
tail migration that addColumnIfNotExists for each. Lesson: never mutate
an existing migration entry; always append a new one.
Kiosk's pipeline.rs needs the gtk4paintablesink element to bridge
GStreamer → GTK4 Picture. Plugin ships in Debian package
gstreamer1.0-gtk4 (separate from plugins-base/good/bad). Without it
ElementFactory::make returns Err → create_camera_pipeline returns None
→ ensure_warm returns None → every camera cell renders
"(no stream)". Add to both the pi-gen image stage and the
setup-pi-kiosk.sh installer.
ONVIF cams legitimately have multiple streams (main+sub) stored in
camera_streams. Dropping the type gate synthesized a single "main"
row for ONVIF cams that lacked rows, hiding the multi-stream design
the kiosk's pick_stream relies on (area >= 0.2 → main, else sub).
The "(no stream)" symptom is a partial-import bug to chase separately;
the bundle fallback is for single-URL RTSP cams only. Also drop the
backfill migration that did the same insert at the DB layer.
ONVIF-imported cameras with rtsp_url but no camera_streams rows showed
"(no stream)" in the kiosk because the bundle fallback was gated to
type=rtsp only. Drop the type check + backfill existing rows so old
imports get a main stream row created.
feat(kiosk-mgmt): report hostname + all network interfaces
Behind Docker/Angie the server only saw the proxy bridge IP (172.31.0.2).
Kiosk now shells `ip -j addr show`, reports every non-loopback IPv4/v6
with CIDR, MAC, and operstate. Plus `hostname` for verifying that
managed-config applies landed. Admin UI renders interface list with
LAN IPs preferred for the copy-paste local-LAN endpoint.
feat(managed-config): auto-sync hostname from kiosk name
When admin renames a managed-image kiosk, slugify the name → DNS-safe
hostname and bump managed_config_version so the kiosk applies it on
next heartbeat. Empty form hostname now falls back to slug too, so
DHCP shows the friendly name.
feat(events): forward firmware + OS update outcomes as kiosk.log
Kiosk POSTs `/api/kiosk/event` with topic=kiosk.log on firmware-apply
attempts. Server-side firmware/os-update endpoints also insert into
event_log so admins can audit upgrades without correlating per-source.
Wire schema heartbeat gains reported_hostname + network_interfaces for
Rust import parity.
Kiosks running our pre-built image (managed_image=true at pairing) can
have their hostname, timezone, network (DHCP/static + VLAN), and Wi-Fi
configured from the admin UI. Pull-model: server stores desired-state
JSON, kiosk heartbeat returns pending_config when version exceeds
applied_version, kiosk echoes applied_version back. Wi-Fi PSK encrypted
with the cluster key so ciphertext at rest is shipped to the kiosk
without per-kiosk re-encryption.
Server side only — kiosk Rust applier (betterframe-apply-config helper
+ rollback timer) and pair-initiate marker file are next.
ci(pi-gen): use action's image-path output for asset upload
pi-gen writes the .img.xz into pi-gen-action's own working dir, not our
repo deploy/. Glob never matched. Use steps.pigen.outputs.image-path
directly — no glob needed.
Pi-gen doesn't auto-copy a sub-stage's files/ dir into the chroot. The
chroot script's install commands were reaching for /tmp/bf-files/... which
never existed. Add a host-side 00-run.sh that bulk-copies files/* into
ROOTFS_DIR/tmp/bf-files, then rename the chroot script to 01-run-chroot.sh
so it sorts AFTER the host copy ('-' < '.' bites you otherwise).
usimd/pi-gen-action#179: Trixie + QEMU breaks on x86 runners (arch-test
"arm64: not supported"). Native arm64 runner means no qemu, no binfmt
registration dance — pi-gen runs the chroot directly. Faster too.
tonistiigi/binfmt registers /usr/bin/qemu-aarch64 (dynamic). Even with F-flag
preload, qemu still dlopen's its libs at exec time — fails inside pi-gen's
chroot. Debian's qemu-user-static ships /usr/bin/qemu-aarch64-static and
post-install sets F flag automatically. Pi-gen's dependencies_check needs
the static path.
Pi-gen container exits in 0.288s after image build with no logs printed.
Default action input verbose-output=false suppresses pi-gen output;
flipping to true should show what build.sh trips on inside the container.
Log diagnosis on run 26130391965:
##[error]The process '/usr/bin/sudo' failed with exit code 100
Failure was inside the action's 'Installing build dependencies on host'
step. extra-host-dependencies: qemu-user-static binfmt-support broke
apt — possibly conflicting locks or the action's input handling.
tonistiigi/binfmt --install arm64 already registered qemu-aarch64 with
'flags: POCF' (F = kernel-resident static binary). That's enough; no
need for the inside-container qemu packages.
Host-side tonistiigi/binfmt registration doesn't propagate into the
pi-gen-action's nested Docker container's view of /proc/sys/fs/binfmt_misc.
usimd/pi-gen-action's extra-host-dependencies input runs apt-get inside
the pi-gen container before pi-gen launches — install qemu-user-static
+ binfmt-support there so the chroot's arm64 binaries can execute.
apt's qemu-user-static + update-binfmts produces a registration that
pi-gen's nested Docker container still couldn't see. Switch to the
canonical tonistiigi/binfmt approach: privileged container that
installs QEMU statically with the F (fix-binary) flag, so the kernel
opens the qemu-aarch64-static binary at registration time and uses it
for all subsequent arm64 execs — independent of which container the
exec happens in.
Plus diagnostic: ls /proc/sys/fs/binfmt_misc + cat qemu-aarch64
detail, so next run's log surfaces whether registration actually
landed.
docker/setup-qemu-action registers binfmt via a privileged side container;
pi-gen-action's own nested Docker container doesn't inherit the
registration. Result: arm64 ELFs in the pi-gen chroot still fail to
exec, exit 1 before any stage runs.
apt-installed qemu-user-static + binfmt-support writes persistent
binfmt_misc entries to the kernel that propagate to every container
share. Pair with update-binfmts --enable qemu-aarch64 and a sanity
ls -la /proc/sys/fs/binfmt_misc/qemu-aarch64.
Real cause of last pi-gen failure was surfaced by verbose-output:
WARNING: Only a native build environment is supported.
arm64: not supported on this machine/kernel
ubuntu-latest is x86_64; pi-gen builds an arm64 image and chroots into
it during stages, requiring binfmt_misc handlers for arm64. Add
docker/setup-qemu-action before the pi-gen step.
While here, audit + bump every action version (pinned to current
majors):
actions/checkout v4 → v6
actions/upload-artifact v4 → v7
actions/download-artifact v4 → v8
softprops/action-gh-release v2 → v3
docker/setup-qemu-action @v4 (new)
usimd/pi-gen-action @v1 (already current major)
dtolnay/rust-toolchain @stable (rolling channel — keep)
Reverts misdiagnosis. pi-gen defaults to trixie since the Debian 13
release, which has gtk4 4.14 + libwebkitgtk-6.0 stock — no backports
needed. Build container, kiosk gtk feature gate, and pi-gen target all
realigned to trixie.
Actual reason last image run failed: our custom stage was missing the
mandatory prerun.sh (pi-gen calls it to seed ROOTFS_DIR from the
previous stage) and the EXPORT_IMAGE marker file (signals 'bake an
image at the end of this stage'). Both added.
Asset upload now globs deploy/*.img.xz so any extra exports stage2
produces ship alongside our customised one.
Repo is public → unlimited Actions minutes, so the 30-60 min pi-gen
bake doesn't have a cost gate. Master pushes now produce the full
asset set (binaries + image), same as tag releases.
Replaces release-kiosk.yml + release-image.yml with two coupled workflows:
release.yml — entrypoint. Computes version/channel/tag:
- master push → semver patch bump from latest stable tag, append
-dev.<shortsha>, create lightweight tag + prerelease record
- v* tag push → use tag verbatim, channel from suffix (-beta./-dev. or
stable), create release if missing
Then invokes build.yml via uses: ./.github/workflows/build.yml.
build.yml — reusable (workflow_call). Single source of truth for asset
production:
- kiosk binary matrix (aarch64, x86_64) in debian:trixie-slim
- flashable .img.xz via pi-gen using the aarch64 artifact (gated by
build-image input; master pushes default false to keep dev cycles
fast, tag pushes default true for a full release)
Both jobs attach to the release at tag_name=${{ inputs.tag }}.
Concurrency: master-branch runs cancel superseded peers; tag runs never
cancel. CI auto-import to a running BF server (BF_AUTOIMPORT_URL +
BF_AUTOIMPORT_API_KEY repo secrets) still wired.
New workflow .github/workflows/release-image.yml takes a tagged kiosk
release binary, layers it onto Raspberry Pi OS Trixie Lite via a custom
pi-gen stage, and publishes the resulting .img.xz back to the GitHub
Release.
Custom stage deploy/pi-gen/stage-betterframe-client/:
- 00-install-packages: cage, seatd, plymouth, gtk4 runtime, gstreamer,
libwebkitgtk-6.0, wlr-randr, ca-certificates
- 01-install-kiosk: drops the prebuilt kiosk binary, systemd unit,
cage PAM stack, firmware-rollback hook, plymouth theme. Creates
bfkiosk user, sets multi-user.target, masks all display managers,
purges piwiz, edits cmdline/config for the BF splash. Mirrors
setup-pi-kiosk.sh but baked into the image.
End state: rpi-imager → SD → boot → pairing screen on the HDMI display,
no operator setup steps. Kiosk auto-discovers server via discover_server()
(localhost → mDNS → frame-eu.betterportal.net).
Heavy build (~30-60 min on GH-hosted Ubuntu) so tag-push triggered, not
master. Workflow_dispatch also supports baking an existing release tag's
binary into a fresh image without re-tagging.
