testbed-qemu: prplOS in QEMU, in CI and on your desk

testbed-qemu · prplOS CI developer session

prplOS in QEMU, in CI and on your desk

CI capacity

Too little CI capacity for runtime tests

One beefy server runs 56 QEMU DUTs, so 56 CI jobs run in parallel

From testing platform to development platform

The test lab is hard to rebuild at your desk, or for your agent

The same environment in CI and on your desk

ec11a42 · 2026-09-23

The idea

One stack, two places it runs

GitLab CI job

differs

you · or your agent + makeclaudecodexagy

cram tests

same

cram tests

testbed-qemu CLI wait · ssh · cram

testbed-qemu container image

QEMU + prplOS firmware image

56 DUTs /dev/kvm

differs

1 DUT /dev/kvm

same entry point

The idea · detail

Same entry point

CI · .gitlab/testbed/qemu-x86-64.yml

image: …/testbed-qemu@sha256:…
script:
  - testbed-qemu run
      --image "$TESTBED_QEMU_IMAGE"
      --artifacts artifacts
      --accel kvm
      --debug-information always
      -- testbed-qemu cram $CRAM_TEST_SUITE

Desk · make test-qemu

docker compose run testbed-qemu \
  testbed-qemu run \
    --image /firmware/prplos.img \
    --artifacts /artifacts \
    --accel auto \
    -- testbed-qemu cram $(TESTS)

Both resolve the test list with resolve-component-tests.py from the prplOS tree.

How it works

A router and its test host in one container

capabilities

How it works · detail

What the container gets

gets

NET_ADMIN NET_RAW /dev/net/tun /dev/kvm

For the private LAN and WAN, and for KVM when the host has it.

never gets

--privileged host network new privileges

The firmware image is mounted read-only. Batch runs start factory-fresh.

Quick start

Three commands to a DUT shell

on the host

git clone https://gitlab.com/prpl-foundation/\
  prplos/ci/containers/testbed-qemu.git
cd testbed-qemu
make fetch test-qemu-shell

inside the shell

testbed-qemu wait && testbed-qemu ssh

back on the host

make test-qemu TESTS=connectivity

Needs Docker Compose, /dev/kvm, git and make. README

shell

Quick start · detail

A shell next to the DUT

⏱

testbed-qemu wait   # until SSH answers

testbed-qemu ssh    # root@192.168.1.1

testbed-qemu console # ttyS0, Ctrl-] exits

The prompt comes back at once. The DUT boots in the background.

in CI

Quick start · detail

The same run, in CI

✓ passede2e-testbed-qemu-kvm

55⌄📦 Test Execution [network,busybox,connectivity,reboot,pvpanic]

56$ PRPLOS_SOURCE_ROOT="$CI_PROJECT_DIR/.prplos-release" testbed-qemu run --image "$CI_PROJECT_DIR/prplos.img" …

57[+ 0.1s] run | accel=kvm cpu=host ovmf=/usr/share/OVMF/OVMF_CODE_4M.fd

58[+ 0.3s] run | lab up: br-lan 192.168.1.2/24, br-wan 10.0.0.1/24

59[+ 0.3s] run | qemu starting (pid=49)

60[+ 6.4s] run | lan ready after 6.1s

61[+ 13.4s] run | waiting for ssh (10s/300s)

62[+ 15.5s] run | ssh ready after 15.1s

63[+ 20.7s] run | waiting for datamodel (5s/120s)

64[+ 31.0s] run | waiting for datamodel (16s/120s)

65[+ 32.0s] run | datamodel ready after 31.7s (6090 lines)

66[+ 32.0s] run | running: testbed-qemu cram network busybox connectivity reboot

67[+ 32.1s] cmd | cram: running 4 tests (list in cram.txt)

68[+ 34.2s] cmd | .gitlab/tests/cram/generic/network.t: passed

69[+ 34.3s] cmd | .gitlab/tests/cram/generic/busybox.t: passed

70[+ 35.1s] cmd | .gitlab/tests/cram/testbed-qemu/connectivity.t: passed

71[+ 52.7s] cmd | .gitlab/tests/cram/testbed-qemu/reboot.t: passed

72[+ 52.7s] cmd | # Ran 4 tests, 0 skipped, 0 failed.

73[+ 52.7s] run | command exited rc=0 after 20.7s

74[+ 53.2s] run | cleanup done, artifacts in /builds/prpl-foundation/prplos/ci/containers/testbed-qemu/artifacts

Verbatim excerpt, lines 55–74; the long command line is cut at … Verify: job 16677383915 ↗

Make targets

Four targets, two variables

Docker Compose

rootless Podman

batch

make test-qemu

make test-qemu-podman

shell

make test-qemu-shell

make test-qemu-shell-podman

PRPLOS_DIR your prplOS checkout

PRPLOS_IMAGE the firmware .img

Your first test

Write it, break it, run the smoke bucket

1 · write

# generic/hello-world.t
Hello world smoke test:
  $ echo hello world
  hello world
  $ alias R="${CRAM_REMOTE_COMMAND:-}"
  $ R "uname -m"
  x86_64

$ testbed-qemu cram hello-world
passed

2 · break

$ sed -i 's/^  hello world$/  hello wrld/' \
    …/generic/hello-world.t
$ testbed-qemu cram hello-world

   $ echo hello world
-  hello wrld
+  hello world
# Ran 1 tests, 0 skipped, 1 failed.

3 · run smoke

$ make test-qemu \
    TEST_COMPONENT=smoke

cram: running 6 tests
generic/busybox.t: passed
generic/network.t: passed
  …
qemu-x86-64/reboot.t: passed
# Ran 6 tests, 0 skipped, 0 failed.

One test as a batch run: make test-qemu TESTS=hello-world

buckets

Your first test · detail

Buckets, from components.yml

full: every test for the board

sanityfirst in every bucket

lcminclude list

prplmeshinclude list

prplosthe rest

smoke6 named tests

post/runs after every bucket

hw_onlyskipped on QEMU

Unions work: TEST_COMPONENT=smoke,prplmesh. A new test under generic/ lands in prplos and full.

Red QEMU job in your MR

Reproduce it at your desk

Check out the pipeline's commit

›

Fetch the .img from its build job

›

Run only that testmake test-qemu TESTS=<test>

›

Rerun in the shell, then ssh and consoletestbed-qemu cram <test>

›

Fix, rerun, push

Your agent runs the tests for you.

AGENTS.md + skills/testbed-qemu/SKILL.md

claudecodexagy

> run connectivity test in qemu and provide me result

artifacts

Red QEMU job · detail

Artifacts and exit codes

run.logrun timeline

serial.logfull DUT console

cram.txt · cram.xmlresults, xUnit

*.t.erractual output

command.logpayload output

manifest.jsoninputs and result

debug-information-*.txtDUT diagnostics

coredumps-*.tar.gzwhen present

rcpayload status

1runtime failure

2bad argument or config

124readiness timeout

130Ctrl-C

On demand in the shell: testbed-qemu debug-information

Where it stands

The merge request chain

snapshot 2026-09-23

draft open merged · click a pill to open the MR

Scope

What it is, what it isn't, what's next

It is

The real prplOS x86-64 image
Full userspace: datamodel, USP, LCM, firewall, DHCP, tunnels
A fresh LAN and WAN on every run
Fast
The same container as CI
Runs on any Linux laptop
Wi-Fi through mac80211_hwsim is there, but untested

It isn't

Switch, SoC acceleration, cellular or xPON hardware (the hw_only tests)
A replacement for the hardware boards
A performance testbed

Wi-Fi tests on hwsim
Make the QEMU jobs blocking
More QEMU coverage (CI-84)
Other targets and architectures (armv8/arm64)
Firmware upgrade testing
Longevity, soak and stress testing