# Use the virtual pendant

> Run the browser Steam Deck pendant against the simulated core, take control, arm, home and jog joints, and know what it cannot do.

URL: https://advancedmetalresearch.com/docs/guides/virtual-pendant
Section: RosieOS docs / Guides
Last updated: 2026-10-10

The virtual pendant is a browser copy of the Steam Deck v4 pendant's screen, drawn at the Deck's native 1280×800. A small loopback bridge connects it to the simulated core. Use it to try the pendant workflow, or to work on its layout, without a Deck or a robot.

It is simulation-only by construction:

- The bridge binds only to a loopback address.
- It checks that the backend it drives is the simulation, with the expected configuration digest and pair.
- The browser never sees a control socket or a credential.

## Start it

Start the simulated core, `rt-control` and the bridge with the dev stack. Then start the browser UI:

```bash
export DEV_STACK_UI_HOST=127.0.0.1 DEV_STACK_FIREWALL=0
./dev-stack.sh start rt-sim rt-control pendant
cd steamdeck/virtual/v1/ui
npm ci
npm run dev
```

Open **http://127.0.0.1:51711/steamdeck/virtual/v1/**. Vite serves the page on port 51711. It proxies `/healthz` and `/api/v1` (including the WebSocket session) to the bridge on `127.0.0.1:51712`.

The pendant's 3D view loads the Rosie 1400 model from `urdf/v1/ui/public/robot/rosie_1400_v3`.

## Drive the simulated robot

On the rt-core backend, the pendant does joint jog only. The usual sequence:

1. **Take the leader.** The bridge acquires the rt-control lease for your browser session and renews it every 100 ms. A second browser gets `control_already_owned`.
2. **Home.** This sends a public `home` for all axes. The dev-stack simulation requires Home before anything moves.
3. **Arm.** This sends `enable` for all axes, then `arm`.
4. **Jog.** Select a joint, J1 to J9, and push the left stick up or down, or the right stick sideways. Holding one trigger scales the speed to 50%, both triggers to 10%.
5. **Disarm or Stop.** Either one sends a Stop and releases the lease. **Release the leader** when you are done.

Jog speed is capped at 10% of the joint's maximum velocity from Describe, times the stick deflection and the trigger scale. The bridge refuses any input sample older than 150 ms. If samples stop arriving, the core's jog deadline ramps the joint to a hold.

**Clear Fault** maps to `reset_fault`, which is an interim capability in rt-core.

## Keyboard and mouse

| Key | Deck control |
|---|---|
| `[` / `]`, or Page Up / Page Down | LB / RB |
| Arrow keys | D-pad |
| A (or Space, Enter), B (or Esc), X, Y | Face buttons |
| Tab | Select (View) |
| M | Menu |
| Shift (hold) | LT |
| Ctrl (hold) | RT |
| 1 / 2 | Latch LT / RT on or off |
| G | Toggle the layout grid |

Drag the on-screen sticks with the mouse, or plug in a gamepad. The browser Gamepad API maps its sticks, face buttons, bumpers and D-pad.

## What it cannot do

On the rt-core backend, these return a typed refusal instead of acting:

| Feature | Refusal code |
|---|---|
| Cartesian or TCP jog | `rt_core_cartesian_unavailable` |
| Program list, program editing, planning | `rt_core_planning_unavailable` |
| Reading or applying drive configuration | `rt_core_configuration_unavailable` |
| Log sessions | `rt_core_logs_unavailable` |

Use the [offline programming app](https://advancedmetalresearch.com/docs/get-started/quickstart) for Cartesian jog and programs. Use `rtctl` and the events and telemetry streams for logs and configuration.

## Run the bridge by hand

The dev-stack `pendant` service builds the bridge from `steamdeck/virtual` and starts it with the simulator's binding. To do the same yourself:

```bash
(cd steamdeck/virtual && go build -o ../../rt-core/build/virtual-deck ./cmd/local)
source rt-core/build/dev-stack/binding.env
rt-core/build/virtual-deck bridge start --adapter robot-v4-sim --backend rt_core \
  --rt-core-socket "$ROSIE_RT_CONTROL_SOCKET" --rt-core-pair "$ROSIE_RT_PAIR_ID" \
  --rt-core-revision "$ROSIE_RT_PAIR_REVISION" --rt-core-sha256 "$ROSIE_RT_CONFIGURATION_SHA256"
```

`bridge start` flags:

| Flag | Default | Description |
|---|---|---|
| `--adapter` | `fixture` | `robot-v4-sim` drives the simulated core. `fixture` serves a deterministic, command-disabled fixture set. |
| `--backend` | `rt_core` | The only supported backend. Any other value is refused as `backend_retired`. |
| `--rt-core-socket` | — | The public `control.sock`. Required for `robot-v4-sim`. |
| `--rt-core-pair`, `--rt-core-revision` | — | The pair binding. Required for `robot-v4-sim`. |
| `--rt-core-sha256` | — | The configuration digest, 64 hex. Required for `robot-v4-sim`. |
| `--fixture` | `v1-complete` | Fixture set for the `fixture` adapter |
| `--host` | `127.0.0.1` | Bind host. Must be a loopback address. |
| `--port` | `51712` | Bind port |
| `--dry-run` | off | Validate the flags and print the address without starting |

`bridge status [--host] [--port]` probes a running bridge's `/healthz`. `bridge contract` prints the bridge's protocol contract as JSON. The rt-core adapter needs Linux or WSL. On other platforms it fails with `rt_core_platform_unavailable`.

To point the UI at a bridge on another port, set `STEAMDECK_VIRTUAL_BRIDGE_URL` before `npm run dev`, for example `http://127.0.0.1:51713`. Start the bridge on that port with `DEV_STACK_PENDANT_PORT=51713`.

## Sources

Written from these files in the RosieOS repository (https://github.com/advanced-metal-research/RosieOS):

- `steamdeck/virtual/bridge/cli.go:20-45,99-150`
- `steamdeck/virtual/bridge/backend.go:25-27`
- `steamdeck/virtual/bridge/server.go:119-128,731-739`
- `steamdeck/virtual/bridge/rt_core.go:20-32,108,133-340,378-404,465-523`
- `steamdeck/virtual/bridge/rt_core_unsupported.go:1-9`
- `steamdeck/virtual/cmd/local/main.go`
- `steamdeck/virtual/v1/ui/vite.config.ts:10-20,108-131`
- `steamdeck/virtual/v1/ui/package.json:6-10`
- `steamdeck/virtual/v1/ui/src/input.ts:69-160,800-851`
- `steamdeck/virtual/v1/ui/src/state.ts:651-653`
- `steamdeck/virtual/v1/ui/src/sticks.ts:1-90`
- `dev-stack.sh:38-40,262-264,356-360`