Advanced Metal Research
GitHub Contact AMR

Dense trajectory daemon

On this page
  1. Quick start
  2. Command line
  3. Config file
  4. TCP ingest
  5. upload
  6. validate
  7. preload
  8. status
  9. NATS control plane
  10. Leader lease
  11. Play a program
  12. Other commands
  13. Status document
  14. consumer_action
  15. Refusal codes
  16. Related pages

joint_trajectory_daemon stores immutable .rdt programs and plays them on the robot through rt-control. It has two planes that never overlap:

  • Data plane: TCP, on 127.0.0.1:8797 by default. Upload, validate, preload and status. Nothing on this socket can start motion.
  • Control plane: NATS, on the cell's robot command subject. A controller acquires the daemon's leader lease, then sends play with the exact plan identity. stop is always accepted.

The daemon uses rt_core as its only backend. Retired backends and options exit with backend_retired or option_retired.

Warning

Energised motion. This moves the robot. RosieOS has no software E-stop: keep the hardware E-stop within reach and clear the cell before you arm. Jog, moves and Home are checked against joint limits only, not against collisions. See the safety model.

Note

Programs played through this daemon are not re-verified. The daemon validates the .rdt format, and rt-control checks native position, velocity and continuity limits. Neither checks collisions or the plan's robot and cell identity. Only OLP's Load path requires a weld planner verifier PASS. See What is verified before motion. Also, play runs native Home on any axis whose Home is not valid.

Quick start#

The dev stack runs the daemon against the simulated core like this:

joint_trajectory_daemon --backend rt_core \
  --config "$ROSIE_LOCAL_RT_BUILD/daemon-rt-core.json" \
  --dense-ingest-listen 127.0.0.1:8797 \
  --plan-store-dir "$ROSIE_LOCAL_RT_BUILD/dense-plans" \
  --nats-url nats://127.0.0.1:14222 \
  --robot-cell dev-cell \
  --robot-command-subject robot/v4/robot.dev-cell.command \
  --status-publish-subject robot/v4/robot.dev-cell.status

Build it with make -C motion-server/joint-trajectory/v1 all. The binary goes to rt-core/build/dense (make ... print-bin-dir prints the path).

Then upload a program from Python. The weld planner package has a client for the TCP framing:

upload.pyPython
from weldplan.dense_joint_trajectory import upload, request

blob = open("hold.rdt", "rb").read()
print(upload("127.0.0.1", 8797, blob))
# {'ok': True, 'kind': 'upload', 'trajectory_digest': 'sha256:…',
#  'segment_count': 1, 'total_sample_count': 3, 'total_duration_s': 0.02}
print(request("127.0.0.1", 8797, "status"))

To play it, a NATS client acquires the leader lease, preloads over TCP, and sends play. The sequence is in Play a program.

Command line#

Settings come from the JSON config file first. Command-line flags override it.

FlagDefaultDescription
--config PATHenv JOINT_TRAJECTORY_DAEMON_CONFIGThe config file. Without one, compiled defaults apply and the native binding is empty, so startup fails with native_binding_required.
--backend rt_corert_core (env JOINT_TRAJECTORY_BACKEND)Anything else exits with backend_retired
--dense-ingest-listen HOST:PORT127.0.0.1:8797TCP ingest address. The host must be an IPv4 literal.
--plan-store-dir DIR$HOME/.rosie/motion-server/joint-trajectory/v1/dense-plansWhere uploaded blobs are stored, as <digest-hex>.rdt
--ready-file PATHnoneWritten after the listener binds: {"schema":"robot-v4.dense-joint-trajectory-ingest-ready.v1","host":…,"port":…}
--nats-url nats://HOST:PORTnoneWithout it the daemon is ingest-only and nothing can deliver a play
--robot-cell NAMEnoneCommands whose robot field differs are ignored
--robot-command-subject SUBJnoneThe NATS subject the daemon subscribes to for commands
--status-publish-subject SUBJnoneStreams the full status document at status.publish_hz. Off when unset.
--helpPrint usage

--joint-target and --rtcore-status-subject are retired and exit with option_retired. Exit code 2 means a configuration or usage error.

Config file#

The canonical, annotated copy is motion-server/joint-trajectory/v1/config/joint_trajectory_daemon.config.json. Every REPLACE_… value must be replaced with the cell's approved binding before the daemon will start.

joint_trajectory_daemon.config.jsonJSON
{
  "schema": "robot.v4.joint-trajectory-daemon.config.v1",
  "ingest": { "listen": "127.0.0.1:8797" },
  "status": { "publish_hz": 20.0 },
  "leader": { "ttl_min_ms": 500, "ttl_max_ms": 2000 },
  "backend": "rt_core",
  "rt_core": {
    "socket": "/run/rosie-rt-core/control.sock",
    "controller": "motion-server",
    "pair_id": "REPLACE_WITH_APPROVED_PAIR",
    "pair_revision": 1,
    "machine_sha256": "REPLACE_WITH_APPROVED_MACHINE_SHA256",
    "deployment_sha256": "REPLACE_WITH_APPROVED_DEPLOYMENT_SHA256",
    "configuration_sha256": "REPLACE_WITH_APPROVED_CONFIGURATION_SHA256",
    "expected_backend": "simulation",
    "home_policy": "home",
    "status_subject": "motion-server.status",
    "control_timeout_ms": 10000
  }
}
KeyTypeDefaultDescription
schemastringrequiredrobot.v4.joint-trajectory-daemon.config.v1
backendstringrequiredrt_core
ingest.listenstring127.0.0.1:8797TCP ingest HOST:PORT
ingest.plan_store_dirstringsee --plan-store-dirBlob store directory
status.publish_hznumber, Hz20Rate of the --status-publish-subject stream
leader.ttl_min_msinteger, ms500Shortest leader lease the daemon grants. Requested TTLs are clamped to this range.
leader.ttl_max_msinteger, ms2000Longest leader lease
rt_core.socketstring/run/rosie-rt-core/control.sockThe rt-control Unix socket
rt_core.controllerstringmotion-serverThe controller name the daemon acquires rt-control with
rt_core.pair_idstringrequiredThe pair binding, as rt-control was started with
rt_core.pair_revisionintegerrequired, > 0The pair revision
rt_core.machine_sha25664 hexrequiredMust equal Describe's machine_sha256
rt_core.deployment_sha25664 hexrequiredMust equal Describe's deployment_sha256
rt_core.configuration_sha25664 hexrequiredMust equal Describe's configuration_sha256
rt_core.expected_backendstringsimulationMust equal Describe's backend: simulation or ethercat
rt_core.home_policystringhomeWhat play does for an axis without valid Home: home runs native Home, restore_anchor restores the saved anchor
rt_core.status_subjectstring<robot-command-subject>.statusSubject for the executor status, published every 200 ms
rt_core.control_timeout_msinteger, ms10000Receipt and observation budget for each rt-control call

Integers must be positive and strings non-empty. A missing file that was asked for, a wrong schema or a malformed value is a startup error; the daemon never falls back to compiled defaults.

On leader_acquire the daemon also checks that rt-control implements describe, acquire, renew, enable, arm, home, restore_anchor, recovery_status, prepare_program, start_program, status, subscribe_events, stop and release, and that Describe lists exactly nine axes J1…J9 in rad. A six-axis cell is refused with native_axis_map_invalid.

TCP ingest#

One request and one response per connection, then the daemon closes it. Each exchange must finish within 240 s.

frame   = [u64 BE payload_size][payload]            payload_size ≤ 64 MiB
payload = [u32 BE json_len][control JSON][optional binary body]

The response uses the same framing, with a JSON body and no binary part. A rejection is:

{"ok": false, "reason": "block_sha256_mismatch", "segment_index": 0, "detail": "stored sha256:… computed sha256:…"}

segment_index is present only when one segment is at fault.

upload#

Validates the binary body as a .rdt and stores it. Upload never causes motion.

request control JSONJSON
{"kind": "upload"}
responseJSON
{"ok": true, "kind": "upload", "trajectory_digest": "sha256:…", "segment_count": 1, "total_sample_count": 3, "total_duration_s": 0.02}

Refusals: any .rdt validation reason (the daemon reports a wrong schema as schema_mismatch), and store_write_failed.

validate#

The same checks as upload, without storing. Answers "kind": "validate".

preload#

Stages a stored blob by reference and prepares it on rt-control (prepare_program). It needs the leader lease to be held, because preparation happens under the daemon's rt-control grant.

request control JSONJSON
{
  "kind": "preload",
  "trajectory_digest": "sha256:…",
  "plan_id": "demo:1",
  "program_id": "demo",
  "program_digest": "sha256:…",
  "manifest_revision": 1,
  "plan_revision": 1
}
FieldTypeRequiredDescription
trajectory_digeststringyessha256:<64 hex> of a stored blob
plan_id, program_id, program_digeststringyesMust equal the stored header exactly
manifest_revision, plan_revisionintegeryesMust equal the stored header. JSON integers, not strings.

The daemon re-verifies the stored bytes in full, so a blob corrupted on disk fails here rather than at play.

responseJSON
{"ok": true, "kind": "preload", "trajectory_digest": "sha256:…", "total_sample_count": 3}
ReasonMeaning
request_invalidMissing trajectory_digest, bad framing or unknown kind
plan_not_foundNo stored blob with that digest
identity_mismatchA named identity field differs from the stored header
native_acquisition_requiredNo rt-control grant: acquire the leader lease first
leader_lease_expiredThe leader lease is not current
native_commissioning_in_progressA play is still homing, enabling or arming
native_playback_in_progressA program is playing. Stop first.
native_torch_unsupportedThe program has torch samples. Remove them.
native_program_identity_mismatchrt-control prepared a program whose identity, axis mask or sample counts differ
any rt-control reasonFor example native_limit_exceeded. See Error codes.

A refused native_limit_exceeded leaves any previously prepared program in place. Other native refusals stop the executor and release the grant.

status#

request control JSONJSON
{"kind": "status"}

Returns the status document.

NATS control plane#

Commands are JSON messages on --robot-command-subject. Send them as NATS requests: the daemon replies on the message's reply subject. It ignores messages with a different robot, a missing envelope field or a command it does not own.

Every command carries this envelope:

FieldTypeRequiredDescription
schemastringyesrobot.v4.robot-command.v1
commandstringyesleader_acquire, leader_renew, leader_release, play, pause, stop or go_home
robotstringyesMust equal --robot-cell
command_idstringyesUnique per command. A resend with the same ID gets the same reply without running again, except stop, which always runs. The daemon remembers the last 256 IDs.
sender_idstringyesoffline-programming:<instance>, where the instance is 1–31 characters from A-Za-z0-9-. No other role may hold the lease.

Replies have this shape:

{"schema": "robot.v4.command-reply.v1", "command": "play", "command_id": "c-42", "ok": false, "reason": "fresh_exact_leader_lease_required"}

Leader lease#

The daemon mints the lease itself. A client never supplies an epoch or lease ID to leader_acquire.

leader_acquireJSON
{"schema": "robot.v4.robot-command.v1", "command": "leader_acquire", "robot": "dev-cell",
 "command_id": "c-1", "sender_id": "offline-programming:bench-1",
 "controller_boot_id": "boot-7f3a", "ttl_ms": 1000}
replyJSON
{"schema": "robot.v4.command-reply.v1", "command": "leader_acquire", "command_id": "c-1", "ok": true,
 "lease_id": "motion-server-…-0000000000000001", "leader_fence_epoch": 1, "expires_in_ms": 1000}
CommandExtra fieldsEffect
leader_acquirecontroller_boot_id, ttl_msStops any motion first, then grants a new lease with a new fence epoch. The daemon then acquires rt-control under its configured binding and checks the deployment identity. If that fails, no lease is granted and the reply carries the native reason.
leader_renewcontroller_boot_id, lease_id, leader_fence_epoch, ttl_msExtends the lease. Refused with lease_not_held unless every field matches the current holder.
leader_releasecontroller_boot_id, lease_id, leader_fence_epochStops motion and ends the lease. The daemon then stops and releases its rt-control grant.

ttl_ms is clamped to leader.ttl_min_ms…leader.ttl_max_ms. Renew well inside the TTL. If the lease expires, the executor aborts any playback with leader_lease_expired, then stops and releases its rt-control grant. Note the field names: lease commands use lease_id, motion commands use leader_lease_id.

Lease refusals: leader_identity_invalid, ttl_ms_required, client_fence_not_accepted (an acquire that carried a lease ID or epoch), lease_not_held.

Play a program#

  1. leader_acquire on NATS. Keep renewing.
  2. upload the blob over TCP, then preload it with its identity.
  3. play on NATS:
playJSON
{
  "schema": "robot.v4.robot-command.v1", "command": "play", "robot": "dev-cell",
  "command_id": "c-3", "sender_id": "offline-programming:bench-1",
  "controller_boot_id": "boot-7f3a",
  "leader_lease_id": "motion-server-…-0000000000000001", "leader_fence_epoch": 1,
  "motion_intent_seq": 1,
  "plan_id": "demo:1", "program_id": "demo", "program_digest": "sha256:…",
  "trajectory_digest": "sha256:…", "manifest_revision": 1, "plan_revision": 1
}
FieldTypeDescription
controller_boot_id, leader_lease_id, leader_fence_epochstring, string, integerThe current lease, exactly
motion_intent_seqintegerGreater than zero and strictly greater than the last one this sender used, so a delayed duplicate can never run
plan_id, program_id, program_digest, trajectory_digeststringMust equal the preloaded plan
manifest_revision, plan_revisionintegerMust equal the preloaded plan

A play the daemon accepts runs these phases, reported in commissioning_phase:

  1. preparation: status and the prepared identity are read back from rt-control.
  2. home or restore_anchor: only for axes whose Home is not valid, as home_policy says. Refused with native_home_unavailable if an axis has no native Home.
  3. enable_arm: recovery status must show no faults, then enable and arm.
  4. readiness: waits until every program axis reports ready.
  5. start: start_program with the full identity.

The reply comes when start_program has been accepted, or when a phase fails. Progress is then visible in the status document.

Other commands#

CommandFencedEffect
stopNoAlways accepted and always executed, even on a resend. Aborts playback, then stops and releases the rt-control grant.
pauseYesRefused with pause_unsupported; use stop
go_homeYesRefused with go_home_unsupported on the rt_core backend

Fenced commands are refused with fresh_exact_leader_lease_required, ordered_motion_intent_sequence_required, stale_motion_intent_sequence or exact_staged_plan_identity_required before the executor sees them. play, leader_acquire and leader_release run on a queue; when 256 are already waiting, a new one is refused with command_queue_full.

Status document#

The TCP status reply, the --status-publish-subject stream and the rt_core.status_subject stream carry the same document.

{
  "ok": true, "kind": "status",
  "schema": "robot.v4.dense-joint-trajectory.v1",
  "store_dir": "/srv/rosie/dense-plans",
  "staged": {"trajectory_digest": "sha256:…", "plan_id": "demo:1", "total_sample_count": 3},
  "executor": {
    "state": "playing", "backend": "rt_core", "native_state": "owned",
    "execution_state": "executing", "commissioning_phase": "",
    "execution_generation": 4, "native_sequence": 118,
    "segment_index": 0, "sample_index": 1, "t_s": 0.01,
    "detail": "", "consumer_action": "", "native_result": null,
    "program_identity": {"plan_id": "demo:1", "…": "…"},
    "authority_fresh": true, "leader_fresh": true
  },
  "last_error": null
}

--status-publish-subject carries this whole document at status.publish_hz. rt_core.status_subject carries only the executor object, every 200 ms.

FieldDescription
stagedThe preloaded plan, or null
last_errorThe last ingest refusal as {reason, detail}, or null
executor.stateidle, staged, playing, done or aborted
executor.native_stateThe rt-control grant: inactive, acquiring, owned, reconciling, cleanup_uncertain or grant_active
executor.execution_statert-control's execution state, for example prepared, executing, completed
executor.commissioning_phaseThe current play phase, or empty
executor.segment_index, sample_index, t_sThe playback cursor on the program's own segment clock
executor.execution_generation, native_sequenceThe native execution identity of the running program
executor.program_identityThe prepared identity, including source_digest and normalised_digest
executor.abort_reason, detailWhy the executor last stopped or refused
executor.native_resultThe native refusal result, when there was one
executor.consumer_actionWhat the client should do next (see below)
executor.authority_freshThe grant is owned under the current leader epoch and the lease is fresh
executor.leader_freshA leader lease is active and unexpired

The executor document also carries compatibility fields from an earlier backend: armed, torch_on, feedback_age_ms, tracking_error_rad, tracking_error_axis, transient, and the authority and feedback objects. The rt_core executor does not fill them; they keep their defaults (armed is false, feedback_age_ms is -1). Read native_state, execution_state and rt-control's own status instead.

consumer_action#

ValueMeaning
acquire_and_preloadIdle and clean. Acquire the leader lease and preload.
remove_torch_samplesThe program has torch samples
use_stoppause is not supported
correct_requestThe request was refused but nothing was stopped. Fix it and retry.
observe_standstill_before_acquireCleanup is in progress
reconcile_status_then_acquireStop or Release could not be confirmed. Check rt-control status before acquiring again.
stop_reconcile_reacquire_uploadThe rt-control session was fenced or expired
invalidate_describe_bind_reacquire_uploadrt-control or the core restarted
abort_stop_reconcileAny other native failure

After an abort the daemon stops the grant, releases it, and waits for two consecutive, newer status samples that show no grant, the axes disarmed and disabled, and unchanged positions before it reports inactive.

Refusal codes#

Beyond the ingest and lease reasons above, abort_reason and play replies can carry:

ReasonMeaning
native_binding_requiredStartup: the config does not name a complete binding
native_session_already_presentleader_acquire while the daemon still holds a grant
native_contract_mismatchrt-control's contract version or capabilities digest differs from the client's
native_deployment_identity_mismatchDescribe's machine, deployment or configuration digest, or backend, differs from the config
native_capability_unavailableA required capability is not implemented
native_axis_map_invalidDescribe does not list exactly nine rad axes J1…J9 with valid limits
invalid_home_policyhome_policy is not home or restore_anchor
native_binding_mismatch, native_grant_mismatchThe grant returned by rt-control differs from the binding
native_grant_activeAnother controller holds rt-control
native_preparation_requiredplay before a successful preload
native_preparation_retiredThe prepared program is no longer prepared on rt-control
native_home_unavailable, native_home_unconfirmedHome could not run, or could not be confirmed
native_fault_observedA safety fault or recovery fault is present
native_readiness_<reason>An axis reported faulted, coordinate_invalid, home_required or mode_mismatch, or another non-ready state
readiness_observation_expiredReadiness did not arrive within control_timeout_ms
native_status_unavailable, native_status_staleNo status, or a core sample older than 200 ms
native_incarnation_changedrt-control or the core restarted
native_execution_aborted, execution_identity_mismatchThe running program faulted, was cancelled or changed identity
native_operation_cancelledStop or a lease change cancelled the operation
native_standstill_unconfirmed, native_shutdown_expired, release_uncertainCleanup could not be confirmed
leader_revoked, operator_stop, daemon_shutdownWhy playback was aborted

rt-control's own reasons pass through unchanged. See Error codes.