pty design + probe test

This commit is contained in:
lda
2026-07-15 06:26:17 +07:00 Verified
parent 8a93250781
commit 4ecef8da38
8 changed files with 278 additions and 5 deletions
+18
View File
@@ -60,6 +60,18 @@ _Avoid_: Device when referring to the agent machine
Something an agent host can observe or do on the network. Something an agent host can observe or do on the network.
_Avoid_: Router flag _Avoid_: Router flag
**Terminal Capability**:
An explicitly enabled agent capability that allows the agent host to create remote PTY sessions.
_Avoid_: Command runner, implicit root access
**Terminal Session**:
One operator attachment to one PTY process hosted by an agent.
_Avoid_: Command, SSH session, stored shell
**Terminal Relay**:
The control plane's in-memory pairing of one operator terminal socket with one outbound agent terminal socket.
_Avoid_: Terminal session storage, terminal emulator
**Interface Telemetry**: **Interface Telemetry**:
Current operational measurements for a network interface visible to an agent. Current operational measurements for a network interface visible to an agent.
_Avoid_: Endpoint state, device presence _Avoid_: Endpoint state, device presence
@@ -84,6 +96,9 @@ _Avoid_: Endpoint state, device presence
- An **Identifier** can link observed **Devices** or **Endpoints** to one **Known Device**. - An **Identifier** can link observed **Devices** or **Endpoints** to one **Known Device**.
- An **Agent** runs on one **Agent Host**. - An **Agent** runs on one **Agent Host**.
- An **Agent Capability** describes what an **Agent Host** can observe or do. - An **Agent Capability** describes what an **Agent Host** can observe or do.
- **Terminal Capability** is an **Agent Capability** and must be explicitly enabled before an agent can host a **Terminal Session**.
- A **Terminal Session** owns one PTY and is independent of one-shot agent commands.
- A **Terminal Relay** pairs exactly one operator socket with exactly one agent socket for a **Terminal Session**.
- **Interface Telemetry** describes an **Agent Host** interface, not a **Device Endpoint**. - **Interface Telemetry** describes an **Agent Host** interface, not a **Device Endpoint**.
## Example dialogue ## Example dialogue
@@ -126,3 +141,6 @@ _Avoid_: Endpoint state, device presence
- An **Endpoint Key** must contain at least one network address value: MAC, IP, or both. Facts without MAC or IP remain **Observation Facts** only. - An **Endpoint Key** must contain at least one network address value: MAC, IP, or both. Facts without MAC or IP remain **Observation Facts** only.
- **Identifiers** are source-independent ownership claims over MAC or permanent IP values, not claims over endpoint sources. - **Identifiers** are source-independent ownership claims over MAC or permanent IP values, not claims over endpoint sources.
- IP **Identifiers** are explicit operator claims; Wakey does not infer whether an IP is permanent, static, or reserved. - IP **Identifiers** are explicit operator claims; Wakey does not infer whether an IP is permanent, static, or reserved.
- **Terminal Sessions** are not agent commands — resolved: command dispatch remains request/result, while terminals use dedicated streaming sockets.
- **Terminal Relay** is ephemeral control-plane state — resolved: terminal bytes and scrollback are not stored or audited.
- Terminal input, output, and terminal-generated control sequences are byte streams; only lifecycle and resize operations are structured terminal protocol messages.
Generated
+11
View File
@@ -1976,6 +1976,16 @@ dependencies = [
"syn", "syn",
] ]
[[package]]
name = "pty-process"
version = "0.5.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "71cec9e2670207c5ebb9e477763c74436af3b9091dd550b9fb3c1bec7f3ea266"
dependencies = [
"rustix",
"tokio",
]
[[package]] [[package]]
name = "quinn" name = "quinn"
version = "0.11.9" version = "0.11.9"
@@ -3690,6 +3700,7 @@ dependencies = [
"futures", "futures",
"lda-ipjs", "lda-ipjs",
"macaddr", "macaddr",
"pty-process",
"serde", "serde",
"serde_json", "serde_json",
"serial_test", "serial_test",
+136
View File
@@ -0,0 +1,136 @@
# Remote Terminal Sessions
Wakey will provide opt-in remote terminal access by pairing a browser terminal WebSocket with a dedicated outbound agent terminal WebSocket. The existing agent WebSocket remains the control channel and does not carry terminal byte streams.
**Status**: accepted
## Context
The control plane already maintains an authenticated outbound WebSocket from each connected agent. Its command protocol is intentionally request/result shaped: the control plane sends one command, the agent executes it, and one result completes the pending request.
An interactive terminal has different semantics. It is long-lived, bidirectional, byte-oriented, and must handle window resizing, process exit, disconnect cleanup, and backpressure. Treating it as an ordinary command would either block the agent session loop or turn the main agent WebSocket into a general stream multiplexer. It would also allow terminal output to compete with heartbeats, snapshots, and wake commands.
Agents commonly run behind NAT and must not expose an inbound listener. Terminal connections therefore remain agent-initiated and outbound.
## Decision
Use one dedicated agent WebSocket and one dedicated operator WebSocket for each terminal session. The control plane holds an in-memory terminal relay that pairs those sockets.
Session establishment follows this sequence:
1. An authenticated operator requests a terminal for a connected, terminal-capable agent.
2. The control plane creates an ephemeral terminal ID plus short-lived, single-use attachment credentials.
3. The control plane sends an open-terminal control message over the existing authenticated agent WebSocket.
4. The agent opens a new outbound terminal WebSocket to the control plane and authenticates it for that terminal ID.
5. The browser opens the corresponding protected operator terminal WebSocket.
6. The control plane pairs the sockets and relays terminal frames until either side exits or disconnects.
The main agent WebSocket carries terminal creation and cancellation control only. It does not carry PTY input or output.
## PTY Ownership
The agent owns the PTY, shell process, process group, and all process cleanup. The control plane never spawns or emulates a shell.
Use `pty-process` with its async feature as the initial PTY abstraction because it provides:
- Tokio `AsyncRead` and `AsyncWrite` integration;
- owned read and write halves for concurrent tasks;
- terminal resizing;
- child session-leader and controlling-terminal setup; and
- a small implementation built on `rustix`.
Adoption remains gated on successfully cross-compiling for the router target and exercising `/bin/ash` or the configured shell on a real device. Wakey does not call `forkpty` or add its own unsafe PTY implementation.
The executable, working directory, environment, UID, and GID are agent-controlled configuration. The browser cannot supply an arbitrary executable or process environment.
## Wire Protocol
WebSocket frame type separates terminal data from terminal control:
- Binary frames carry raw PTY input and output bytes.
- Text JSON frames carry lifecycle, error, and resize messages.
The terminal control vocabulary is deliberately small:
```text
operator -> agent: resize, close
agent -> operator: ready, exited, error
```
Initial terminal dimensions are part of session attachment. Resize carries rows and columns; pixel dimensions may be added later if a demonstrated program requires them.
Terminal behavior such as Ctrl-C, Ctrl-Z, Ctrl-D, cursor keys, mouse reporting, bracketed paste, colors, and terminal-title changes remains in the byte stream. The control plane does not parse ANSI escape sequences. The browser terminal emulator renders output and owns connected-session scrollback.
WebSockets already provide ordered, reliable delivery, so terminal frames do not carry sequence numbers in the initial protocol.
## Lifecycle and Cleanup
Initial terminal sessions are ephemeral and allow only one attached operator.
When the operator socket disconnects, the control plane keeps the agent socket and PTY alive for a short grace period. A protected operator request may obtain a fresh, single-use attachment credential for that existing session. The control plane retains only a bounded in-memory output buffer during the gap, replays it before live output on reattachment, and closes the session if no operator returns before the grace period expires.
An agent terminal socket disconnect closes the session immediately because the control plane can no longer control or observe the PTY. Agent reconnect creates a new agent session and cannot adopt an old terminal.
Closing a session must terminate the entire PTY process group, not only the shell process. Cleanup should send a hangup first, then escalate to termination and forced kill after bounded grace periods. Agent reconnect, agent replacement, control-plane restart, token expiry before attachment, and absolute session timeout all close the session.
The agent reports a normal exit status or terminating signal when available. The UI must distinguish an exited shell from a transport failure.
## Backpressure and Resource Limits
Terminal transport must not use unbounded queues.
Every queue between PTY, agent socket, control-plane relay, and browser socket is bounded. The brief-disconnect replay buffer is bounded as well. When the downstream consumer is slow, the producer waits rather than accumulating output in memory. Backpressure eventually stops PTY reads and allows the kernel PTY buffer to block an abusive producer.
The implementation also enforces:
- maximum terminal frame size;
- per-agent concurrent-session limits;
- idle and absolute session timeouts;
- bounded disconnect grace;
- bounded control-plane relay buffers; and
- cleanup when any relay task exits unexpectedly.
Exact limits are configuration and may be tuned from router testing. The bounded behavior is an architectural requirement.
## Security and Audit
Remote terminal access is root-equivalent on agents that run as root. It is therefore an explicit agent capability and is disabled by default.
Operator terminal endpoints belong to the protected control API surface. Session identifiers are not credentials. Agent and operator attachment credentials are short-lived, scoped to one terminal session, and single-use. The operator WebSocket must enforce the same-origin/protected-control assumptions used when the session was created.
Audit records include terminal request, open, ready, disconnect, close, timeout, and exit metadata. Wakey never stores or audits terminal input, terminal output, command history, or scrollback.
Terminal capability and session state are separate from device inventory, known-device identity, wake routes, and the debug Command Runner.
## Deferred Work
Durable or long-lived terminal sessions are not part of the initial version. A later design may preserve sessions across longer operator absences or control-plane restart. That design must define durable ownership, reconnect credentials, persisted replay bounds, secret handling, and process reconciliation before adding persistence.
File transfer, multi-operator attachment, terminal recording, shell-history storage, and arbitrary process launch are also deferred.
## Alternatives Considered
### Multiplex terminal bytes over the main agent WebSocket
Rejected for the initial version. It requires refactoring the current request/result session into a general concurrent writer and stream router, adds head-of-line coupling with fleet traffic, and requires terminal-byte framing within a shared connection.
### Model a terminal as a long-running agent command
Rejected. Commands have one result and a timeout; terminals have continuous bidirectional traffic and independent lifecycle.
### Accept an inbound connection on the agent
Rejected. Agents run behind NAT and should not expose a new network service.
### Use `portable-pty`
Viable, but not selected initially. Its cross-platform abstraction is broader than the Linux agent requires and its reader/writer API would need blocking-task bridges. It remains a fallback if `pty-process` does not build or behave correctly on the target router.
## Consequences
- Each active terminal consumes two additional WebSockets and one PTY process on the agent.
- Terminal traffic cannot delay main agent heartbeat, snapshot, or command messages.
- Control-plane terminal state is simple and ephemeral; restarting the control plane closes active terminals.
- The agent session protocol gains terminal-open control and capability advertisement but does not become a terminal-data multiplexer.
- Browser and agent terminal transports can be tested independently before adding xterm.js.
- Future resume support can extend terminal-session lifecycle without changing the basic PTY byte protocol.
+1
View File
@@ -1,3 +1,4 @@
* *
!.gitignore !.gitignore
!copy.ps1 !copy.ps1
!wslfix.sh
+6
View File
@@ -0,0 +1,6 @@
#!/usr/bin/env bash
# sometimes it breaks, run to fix
sudo sh -c 'echo :WSLInterop:M::MZ::/init:PF > /usr/lib/binfmt.d/WSLInterop.conf'
sudo systemctl restart systemd-binfmt
+8 -5
View File
@@ -5,12 +5,12 @@ edition = "2024"
publish = ["gitea"] publish = ["gitea"]
[dependencies] [dependencies]
anyhow = "1" anyhow = { workspace = true }
futures = "0" futures = "0"
macaddr = { version = "1", features = ["serde", "serde_std"] } macaddr = { workspace = true }
serde = { version = "1", features = ["derive"] } serde = { workspace = true }
serde_json = "1" serde_json = { workspace = true }
tokio = { version = "1", features = ["fs", "net", "rt", "sync"] } tokio = { workspace = true, features = ["net", "sync"] }
wakey-core = { path = "../wakey-core", registry = "gitea", version = "0"} wakey-core = { path = "../wakey-core", registry = "gitea", version = "0"}
[dev-dependencies] [dev-dependencies]
@@ -21,3 +21,6 @@ path = "../ipjs"
registry = "gitea" registry = "gitea"
version = "0" version = "0"
features = ["experimental-nl"] features = ["experimental-nl"]
[target.'cfg(unix)'.dependencies]
pty-process = { version = "0.5", features = ["async"] }
+4
View File
@@ -3,9 +3,13 @@
pub mod devices; pub mod devices;
pub mod dhcp; pub mod dhcp;
pub mod observations; pub mod observations;
#[cfg(unix)]
pub mod terminal;
pub mod wake; pub mod wake;
pub use devices::*; pub use devices::*;
pub use dhcp::*; pub use dhcp::*;
pub use observations::*; pub use observations::*;
#[cfg(unix)]
pub use terminal::*;
pub use wake::*; pub use wake::*;
+94
View File
@@ -0,0 +1,94 @@
//! Unix PTY ownership for interactive agent terminal sessions.
use std::path::Path;
use anyhow::{Context, Result};
use pty_process::{Command, OwnedReadPty, OwnedWritePty, Size};
use tokio::process::Child;
/// An interactive child process and the independently owned sides of its PTY.
///
/// Keeping the write half available is important: `OwnedWritePty` also owns the
/// resize operation used by terminal WebSocket control frames.
pub struct TerminalPty {
pub reader: OwnedReadPty,
pub writer: OwnedWritePty,
pub child: Child,
}
impl TerminalPty {
/// Starts `program` attached to a newly allocated PTY.
pub fn spawn(program: &Path, rows: u16, cols: u16) -> Result<Self> {
let (pty, pts) = pty_process::open().context("failed to open PTY")?;
pty.resize(Size::new(rows, cols))
.context("failed to set initial PTY size")?;
let command = Command::new(program);
let child = command
.spawn(pts)
.with_context(|| format!("failed to spawn {} in PTY", program.display()))?;
let (reader, writer) = pty.into_split();
Ok(Self {
reader,
writer,
child,
})
}
}
#[cfg(test)]
mod tests {
use std::time::Duration;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use super::*;
/// This test intentionally uses the same PTY implementation on the router.
/// Run it remotely with:
///
/// `./scripts/test_remote.ps1 -Package wakey-linux -Filter terminal::tests::pty_round_trip_and_resize -Exact -ShowOutput`
#[tokio::test]
async fn pty_round_trip_and_resize() {
let TerminalPty {
mut reader,
mut writer,
mut child,
} = TerminalPty::spawn(Path::new("/bin/sh"), 24, 80).expect("spawn PTY shell");
writer
.resize(Size::new(31, 101))
.expect("resize owned PTY writer");
writer
.write_all(b"read line; printf 'WAKEY_PTY_OK:%s\\n' \"$line\"; exit 0\nprobe\n")
.await
.expect("write shell input");
let output = tokio::time::timeout(Duration::from_secs(5), async {
let mut output = Vec::new();
let mut chunk = [0_u8; 4096];
while !String::from_utf8_lossy(&output).contains("WAKEY_PTY_OK:probe") {
// Use a fresh ReadBuf for each PTY read. pty-process 0.5.3's
// AsyncRead implementation cannot safely extend the partially
// filled ReadBuf used internally by Tokio's `read_to_end`.
let count = reader.read(&mut chunk).await.expect("read PTY output");
assert_ne!(count, 0, "PTY closed before emitting probe marker");
output.extend_from_slice(&chunk[..count]);
}
output
})
.await
.expect("PTY output timed out");
let status = tokio::time::timeout(Duration::from_secs(5), child.wait())
.await
.expect("PTY child wait timed out")
.expect("wait for PTY child");
assert!(status.success(), "shell exited with {status}");
let output = String::from_utf8_lossy(&output);
assert!(
output.contains("WAKEY_PTY_OK:probe"),
"unexpected PTY output: {output:?}"
);
}
}