geng

wym glibc 2.38
2026-04-14 09:45:59 +07:00
1 parent 54d3f1e93d
commit ac824235ab
@@ -0,0 +1,100 @@
+# Wakey Checkpoint (2026-04-11)
+
+## Snapshot
+
+This checkpoint captures the current state after control-plane migration, logging hardening, config ergonomics, and state storage upgrades.
+
+## What Is Done
+
+- Legacy router-hosted HTTP/static layer removed from `wakey` crate.
+- New `wakey-control-plane` crate is active for:
+  - enroll-token issuance
+  - agent enrollment
+  - connected-agent websocket registry
+  - command relay to agent
+- `wakey-agent` is active for outbound websocket execution and local command dispatch.
+
+## Logging + Telemetry
+
+- High-signal logs were added across:
+  - control-plane API relay path
+  - control-plane websocket lifecycle
+  - control-plane state lifecycle
+  - agent command lifecycle, session lifecycle, and enrollment
+- Correlated relay spans include command context (`agent_id`, `request_id`, `command`).
+- Control-plane telemetry is config-driven:
+  - optional OTLP endpoint
+  - optional JSON logs
+  - fallback local logs when OTLP endpoint is not set
+
+## Config Ergonomics
+
+### Control-plane
+
+- Config file support is wired (`/etc/wakey-control-plane/config.toml` by default).
+- `serve` can read defaults from config file and CLI can override.
+- New `init-config` command scaffolds a control-plane config file.
+
+### Agent
+
+- Existing `init-config` command scaffolds agent config.
+- Enrollment can optionally signal reload of running daemon.
+
+## State Storage Upgrade
+
+- Control-plane state backend moved from JSON snapshot to embedded `sled` DB.
+- Default state path changed to `/var/lib/wakey-control-plane/state.db`.
+- Legacy JSON migration support has been removed; sled is now the only supported
+  state format.
+- Enroll tokens include persisted expiry timestamps and are validated on enroll.
+- Periodic and explicit garbage collection remove expired tokens.
+
+## Operator Commands
+
+### Control-plane bootstrap
+
+```sh
+wakey-control-plane init-config
+wakey-control-plane serve --config-file /etc/wakey-control-plane/config.toml
+```
+
+### Issue enroll token (live daemon path)
+
+```sh
+wakey-control-plane issue-enroll-token --public-url https://cp.example.com
+```
+
+### Agent bootstrap
+
+```sh
+wakey-agent enroll --server-url https://cp.example.com --enroll-token <token>
+wakey-agent serve --config /etc/wakey-agent/config.toml
+```
+
+## Build Health
+
+- Last verified passing:
+  - `cargo check --workspace`
+  - `cargo clippy --workspace`
+
+## Test Coverage Added In This Pass
+
+- Unit tests in `wakey-control-plane/src/state/store.rs` now cover:
+  - expired-token garbage collection removes persisted stale tokens
+  - enroll rejects expired tokens and consumes stale entries
+  - state stats counters for agents and expired token totals
+
+## Known Tradeoffs / Follow-ups
+
+- Reload semantics with `sled` are now mostly no-op for in-memory state (data is durable in DB).
+- No dedicated state-inspection CLI command yet (suggestion: add `state-stats` command).
+- OTLP configuration is currently control-plane focused; agent parity can be added if needed.
+
+## Suggested Next Steps
+
+1. Add a control-plane `state-stats` command to print DB path, agent count, token count.
+2. Add symmetric telemetry config support in `wakey-agent` config file.
+3. Add integration tests for:
+   - enroll + relay over websocket
+   - legacy JSON-to-sled migration path
+   - init-config command behavior and overrides
@@ -0,0 +1,82 @@
+# Wakey Checkpoint (2026-04-12)
+
+## Snapshot
+
+This checkpoint captures progress after adding audit persistence, alert evaluation and transitions, control-plane alert APIs, and websocket timing diagnostics across agent and control-plane.
+
+## Major Changes Landed
+
+- Control-plane audit system implemented with persistent sled-backed events.
+- Audit emission wired into:
+  - enroll accept/reject
+  - token issue/list/revoke
+  - command dispatch/result/error/timeout
+  - websocket auth accept/reject and disconnect
+- Audit query API added:
+  - `GET /api/v1/control/audit/events`
+- Active alert engine added with deterministic rules over audit + live session state.
+- Alert APIs added:
+  - `GET /api/v1/control/alerts`
+  - `GET /api/v1/control/alerts/history`
+  - `GET /api/v1/control/alerts/ws`
+- Alert transition persistence added (open/resolve transitions tracked across evaluations).
+- Route classes split explicitly in runtime:
+  - public routes (enroll/ws/health)
+  - control routes (`/api/v1/control/*`)
+- Caddy template added for edge policy and Cloudflare Access boundary:
+  - `deploy/Caddyfile.control-plane.example`
+
+## Reliability and Diagnostics Improvements
+
+- Agent websocket connect diagnostics now include:
+  - DNS resolution timing (`dns_resolve_ms`)
+  - websocket connect timing (`ws_connect_ms`)
+- Control-plane websocket lifecycle logs include:
+  - connect-to-hello timing
+  - connect-to-auth timing
+  - hello-to-auth timing
+- Slow connect warnings are now emitted when timing thresholds are exceeded.
+
+## Root-Cause Findings Captured
+
+- Long agent websocket connect delays were reproduced and traced to hostname resolution path.
+- Switching agent `server_url` hostname to direct IP made connect immediate.
+- This confirms app-level relay logic was not the source of the startup delay.
+
+## Verification Status
+
+- `cargo check --workspace` passing after all changes.
+- Added and passing tests include:
+  - audit event append/filter in state store
+  - alert transition open/resolve persistence
+  - alert evaluator rule checks (offline + timeout, auth/enroll rejection spikes)
+
+## Current API Surface for UI Start
+
+- Agents and command execution:
+  - `GET /api/v1/control/agents`
+  - `POST /api/v1/control/agents/{agent_id}/command`
+- Audits:
+  - `GET /api/v1/control/audit/events`
+- Alerts:
+  - `GET /api/v1/control/alerts`
+  - `GET /api/v1/control/alerts/history`
+  - websocket subscribe: `GET /api/v1/control/alerts/ws`
+
+## Remaining Plan Items (Most Significant)
+
+1. UI implementation (`/ui` app shell and pages) is still open.
+2. Alert dedupe/cooldown persistence and tuning are still basic and need hardening.
+3. Audit retention pruning policy and long-run storage controls are not finalized.
+4. Edge auth enforcement tests and deployment rehearsals remain to be added.
+5. Multi-day soak drills and failure-injection validation remain open.
+
+## Suggested Next Actions
+
+1. Build minimal UI shell with three views:
+   - agents/commands
+   - audit timeline
+   - alerts panel (active + history + websocket stream)
+2. Add periodic retention task for audit and alert transition trees.
+3. Add proxy-level integration tests that assert private endpoints are blocked without Access headers.
+4. Run a 48-72h soak with hostname vs IP connect-path metrics collected.
@@ -0,0 +1,14 @@
+# Control Plane Review
+
+## Findings
+
+1. High: the admin/control API is effectively unauthenticated, so anyone who can reach the server can issue enroll tokens, list/revoke tokens, inspect audit/alerts, and send live commands to agents. In [`wakey-control-plane/src/runtime/mod.rs:61`](c:/Users/Admin/Documents/realshit/wakey/wakey-control-plane/src/runtime/mod.rs#L61) through [`wakey-control-plane/src/runtime/mod.rs:109`](c:/Users/Admin/Documents/realshit/wakey/wakey-control-plane/src/runtime/mod.rs#L109), `control_api_routes()` is merged straight into the app with no auth middleware despite the comment saying these routes are “admin-only.” The UI also calls those endpoints directly with plain `fetch` and no auth material in [`ui/src/api.ts:60`](c:/Users/Admin/Documents/realshit/wakey/ui/src/api.ts#L60) through [`ui/src/api.ts:123`](c:/Users/Admin/Documents/realshit/wakey/ui/src/api.ts#L123). This is a full remote-takeover issue for the control plane, not just a missing polish item.
+
+2. High: a second websocket connection for the same `agent_id` silently replaces the current session, but the old authenticated socket is left alive and can still submit `result`/`error` frames against pending requests. In [`wakey-control-plane/src/ws.rs:195`](c:/Users/Admin/Documents/realshit/wakey/wakey-control-plane/src/ws.rs#L195) through [`wakey-control-plane/src/ws.rs:200`](c:/Users/Admin/Documents/realshit/wakey/wakey-control-plane/src/ws.rs#L200), a successful auth simply overwrites `sessions[agent_id] = tx.clone()`. The previous socket is not closed or demoted. Later, any authenticated socket can satisfy pending requests in [`wakey-control-plane/src/ws.rs:229`](c:/Users/Admin/Documents/realshit/wakey/wakey-control-plane/src/ws.rs#L229) through [`wakey-control-plane/src/ws.rs:247`](c:/Users/Admin/Documents/realshit/wakey/wakey-control-plane/src/ws.rs#L247), while requests are correlated only by `request_id` created in [`wakey-control-plane/src/api/commands.rs:91`](c:/Users/Admin/Documents/realshit/wakey/wakey-control-plane/src/api/commands.rs#L91) through [`wakey-control-plane/src/api/commands.rs:97`](c:/Users/Admin/Documents/realshit/wakey/wakey-control-plane/src/api/commands.rs#L97). That creates a split-brain/race condition where a stale or malicious prior session for the same agent can inject or win replies.
+
+3. Medium: configured seed enroll tokens are reinserted into the database on every startup, so “one-time” tokens become reusable after a restart if they remain in config. In [`wakey-control-plane/src/state/store.rs:52`](c:/Users/Admin/Documents/realshit/wakey/wakey-control-plane/src/state/store.rs#L52) through [`wakey-control-plane/src/state/store.rs:68`](c:/Users/Admin/Documents/realshit/wakey/wakey-control-plane/src/state/store.rs#L68), `load_or_init()` blindly seeds `daemon.enroll_tokens` into sled every time the process starts. Enrollment consumes tokens in [`wakey-control-plane/src/state/store.rs:91`](c:/Users/Admin/Documents/realshit/wakey/wakey-control-plane/src/state/store.rs#L91) through [`wakey-control-plane/src/state/store.rs:109`](c:/Users/Admin/Documents/realshit/wakey/wakey-control-plane/src/state/store.rs#L109), but that consumption is undone on the next boot if the token still exists in config. That breaks the “short-lived/one-time” assumption and makes operational mistakes much more likely.
+
+## Residual Risks
+
+- The control-plane/UI surface is growing quickly and currently assumes a trusted environment in multiple places. Even after adding admin auth, I would expect more authz/session-boundary issues to surface.
+- The command relay path is conceptually good, but it needs stronger session ownership rules before it is trustworthy under reconnect races or duplicated agents.