add 003-db-worker-node-cli-orchestration.md

.gitignore

@@ -77,3 +77,4 @@ clj-e2e/.wally
 clj-e2e/e2e-dump
 .dir-locals.el
 .projectile
+*.map

docs/agent-guide/003-db-worker-node-cli-orchestration.md

@@ -0,0 +1,115 @@
+# db-worker-node and logseq-cli Orchestration Plan
+
+Goal: Based on the current `logseq-cli` and `db-worker-node` implementations, refactor db-worker-node to be repo-bound with locking, make logseq-cli fully manage db-worker-node lifecycle, and add server subcommands for management.
+
+## Background and Current State (from existing code)
+
+- `db-worker-node` currently accepts `--repo` but it is optional; it can open/switch graphs via `thread-api/create-or-open-db` at runtime. Entrypoint: `src/main/frontend/worker/db_worker_node.cljs`.
+- `logseq-cli` connects to an existing db-worker-node via `base-url`; it does not start/stop the server. Entrypoint: `src/main/logseq/cli/main.cljs`, `src/main/logseq/cli/commands.cljs`, `src/main/logseq/cli/transport.cljs`.
+- Tests explicitly start db-worker-node (`src/test/logseq/cli/integration_test.cljs`, `src/test/frontend/worker/db_worker_node_test.cljs`).
+
+## Requirements
+
+1. Refactor `db-worker-node`: startup must require `--repo`; on startup it must open or create that graph; it must not switch graphs at runtime; it must create a lock file so a graph can be served by only one db-worker-node instance; it only needs to bind to localhost.
+2. In `logseq-cli`, all commands requiring `--repo` or any graph operations must connect to or create the corresponding db-worker-node server.
+3. db-worker-node server must not be started manually; logseq-cli is fully responsible.
+4. Add `server` subcommand(s) to logseq-cli for managing db-worker-node servers.
+
+## Scope / Non-goals
+
+- In scope: db-worker-node startup and lifecycle, repo binding enforcement, lock files, CLI server orchestration, management commands, tests/docs.
+- Out of scope: changing db-worker core query/write protocol; changing db-worker-node HTTP API semantics beyond repo binding constraints.
+
+## Proposed Design
+
+- **Repo-bound server**: db-worker-node opens a single repo at startup and refuses repo changes for the lifetime of the process. It only listens on localhost.
+- **Lock file**: each repo directory has a lock file to ensure one server per graph. Lock contains metadata for status/cleanup; db-worker-node handles it by default, and logseq-cli handles cases db-worker-node cannot.
+- **CLI orchestration**: logseq-cli discovers/starts/reuses db-worker-node servers per repo. It is the only entrypoint for starting servers.
+- **Server subcommands**: add `server list|status|start|stop|restart` (or similar) to manage servers explicitly.
+
+## Detailed Design
+
+### 1) db-worker-node: required repo, repo binding, lock file
+
+Files:
+- `src/main/frontend/worker/db_worker_node.cljs`
+- `src/main/frontend/worker/platform/node.cljs` (for data-dir / repo dir resolution)
+- Optional new helper: `src/main/frontend/worker/db_worker_node_lock.cljs`
+
+Key changes:
+- **Argument parsing**: `--repo` becomes required. If missing, print help and exit non-zero. Host binding is restricted to localhost (e.g., `127.0.0.1`) regardless of input.
+- **Startup flow**: replace `<maybe-open-repo!` with a required `create-or-open-db` for the repo; store `bound-repo`.
+- **Reject switching**:
+  - In `/v1/invoke`, for repo-scoped thread APIs, validate `args[0]` matches `bound-repo`.
+  - Reject `thread-api/create-or-open-db`, `thread-api/unsafe-unlink-db`, etc. when repo differs.
+  - Return 409/400 with `:repo-mismatch` error shape.
+- **Lock file**:
+  - Location: inside repo dir (e.g. `~/.logseq/db-worker/.logseq-pool-<graph>/db-worker.lock`).
+  - Content: JSON `{repo, pid, host, port, startedAt}`.
+  - Creation: exclusive create (`fs.open` with `wx`) or atomic temp + rename. If exists, fail with “graph already locked”.
+  - Cleanup: delete lock file on stop (`stop!`) and on SIGINT/SIGTERM.
+  - Stale lock: if lock exists but pid is dead, allow replacement (db-worker-node first; CLI can repair when server cannot).
+
+### 2) logseq-cli: auto start/reuse db-worker-node per repo
+
+Files:
+- `src/main/logseq/cli/commands.cljs`
+- `src/main/logseq/cli/main.cljs`
+- `src/main/logseq/cli/config.cljs`
+- New: `src/main/logseq/cli/server.cljs` (process management + lock handling)
+
+Key changes:
+- **Repo resolution**: for all graph/content commands, require `--repo` or resolved repo from config; otherwise error.
+- **Ensure server** (new helper `ensure-server!`):
+  1. Derive data-dir, repo dir, and lock file path from repo.
+  2. If lock file exists, read host/port/pid; probe `/healthz` + `/readyz`.
+  3. If healthy, reuse existing server; set `config :base-url` dynamically.
+  4. If unhealthy or stale, attempt to spawn a new server; if db-worker-node cannot handle the lock situation, CLI repairs the lock then retries.
+  5. Spawn via `child_process.spawn`: `node ./static/db-worker-node.js --repo <repo> --data-dir <...> --host 127.0.0.1 --port 0`.
+  6. Resolve actual port from lock file or server output.
+- **base-url usage**: dynamically set based on repo server; no longer required from user. If `--base-url` is provided, decide if it is ignored or overrides orchestration (see Compatibility section).
+
+### 3) CLI `server` subcommands
+
+Suggested command group:
+- `server list`: list servers from lock files (repo, pid, port, status).
+- `server start --repo <name>`: start server for repo.
+- `server stop --repo <name>`: stop server (SIGTERM or `/v1/shutdown`).
+- `server restart --repo <name>`: stop + start.
+
+Implementation notes:
+- `start|stop|restart` require `--repo`.
+- `list` scans data-dir for repo directories, reads lock files, and verifies status.
+- Consider adding `/v1/shutdown` in db-worker-node for graceful stop.
+
+## Compatibility / Migration
+
+- No need to preserve compatibility for existing env vars, config keys, or flags related to db-worker-node or CLI server connectivity; remove them if they are no longer needed.
+
+## Test Plan
+
+- **Unit tests**:
+  - CLI: repo resolution, server orchestration logic, lock parsing, error codes (`src/test/logseq/cli/*`).
+  - db-worker-node: repo required, repo mismatch rejection, lock file create/cleanup (`src/test/frontend/worker/db_worker_node_test.cljs`).
+- **Integration tests**:
+  - CLI runs graph/content commands without manual server startup (`src/test/logseq/cli/integration_test.cljs`).
+  - Concurrent startup of same repo must fail due to lock.
+
+## Milestones
+
+1. **db-worker-node binding & lock file**: repo required + repo enforcement + lock creation/cleanup.
+2. **CLI server module**: `ensure-server!` with lock/health checks and spawning.
+3. **CLI command updates**: graph/content commands require repo and auto-start server; add `server` subcommands.
+4. **Tests + docs**: update/extend tests and adjust CLI docs (`docs/cli/logseq-cli.md`).
+
+## Open Questions
+
+1. Should `graph list` require `--repo`? If not, define a “global” server or out-of-band access to data-dir.
+   - Answer: No --repo needed, using 'out-of-band access to data-dir' way
+2. Lock file format and location: confirm cross-platform expectations (Windows paths/permissions).
+   - lockfile name:`db-worker.lock`,
+   - Location: inside repo dir (e.g. `~/.logseq/db-worker/.logseq-pool-<graph>/db-worker.lock`).
+   - only consider linux/macos for now
+3. Who owns lock cleanup and stale lock handling: primarily db-worker-node; CLI only steps in for cases db-worker-node cannot handle.
+4. Add `/v1/shutdown` for graceful stop vs. SIGTERM from CLI?
+5. db-worker-node servers should keep running unless `logseq-cli server stop` is invoked or the process exits unexpectedly; in the latter case, CLI should handle lockfile cleanup on restart.