Home Verkennen Help
Inloggen
Apq
/
logseq
spiegel van https://github.com/logseq/logseq.git
1
0
Vork 0
Bestanden Issues 0 Wiki
Boom: 50535ae181
Aftakkingen Labels
logseq
/
docs
/
agent-guide
/
003-db-worker-node-cli-orchestration.md

003-db-worker-node-cli-orchestration.md 7.3 KB
Geschiedenis Ruwe

db-worker-node and logseq-cli Orchestration Plan

Goal: Based on the current logseq-cli and db-worker-node implementations, refactor db-worker-node to be repo-bound with locking, make logseq-cli fully manage db-worker-node lifecycle, and add server subcommands for management.

Background and Current State (from existing code)

  • db-worker-node currently accepts --repo but it is optional; it can open/switch graphs via thread-api/create-or-open-db at runtime. Entrypoint: src/main/frontend/worker/db_worker_node.cljs.
  • logseq-cli connects to an existing db-worker-node via base-url; it does not start/stop the server. Entrypoint: src/main/logseq/cli/main.cljs, src/main/logseq/cli/commands.cljs, src/main/logseq/cli/transport.cljs.
  • Tests explicitly start db-worker-node (src/test/logseq/cli/integration_test.cljs, src/test/frontend/worker/db_worker_node_test.cljs).

Requirements

  1. Refactor db-worker-node: startup must require --repo; on startup it must open or create that graph; it must not switch graphs at runtime; it must create a lock file so a graph can be served by only one db-worker-node instance; it only needs to bind to localhost.
  2. In logseq-cli, all commands requiring --repo or any graph operations must connect to or create the corresponding db-worker-node server.
  3. db-worker-node server must not be started manually; logseq-cli is fully responsible.
  4. Add server subcommand(s) to logseq-cli for managing db-worker-node servers.

Scope / Non-goals

  • In scope: db-worker-node startup and lifecycle, repo binding enforcement, lock files, CLI server orchestration, management commands, tests/docs.
  • Out of scope: changing db-worker core query/write protocol; changing db-worker-node HTTP API semantics beyond repo binding constraints.

Proposed Design

  • Repo-bound server: db-worker-node opens a single repo at startup and refuses repo changes for the lifetime of the process. It only listens on localhost.
  • Lock file: each repo directory has a lock file to ensure one server per graph. Lock contains metadata for status/cleanup; db-worker-node handles it by default, and logseq-cli handles cases db-worker-node cannot.
  • CLI orchestration: logseq-cli discovers/starts/reuses db-worker-node servers per repo. It is the only entrypoint for starting servers.
  • Server subcommands: add server list|status|start|stop|restart (or similar) to manage servers explicitly.

Detailed Design

1) db-worker-node: required repo, repo binding, lock file

Files:

  • src/main/frontend/worker/db_worker_node.cljs
  • src/main/frontend/worker/platform/node.cljs (for data-dir / repo dir resolution)
  • Optional new helper: src/main/frontend/worker/db_worker_node_lock.cljs

Key changes:

  • Argument parsing: --repo becomes required. If missing, print help and exit non-zero. Host binding is restricted to localhost (e.g., 127.0.0.1) regardless of input.
  • Startup flow: replace <maybe-open-repo! with a required create-or-open-db for the repo; store bound-repo.
  • Reject switching:
    • In /v1/invoke, for repo-scoped thread APIs, validate args[0] matches bound-repo.
    • Reject thread-api/create-or-open-db, thread-api/unsafe-unlink-db, etc. when repo differs.
    • Return 409/400 with :repo-mismatch error shape.
  • Lock file:
    • Location: inside repo dir (e.g. ~/.logseq/db-worker/.logseq-pool-<graph>/db-worker.lock).
    • Content: JSON {repo, pid, host, port, startedAt}.
    • Creation: exclusive create (fs.open with wx) or atomic temp + rename. If exists, fail with “graph already locked”.
    • Cleanup: delete lock file on stop (stop!) and on SIGINT/SIGTERM.
    • Stale lock: if lock exists but pid is dead, allow replacement (db-worker-node first; CLI can repair when server cannot).

2) logseq-cli: auto start/reuse db-worker-node per repo

Files:

  • src/main/logseq/cli/commands.cljs
  • src/main/logseq/cli/main.cljs
  • src/main/logseq/cli/config.cljs
  • New: src/main/logseq/cli/server.cljs (process management + lock handling)

Key changes:

  • Repo resolution: for all graph/content commands, require --repo or resolved repo from config; otherwise error.
  • Ensure server (new helper ensure-server!):
    1. Derive data-dir, repo dir, and lock file path from repo.
    2. If lock file exists, read host/port/pid; probe /healthz + /readyz.
    3. If healthy, reuse existing server; set config :base-url dynamically.
    4. If unhealthy or stale, attempt to spawn a new server; if db-worker-node cannot handle the lock situation, CLI repairs the lock then retries.
    5. Spawn via child_process.spawn: node ./static/db-worker-node.js --repo <repo> --data-dir <...> --host 127.0.0.1 --port 0.
    6. Resolve actual port from lock file or server output.
  • base-url usage: dynamically set based on repo server; no longer required from user. If --base-url is provided, decide if it is ignored or overrides orchestration (see Compatibility section).

3) CLI server subcommands

Suggested command group:

  • server list: list servers from lock files (repo, pid, port, status).
  • server start --repo <name>: start server for repo.
  • server stop --repo <name>: stop server (SIGTERM or /v1/shutdown).
  • server restart --repo <name>: stop + start.

Implementation notes:

  • start|stop|restart require --repo.
  • list scans data-dir for repo directories, reads lock files, and verifies status.
  • Consider adding /v1/shutdown in db-worker-node for graceful stop.

Compatibility / Migration

  • No need to preserve compatibility for existing env vars, config keys, or flags related to db-worker-node or CLI server connectivity; remove them if they are no longer needed.

Test Plan

  • Unit tests:
    • CLI: repo resolution, server orchestration logic, lock parsing, error codes (src/test/logseq/cli/*).
    • db-worker-node: repo required, repo mismatch rejection, lock file create/cleanup (src/test/frontend/worker/db_worker_node_test.cljs).
  • Integration tests:
    • CLI runs graph/content commands without manual server startup (src/test/logseq/cli/integration_test.cljs).
    • Concurrent startup of same repo must fail due to lock.

Milestones

  1. db-worker-node binding & lock file: repo required + repo enforcement + lock creation/cleanup.
  2. CLI server module: ensure-server! with lock/health checks and spawning.
  3. CLI command updates: graph/content commands require repo and auto-start server; add server subcommands.
  4. Tests + docs: update/extend tests and adjust CLI docs (docs/cli/logseq-cli.md).

Open Questions

  1. Should graph list require --repo? If not, define a “global” server or out-of-band access to data-dir.
    • Answer: No --repo needed, using 'out-of-band access to data-dir' way
  2. Lock file format and location: confirm cross-platform expectations (Windows paths/permissions).
    • lockfile name:db-worker.lock,
    • Location: inside repo dir (e.g. ~/.logseq/db-worker/.logseq-pool-<graph>/db-worker.lock).
    • only consider linux/macos for now
  3. Who owns lock cleanup and stale lock handling: primarily db-worker-node; CLI only steps in for cases db-worker-node cannot handle.
  4. Add /v1/shutdown for graceful stop vs. SIGTERM from CLI?
  5. db-worker-node servers should keep running unless logseq-cli server stop is invoked or the process exits unexpectedly; in the latter case, CLI should handle lockfile cleanup on restart.