test(app): smoke tests spec

specs/08-app-e2e-smoke-suite.md

@@ -0,0 +1,253 @@
+## App E2E Smoke Suite (CI)
+
+Implement a small set of high-signal, low-flake Playwright tests to run in CI.
+
+These tests are intended to catch regressions in the “core shell” of the app (navigation, dialogs, prompt UX, file viewer, terminal), without relying on model output.
+
+---
+
+### Summary
+
+Add 6 smoke tests to `packages/app/e2e/`:
+
+- Settings dialog: open, switch tabs, close
+- Prompt slash command: `/open` opens the file picker dialog
+- Prompt @mention: `@<file>` inserts a file pill token
+- Model picker: open model selection and choose a model
+- File viewer: open a known file and assert contents render
+- Terminal: open terminal, verify Ghostty mounts, create a second terminal
+
+---
+
+### Progress
+
+- [x] 1. Settings dialog open / switch / close (`packages/app/e2e/settings.spec.ts`)
+- [ ] 2. Prompt slash command path: `/open` opens file picker
+- [ ] 3. Prompt @mention inserts a file pill token
+- [ ] 4. Model selection UI works end-to-end
+- [ ] 5. File viewer renders real file content
+- [ ] 8. Terminal init + create new terminal
+
+---
+
+### Goals
+
+- Tests run reliably in CI using the existing local runner (`packages/app/script/e2e-local.ts`).
+- Cover “wiring” regressions across UI + backend APIs:
+  - dialogs + command routing
+  - prompt contenteditable parsing
+  - file search + file read + code viewer render
+  - terminal open + pty creation + Ghostty mount
+- Avoid assertions that depend on LLM output.
+- Keep runtime low (these should be “smoke”, not full workflows).
+
+---
+
+### Non-goals
+
+- Verifying complex model behavior, streaming correctness, or tool call semantics.
+- Testing provider auth flows (CI has no secrets).
+- Testing share, MCP, or LSP download flows (disabled in the e2e runner).
+
+---
+
+### Current State
+
+Existing tests in `packages/app/e2e/` already cover:
+
+- Home renders + server picker opens
+- Directory route redirects to `/session`
+- Sidebar collapse/expand
+- Command palette opens/closes
+- Basic session open + prompt input + (optional) prompt/reply flow
+- File open via palette (but shallow assertion: tab exists)
+- Terminal panel toggles (but doesn’t assert Ghostty mounted)
+- Context panel open
+
+We want to add a focused smoke layer that increases coverage of the most regression-prone UI paths.
+
+---
+
+### Proposed Tests
+
+All tests should use the shared fixtures in:
+
+- `packages/app/e2e/fixtures.ts` (for `sdk`, `directory`, `gotoSession`)
+- `packages/app/e2e/utils.ts` (for `modKey`, `promptSelector`, `terminalToggleKey`)
+
+Prefer creating new spec files rather than overloading existing ones, so it’s easy to run these tests as a group via grep.
+
+Suggested file layout:
+
+- `packages/app/e2e/settings.spec.ts`
+- `packages/app/e2e/prompt-slash-open.spec.ts`
+- `packages/app/e2e/prompt-mention.spec.ts`
+- `packages/app/e2e/model-picker.spec.ts`
+- `packages/app/e2e/file-viewer.spec.ts`
+- `packages/app/e2e/terminal-init.spec.ts`
+
+Name each test with a “smoke” prefix so CI can run only this suite if needed.
+
+#### 1) Settings dialog open / switch / close
+
+Purpose: catch regressions in dialog infra, settings rendering, tabs.
+
+Steps:
+
+1. `await gotoSession()`.
+2. Open settings via keybind (preferred for stability): `await page.keyboard.press(`${modKey}+Comma`)`.
+3. Assert dialog visible (`page.getByRole('dialog')`).
+4. Click the "Shortcuts" tab (role `tab`, name "Shortcuts").
+5. Assert shortcuts view renders (e.g. the search field placeholder or reset button exists).
+6. Close with `Escape` and assert dialog removed.
+
+Notes:
+
+- If `Meta+Comma` / `Control+Comma` key name is flaky, fall back to clicking the sidebar settings icon.
+- Favor role-based selectors over brittle class selectors.
+- If `Escape` doesn’t dismiss reliably (tooltips can intercept), fall back to clicking the dialog overlay.
+
+Implementation: `packages/app/e2e/settings.spec.ts`
+
+Acceptance criteria:
+
+- Settings dialog opens reliably.
+- Switching to Shortcuts tab works.
+- Escape closes the dialog.
+
+#### 2) Prompt slash command path: `/open` opens file picker
+
+Purpose: validate contenteditable parsing + slash popover + builtin command dispatch (distinct from `mod+p`).
+
+Steps:
+
+1. `await gotoSession()`.
+2. Click prompt (`promptSelector`).
+3. Type `/open`.
+4. Press `Enter` (while slash popover is active).
+5. Assert a dialog appears and contains a textbox (the file picker search input).
+6. Close dialog with `Escape`.
+
+Acceptance criteria:
+
+- `/open` triggers `file.open` and opens `DialogSelectFile`.
+
+#### 3) Prompt @mention inserts a file pill token
+
+Purpose: validate the most fragile prompt behavior: structured tokens inside contenteditable.
+
+Steps:
+
+1. `await gotoSession()`.
+2. Focus the prompt.
+3. Type `@packages/app/package.json`.
+4. Press `Tab` to accept the active @mention suggestion.
+5. Assert a pill element is inserted:
+   - `page.locator('[data-component="prompt-input"] [data-type="file"][data-path="packages/app/package.json"]')` exists.
+
+Acceptance criteria:
+
+- A file pill is inserted and has the expected `data-*` attributes.
+- Prompt editor remains interactable (e.g. typing a trailing space works).
+
+#### 4) Model selection UI works end-to-end
+
+Purpose: validate model list rendering, selection wiring, and prompt footer updating.
+
+Implementation approach:
+
+- Use `/model` to open the model selection dialog (builtin command).
+
+Steps:
+
+1. `await gotoSession()`.
+2. Focus prompt, type `/model`, press `Enter`.
+3. In the model dialog, use the search field to filter to a deterministic model, e.g. `qwen3-coder`.
+4. Select the first matching model.
+5. Assert dialog closed.
+6. Assert the prompt footer now shows the chosen model name.
+
+Acceptance criteria:
+
+- A model can be selected without requiring provider auth.
+- The prompt footer reflects the new selection.
+
+#### 5) File viewer renders real file content
+
+Purpose: ensure file search + open + file.read + code viewer render all work.
+
+Steps:
+
+1. `await gotoSession()`.
+2. Open file picker (either `mod+p` or `/open`).
+3. Search for `packages/app/package.json`.
+4. Click the matching file result.
+5. Assert the code viewer contains a known substring:
+   - `"name": "@opencode-ai/app"`.
+6. Optionally assert the file tab is active and visible.
+
+Acceptance criteria:
+
+- Code view shows expected content (not just “tab exists”).
+
+#### 8) Terminal init + create new terminal
+
+Purpose: ensure terminal isn’t only “visible”, but actually mounted and functional.
+
+Steps:
+
+1. `await gotoSession()`.
+2. Open terminal with `terminalToggleKey` (currently `Control+Backquote`).
+3. Assert terminal container exists and is visible: `[data-component="terminal"]`.
+4. Assert Ghostty textarea exists: `[data-component="terminal"] textarea`.
+5. Create a new terminal via keybind (`terminal.new` is `ctrl+alt+t`).
+6. Assert terminal tab count increases to 2.
+
+Acceptance criteria:
+
+- Ghostty mounts (textarea present).
+- Creating a new terminal results in a second tab.
+
+---
+
+### CI Stability + Flake Avoidance
+
+These tests run with `fullyParallel: true` in `packages/app/playwright.config.ts`. Keep them isolated and deterministic.
+
+- Avoid ordering-based assertions: never assume a “first” session/project/file is stable unless you filtered by unique text.
+- Prefer deterministic targets:
+  - use `packages/app/package.json` rather than bare `package.json` (multiple hits possible)
+  - filter model search to a specific id/name (e.g. `qwen3-coder`)
+- Prefer robust selectors:
+  - role selectors: `getByRole('dialog')`, `getByRole('textbox')`, `getByRole('tab')`
+  - stable data attributes already present: `promptSelector`, `[data-component="terminal"]`
+- Keep tests local and fast:
+  - do not submit prompts that require real model replies
+  - avoid `page.waitForTimeout`; use `expect(...).toBeVisible()` and `expect.poll` when needed
+- Watch for silent UI failures:
+  - capture `page.on('pageerror')` and fail test if any are emitted
+  - optionally capture console errors (`page.on('console', ...)`) and fail on `type==='error'`
+- Cleanup:
+  - these tests should not need to create sessions
+  - if a test ever creates sessions or PTYs directly, clean up with SDK calls in `finally`
+
+---
+
+### Validation Plan
+
+Run locally:
+
+- `cd packages/app`
+- `bun run test:e2e:local -- --grep smoke`
+
+Verify:
+
+- all new tests pass consistently across multiple runs
+- overall e2e suite time does not increase significantly
+
+---
+
+### Open Questions
+
+- Should we add a small helper in `packages/app/e2e/utils.ts` for “type into prompt contenteditable” to reduce duplication?
+- Do we want to gate these smoke tests with a dedicated `@smoke` naming convention (or `test.describe('smoke', ...)`) so CI can target them explicitly?