diff options
| author | Yuval Adam <_@yuv.al> | 2026-07-24 12:51:42 +0200 |
|---|---|---|
| committer | Yuval Adam <_@yuv.al> | 2026-07-24 12:51:42 +0200 |
| commit | b509b20e05dcc481b20ae1e8181553e860ca5e34 (patch) | |
| tree | 81661f983c2abfaa555d7c3ecc8f7f85c75c7aa6 | |
| parent | 6608dbd48c3e50d184ab984c56a84e090f49e40b (diff) | |
Tighten project documentation around the working product
| -rw-r--r-- | README.md | 175 | ||||
| -rw-r--r-- | SPEC.md | 32 | ||||
| -rw-r--r-- | docs/IBUS_BAKEOFF.md | 2 |
3 files changed, 80 insertions, 129 deletions
@@ -1,155 +1,104 @@ # Parley -Parley is a planned local GNOME/Wayland dictation tool built around the Parakeet transcription runtime. It will provide a per-user D-Bus daemon, global shortcut, GNOME top-bar extension, transcript history, and tiered text insertion through IBus, clipboard, or an opt-in portal paste backend. +Parley is local, push-to-dictate text input for GNOME on Wayland. Put the caret in a text field, press a shortcut, speak, and press it again. Parley records through PipeWire, transcribes locally with Parakeet, and inserts the result through IBus. -See [SPEC.md](SPEC.md) for the architecture, delivery phases, insertion experiments, and acceptance criteria. +Audio and transcription stay on your machine. Every successful transcript is also saved and copied to the clipboard before insertion is attempted. -## Status - -Phase 1's core and the useful portion of Phase 2 are implemented. The repository contains the standalone frontend, asynchronous per-user D-Bus daemon, `parleyctl`, clipboard copying, prototype notifications, service definitions, and a GNOME Shell 50 top-bar extension. The custom shortcut and insertion experiments are not implemented yet. - -## Development environment +## Platform -The initial target is Arch Linux with GNOME 50 on Wayland. Expected host dependencies include: +Parley deliberately targets one current desktop stack: -- Python 3 and PyGObject -- GNOME Shell and `gnome-extensions` -- IBus -- PipeWire/WirePlumber with PulseAudio compatibility +- fully updated Arch Linux +- GNOME Shell 50 on Wayland +- IBus 1.5.34 +- PipeWire and WirePlumber +- Python 3.14 with PyGObject - FFmpeg -- systemd user services -- XDG Desktop Portal and GNOME's portal backend -- libei/EIS development bindings for the portal-paste experiment - -The existing runtime assets are currently located at: - -```text -../transcribe-parakeet/bin/transcribe-cli -../transcribe-parakeet/models/parakeet-unified-en-0.6b-Q8_0.gguf -``` - -That binary presently requires host CUDA 13 and Intel MKL libraries. Do not copy the large model into this Git repository; development configuration should reference its existing path. - -## Current developer usage - -Set up the development checkout with `uv` and Arch's current system Python (Python 3.14). PyGObject is supplied by Arch, so the uv environment must inherit system site packages: - -```bash -cd ../parley -uv venv --python /usr/bin/python --system-site-packages -uv pip install -e . -``` +- the Parakeet `transcribe-cli` runtime and Unified EN model -Use `uv run` for local Python commands. +Other distributions, desktop environments, X11 sessions, older GNOME releases, and older Python versions are currently unsupported. -Run the standalone frontend (Enter stops and transcribes; Escape cancels): +## Usage -```bash -uv run parley -``` +With Parley installed and the shortcut configured: -By default it finds `transcribe-cli` on `PATH` and the model at `$XDG_DATA_HOME/parley/models/parakeet-unified-en-0.6b-Q8_0.gguf`. During development, each falls back to the corresponding asset in `../transcribe-parakeet`. Transcripts are saved under `$XDG_DATA_HOME/parley/transcripts` (normally `~/.local/share/parley/transcripts`). Paths can be overridden with: +1. Put the caret in an editable field. +2. Press `Super+Shift+D` to start recording. +3. Speak. +4. Press `Super+Shift+D` again. +5. Wait for local transcription and insertion. -- `PARLEY_FFMPEG` -- `PARLEY_TRANSCRIBE_CLI` -- `PARLEY_MODEL` -- `PARLEY_TRANSCRIPT_DIR` -- `PARLEY_DEVICE` -- `PARLEY_LANGUAGE` -- `PARLEY_AUTO_INSERT` -- `PARLEY_INSERTION_MODE` (`clipboard` or `ibus`) +GNOME displays its microphone privacy indicator while recording. The Parley top-bar indicator shows idle, recording, transcribing, and error states and provides start/stop, cancel, copy-last, and open-folder actions. -Run tests without desktop, microphone, or model dependencies: +Useful fallback commands: ```bash -uv run python -m unittest discover -s tests +parleyctl status +parleyctl toggle +parleyctl cancel +parleyctl copy +parleyctl insert ``` -Run the daemon in a terminal for development: +Transcripts are stored in: -```bash -uv run parleyd +```text +~/.local/share/parley/transcripts/ ``` -In another terminal, control it over the session bus: +If insertion is unavailable, the transcript remains saved and on the clipboard, ready for `Ctrl+V`. -```bash -uv run parleyctl status -uv run parleyctl toggle # start recording -uv run parleyctl toggle # stop and transcribe -uv run parleyctl cancel -uv run parleyctl copy -``` +## Runtime assets -For the normal development loop, install an editable local user service once: +Parley expects: -```bash -./scripts/install-dev +```text +~/.local/bin/transcribe-cli +~/.local/share/parley/models/parakeet-unified-en-0.6b-Q8_0.gguf ``` -This writes only to `~/.config/systemd/user` and `~/.local/share/dbus-1/services`; it does not install a system package. Python source changes are visible through the editable uv install. Restart the daemon after changes and follow its journal with: +The current runtime binary requires host CUDA 13 and Intel MKL libraries. The approximately 700 MiB model is not stored in this repository. -```bash -systemctl --user restart parley.service -journalctl --user -u parley.service -f -``` +## Status -Rerun `./scripts/install-dev` after changing entry points or service files. +The working development build currently provides: -Install the development GNOME Shell extension as a symlink into your user extension directory: +- asynchronous recording and local transcription through the `parleyd` user service +- collision-safe transcript persistence and clipboard fallback +- D-Bus control through `parleyctl` +- a GNOME Shell 50 top-bar extension +- a persistent passthrough IBus engine with automatic semantic text insertion +- local systemd user units and an Arch PKGBUILD -```bash -./scripts/install-extension -gnome-extensions info parley@org.parley -``` +End-to-end IBus insertion is confirmed in Firefox and Ghostty. Broader application, keyboard-layout, compose-key, shortcut, and sensitive-field testing is ongoing; results live in [docs/IBUS_BAKEOFF.md](docs/IBUS_BAKEOFF.md). -The top-bar microphone shows daemon state and its menu provides start/stop, cancel, copy-last, and open-folder actions. Source changes are visible through the symlink, but GNOME Shell must reload the extension; toggle it off and on with the Extensions app. A newly installed extension may require one logout/login on Wayland before Shell discovers it. +This is not yet a polished end-user release. See [SPEC.md](SPEC.md) for architecture, security constraints, remaining work, and acceptance criteria. -### IBus insertion experiment +## Development -The development installer also starts a minimal persistent passthrough IBus engine. Enable the experiment with: +Use Arch's system Python through `uv` so the environment can access system PyGObject bindings: ```bash -./scripts/enable-ibus-test -``` - -This saves the current engine, selects Parley, and enables automatic IBus insertion in the daemon. Put the caret in a normal text field and use the recording shortcut twice. Successful transcripts are still saved and copied before insertion. Password and PIN purposes are refused where the application reports them. - -This is a bake-off prototype: while it is enabled, explicitly test normal typing, modifiers, shortcuts, compose/dead keys, layout switching, terminals, browsers, and Electron applications. Restore the previous engine and clipboard-only mode with: - -```bash -./scripts/disable-ibus-test +uv venv --python /usr/bin/python --system-site-packages +uv pip install -e . +./scripts/install-dev ``` -## Arch Linux package - -The local PKGBUILD installs `parleyd` and the other executables under `/usr/bin`, the systemd **user** unit under `/usr/lib/systemd/user`, and the session D-Bus service under `/usr/share/dbus-1/services`. It builds from the latest committed state of this checkout: +Run tests and inspect the daemon: ```bash -git status # commit the source you want packaged -cd packaging/arch -makepkg -si -systemctl --user daemon-reload -systemctl --user enable --now parley.service +uv run python -m unittest discover -s tests +systemctl --user restart parley.service +journalctl --user -u parley.service -f ``` -The package deliberately does not bundle the runtime or model. Put `transcribe-cli` on `PATH` (for example at `~/.local/bin/transcribe-cli`) and the model at `~/.local/share/parley/models/parakeet-unified-en-0.6b-Q8_0.gguf`, or set the documented environment overrides. User-manager environment overrides can be placed in `~/.config/environment.d/parley.conf`; log in again after changing them. - -The prototype daemon uses `xclip` through GNOME's XWayland clipboard bridge and `notify-send` for notifications. - -## Planned first milestone - -1. Extract recording, transcription, and persistence from the TTY frontend. -2. Preserve a standalone CLI using the shared core. -3. Add automated tests that do not require GNOME, a microphone, or the model. -4. Implement the user daemon, D-Bus interface, and `parleyctl toggle` workflow. +Development helpers for the Shell extension and IBus experiment are in [`scripts/`](scripts/). The local Arch package recipe is in [`packaging/arch/`](packaging/arch/). -## Design principles +## Principles -- Local-only transcription -- Never lose a successful transcript because insertion failed -- Keep model/audio work outside GNOME Shell -- Respect Wayland's security model -- Use IBus only if the passthrough/activation feasibility tests prove reliable -- Keep clipboard-only operation as the safe fallback -- Make synthetic portal paste explicit and revocable +- local-only audio and transcription +- visible recording state +- save and copy before attempting insertion +- never destabilize GNOME Shell with model or audio work +- refuse known password and PIN fields +- preserve a clipboard fallback when an application rejects IBus insertion @@ -14,13 +14,13 @@ The normal interaction is: 6. Parley inserts through the configured insertion backend. 7. Parley also saves the transcript and copies it to the clipboard. -The initial target environment is: +The supported target environment is deliberately narrow: -- Arch Linux +- fully updated Arch Linux - GNOME Shell 50 on Wayland - IBus 1.5.34 - PipeWire/WirePlumber with PulseAudio compatibility -- Python 3 with PyGObject +- Python 3.14 with PyGObject - The existing `transcribe-cli` binary and Parakeet Unified EN model ## 2. Goals @@ -43,18 +43,18 @@ The initial target environment is: - Cloud transcription is out of scope. - The Shell extension will not load the model, capture audio, or run transcription. -## 4. Existing transcription pipeline +## 4. Transcription pipeline -The source prototype lives in the sibling `transcribe-parakeet` project. It currently: +The terminal-independent Python core now: - records the default input with FFmpeg's `pulse` input at mono 16 kHz PCM; -- invokes a bundled `transcribe-cli` executable; +- invokes `transcribe-cli` as a cancellable child process; - uses `parakeet-unified-en-0.6b-Q8_0.gguf` (approximately 698 MiB); -- reads JSONL batch output; -- saves uniquely named text files; and -- uses terminal Enter/Escape handling to stop or cancel. +- parses JSONL batch output; +- persists collision-safe UTF-8 transcript files atomically; and +- exposes the same core to the standalone terminal frontend and daemon. -The runtime binary currently links to CUDA 13 and Intel MKL libraries. Packaging must either declare those host dependencies or later supply a more portable build. +Development can use runtime assets from the sibling `transcribe-parakeet` checkout. Installed assets default to `~/.local/bin/transcribe-cli` and `~/.local/share/parley/models/`. The current runtime binary links to CUDA 13 and Intel MKL libraries. Packaging must declare those host dependencies or later supply a more portable build. ## 5. Architecture @@ -239,7 +239,7 @@ Regardless of selected backend, a successful transcript is saved and copied to t ### 7.1 Primary candidate: IBus `commit_text` -Implement a minimal IBus dictation engine. When it owns the focused input context, it commits the transcript as semantic text. +A minimal persistent passthrough IBus engine is implemented. When it owns the focused input context, it commits the transcript as semantic text. Initial end-to-end insertion and ordinary passthrough typing work in Firefox and GNOME Console; the full application and input-behavior matrix is still required. Benefits: @@ -366,7 +366,7 @@ Secrets and portal restore tokens must not be exposed over D-Bus or written to l ## 12. Delivery phases -### Phase 1 — Refactor +### Phase 1 — Refactor (complete) - Create the terminal-independent core. - Implement start/stop/cancel recording operations. @@ -376,7 +376,7 @@ Secrets and portal restore tokens must not be exposed over D-Bus or written to l Exit criterion: CLI behavior works through the shared core without core TTY dependencies. -### Phase 2 — Daemon, D-Bus, and early hotkey +### Phase 2 — Daemon, D-Bus, and early hotkey (complete for development) - Implement the GLib daemon and state machine. - Add the systemd user unit. @@ -387,7 +387,7 @@ Exit criterion: CLI behavior works through the shared core without core TTY depe Exit criterion: hotkey-driven recording/transcription reliably ends with a saved transcript on the clipboard. -### Phase 3 — Insertion bake-off +### Phase 3 — Insertion bake-off (in progress) Build two deliberately small prototypes: @@ -413,7 +413,7 @@ For portal paste, test first authorization, restored sessions, logout/login, per Exit criterion: record results, choose the default backend, and document unsupported applications. If IBus passthrough or activation is flaky, use clipboard as the safe default and retain portal paste as opt-in. -### Phase 4 — GNOME Shell extension +### Phase 4 — GNOME Shell extension (baseline implemented) - Implement the top-bar indicator and menu. - Synchronize state over D-Bus. @@ -481,7 +481,7 @@ Runtime and desktop dependencies should be abstracted so core tests do not requi These must be resolved by prototypes or packaging work rather than assumption: -- whether IBus persistent passthrough or temporary activation is reliable enough to be default; +- whether the promising IBus persistent passthrough results remain reliable across the full test matrix; - exact compatibility of IBus and portal paste across the target application's real versions; - GNOME portal restore-token lifetime and user experience across login sessions; - final reverse-DNS application/D-Bus identifier; diff --git a/docs/IBUS_BAKEOFF.md b/docs/IBUS_BAKEOFF.md index 3dcdf24..947a9d2 100644 --- a/docs/IBUS_BAKEOFF.md +++ b/docs/IBUS_BAKEOFF.md @@ -6,11 +6,13 @@ Environment for the initial test round: - GNOME Shell 50.3 on Wayland - IBus 1.5.34 - Firefox 152.0.6 +- Ghostty terminal - Parley persistent passthrough engine | Application | Semantic insertion | Ordinary typing | Notes | | --- | --- | --- | --- | | Firefox | Pass | Pass (basic) | First end-to-end two-toggle insertion succeeded. | +| Ghostty | Pass | Pass (basic) | End-to-end insertion succeeded at the terminal prompt. | | GTK 4 editor | Not tested | Not tested | | | Chromium | Not tested | Not tested | | | LibreOffice | Not tested | Not tested | | |
