Skip to content

Commit 3e16652

Browse files
committed
feat: expand dictation tooling and TypeScript SDK
1 parent a8ef024 commit 3e16652

62 files changed

Lines changed: 4326 additions & 1069 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44
/perf/transcription-corpus
55
/dist
66
/.wrangler
7+
node_modules/
78
__pycache__/
89
*.py[cod]
910
/ios/*DerivedData

‎.markdownlint-cli2.jsonc‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
{
2+
"config": {
3+
"MD013": false
4+
}
5+
}

‎AGENTS.md‎

Lines changed: 26 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -15,8 +15,8 @@ only in debug builds.
1515

1616
## Architecture
1717

18-
- `audio`: `cpal` device enumeration and capture, mono float PCM, and
19-
dropped-chunk accounting.
18+
- `audio`: `cpal` device enumeration and capture, mono float PCM, dropped-chunk
19+
accounting, live selection, and bounded stream recovery.
2020
- `moonshine`: the only Moonshine C adapter and the streaming recognizer.
2121
- `suppression`: the bounded macOS event tap, shortcut suppression, and the
2222
configurable dictation-hotkey state machine.
@@ -31,8 +31,16 @@ only in debug builds.
3131
`opencode2 api get` to discover or start its managed service.
3232
- `parakeet`: the strict-Metal `transcribe.cpp` adapter plus bounded inference,
3333
processing, ordered output, paste, last-result, and meeting-delta workers.
34+
- `transcription`: runtime selection and transactional warm-model activation.
3435
- `transcription_models`: the compiled model catalog, language recommendations,
3536
pinned artifact verification, and lazy installation.
37+
- `transcription_service`: bounded host-audio admission, hostile WAV validation,
38+
normalization, cancellation, and warm inference ownership.
39+
- `local_api`: authenticated loopback discovery or direct-child endpoint
40+
handoff, bounded HTTP parsing, model preparation progress, and raw
41+
transcription routes.
42+
- `sdk/typescript`: Promise and Effect v4 host wrappers for direct-child
43+
lifecycle, authenticated model preparation, and host-audio transcription.
3644
- `paste`: clipboard insertion, continuation joins, and generation-safe
3745
clipboard restoration.
3846
- `selected_text`: bounded selected-text capture through Accessibility without
@@ -57,8 +65,8 @@ only in debug builds.
5765
- `sparkle`: packaged-app-only Sparkle lifecycle and manual update checks.
5866
- `linux_updater`: signed direct-install updates, bounded downloads, atomic
5967
version activation, and restart handoff for user-local Linux installs.
60-
- `events`: append-only NDJSON observations; `dashboard` and the GPUI Activity
61-
pane are read-only projections.
68+
- `events`: append-only NDJSON observations and bounded incremental reading;
69+
`dashboard` and the GPUI Activity pane are read-only projections.
6270
- `app_window`: the production Settings, Modes, Replacements, and opt-in
6371
Commands shell plus developer-only Meetings and Activity panes.
6472
- `dictation_indicator`: the click-through Metal/GPUI capture and processing HUD.
@@ -91,8 +99,8 @@ CoreAudio formats, AppleScript details, or event serialization.
9199
phrases. Dictation and explicit paste shortcuts remain available.
92100
- Unmatched completed speech is ignored and logged.
93101
- The dictation shortcut defaults to Option but supports modifier-only,
94-
modifier-plus-key, and standalone function-key bindings. Capturing a new
95-
binding suspends global matching.
102+
modifier-plus-key, standalone Globe/Fn, and standalone function-key bindings.
103+
Capturing a new binding suspends global matching.
96104
- Hold the shortcut to dictate and release to transcribe. Captures shorter than
97105
300 ms discard. A 450 ms hotkey pre-roll and one-second voice-trigger pre-roll
98106
protect speech onset. Capture has no automatic duration limit; release,
@@ -129,12 +137,13 @@ CoreAudio formats, AppleScript details, or event serialization.
129137
`NotRegistered` and `NotFound` as disabled before registration, represent
130138
`RequiresApproval` with a link to Login Items settings, and poll macOS as the
131139
source of truth instead of persisting a parallel Boolean.
132-
- Voice edit is an explicit second capture path, defaulting to Option-Command.
133-
Snapshot a non-empty selection before recording, use the spoken transcript as
134-
an edit instruction, and replace only after the foreground application and
135-
exact selection are revalidated. Missing, oversized, changed, or unsupported
136-
selections and failed processing must leave the document unchanged. Edit jobs
137-
share normal queueing and cancellation but never update the last dictation.
140+
- Voice Action is an explicit second capture target, defaulting to
141+
Option-Command. Promoting an active Option capture preserves all recorded
142+
audio. Selected text is optional prompt context; inaccessible or empty
143+
selections act as no selection. Use the dedicated OpenCode model and deadline,
144+
return only paste-ready text, and paste at the current focus. Failed, empty,
145+
cancelled, or timed-out actions paste nothing. Voice Action jobs share normal
146+
queueing and cancellation but never update the last dictation.
138147
- Post-processing is optional and best-effort. Empty, failed, or timed-out
139148
processing falls back to the raw local transcript. It applies to Paste and
140149
Send, but not meetings.
@@ -194,6 +203,7 @@ cargo fmt --check
194203
cargo test
195204
cargo clippy --all-targets --all-features -- -D warnings
196205
git diff --check
206+
cd sdk/typescript && bun run check && bun run test && bun run build
197207
```
198208

199209
The supported Linux beta and release host use x86_64 Arch Linux. Install its
@@ -268,9 +278,9 @@ seam once there are two real adapters or a current test requires substitution.
268278
- Finish validated legacy import and a persistent permission-health surface.
269279
- Physically smoke-test custom shortcut injection and last-dictation paste in
270280
release; test the meeting paste shortcut separately in developer builds.
271-
- Move observation writes off the microphone loop, tail bounded event
272-
projections, and distinguish a stale or crashed listener from a normal
273-
persisted `Stopping` state.
281+
- Move observation writes off the microphone loop and distinguish a stale or
282+
crashed listener from a normal persisted `Stopping` state. Read-side event
283+
projections are already bounded and incremental.
274284
- Replace System Events foreground polling with
275285
`NSWorkspace.frontmostApplication`; add a second browser adapter before
276286
generalizing the context interface further.
@@ -281,4 +291,4 @@ seam once there are two real adapters or a current test requires substitution.
281291
- Add edge tests for shutdown during capture, final-transcription failure, and
282292
voice-triggered meeting start/stop.
283293
- Do not generalize the concrete macOS and Linux X11 adapters into hypothetical
284-
platform or plugin seams. Follow `LINUX_PORT_PLAN.md` for the selected target.
294+
platform or plugin seams. Follow `docs/plans/linux.md` for the selected target.

0 commit comments

Comments
 (0)