VoiceWiz

VoiceWiz began as a fork of OpenWhispr. The upstream project already did local transcription well, but it also carried a paid tier — its own locale strings advertise a Pro plan — and routed value through accounts, usage APIs and a first-party cloud. A local speech model that costs a few gigabytes of memory should not require a subscription, so the fork kept the parts that were right and removed the parts that contradicted the premise.

The upstream MIT licence and attribution were kept. This is a fork, not a rebrand: the work removed a business model, it did not claim someone else's engineering.

Every mode in the product is a branch on one spine. That constraint is deliberate: a new mode should never mean a second capture path, a second transcription path or a second prompt. When the pipeline is wrong, it is fixed once.

• Push-to-talk — a global hotkey over whatever application holds focus, with slot hotkeys so one key combination can address a specific project, book or recipient.
• Capture — microphone by default, with system-audio taps where the platform allows one (CoreAudio process taps on macOS, the desktop portal on Linux). Voice-activity detection exists but is disabled for push-to-talk, because silence scanning only adds latency when the user is already telling you when to start and stop.
• Transcription — whisper.cpp on CPU or CUDA, Parakeet through sherpa-onnx, or a bring-your-own-key batch API. GPU presence is probed at startup, and if the CUDA server dies on launch the app falls back to CPU instead of failing the dictation.
• Normalisation — fillers, stutters, spoken punctuation, learned correction pairs, the custom dictionary and numeric formatting, applied in that order and skipped entirely in raw mode.
• Gate — a deterministic pre-flight check that can reject before a single token is spent (section 10).
• Context — repository metadata, retrieved examples, style windows, reference literature or thread context, depending on the mode.
• Assembly — a five-section prompt with hard budgets (section 5).
• Inference — a local runtime by default, or the user's own key if they have supplied one.
• Delivery — paste into the focused application, or fall back to the raw dictation when the model declines.

Running a speech model, an embedding model, a vector database and a generator on a consumer GPU is a resource problem before it is an AI problem. Each model runs as a supervised child process rather than a linked library, so one crash costs a capability instead of the application.

Startup is deliberately sequential: the embedding worker, then the vector store, then the index. One failure disables retrieval rather than the app, and the index can always be rebuilt from SQLite because the database is canonical and the vectors are derived.

Dictation lives or dies on the last hundred milliseconds. Global hotkeys and paste are the two places where desktop platforms disagree most, so each has a backend per environment and a fallback chain.

• Wayland was the sharpest edge: the main-process clipboard uses X11 selections, which native Wayland applications cannot read, so the Linux build depends on a Wayland-native clipboard tool.
• Paste timings are hand-tuned per platform and per fallback, including a longer clipboard-restore delay on KDE Wayland, because restoring too early corrupts what the user had copied.
• A microphone/session monitor distinguishes real capture from a device that silently produced no audio.

The first pipeline was honest and useless: a small local model handed a single sentence produced text with no information in it. The fix was not a bigger model. It was giving the model a role, a task, context, constraints and examples — and then measuring every block that goes into the prompt.

• A framing instruction establishes that the model is a text transformation engine, not a conversational assistant, which removes most of the chatter small local models default to.
• Modes differ by output contract: programming builds a prompt for an external agent, writing and research produce finished prose or notes, communication produces a subject and a body, and direct pastes the dictation untouched.
• Sampling is per mode: 0.15 for programming, where drift is expensive, and 0.6 for writing, where voice matters.
• The ghostwriter template orders its blocks for local models — style reference, prior context, rough dictation, then instructions — because recency matters more to small models than abstract ordering does.

The five-section structure (role, task, context, constraints, examples) and the discipline of tagging untrusted blocks in XML follow the prompting practices Anthropic publishes. Everything built on top of it here — the rejection sentinel, the budgets, the per-mode sampling and the gates in section 10 — is my own work, and no Anthropic document is a dependency of the codebase.

This mode was never meant to write the code. Agents already do that. It exists to write the prompt: to turn a spoken sentence into something an agent can act on, with the repository's own conventions attached, so the developer knows what they asked for and can review it.

• A repository is bound by a blobless, no-checkout clone, restricted to public GitHub URLs, with a bounded commit window. Project type is detected from git objects without checking the tree out.
• Commits are filtered for significance — lockfile-only and changelog-only changes are noise — and deduplicated before they ever become examples.
• Examples are embedded and stored in a vector collection, but SQLite stays canonical: the vector index is derived and can be rebuilt at any time.
• Retrieval is strict: a candidate must clear a similarity floor and there must be more than one of them, otherwise the mode falls back to static, language-appropriate examples or emits no example block at all.
• For external repositories, examples are skeletonised so paths, imports and hunk headers are redacted — the model sees the shape of a change, not someone else's project structure.
• Output is grounded: every path claim is classified as supported, proposed or unsupported, and unsupported claims are rewritten as instructions to verify rather than asserted as fact.

A friend was writing a biography by hand. That is the origin of this mode: speech is faster than typing, but dictation alone loses the voice of the book. Writing mode adds a style reference, a continuity anchor that knows where you are in the document, and hard rules about what may and may not be borrowed.

• Continuity is tracked per mode and slot, with a manual anchor so a user can say 'continue from here' after editing by hand.
• Domain classification is deterministic first, with a learned classifier and a single refinement call only when the deterministic pass abstains.
• Absent or corrupt model artifacts fall back to heuristics rather than failing the request.

This one is personal. Writing a thesis means every claim has to point at something, and matching citations to prose by hand is the most tedious part of it. Research mode dictates into your own reference literature and keeps the citations attached.

• Documents are indexed opt-in, chunked heading-aware with overlap, and retrieved as excerpts above a similarity floor, with the working document excluded so the model cannot cite the thing you are writing.
• A topic gate decides whether a source is relevant enough to cite at all, using both an absolute floor and a relative one; when the embedding engine is unavailable the gate fails open rather than stripping every citation.
• Citations are validated against a source table, repaired once by the model if they are malformed, and then stripped deterministically — an invalid marker never ships.
• References are rendered per style (APA, MLA, Chicago, IEEE, Harvard) from structured metadata, and BibTeX is supported in both directions: import your library, export what you cited.
• Output contracts check that required concepts and section headings actually appear, so a template is not considered satisfied because the prose sounds confident.

The smallest mode by surface and the most unforgiving: an email that reads slightly wrong is worse than one written by hand. The mode carries recipient, recipient role, tone, purpose and thread context into the prompt, returns a structured subject and body, and scores the result across a fixed set of quality dimensions rather than trusting the first draft.

Every mode reuses the same capture, transcription and validation path. The evaluator exists because communication is the one mode where a confidently wrong output has a social cost.

Dictation produces noise: half-sentences, throat-clearing, and sometimes a sentence said to someone else in the room. Sending all of it to a model wastes time and invites invention, so the cheapest possible checks run first.

• Availability is checked before inference, so a missing key or an undownloaded model produces an explanation instead of a prompt printed to the screen.
• Thinking tags are suppressed at the user turn, because several local chat templates only honour the instruction from there.
• Custom system-prompt overrides were removed: they replaced the whole system prompt and bypassed the rejection protocol.

Hybrid search fuses vector results with full-text results rather than choosing between them: vectors for meaning, full text for exact phrases, combined so neither failing alone collapses the feature.

• Unit and integration suites cover the guards, prompt assembly, citation handling, retrieval, storage and every provider adapter.
• End-to-end runs drive the real Electron application serially — single-instance locks and sidecars make parallelism a lie — with only the model boundary and the microphone stubbed.
• Continuous integration enforces lint, types, tests, all ten locales, dependency hygiene and a licence allowlist, so a new dependency cannot quietly change the terms of the product.
• A weekly canary re-runs the guard suites to catch accuracy regressions that no type checker would notice.
• An evaluation harness binds real public repositories, records the exact prompt and output for each case, and scores them pass or fail — the pipeline is judged on artefacts, not impressions.

Engineering debt and genuine limitations are different things, and a reviewer deserves to see the difference. Work actively being implemented is not a limitation of the product; work blocked on credentials is deferred, not broken; and only what survives remediation is a real limitation.