VOICE / AI / DESKTOP
VoiceWiz
Local Desktop Dictation
A local-first desktop dictation application that turns speech into text without requiring recordings to leave the user's machine.
Visit VoiceWizAt a glance
- Role
- Solo engineer — desktop, native, inference and product surfaces
- Timeline
- 2026 — fork to release
- Platforms
- Linux published · Windows in release validation · macOS build target
- Scope
- Local dictation, four writing modes, retrieval over your own material, and citations
- State
- Version 1.0.0 published for Linux
- Read time
- 7 min
Marketing site — hero
The marketing hero introduces spoken thought to finished work.
Why this screen
The public surface that explains the local-first workflow before install.
Engineering highlights
- Global push-to-talk across any application
- Local speech models from 75 MB to 3 GB with CUDA acceleration
- Four writing modes built on one voice pipeline
- Retrieval over your own codebase, books and reference literature
- Citation-aware research mode with APA/MLA/Chicago/IEEE/Harvard and BibTeX
- Keys only ever leave the machine if you bring your own
Why it exists
Dictation tools usually trade privacy for convenience: the reliable ones send every recording to a cloud service, and the local ones are frequently locked behind a subscription — the project this one forked from charged for its Pro tier and routed value through an account. VoiceWiz asks whether a desktop application can keep the entire speech workflow on the user's machine, run it on hardware they already own, and still be practical to install and use every day.
Where it started
Original objective
A local dictation application that runs on the hardware people already have — an NVIDIA card with 4 GB of VRAM, or a plain CPU — with no account, no telemetry and no first-party cloud. You speak, and text lands wherever your cursor is.
The technique
One invariant spine: push-to-talk → capture → transcription → normalisation → a gate → context → prompt → inference → paste. Modes are branches off that spine rather than separate applications, so adding a mode never means building another pipeline.
Final product
The result is a local-first desktop dictation orchestrator: global push-to-talk, Whisper or Parakeet transcription, normalised text, programming/writing/research/communication modes, retrieval over code and reference material, optional bring-your-own-key providers, and safe paste into the focused application. Linux builds are published today; Windows validation and the macOS release path are separate status tracks.
How the scope changed
Fork and strip
Started from OpenWhispr and removed what contradicted the premise: first-party cloud transcription, billing and usage APIs, sign-in state, and streaming providers. Cloud survived only as bring-your-own-key providers, and the upstream MIT attribution stayed in the licence.
Make local fast enough to use
CUDA detection with automatic CPU fallback, model tiers from 75 MB to 3 GB, voice-activity detection disabled for push-to-talk, and a benchmark script so speed claims are measured instead of asserted.
Programming mode
The first mode: dictation became a paste-ready prompt for a coding agent, assembled from repository metadata, retrieved examples and a strict rejection protocol.
Writing mode
For long-form authors: a style corpus, a continuity anchor that knows where you are in the document, and an explicit protocol that transfers style while forbidding copied content.
Research mode
Born from writing my own thesis: dictation grounded in your own reference literature, with topic gating, citation identity enforcement, five citation styles and BibTeX in both directions.
Communication mode
Subject and body envelopes carrying recipient, role, tone and purpose, scored by a deterministic quality evaluator instead of a vibe check.
Bring your own model
A provider registry spanning local runtimes and hosted APIs, with secrets encrypted at rest in the OS keychain, so quality can be upgraded without the pipeline changing.
How it is built
An Electron shell hosts a React interface over a set of local services: one speech engine, one text pipeline, one retrieval stack and one inference router. Five native sidecars run as supervised child processes with their own ports, memory budgets and watchdogs, and the renderer never talks to a model directly. Everything the pipeline needs resolves locally first and only touches a network when the user has supplied their own key.
Data flow
Modules
Engineering decisions
Keep the primary transcription path fully local
- Why
- Privacy is the premise of the product, so routing audio to a cloud service would contradict it
- Trade-off accepted
- Larger installs, model downloads, and inference bound to the user's hardware
Treat cloud as a bring-your-own-key provider, never a dependency
- Why
- The core workflow must not depend on an account, and quality should be upgradeable without a rewrite
- Trade-off accepted
- Two paths to test, and real quality differences between local and hosted models
Run every model as a supervised sidecar instead of linking it in
- Why
- A crash, a leak or an out-of-memory kill should cost one capability rather than the application
- Trade-off accepted
- Port ranges, process lifecycles, orphan reaping and a watchdog to maintain
Degrade instead of refusing when memory is tight
- Why
- Someone mid-sentence should lose context size, not their dictation
- Trade-off accepted
- Admission control estimates rather than measures, so budgets are deliberately conservative
Make every gate fail open
- Why
- A broken embedding engine must never leave the user worse off than the earlier ungated behaviour
- Trade-off accepted
- Some nonsense slips past a gate instead of being blocked outright
Data decisions
What was hard
Five local models have to share one machine without taking it down
Global push-to-talk and paste behave differently on every desktop
Early output was shallow: small models with no context produced text with no information in it
Gibberish and out-of-context dictation reached the model and returned as invented detail
Citations were the easiest thing for a model to invent
AI engineering
The first pipeline was honest and useless: a small local model, handed a sentence and nothing else, produced text with no information in it. Fixing that was not a model problem — it was a context, structure and verification problem. Every stage since has been about making the input worth generating from and the output provable: structured prompts, retrieved context, per-mode sampling, deterministic gates before and after inference, and a rejection path that hands the user their own words when the model has nothing honest to add.
Local models produced shallow, content-free output
- Control
- A five-section prompt body — role, task, context, constraints, examples — with XML-tagged blocks, context slots per mode, retrieved examples and style windows, and per-mode sampling (0.15 for programming, 0.6 for writing)
- Effect
- Output is a paste-ready artefact rather than a chat reply, and the framing forbids the model from answering conversationally
Gibberish and off-topic dictation reached the model
- Control
- A pre-flight gate of five escalating checks — empty input, an embedding centroid test, an imperative heuristic, file-token detection and a 1,164-term development lexicon — plus a rejection token the model can return
- Effect
- Noise is dropped before a token is spent, and a rejected dictation is pasted back exactly as spoken: no error, no invention
Models invented files, paths and citations
- Control
- A grounding validator that classifies each claim against verified paths, a topic gate for references, and a citation guard that validates identity, attempts one repair, then strips deterministically
- Effect
- Unsupported claims become instructions to verify, and an invalid citation marker never ships
One machine has to host five models at once
- Control
- Admission control against a reserved-memory budget, a context degradation ladder, five-minute idle unloads, and a watchdog that restarts leaky sidecars or sheds them in priority order
- Effect
- Under memory pressure the application gives up capabilities rather than the dictation
Retrieved examples could leak foreign code, and prose style could leak copyrighted text
- Control
- Pattern skeletonisation for external repositories, an example protocol that permits structure but forbids names, facts and phrasing, and a corpus built only from license-screened sources
- Effect
- The model sees shape and style without importing anyone else's content
Local quality is bounded by the user's hardware
- Control
- Model tier presets and a provider registry covering local runtimes and bring-your-own-key APIs, with secrets stored in the OS keychain
- Effect
- Quality can be upgraded without the pipeline changing, while the default path stays on the machine
Budgets
Guardrails
- Every gate fails open: a disabled embedding engine degrades to the earlier behaviour, never to a worse one
- External repositories are skeletonised before they reach a prompt, so foreign paths and imports are redacted
- Style references are anonymised and drawn only from license-screened sources — public domain, CC BY/CC0, or pre-1926
- No first-party cloud: transcription, retrieval and generation are local unless the user supplies their own key
- Secrets are encrypted at rest in the OS keychain, never written to a database or a plaintext env file
- A rejected dictation returns the user's own words, so a refusal is never a dead end
How it is verified
Correctness is a release gate: unit and integration suites over the pipeline, serial end-to-end runs against the real Electron application, prompt-quality gates in continuous integration, and an evaluation harness that binds real public repositories and scores each case pass or fail.
- 141 test suites over guards, prompts, citations, retrieval, storage and providers
- 13 serial Electron end-to-end specs with only the model boundary stubbed
- Continuous integration enforces lint, types, tests, ten locales, dependency and licence checks
- A weekly benchmark canary watches the guard suites for accuracy regressions
- An evaluation harness binds public repositories and records prompts, outputs and pass/fail per case
Where it stands
VoiceWiz is published for Linux as an AppImage and a Debian package, with the product surface at voicewiz.app. It began as a fork that removed a subscription and became an orchestrator: one voice pipeline, four modes, retrieval over the user's own material, and gates that decide when a model should say nothing at all. The repository stays private.
Known limitations
- Local inference is hardware-bound: models have to be downloaded, and speed and context size follow the machine's CPU, GPU and memory
- The macOS build is staged behind Apple Developer credentials — the application targets macOS, but packaging, signing and notarisation are deferred
- Style coverage per domain depends on finding genuinely licensed sources, so breadth is a curation task as much as an engineering one
- The source repository is private; the product, the site and this write-up are the public surface
Next
The near-term work is completing the validated Windows release, finishing the macOS packaging path once credentials are available, and extending the licensed style corpus domain by domain.
In short
VoiceWiz is not a model — it is the orchestration around one. The engineering lives in what reaches the model and what is allowed to leave it: which context gets assembled, which gate runs first, which budget gives way under pressure, and when the honest answer is to hand the user their own words back unchanged.