+
+
+
+ Local voice dictation for Windows, made for developers who speak English with an accent. +
+ +
+ 
+ 
+ Install · + Guide · + Report a mishearing +
-Hold a key, talk, let go: the words paste into the window you were working in. Or tap the -pill to hand them to the agent CLI you already use — shaped into a prompt for your -project, or asked as a question and answered above the pill. - -Speech recognition runs on your machine. No API key. + -**English only · three dependencies · Windows in full, macOS and Linux in -[Lite](#flow-lite-macos-and-linux)** +Hold Ctrl+Win, say what you want to write, and let go. Flow pastes the +words wherever you were typing: your editor, a terminal, Slack, a browser tab. If it heard +something wrong, hold the keys again and say *"change Tuesday to Thursday"* or *"scratch +that"*, and it fixes the text it just pasted. - +Flow sits in a small pill at the bottom of the screen, and the pill has two more modes. In +Refine, what you say is rewritten into a proper prompt for your project. In Ask, your question goes to the agent +CLI you already use (`codex` or `claude`) and the answer is read back to you. -*Flow Home and the real pill against a scripted session, recorded by [`scripts/home_reel.py`](scripts/home_reel.py).* +I made Flow for people whose English has an accent that speech recognition keeps tripping +over, often enough that they give up and type. Spanish, Indian, Russian and Japanese +speakers are who it's designed and benchmarked for. Recognition runs on your own machine, +there's no account or API key, and your audio never leaves your PC. ## Install -```bash -uv tool install git+https://github.com/samartomar/flow -flow -``` +### Windows + +You need Windows 10 or 11 and a microphone. Download +[`flow-windows-x64.zip`](https://github.com/samartomar/flow/releases/latest/download/flow-windows-x64.zip), +unzip it anywhere and run `flow.exe`. No Python required. -You need [`uv`](https://docs.astral.sh/uv/), which fetches Python and the three -dependencies itself, and a microphone. No Python? On Windows, [Scoop](https://scoop.sh) -installs the release build: +> [!NOTE] +> The zip isn't code-signed, so the first launch shows a SmartScreen warning. Click +> **More info**, then **Run anyway**. + +Or get the same build through [Scoop](https://scoop.sh), which makes updating one command: ```powershell scoop bucket add flow https://github.com/samartomar/scoop-flow scoop install flow/flow # flow/ matters: Scoop's main bucket has a different "flow" ``` -Or download -[`flow-windows-x64.zip`](https://github.com/samartomar/flow/releases/latest/download/flow-windows-x64.zip), -unzip it and run `flow.exe`. It is unsigned, so the first launch shows SmartScreen: -**More info → Run anyway**. +If you already use [uv](https://docs.astral.sh/uv/), you can install from source instead. +It fetches Python and Flow's three dependencies (faster-whisper, sounddevice and numpy): -An agent CLI — `codex` or `claude` on PATH, signed in — is optional. It adds Refine, Ask -and rewrites; dictation and spoken corrections work without it. -[The guide](docs/guide.md#install) has the rest. +```bash +uv tool install git+https://github.com/samartomar/flow +flow +``` + +The speech models aren't part of any of these. Flow downloads them on first run and shows +you the progress. + +### macOS and Linux -## Flow Lite (macOS and Linux) +Flow Lite, the version for macOS and Linux, is still in progress. Install it with uv, as +above. It has the same recognition, corrections and agent features, but it can't paste into +other apps or listen for global hotkeys. You hold the pill to talk, and Send copies the +text for you to paste. The microphone is the only permission it needs. There's no Mac or +Linux download yet. [More about Lite](docs/product.md#flow-lite--the-portable-body). -Off Windows, Flow runs as **Lite**: the same recognition, corrections and agent loop, but -Send copies the draft and you paste it yourself. No injection and no global hotkeys, so -there is nothing to grant but the microphone — hold the pill to talk. There is no Mac or -Linux download; install with uv. [What Lite is, and is not](docs/product.md#flow-lite--the-portable-body). +### The agent CLI (optional) -## The loop +Refine and Ask need `codex` or `claude` on your PATH, already signed in. Dictation and +spoken corrections work without either. [The guide](docs/guide.md#install) has the details. -1. **Hold `ctrl+win`** (or the pill) and talk. -2. **Let go.** The words paste where you were typing. -3. **Said it wrong?** Hold again and say *"change Tuesday to Thursday"* or *"scratch - that"*. Flow fixes its last paste until you type or click there. -4. **Wrong window?** Click the right one and press **Paste last** (`alt+shift+Z`). -5. **Want an answer?** Hold **`ctrl+alt+win`** and ask. +## Using it -On first run, Flow Home walks you through the microphone, the model download, a -45-second tuning to your voice, and whether to keep a history. Every step can be skipped. +1. Hold Ctrl+Win (or hold the pill) and talk. +2. Let go, and the text is pasted where your cursor was. +3. Heard you wrong? Hold again and say *"change Tuesday to Thursday"* or *"scratch that"*. + Flow can fix its last paste until you type or click there. +4. Pasted into the wrong window? Click the right one and press + Alt+Shift+Z. +5. Want an answer rather than text? Hold Ctrl+Alt+Win and + ask. -## Three modes, one tap +Those last two are Wispr Flow's keys on Windows as well, so if you've used it, your hands +already know them. -- **Type** (white): speak, let go, it pastes. -- **Refine** (gold): `codex` or `claude` turns what you said into a prompt for your - project. Nothing pastes until you press Send. -- **Ask** (violet): the question goes to your agent CLI, and the answer appears above the - pill and is read aloud. **Continue in Flow** opens the conversation in a window. +The first run walks you through the microphone, the model download, a 45-second tuning to +your voice, and whether to keep a history. You can skip any step. -Point Refine and Ask at a project (Flow Home ▸ Settings ▸ Workspaces, or `--cwd`) and the -answers are about your code. The Classic pill — a draft you correct by voice before -sending — is one switch away in Settings. +## Type, Refine, Ask + +Tap the pill to cycle through its three modes. Its colour tells you which one it's in. + +| Mode | Pill | When you let go | +|:---|:---|:---| +| Type | white | The words are pasted. | +| Refine | gold | `codex` or `claude` turns them into a prompt for your project. Nothing is pasted until you press Send. | +| Ask | violet | The question goes to your agent CLI. The answer appears above the pill and is read aloud, and **Continue in Flow** opens the conversation in a window. | + +Point Refine and Ask at a project folder (Flow Home ▸ Settings ▸ Workspaces, or `--cwd`) +and the answers are about your code. If you'd rather correct a draft by voice before +anything is sent, switch to the Classic pill in Settings. ## Flow Home -Right-click the pill ▸ **Open Flow**, or run `flow --home`: one window for everything that -isn't talking. Speech models download with progress and swap without a restart; the -microphone, shortcuts, workspaces, agent CLI and reading voice are all set there. -**Voice** tunes Flow to you, **History** shows what you dictated (if you keep one), and -**Conversations** is Ask in a window. The pill never grows a setting. +Everything that isn't talking happens in one window, the one in the GIF above. Right-click +the pill and choose **Open Flow**, or run `flow --home`. Speech models download there with +a progress bar and swap in without a restart, and it's where you set the microphone, +shortcuts, workspaces, agent CLI and the voice that reads answers. Voice tunes Flow to how +you speak, History shows what you've dictated if you chose to keep it, and Conversations +is Ask in a full window. -## Who it is for +## Privacy -Developers who speak English as a second language, with a strong accent. Spanish, Indian, -Russian and Japanese L1 speakers anchor the design and the benchmarks -([why](docs/product.md)). +By default nothing leaves your PC. Audio, recognition, corrections, your word list and your +voice profile all stay local, and no API key is read, stored or passed anywhere in the code. -## What leaves your machine +A few things do go out, and only when you use them: -- **Nothing, by default.** Audio, recognition, corrections, your lexicon and your profile - stay local. No API key is read, stored or passed anywhere in this codebase. -- **Whatever you hand an agent CLI** goes to its cloud: the draft or the question, plus - your workspace path if you set one. -- **History** is kept only if you choose to, in `~/.flow/history.jsonl`. -- **Pasting** goes through the Windows clipboard for a moment, where clipboard managers +- A draft you refine or a question you ask goes to that agent CLI's cloud, along with your + workspace path if you set one. +- Pasting puts the text on the Windows clipboard for a moment, where clipboard managers and cloud sync can see it. -- **A Microsoft voice**, if you choose one, sends each spoken answer's text to Microsoft. - Every other voice is local. +- If you choose one of Microsoft's natural voices to read answers, the text of each answer + is sent to Microsoft. Every other voice runs locally. -Flow Home is served on `127.0.0.1`, only to the window Flow opens, with a new token each -launch. [The exact boundary](docs/architecture.md#what-leaves-the-machine). +History is kept only if you say so, in `~/.flow/history.jsonl`. Flow Home is served on +`127.0.0.1` to the window Flow opens and nothing else, with a new token every launch. +[The exact boundary](docs/architecture.md#what-leaves-the-machine) is in the architecture +doc. ## Known limits -- **Corrections must be phrased as commands.** *"delete the bit about the standup"* works; - *"I feel it should not contain the summary"* is typed as words. -- **A personal lexicon cuts both ways.** It recovers 27–34% of the rare words the model - missed, and makes WER 14–38% relatively worse on speech without them (EdAcc, - `small.en`). Add the words you say often. -- **Live text refreshes about once a second**, is shown dimmed, and is always replaced by - the final text. -- **Accuracy depends on the voice.** Flow Home ▸ Voice ▸ **How well Flow hears you** - measures yours in five sentences. +- Flow does English only. +- Corrections have to be phrased as commands. *"delete the bit about the standup"* works; + *"I feel it should not contain the summary"* gets typed as words. +- Your personal word list cuts both ways. It recovers 27–34% of the rare words the model + missed, but raises the error rate by 14–38% on speech that doesn't contain them + (measured on EdAcc with `small.en`). Add the words you say often, not every word you know. +- The live preview refreshes about once a second and is shown dimmed. The final text + always replaces it. +- How accurate it is depends on your voice. Flow Home ▸ Voice ▸ **How well Flow hears you** + measures yours with five sentences. -All of them are in [the guide](docs/guide.md#known-limitations). +The rest are in [the guide](docs/guide.md#known-limitations). ## Docs -| | | -|---|---| -| [guide](docs/guide.md) | the manual: every flag, spoken command, mode and setting | -| [product](docs/product.md) | who Flow is for, the P1–P9 requirements, the non-goals | -| [architecture](docs/architecture.md) | data flow, threads, events, and the tuned constants with their measurements | -| [roadmap](docs/roadmap.md) | what is left, and the accent benchmark that measures it | -| [development](docs/development.md) | layout, tests, harnesses, and building a release | -| [decisions](docs/decisions.md) | why things are the way they are | +[The guide](docs/guide.md) is the manual: every flag, spoken command, mode and setting. +[product.md](docs/product.md) says who Flow is for and what it deliberately won't do, +[architecture.md](docs/architecture.md) how it fits together and the measurements behind +its tuned numbers, [roadmap.md](docs/roadmap.md) what's left and the accent benchmark that +tracks it, [development.md](docs/development.md) how to work on it and cut a release, and +[decisions.md](docs/decisions.md) why things are the way they are. ## Contributing -Especially welcome: anyone who speaks accented English and can tell me where Flow -mishears them. That is the one thing I cannot measure alone. +The most useful thing you can send me is a mishearing. If you speak English with an +accent, [open an issue](https://github.com/samartomar/flow/issues/new) with what you said +and what Flow typed. That's the one thing I can't measure on my own. + +Pull requests are welcome for anything else. To run it from a clone: ```bash git clone https://github.com/samartomar/flow && cd flow uv sync && uv run flow # run it -uv run python -m unittest discover -s tests # ~3,200 tests, ~90 s, no mic needed +uv run python -m unittest discover -s tests # ~3,200 tests, about 90 s, no mic needed ``` -Open an [issue](https://github.com/samartomar/flow/issues) for a mishearing (what you -said, and what appeared) or a pull request for anything else. +The GIF at the top is recorded by [`scripts/home_reel.py`](scripts/home_reel.py) from a +scripted session, so it can be re-shot whenever the app changes. + +## License -MIT licensed. +MIT. Flow is built on [faster-whisper](https://github.com/SYSTRAN/faster-whisper) and +OpenAI's [Whisper](https://github.com/openai/whisper) models, and reads answers aloud with +[Piper](https://github.com/OHF-Voice/piper1-gpl) if you add it. diff --git a/docs/flow.gif b/docs/flow.gif index 1bbb5b9..62ab649 100644 Binary files a/docs/flow.gif and b/docs/flow.gif differ diff --git a/docs/guide.md b/docs/guide.md index 7749d78..70cedd4 100644 --- a/docs/guide.md +++ b/docs/guide.md @@ -49,7 +49,7 @@ asking and Mac hardware to measure on. **Off Windows, Flow starts in Lite** rather than refusing. Everything that does not need hands is there — both decoder tiers, the correction loop, the lexicon, the profile, the polish, the thread, converse mode — and Send copies the draft for you to paste. See -[README](../README.md#flow-lite-macos-and-linux) for the short version and +[README](../README.md#macos-and-linux) for the short version and [product.md](product.md#flow-lite--the-portable-body) for the definition and the fence. `--lite` runs the same code on Windows. diff --git a/flow/home/static/app.js b/flow/home/static/app.js index 9a1a070..a46e8be 100644 --- a/flow/home/static/app.js +++ b/flow/home/static/app.js @@ -360,17 +360,16 @@${onGpu ? (sp.compute_types.includes("float16") ? "This card has fast half-precision; Flow runs int8 on it." : "This card has no fast half-precision, so int8 is its fast path. Flow picks it for you.") : esc(sp.why_cpu || "The CPU runs the smaller models in time; the large ones need a GPU.")}
Now: ${esc(finalNow)} for the words that get pasted, ${esc(partialNow)} for the live preview.
${sp.swappable ? `Local voice dictation for Windows, made for developers who speak English with an accent.
+