Skip to content

Latest commit

 

History

History
481 lines (344 loc) · 26.9 KB

Documentation.md

File metadata and controls

481 lines (344 loc) · 26.9 KB

FlyCoder 0.3 documentation

FlyCoder is a local coding model that runs in Ollama, tuned for MacBooks and Mac minis. This documentation covers installation, usage, settings, the benchmark, publishing and troubleshooting. For a quick overview, see the README.

Contents

  1. What FlyCoder is
  2. Requirements
  3. Installation
  4. Choosing a variant
  5. Usage
  6. Settings
  7. Benchmark
  8. Publishing on ollama.com
  9. Troubleshooting
  10. Development
  11. Known limitations
  12. Version history
  13. Licenses

1. What FlyCoder is

FlyCoder 0.3 is an Ollama profile: an open base model, the sampling settings its authors recommend for code, a large context window and a short system prompt dedicated to code. You use it like any Ollama model: in the terminal, through the API, or behind a coding agent.

It comes in three sizes, each published as its own model on ollama.com:

Model Base Weights Mac Context
Tromset/flycoder0.3 Qwen3.5 9B 6.6 GB 16 GB 65,536 tokens
Tromset/flycoder0.3fast Qwen3.5 4B 3.3 GB 8 GB 32,768 tokens
Tromset/flycoder0.3pro Qwen3.8 27B 18 GB 32 GB or more 65,536 tokens

FlyCoder does not retrain the weights. The gains come from the choice of base and settings, and are measured by the repository's benchmark (section 7).

2. Requirements

Item Minimum Note
Computer Any Mac (Apple Silicon or Intel), Linux or Windows The published models are GGUF: they run everywhere
Memory 8 GB (fast), 16 GB (flycoder0.3), 32 GB (pro) The installer picks for you
Ollama 0.30 (flycoder0.3, fast), 0.32.12 (pro) ollama --version; update from https://ollama.com/download
Node.js 22 Only for FlyBrain, the benchmark and the tests
Python 3 The one shipped with macOS Only for the benchmark

3. Installation

3.1 From ollama.com

One command is enough:

ollama run Tromset/flycoder0.3       # Macs with 16 GB
ollama run Tromset/flycoder0.3fast   # Macs with 8 GB
ollama run Tromset/flycoder0.3pro    # Macs with 32 GB or more

To call it simply flycoder: ollama cp Tromset/flycoder0.3 flycoder.

3.2 One-command installer

curl -fsSL https://raw.githubusercontent.com/Tromset/flycoder/main/install.sh | sh
ollama run flycoder

The installer:

  1. checks that Ollama is installed, running and recent enough for the chosen variant;
  2. detects the system and the memory;
  3. picks flycoder0.3pro from 30 GB of memory up, flycoder0.3 from 15 GB, otherwise flycoder0.3fast;
  4. downloads the base model, creates the variant and copies it to the short name flycoder;
  5. installs FlyBrain's experts and server (section 5.5), except with pro.

Options, placed after sh -s -- when the script is read from curl:

Option Effect
--fast installs flycoder0.3fast
--pro installs flycoder0.3pro
--all installs all three; flycoder points to flycoder0.3
--mlx uses Ollama's MLX engine on Apple Silicon (faster, needs Ollama 0.31) instead of the portable GGUF weights
--no-brain skips the FlyBrain experts (flycoder0.3:router, flycoder0.3:lite)

From a copy of the repository, sh install.sh uses the local Modelfiles. The FLYCODER_REF variable picks another GitHub branch or tag to download them from.

3.3 By hand

git clone https://github.com/Tromset/flycoder.git && cd flycoder
ollama create flycoder0.3 -f Modelfile
ollama create flycoder0.3fast -f Modelfile.fast
ollama create flycoder0.3pro -f Modelfile.pro

3.4 Checking the installation

ollama list                             # flycoder0.3, flycoder...
ollama show flycoder0.3                 # base, quantization, capabilities
ollama show flycoder0.3 --parameters    # applied settings
ollama run flycoder0.3 "Write a Python function that reverses a string."
ollama ps                               # memory used, GPU share, allocated context

On a Mac, the PROCESSOR column of ollama ps should show 100% GPU. A CPU/GPU split means memory is short (section 9).

3.5 Uninstalling

ollama rm flycoder flycoder0.3 flycoder0.3fast flycoder0.3pro flycoder0.3:lite flycoder0.3:router
ollama rm qwen3.5:9b qwen3.5:4b qwen3.5:2b qwen3.5:0.8b qwen3.8:27b   # bases, if no other model uses them

4. Choosing a variant

Take the largest variant your memory allows:

  • flycoder0.3 (Qwen3.5 9B, 16 GB Macs): the default. With its 64K context it uses about 9 GB in memory (measured: 8.8 GB).
  • flycoder0.3fast (Qwen3.5 4B, 8 GB Macs): about twice as fast per token, a lower pass rate on hard problems.
  • flycoder0.3pro (Qwen3.8 27B, 32 GB Macs or more): the latest Qwen generation, set up for hard bugs, algorithms and coding agents (section 6.5).

Every Modelfile declares the Ollama version it needs (REQUIRES): an older Ollama names the version to install instead of failing to load the model.

All three bases support tool calling, thinking and images.

5. Usage

5.1 In the terminal

ollama run flycoder0.3

Type your request, then press Enter. For multi-line text, wrap it in """. Useful commands in a conversation:

Command Effect
/set nothink / /set think turns thinking off or back on
/set parameter num_ctx 131072 enlarges the context for the session
/set system "…" replaces the system prompt for the session
/show parameters shows the current settings
/clear clears the conversation
/bye quits

Launch options: --think=false (immediate answer), --hidethinking (hides the thinking but keeps it), --verbose (shows the tokens-per-second rate after each answer).

Giving the model a file:

ollama run flycoder0.3 "Find the bugs in this file and suggest a fix: $(cat src/parser.py)"

Giving it an image (screenshot, diagram): write its path in the request, for example ollama run flycoder0.3 "Write the HTML for this mockup: ./mockup.png".

5.2 Ollama REST API

curl http://localhost:11434/api/chat -d '{
  "model": "Tromset/flycoder0.3",
  "messages": [{"role": "user", "content": "Write a JavaScript function debounce(fn, ms)."}],
  "think": false,
  "stream": false
}'
  • "think": true or false turns thinking on or off; the thinking comes back separately in message.thinking.
  • "stream": true (the default) sends the answer as it is generated, as JSON lines.
  • "options": {"num_ctx": 131072} changes a setting for one request.
  • A {"role": "system", ...} message replaces FlyCoder's system prompt for that conversation.

5.3 OpenAI-compatible API

Ollama also exposes http://localhost:11434/v1. Example in Python with the openai package:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")
reply = client.chat.completions.create(
    model="Tromset/flycoder0.3",
    messages=[{"role": "user", "content": "Write a pytest test for a slugify function."}],
)
print(reply.choices[0].message.content)

5.4 Coding agents and editors

ollama launch sets up and opens an agent with FlyCoder:

ollama launch claude --model Tromset/flycoder0.3      # Claude Code
ollama launch codex --model Tromset/flycoder0.3       # Codex
ollama launch opencode --model Tromset/flycoder0.3    # OpenCode

ollama launch --help lists the other integrations. Ollama recommends at least 64,000 tokens of context for agents: flycoder0.3 and flycoder0.3pro have 65,536 out of the box. For agents, flycoder0.3pro gives the best results if your Mac can hold it.

5.5 FlyBrain: the router that lowers RAM use

FlyBrain is a small, dependency-free Node 22 server that sits in front of Ollama on port 11435. It answers to the names flycoder0.3 (and flycoder) and flycoder0.3fast, picks an expert for each request and keeps only one expert in memory. Other models, flycoder0.3pro included, and other routes pass through unchanged.

You call Simple request Hard request, tools (agents) or long context Unclear request
flycoder0.3 flycoder0.3fast (4B), with thinking flycoder0.3 (9B), with thinking the flycoder0.3:router micro-model (Qwen3.5 0.8B) decides
flycoder0.3fast flycoder0.3:lite (Qwen3.5 2B) flycoder0.3fast (4B) the 4B (no micro-model)
node ~/.flycoder/brain/flybrain.mjs          # copied by install.sh; or npm run brain in this repository
node brain/flybrain.mjs --prefix Tromset/   # with the models pulled from ollama.com

With --prefix Tromset/, first pull the experts: ollama pull Tromset/flycoder0.3:router, Tromset/flycoder0.3:lite, Tromset/flycoder0.3fast and Tromset/flycoder0.3.

Options: --port (11435), --ollama (http://127.0.0.1:11434), --prefix, --max-expert auto|full|fast. With auto, the default, a machine with less than 16 GB caps flycoder0.3 at the 4B.

How a request is routed:

  1. Rules (free): a request is hard if it carries tools (agents), more than 6,000 characters, several code blocks, a bulleted specification, or a word such as debug, algorithm, optimize, security, error. It is simple if it is a short question without code ("what is", "how do I", "explain"…). The rules recognize these cues in both English and French.
  2. The micro-model for the rest. It answers {"level":"simple"} or {"level":"hard"} in an enforced JSON format. If it fails, takes longer than 8 s or answers anything else, the request goes to the large model. It is not called when the large model is already loaded.
  3. Conversation memory: a conversation keeps its expert and can only move up to the stronger one.
  4. A single expert: before loading another expert, FlyBrain waits for the running answers to finish, then unloads the old one. It also unloads the micro-model before the large model.

On 60 labelled requests (the 20 bench problems plus tests/fixtures/route-prompts.json), the rules and the micro-model route 59 correctly; no hard request goes to the small model. Measure it on your own requests with node brain/eval-router.mjs.

Connecting tools:

OLLAMA_HOST=127.0.0.1:11435 ollama run flycoder0.3
ANTHROPIC_BASE_URL=http://127.0.0.1:11435 ANTHROPIC_AUTH_TOKEN=ollama ANTHROPIC_API_KEY="" claude --model flycoder0.3
curl http://127.0.0.1:11435/v1/chat/completions -d '{"model":"flycoder0.3","messages":[{"role":"user","content":"Hello"}]}'

Each response carries the x-flybrain-expert and x-flybrain-reason headers, and FlyBrain's terminal prints one line per decision.

6. Settings

6.1 What the Modelfiles set

Parameter flycoder0.3 fast pro Why
num_ctx 65,536 32,768 65,536 Otherwise Ollama limits machines under 24 GB of memory to 4,096 tokens
temperature 0.6 0.6 0.6 Qwen's settings for coding in thinking mode
top_k, top_p, min_p 20, 0.95, 0 same same Same
presence_penalty, repeat_penalty 0, 1 same same Same
draft_num_predict — — 4 Multi-token prediction: Qwen3.8's built-in draft layer proposes 4 tokens per pass

Avoid lowering the temperature: Qwen advises against it, and FlyCoder 0.1 (temperature 0.2) produced looping answers.

6.2 Enlarging the context

The bases support up to 262,144 tokens. For one session: /set parameter num_ctx 131072. For a permanent model, create a variant:

printf 'FROM Tromset/flycoder0.3\nPARAMETER num_ctx 131072\n' > Modelfile.128k
ollama create flycoder0.3:128k -f Modelfile.128k

Qwen3.5 and Qwen3.8 use a hybrid attention where only one layer in four keeps a growing cache, so a large context costs little memory: flycoder0.3 uses 8.8 GB in total at 65,536 tokens (measured with ollama ps). For flycoder0.3pro, the GGUF header gives 16 cache layers of 4 KB per token each: about 4.4 GB at 65,536 tokens, 22 GB in total with the weights (computed, not measured). On a 32 GB Mac, if ollama ps shows a CPU share, use /set parameter num_ctx 32768. Check with ollama ps that the model stays at 100% GPU after a change.

6.3 Thinking

Thinking is on by default: the model reasons before answering, which clearly improves code on hard problems but lengthens the wait. Turn it off for simple questions (--think=false, /set nothink, or "think": false in the API).

6.4 System prompt

The system prompt of flycoder0.3, fast and lite (ollama show flycoder0.3 --system) asks the model to:

  • keep the requested names, signatures and files exactly;
  • deliver complete, runnable code, without "TODO";
  • handle edge cases;
  • prefer the standard library and never invent APIs;
  • choose a suitable algorithm;
  • check its code once more before answering, and give a single final version;
  • show only the changed parts of an existing file;
  • never claim to have run tests;
  • avoid common vulnerabilities;
  • answer in the user's language.

Replace it for a session with /set system, or with a system message in the API.

6.5 What makes flycoder0.3pro different

flycoder0.3pro is not Qwen3.8 with a new name. On top of the base, its Modelfile sets:

  • FlyCoder's system prompt (section 6.4), the one that scored best in the prompt comparison of section 7, with the base model named.
  • Rules for coding agents: read files before editing them, change as little as the task needs, run the project's tests and report their real output, never claim a result it did not see, never run a destructive command without the user's agreement.
  • Qwen's coding sampling (temperature 0.6 instead of the library's 1.0), 64K tokens of context (instead of Ollama's 4,096 default under 24 GB) and multi-token prediction (draft_num_predict 4).
  • REQUIRES 0.32.12: an older Ollama names the version to install.

A first pro recipe had a six-step workflow (contract, plan, code, tests, check, report) and a worked example conversation. It was compared on the 9B base, since the test server cannot hold the 27B: it solved 6 of the 20 bench problems, against 9 for FlyCoder's prompt. It was dropped. FlyCoder does not retrain the weights.

6.6 Ollama server settings

These variables apply to every model. On a Mac running the Ollama app, set them with launchctl setenv, then restart Ollama:

launchctl setenv OLLAMA_KV_CACHE_TYPE q8_0      # context cache half the size, small loss of precision
launchctl setenv OLLAMA_KEEP_ALIVE 30m          # keeps the model loaded for 30 minutes

7. Benchmark

The bench/ folder measures the pass rate and speed of any Ollama model on 20 coding problems:

  • 13 in Python and 7 in JavaScript;
  • each with hidden tests that the model never sees;
  • each test is proven by a reference solution and rejects an empty solution (npm test).
npm run bench -- --models flycoder0.3,flycoder0.3fast
Option Effect
--models a,b models to compare, one after the other (memory is freed in between)
--think on|off|default forces or disables thinking; default follows the model
--samples 3 several attempts per problem, to smooth out randomness
--only py-lcs,js-evaluate subset of problems
--max-tokens 8192 token cap per answer; a truncated answer counts as a failure
--num-ctx 8192 smaller context for machines short on memory, with no effect on these short answers
--out file.json report location; bench/results/ by default
--no-sandbox disables the macOS sandbox

The bench runs the code written by the models on your machine, in a temporary folder, with a time limit. On macOS, sandbox-exec blocks network access and writes outside that folder.

Reference results, recorded on a 4-core Linux server without a GPU (Xeon at 2.8 GHz, GGUF Q4_K_M weights, thinking off, one attempt per problem, at most 2,048 tokens per answer). On a Mac, the absolute speeds are much higher; only the comparisons between profiles matter.

Profile Base Passed Generation Tokens per answer Time per problem
flycoder0.3 Qwen3.5 9B 9/20 3.9 tok/s 900 254 s
flycoder0.3fast Qwen3.5 4B 5/20 6.1 tok/s 689 126 s
0.2 beta, for comparison Gemma 4 12B 14/20 3.4 tok/s 739 235 s
0.2 beta fast, for comparison Qwen3.5 4B 5/20 6.5 tok/s 537 89 s

Results: docs/bench/cpu-nothink-0.3.json; the 0.2 rows come from docs/bench/cpu-nothink.json, measured on a slower Xeon (2.1 GHz), so their speeds compare only roughly.

What these numbers say:

  • Gemma 4 12B solved more problems (14 against 9). It was replaced because it did not run on every 16 GB Mac; flycoder0.3 runs on any Ollama 0.30 or later and leaves room for a 64K context.
  • 3 of the 9B's 11 failures are cut-off answers: with thinking off, the model reasons in code comments until it hits the 2,048-token cap. With thinking on (the default of ollama run), that reasoning goes to the thinking channel instead. The other 8 failures are wrong logic.
  • flycoder0.3fast matches 0.2's fast variant (5/20), with a context twice as large.

Prompt comparison on the same 9B base and settings, recorded in docs/bench/cpu-nothink-0.3-prompts.json:

System prompt Passed Cut-off answers Tokens per answer
FlyCoder 0.3 (shipped) 9/20 3 900
The same, without the final-check line 8/20 3 890
First pro recipe: six-step workflow and a worked example 6/20 2 772

With one attempt per problem, a gap of one problem is noise; the gap to the first pro recipe is why it was dropped.

8. Publishing on ollama.com

Publishing is automatic. On every change to main that touches a Modelfile, a page in docs/ollama/ or the publishing scripts, the publish-ollama workflow:

  1. rebuilds the changed models from their Modelfiles and pushes them as Tromset/flycoder0.3, Tromset/flycoder0.3fast, Tromset/flycoder0.3pro, Tromset/flycoder0.3:lite and Tromset/flycoder0.3:router;
  2. updates each model page on ollama.com from docs/ollama/<model>.md.

It can also be started by hand from the Actions tab (Run workflow), with a list of models or none to update only the pages.

One-time setup, in the repository's Settings > Secrets and variables > Actions:

Secret Value
OLLAMA_KEY the private key ~/.ollama/id_ed25519 of a machine linked to the Tromset account (ollama signin on that machine first)
OLLAMA_API_KEY an API key created at https://ollama.com/settings/keys, used to update the pages

Updating a page uses the same request as the page's Edit button; Ollama does not document it. If it is refused, the workflow shows a warning and the page can still be pasted by hand.

By hand, from any machine signed in to the account: sh scripts/publish.sh (all models) or sh scripts/publish.sh Tromset flycoder0.3fast (one model), then OLLAMA_API_KEY=... sh scripts/update-ollama-pages.sh.

For a new version, update the version in package.json, install.sh, scripts/publish.sh, scripts/publish-hf.sh, the system prompt of the Modelfiles and the pages. npm test checks that they agree.

8.1 Publishing on Hugging Face

After each successful run of publish-ollama, the publish-huggingface workflow mirrors flycoder0.3, flycoder0.3fast and flycoder0.3pro on Hugging Face. It also runs when docs/huggingface/ changes, and by hand from the Actions tab. For each model, scripts/publish-hf.sh downloads the published layers of Tromset/<model> from the ollama.com registry, checks their SHA-256, and uploads to <namespace>/<model>:

File Content
<model>-Q4_K_M.gguf the weights, byte for byte the ollama.com file
mmproj-<model>.gguf the vision projector
system, params the system prompt and the sampling settings, which Ollama applies to ollama run hf.co/<namespace>/<model>
LICENSE the base's Apache 2.0 license
README.md the model card, from docs/huggingface/<model>.md (@HF_NAMESPACE@ is replaced by the namespace)

The template layer is not copied: on ollama.com it is a placeholder and Ollama formats Qwen with its built-in renderer. From Hugging Face, Ollama and llama.cpp (--jinja) use the chat template inside the GGUF; for tool calling and thinking in Ollama, Tromset/<model> stays the reference build.

One-time setup, in Settings > Secrets and variables > Actions:

Name Kind Value
HF_TOKEN secret a token with write access, created at https://huggingface.co/settings/tokens
HF_NAMESPACE variable, optional the user or organization to publish under; by default, the owner of the token

By hand: pip install huggingface_hub, then HF_TOKEN=... sh scripts/publish-hf.sh (all models, under the token's owner) or HF_TOKEN=... sh scripts/publish-hf.sh <namespace> flycoder0.3fast. The largest model needs 18 GB of free disk.

9. Troubleshooting

Symptom Likely cause Fix
pull model manifest: file does not exist wrong name the names are Tromset/flycoder0.3, Tromset/flycoder0.3fast and Tromset/flycoder0.3pro
requires a newer version of Ollama or unknown model architecture Ollama too old update from https://ollama.com/download (0.30 or later; 0.32.12 for pro)
The installer says "Ollama is not responding" the app is closed open Ollama, or run ollama serve
Very slow answers, ollama ps shows a CPU share not enough memory, the model spills out of the GPU close applications, lower num_ctx, or switch to flycoder0.3fast
The model stops or crashes while loading memory full same; only one large model loaded at a time
The model forgets the start of a long file context too small enlarge num_ctx (section 6.2)
Thinking takes too long simple question handled in thinking mode --think=false or /set nothink
Endless repetition temperature changed to a value that is too low go back to the Modelfile's settings
npm run bench: "Reference solution failed" python3 or node missing, or sandbox refused xcode-select --install, Node 22, or --no-sandbox
The publish workflow fails at "Check the secrets" OLLAMA_KEY is missing add it (section 8)
publish-huggingface warns "HF_TOKEN is not set" and publishes nothing HF_TOKEN is missing add it (section 8.1)

10. Development

10.1 Repository layout

Path Role
Modelfile, Modelfile.fast, Modelfile.pro the three variants
Modelfile.lite, Modelfile.router FlyBrain's 2B expert and micro-model
brain/flybrain.mjs, brain/router.mjs FlyBrain server and routing rules
brain/eval-router.mjs, tests/fixtures/route-prompts.json router accuracy measurement
install.sh one-command installer
scripts/publish.sh, scripts/update-ollama-pages.sh publishing models and pages on ollama.com
docs/ollama/ the ollama.com pages, one file per model
docs/huggingface/, scripts/publish-hf.sh the Hugging Face model cards and their publisher
bench/bench.mjs, bench/problems.mjs benchmark and problems
bench/baselines/ 0.1 profile, for comparison
docs/bench/ reference results
tests/ tests for the bench, the Modelfiles, the scripts, the pages and FlyBrain
.github/workflows/test.yml, .github/workflows/publish-ollama.yml, .github/workflows/publish-huggingface.yml CI on Linux and macOS, automatic publishing
training/, scripts/build_language_dataset.js, server/fly-language.js former FlyBrain voice pipeline, independent of FlyCoder; the voice speaks French by design

10.2 Tests

npm test

The tests check:

  • that each reference solution passes its hidden tests, and that an empty solution fails;
  • code extraction from answers and the statistics;
  • that the Modelfiles start from the right bases, with the expected settings and an identical system prompt apart from the base name;
  • that the installer, the publishing script and the workflow agree with the Modelfiles and the version;
  • that each ollama.com page shows the right name, size, memory and context;
  • that every Modelfile declares its minimum Ollama version, and that flycoder0.3pro keeps every shared rule plus the agent rules;
  • FlyBrain's rules, the fallback to the large model when the micro-model fails, conversation memory, the scheduler, and the server against a fake Ollama.

CI runs them on Linux and on macOS, where the reference solutions run inside the sandbox.

10.3 Changing the system prompt

Edit the SYSTEM block of the Modelfiles identically (only the sentence naming the base differs), run npm test, rebuild with ollama create, then compare before and after with npm run bench. Once merged into main, the new prompt is published automatically.

10.4 Adding a problem to the bench

Add an object to bench/problems.mjs with id (py- or js- prefix), language (python or javascript), entry (expected function or class), prompt (the only text sent to the model), reference (correct solution) and test (assertions). npm test rejects the problem if the reference fails or if an empty solution passes.

11. Known limitations

  • The weights are not retrained: FlyCoder is a profile of existing models.
  • The bench has 20 problems and one attempt per problem: it separates profiles without replacing a public benchmark.
  • The reference measurements come from a Linux server without a GPU; speed on a Mac has not been measured for this version.
  • flycoder0.3pro was not run by the bench: its 27B base needs more memory than the test server has. Its prompt choices were measured on the 9B base instead.
  • The bench runs with thinking off and one attempt per problem; quality with thinking on (the default) is not measured yet.
  • FlyBrain does not lower the memory peak of hard requests, and switching experts costs a few seconds.

12. Version history

Version Contents
0.1 beta Full workshop: Qwen3.5 4B, trainable FlyBrain controller, CLI, web and Electron interfaces, architect, coder and reviewer agents.
0.2 beta FlyCoder becomes an Ollama model only: Gemma 4 12B variant (MLX, multi-token) and a fixed Qwen3.5 4B variant, one-command install, hidden-test benchmark. Then the optional FlyBrain router.
0.3 Three models under clear names (Tromset/flycoder0.3, fast, pro), all portable GGUF that run on any recent Ollama; Qwen3.5 9B replaces Gemma 4 12B, which did not run on every 16 GB Mac; Qwen3.8 27B for pro; 64K context; automatic publishing from GitHub; short ollama.com pages.

13. Licenses

  • Repository code: MIT, see license.md.
  • Weights: Apache 2.0 for Qwen3.5 and Qwen3.8. Each base's terms are shown by ollama show flycoder0.3 --license.