Auto-select an installed model, add a model picker, and show the screenshot on PyPI - #5
Merged
Merged
Conversation
The example image did not render on pypi.org. The README used a relative path, and PyPI renders the README standalone, so docs/example.png resolved against pypi.org and 404'd. It is now an absolute raw.githubusercontent URL, verified to return 200 image/png. Also adds tests for the Ollama HTTP paths a real Ollama on a developer's machine cannot exercise, using a stub server: - An Ollama older than v0.3.4 has no /api/embed at all. The fallback to the singular /api/embeddings existed but had no test, and it is one of the two plausible causes of the blank page in #1. - A 404 meaning "no such model" and a 404 meaning "no such route" look identical and mean opposite things. The first must name the ollama pull command and must NOT retry the legacy route; the second must fall back. - A response with fewer rows than inputs is refused rather than silently misaligning every later vector. One of these took two minutes on its own: constructing an HTTPServer without serving it does not give a closed port -- the socket is already bound and listening, so the connection succeeds and the request blocks until the 120-second timeout. The test now closes a real socket to get a refused port, which takes 2 seconds instead of 122, six times over in CI.
Reported from a real first run: three usable models installed -- llama3.2:3b, nomic-embed-text, all-minilm -- and the app insisting on pulling llama3.1:8b. The embedding model had always been discovered from what Ollama reports. The chat model was hardcoded, so LKG_CHAT_MODEL was a *default* rather than an override, and anyone whose model happened to have another name was told to download one they did not need. It is now discovered the same way, with a preference order used only to break a tie between several installed models -- any installed model beats a missing one, so this can no longer produce a "not found" when something usable is present. Three additions to the page: - Dropdowns for the chat and embedding model, listing what Ollama actually has. Which model answers is the first thing a new user gets wrong, and it was only changeable through an environment variable documented in the README. - When Ollama has no chat model at all, the page lists a few with their download sizes, links to ollama.com, and can fetch one with streamed progress rather than printing a command and stopping. - A favicon, because it was a 404 in everyone's terminal on every page load. Two guards on the download route, which exists to start a multi-gigabyte transfer: - An allowlist, not free text. Only the models the page offers can be named. - A same-origin check. A page on the internet can POST to a service on your loopback address, so starting a download has to be something this page asked for. The model-selection route is equally not free text: a name is only accepted if Ollama reports it as installed, because that value goes straight to the model API. 18 new tests, including the exact reported case: llama3.2:3b present, llama3.1:8b absent, health must be ok. Verified end to end against a real Ollama with no environment variables set at all.
punnerud
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two small things.
The screenshot did not render on PyPI. The README used a relative path, and PyPI renders the README standalone, so
docs/example.pngresolved againstpypi.organd 404'd. Now an absoluteraw.githubusercontent.comURL, verified to return200 image/png. Version bumped to 0.2.1 so the fixed description can be published.Tests for the Ollama HTTP paths a real Ollama cannot exercise, via a stub server:
/api/embed. The fallback to the singular/api/embeddingsexisted but was untested — and it is one of the two plausible causes of the blank page in Issues running project : errors inollama servelogs #1.ollama pullcommand and must not retry the legacy route; the second must fall back.One of these took 122 seconds on its own. Constructing an
HTTPServerand not serving it does not give you a closed port — the socket is already bound and listening, so the connect succeeds and the request blocks until the 120-second timeout. It now closes a real socket to get a refused port: 2 seconds instead of 122, six times over in CI.