Repository navigation
cache sample rows on disk - #3
Merged
Merged
Conversation
`wlens generate` used to re-query every table for sample rows on every run. The rows came back in different order each time (no ORDER BY), producing hundreds of lines of diff churn, and on big tables the re-query was just slow. Now sample rows are written to wlens/.cache/samples/<schema>.<table>.json on the first run and reused on every subsequent run. The cache is committed, so teammates and CI share the same pinned rows. We re-fetch when the table's columns change, when its PII tags change, or when the obfuscation rules change. `--refresh-samples` forces a re-fetch (pass no value for all tables, or `=schema.table[,...]` for specific ones). Existing SQL query cache also moves from wlens/cache/sql/ to wlens/.cache/sql/ so both caches live under one wlens-internal root. `wlens init` writes a wlens/.gitignore that keeps the SQL cache out but tracks the sample cache. `wlens generate` migrates an older wlens/.gitignore in place.
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 79102b8. Configure here.
`--refresh-samples` followed by an empty fetch (transient query error or genuinely-empty table) used to drop the sample-rows section to [] without touching the cache file. Next run would silently put the old rows back, so the .md flipped between "with rows" and "without rows" across runs. Now an empty fetch falls back to the existing cache and logs a warning. Disk state and the rendered markdown stay in sync.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This prevents refetching data in unchanged models, which prevents having a large git diff on
wlens generate.wlens/.cache/samplesto keep track of table's head. Helps us prevent refetching data and large git diffs onwlens generate--refresh-samplesto force-refresh all cached samples (and table heads)wlens/cache/sqltowlens/.cache/sql