Repository navigation
fix(index): download the catalogue without a 60 second cap - #1
Open
atsyplenkov wants to merge 1 commit into
Open
atsyplenkov wants to merge 1 commit into
atsyplenkov wants to merge 1 commit into
Conversation
download.file() stops at options(timeout), which defaults to 60 seconds. curl writes the index to a temporary file and moves it into the cache only when the size matches Content-Length.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
index()downloads the AlphaEarth index once and keeps it in the cache.download.file()stops after 60 seconds. The index is too large for that limit, so the download stopped before a complete file reached the cache.curl_download()has no total time limit. The file moves into the cache only after the download finishes and the size matches the serverContent-Length. A response with noContent-Lengthis accepted. A short file is not stored in the cache.Evidence
download.file()used the 60 second timeout. A missingContent-Lengthmade the size check stop with "missing value where TRUE/FALSE needed".After:
pkgload::load_all()andtestthat::test_file("tests/testthat/test-index.R")reported 7 passes and 0 failures. The new tests mock the server. One accepts a response with noContent-Length. One rejects a 4 byte file when the server reports 12 bytes.air formatandjarl checkpassed on the edited R files.Merge Danger
Door: two-way
Revert this commit. An index that is already in the cache is reused. The new download runs only when the cache is empty or
overwrite = TRUE.Blast Radius: the first download
People who already have the index see no change. A new download uses
curlinstead ofdownload.file().