This repository was archived by the owner on Apr 20, 2023. It is now read-only.
Fix caching on JSON body read - #104
Open
m4ns0ur wants to merge 1 commit into
Open
Conversation
When response is JSON and it doesn't have Content-length header and Transfer-encoding header is not chunked, reader will use bufio where we miss the EOF and it never set the refreshed cache. This fix will check the read length besides EOF err.
Author
|
We've experience the issue described in the linked issue of go-github. Is there any reason this wasn't merged in? |
|
@dmitshur - Do you have powers to merge this? Without it, we are forced to use the forked version of the repo as you can see in the linked PR. |
|
Can this be merged? I have been forced to use the forked version! |
|
I'll add another voice to this, as I'm hitting the same issue. |
idefixcert
approved these changes
Feb 3, 2021
|
I tried this fix but I get the following error when executing the testcases: |
Author
|
@idefixcert tests are passing for me Which version do you have? It seems you have this issue: golang/go#40735 |
|
your right, that was the problem. |
1 task
Closed
whywaita
added a commit
to whywaita/myshoes
that referenced
this pull request
Jan 17, 2022
saschagrunert
added a commit
to saschagrunert/release-sdk
that referenced
this pull request
Jun 13, 2022
The httpcache transport is breaking authentication where no fix seems to be implemented yet. I'd propose to revert the change for now to be able to use authenticated requests in tools like `k/release/cmd/release-notes`. Refers to: google/go-github#1503, gregjones/httpcache#104 Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
Repository owner
deleted a comment from
yuriy-yarosh
Aug 11, 2022
matejvasek
added a commit
to openshift/faas-console-plugin
that referenced
this pull request
Sep 11, 2026
The ETag cache added earlier on this branch only helps when it actually stores a response, and httpcache stores one only once its body reaches EOF. go-github decodes with json.Decoder, which stops as soon as the top-level value is complete and need never read that far, so whether a response lands in the cache depends on how the body is framed. That makes it a size lottery rather than a threshold. Measured against this client, 8 KB and 16 KB bodies were cached while 14 KB and 100 KB were not, and the payload only has to grow by a few bytes to flip. A lost poll costs a full 200 against the primary rate limit instead of a free 304. Production against api.github.com is unaffected, because gzip drains to EOF, so the exposure is GitHub Enterprise Server, the fake used in dev, or anything that strips Content-Encoding in between. A workflow_runs entry embeds whole repository objects and runs 10-20 KB, right in the range where this bites. The fix wraps each response body so Close drains whatever the caller left unread. It lives in the transport we already wrap for forced revalidation, because upstream cannot be fixed: gregjones/httpcache#104 is an open fix for this from 2020 and the repository was archived in 2023. The existing revalidation spec passed for the wrong reason. Its fixture was a few dozen bytes, a size that happens to cache either way, so it never exercised the failure. It now runs at three sizes to state the property that matters, that caching must not depend on payload size, and two of the three fail without this change. Signed-off-by: Matej Vašek <matejvasek@gmail.com> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When response is JSON and it doesn't have Content-length header
and Transfer-encoding header is not chunked, reader will use bufio
where we miss the EOF and it never set the refreshed cache.
This fix will check the read length besides EOF err.