Repository navigation
Fix inaccuracies in RoPE and block hash explanation - #34
Merged
Merged
Conversation
Clarified the relationship between K vectors and block hashes in the context of vLLM APC.
Contributor
|
感谢指出,你的理解是对的。我们对照最新 main(8b365ff949)核实了源码。 当前实现( def hash_block_tokens(
hash_function: Callable[[Any], bytes],
parent_block_hash: BlockHash | None,
curr_block_token_ids: Sequence[int],
extra_keys: tuple[Any, ...] | None = None,
) -> BlockHash:
...
if not parent_block_hash:
parent_block_hash = NONE_HASH
curr_block_token_ids_tuple = tuple(curr_block_token_ids)
return BlockHash(
hash_function((parent_block_hash, curr_block_token_ids_tuple, extra_keys))
)哈希输入只有三样:父块哈希、当前块的 token IDs、extra keys(后者仅多模态 / LoRA 请求生成,见 所以正文 §1.3 那句「K 向量不同导致 block hash 不同」确实不成立——而且与我们自己在脚注 [^1] 和「hash chaining 隐式编码位置」处的描述相互矛盾。准确的机制是两层:
修改方向建议(供参考):
|
§1.3 原文「RoPE 编码不同 → K 向量不同 → block hash 不同」与 vLLM 实现不符:block hash 仅由父块哈希、token IDs 和 extra keys 算出(vllm/v1/core/kv_cache_utils.py hash_block_tokens),K 向量 不参与;位置的影响经由哈希链进入,RoPE 的障碍在跨位置复用的 正确性上(后文方案 B)。同时移除追加的讨论段,行文并入正文。 依据 PR ForceInjection#34 由 YangooSen 指出的错误,实现细节经最新 main 源码 复核。 Co-authored-by: YangooSen <154312675+YangooSen@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Clarified the relationship between K vectors and block hashes in the context of vLLM APC.