Feat spdiag logging - #36
Closed
ZhiningPan wants to merge 291 commits into
Closed
Conversation
add UbDiag Perf Points
- 新增日志配置章节，说明环境变量 MC_LOG_LEVEL 和 MC_LOG_DIR 的用法 - 新增异步日志改造方案章节，提供 spdlog 和 glog 包装两种方案 - 详细对比两种方案的优缺点，为后续性能优化提供参考
- 新增 mooncake_common 日志库，提供 MC_LOG/MC_VLOG 宏，支持异步输出和 TraceId 跟踪 - 为 get/put 等读写操作自动生成唯一 TraceId，通过线程本地和 ScopedTraceId 在异步任务中传递 - 添加环境变量 MC_LOG_ENABLE 控制日志开关，支持 off/0/false/no 值关闭普通日志 - 在 transfer_task 中为 MemcpyTask 和 FilereadTask 添加 trace_id 字段，确保异步任务日志可追踪 - 新增性能监控日志：transfer_future_wait、open_segment_breakdown、submit_transfer_breakdown - 更新 CMakeLists.txt 链接依赖，添加相关文档说明日志格式和 Trace 机制
- 在 UB/URMA 建连路径添加 UbDiag 性能监控点 - 为 RealClient 公开操作添加慢操作告警（阈值 3000us） - 更新日志逻辑以支持 FLAGS_minloglevel - 清理 Python binding 中的冗余日志 - 更新相关文档说明
- 在 real_client.cpp 中，将 replica_selected 日志的副本类型从 "memory" 和 "local_disk" 细化为 "memory_local"、"memory_remote"、"local_disk_local" 和 "local_disk_remote"，以区分本地与远程副本 - 更新 docs/yh/log-reference.md 文档，补充新增的传输任务层、URMA 建连层及 Client 创建日志的详细说明，并同步其他日志格式的更新
- 在 OperationState 及其子类构造函数中增加 trace_id 参数，用于传递调用链追踪标识 - 在 TransferSubmitter 提交 MemcpyTask 和 FilereadTask 时捕获当前线程的 trace_id - 在传输任务执行时通过 RestoreTraceIfMissing 恢复缺失的 trace 上下文 - 更新文档说明 trace_id 在同步/异步调用链中的传播机制
[WIP] Ssd balance + hot-cold
# Conflicts: # mooncake-store/benchmarks/CMakeLists.txt # mooncake-store/src/real_client.cpp # mooncake-store/src/utils.cpp
- 更新 build.sh 脚本，在 EXT_LDFLAGS 链接标志中增加 -lubdiag - 更新 CMakeLists.txt，为 mooncake_master 和 mooncake_client 目标链接 UbDiag::ubdiag_lib 库
在传输层和元数据处理中添加 trace_id 字段，用于跨组件的请求追踪： - 在 TransferMetadata 和 Transport::Slice 结构中添加 trace_id 字段 - 在 UB 传输提交任务时设置当前 trace_id - 在握手过程中传递和恢复 trace_id - 在 worker 处理时根据 slice 的 trace_id 恢复上下文 - 将相关日志输出从 LOG 更新为 MC_LOG
Remove the temporary pre-remount recovery lease and require memory replicas to be readable before ordinary eviction and offload. Keep oplog descriptor prediction aligned with the actual eviction predicate.
chore(spdiag): pin L0 source to v1.0.0 commit
store： ha for ub
fix(tracing): 添加缺失的 span 结束调用
依据 vllm_spdiag_logging_plan.md，按 Q1b/Q2A/Q3b 方案为 Mooncake Store vLLM 调用链添加 SpDiag 性能打点与日志输出： - mooncake_perf_points.def: 追加 25 个新 PerfKey，覆盖 store_py 入口 层（STORE_PY_*）、RealClient 核心层（RC_*）、Client 服务层 （CLIENT_BATCH_QUERY）。 - store_py.cpp: 为 setup/register_buffer/batch_put_from_multi_buffers/ batch_get_into_multi_buffers/batchIsExist/remove_all/close 等 8 个 Python 绑定方法添加 PerfPoint（KEY_MODULE），remove_all 与 close 额外输出 MC_LOG 汇总行。 - real_client.cpp: 为 setup_real/batchIsExist/batch_put_from_multi_ buffers/batch_get_into_multi_buffers 等 9 个方法添加 PerfPoint； 下沉层 _internal 方法用 MODULE 级别，并按 Q1b 输出 MC_LOG 汇总行 + per-key 多行（包含 success/key/size/replica/endpoint 字段， 缺失字段不输出）。 - client_service.cpp: 在 L1117 BatchQuery 实际实现中添加 CLIENT_BATCH_QUERY PerfPoint（Q3b，仅实际实现，避免重复统计）。
github-actions
Bot
added
documentation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
10 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Module
mooncake-transfer-engine)mooncake-store)mooncake-reshard)mooncake-ep)mooncake-pg)mooncake-integration)mooncake-p2p-store)mooncake-wheel)mooncake-common)mooncake-rl)Type of Change
How Has This Been Tested?
Test commands:
# Example: bash scripts/run_ci_test.shTest results:
Checklist
./scripts/code_format.shAI Assistance Disclosure