Skip to content

[XPU] add fetch req log #8070

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
cmcamdy wants to merge 1 commit into
base: develop
Choose a base branch
from
+8 −0
Open

[XPU] add fetch req log #8070

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions fastdeploy/engine/common_engine.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -981,6 +981,8 @@ def _fetch_request():
LoggingEventName.ASK_DECODE_RESOURCE_START, task.request_id, getattr(task, "user", "")
)
task.metrics.ask_decode_resource_start_time = time.time()
if envs.FD_PD_LOG_REQUEST:
self.llm_logger.info(f"[PD_LOG] P sends Request: {task.to_dict()}")

@PaddlePaddle-bot PaddlePaddle-bot Jun 23, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 安全 这里会把完整用户请求写入 INFO 日志。

Request.to_dict() 包含 prompt、prompt_token_ids、messages、system、history、tools、multimodal_data 等用户输入字段；一旦 FD_PD_LOG_REQUEST=1，PD 请求的原始内容会进入持久化日志，命中仓库 checklist 的日志泄漏项。

建议修复方式：
改为专用的脱敏/摘要结构，只输出 request_id、prompt_token_ids_len、idx、num_computed_tokens、必要的 block/resource 元数据；不要调用 task.to_dict() 直接落日志。

while True:
self.split_connector.send_splitwise_tasks([task], task.idx)
status, msg = self.split_connector.check_decode_allocated(task)
Expand Down Expand Up @@ -1011,6 +1013,8 @@ def _fetch_request():
LoggingEventName.ASK_DECODE_RESOURCE_START, task.request_id, getattr(task, "user", "")
)
task.metrics.ask_decode_resource_start_time = time.time()
if envs.FD_PD_LOG_REQUEST:
self.llm_logger.info(f"[PD_LOG] P sends Request: {task.to_dict()}")

@PaddlePaddle-bot PaddlePaddle-bot Jun 23, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 安全 这里会把完整用户请求写入 INFO 日志。

Request.to_dict() 包含 prompt、prompt_token_ids、messages、system、history、tools、multimodal_data 等用户输入字段；一旦 FD_PD_LOG_REQUEST=1，PD 请求的原始内容会进入持久化日志，命中仓库 checklist 的日志泄漏项。

建议修复方式：
改为专用的脱敏/摘要结构，只输出 request_id、prompt_token_ids_len、idx、num_computed_tokens、必要的 block/resource 元数据；不要调用 task.to_dict() 直接落日志。

self.split_connector.send_splitwise_tasks([task], task.idx)

for task in tasks:
Expand Down Expand Up @@ -2095,6 +2099,8 @@ def _fetch_requests():
f"D has received tasks to preallocate resource for tasks: {[task.request_id for task in tasks]}"
)
for task in tasks:
if envs.FD_PD_LOG_REQUEST:
self.llm_logger.info(f"[PD_LOG] D received Request: {task.to_dict()}")

@PaddlePaddle-bot PaddlePaddle-bot Jun 23, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 安全 这里会把完整用户请求写入 INFO 日志。

Request.to_dict() 包含 prompt、prompt_token_ids、messages、system、history、tools、multimodal_data 等用户输入字段；一旦 FD_PD_LOG_REQUEST=1，PD 请求的原始内容会进入持久化日志，命中仓库 checklist 的日志泄漏项。

建议修复方式：
改为专用的脱敏/摘要结构，只输出 request_id、prompt_token_ids_len、idx、num_computed_tokens、必要的 block/resource 元数据；不要调用 task.to_dict() 直接落日志。

task.metrics.decode_recv_req_time = time.time()
allocate_resource_requests.extend(tasks)
elif isinstance(tasks[0], RequestOutput):
Expand Down
2 changes: 2 additions & 0 deletions fastdeploy/envs.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -269,6 +269,8 @@ def _validate_split_kv_size(value: int) -> int:
"FD_DETERMINISTIC_LOG_MODE": lambda: bool(int(os.getenv("FD_DETERMINISTIC_LOG_MODE", "0"))),
# Whether to use PD REORDER, can set 0 or 1
"FD_PD_REORDER": lambda: int(os.getenv("FD_PD_REORDER", "0")),
# Whether to enable PD disaggregation request logging on decode side
"FD_PD_LOG_REQUEST": lambda: int(os.getenv("FD_PD_LOG_REQUEST", "0")),
# Whether to probe MoE routing probabilities and use Fleet's fused SwiGLU kernel.
"FD_MOE_PROB_IN_ADVANCE": lambda: bool(int(os.getenv("FD_MOE_PROB_IN_ADVANCE", "0"))),
# Whether to use batch send data in zmq
Expand Down
Loading