Skip to content

[API Compatibility] enhance paddle.enable_compat -part#79391

Open
Manfredss wants to merge 14 commits into
PaddlePaddle:developfrom
Manfredss:patch_compat
Open

[API Compatibility] enhance paddle.enable_compat -part#79391
Manfredss wants to merge 14 commits into
PaddlePaddle:developfrom
Manfredss:patch_compat

Conversation

@Manfredss

Copy link
Copy Markdown
Contributor

PR Category

User Experience

PR Types

Improvements

Description

paddle.enable_compat() currently only installs a sys.meta_path finder so that
import torch resolves to PaddlePaddle (a torch→paddle proxy); it does not touch
the real paddle.* namespace, so paddle.sort/paddle.min/... stay paddle-native
even while compat is enabled.

Because paddle.X and paddle.compat.X share identical positional parameter types, a
call site cannot be auto-classified as "native intent" vs "compat intent", so the clean
design is a global switch: after enable_compat(), paddle.X is paddle.compat.X
and its arguments are interpreted as the torch-aligned compat arguments. Users write
PyTorch-style code and no longer need to care whether an API is paddle.* or
paddle.compat.*. (This also backs the PaConvert migration flow: prefix-only
torch.X → paddle.X + an injected paddle.enable_compat().)

What this PR does

  • Namespace aliasing (compat/proxy.py): on enable_compat() (global scope only),
    alias every public paddle.compat.* symbol onto the live paddle / paddle.nn /
    paddle.nn.functional modules via setattr, mirroring _register_compat_override
    (torch.X → paddle.compat.X). disable_compat() perfectly restores the original
    namespace (including deleting symbols that did not exist before — paddle.slogdet,
    paddle.nn.AvgPool1d/2d/3d, paddle.nn.MultiheadAttention).
  • Self-recursion fix (_native helper): compat wrappers that internally call back
    into a name they alias (paddle.compat.minpaddle.min, Unfold/transformer
    F.linear/F.softmax, unfold, scaled_dot_product_attention) now route through
    _native(module, name) so they always reach the genuine native impl — avoiding
    infinite recursion / ForbidKeywords rejection / a silent double-transpose, while
    keeping paddle.X is paddle.compat.X.
  • Root-package handling: the alias walk (and _register_compat_override) now also
    process the paddle.compat root package that pkgutil.walk_packages skips, so the
    top-level functions (sort/min/max/split/median/unique/...) are mapped for both global
    and scoped enable.
  • Lifecycle correctness: idempotent meta_path finder insert; enable_compat(scope=...)
    while already global no longer silently downgrades to scoped; use_compat_guard
    faithfully restores the original finder state (so importing a scoped module no longer
    destroys the surrounding scoped enable); and a leaked torch ProxyModule is cleared
    when restoring to scoped state.
  • sdpa robustness (nn/functional/flash_attention.py): aliasing
    paddle.nn.functional.softmax to the torch-style compat softmax (whose dim=None
    default picks a different axis than native axis=-1) silently corrupted the sdpa
    math backend, because _math_attention called F.softmax(product) with no axis.
    Pass the axis explicitly (F.softmax(product, -1)) so attention stays correct under
    enable_compat(); with compat off this is identical to the previous behavior.

是否引起精度变化

…alls resolve to the paddle.compat.* implementations at runtime. assisted by Claude

@risemeup1111 risemeup1111 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

发现了需要修复后再合入的兼容性回归,具体问题已放在行级评论中:主要是 paddle.compat 的 lazy attribute 行为被破坏,以及已有的 compat public API 被删除。CI 当前仍有部分任务进行中,且已有 Windows-GPU build/test 失败,请后续一并确认。

Powered by Nyanpasu with gpt-5.5 xhigh, please check the suggestions carefully.

Comment on lines 59 to 61
def __getattr__(name):
if name == "paddle_triton":
return paddle_triton_fun()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 优先级:P1

这里删掉了 distributions 的 lazy import 和最终的 AttributeError,导致两个回归:paddle.compat.distributions 在未预先 import paddle.compat.distributions 时会通过 __getattr__ 直接返回 None,未知属性也不再抛 AttributeError,会破坏 Python 模块属性协议以及已有的 paddle.compat.distributions 访问方式。请保留 distributions 分支,并让未知属性继续抛 AttributeError

Suggested change
def __getattr__(name):
if name == "paddle_triton":
return paddle_triton_fun()
def __getattr__(name):
if name == "paddle_triton":
return paddle_triton_fun()
if name == "distributions":
import importlib
module = importlib.import_module("paddle.compat.distributions")
globals()[name] = module
return module
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")

'BatchNorm1d',
'BatchNorm2d',
'BatchNorm3d',
'MultiheadAttention',

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 优先级:P1

这里把 paddle.compat.nn.BatchNorm1D/2D/3D、小写别名以及 SmoothL1Loss 从 public compat surface 删除了,但这些 API 已经有现有调用和测试覆盖:test/legacy_test/test_api_compatibility_part1.py 在类定义时直接绑定 paddle.compat.nn.BatchNorm*test/legacy_test/test_compat_smooth_l1_loss.py 也直接实例化 paddle.compat.nn.SmoothL1Loss。删除后这些测试会在 import/执行阶段变成 AttributeError,也会破坏已有用户代码。请恢复这些符号及其实现;如果目标是只做命名空间 alias,仍需要保留 compat 侧的实现作为 paddle.* alias 的目标。

建议形状:

__all__ = [
    # ...
    'BatchNorm1D', 'BatchNorm2D', 'BatchNorm3D',
    'BatchNorm1d', 'BatchNorm2d', 'BatchNorm3d',
    'SmoothL1Loss',
]

class BatchNorm1D(nn.BatchNorm1D):
    ...  # 保留 torch-style eps/momentum/track_running_stats 适配

BatchNorm1d = BatchNorm1D
# 同步恢复 2D/3D 以及 SmoothL1Loss

return x

return paddle.nn.functional.unfold(
return _native(paddle.nn.functional, "unfold")(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 优先级:P1

这里删除 paddle.compat.nn.functional.smooth_l1_loss 会直接破坏已有 compat functional API:test/legacy_test/test_compat_smooth_l1_loss.py 覆盖了 F_compat.smooth_l1_loss(..., beta=...)target=size_average/reducebeta=0 和 forbidden keywords。删除后这些用例和用户迁移代码都会变成 AttributeError,并且 paddle.enable_compat() 也无法把 paddle.nn.functional.smooth_l1_loss alias 到 torch-style 语义。请恢复该函数、__all__ 条目,以及所需的 warnings_ReduceMode

建议形状:

import warnings

if TYPE_CHECKING:
    _ReduceMode: TypeAlias = Literal["mean", "sum", "none"]

__all__ = [
    # ...
    'unfold',
    'smooth_l1_loss',
]

@ForbidKeywordsDecorator(
    illegal_keys={"label", "delta", "is_huber", "name"},
    func_name="paddle.compat.nn.functional.smooth_l1_loss",
    correct_name="paddle.nn.functional.smooth_l1_loss",
)
def smooth_l1_loss(input, target, size_average=None, reduce=None, reduction='mean', beta=1.0):
    # 保留原来的 torch-style beta/size_average/reduce 适配逻辑
    ...

PaddlePaddle-bot

This comment was marked as outdated.

@PaddlePaddle-bot

PaddlePaddle-bot commented Jun 29, 2026

Copy link
Copy Markdown

CI 分析结果

本轮达到时间上限,未完成全部失败原因分析;未分析完的 job 标记为“分析省略/待后续深挖”。

PaddlePaddle-bot

This comment was marked as outdated.

PaddlePaddle-bot

This comment was marked as outdated.

@liuhao2638 liuhao2638 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已复查当前提交。此前指出的 paddle.compat lazy attribute、compat.nn BatchNorm/SmoothL1Loss、compat.nn.functional.smooth_l1_loss 删除问题在当前代码中已经恢复。

不过这轮发现了一个新的全局 alias 下的 P1 回归,具体见行级评论:smooth_l1_loss wrapper 需要通过 native 实现回调,避免 enable_compat() 后自引用到 compat wrapper。CI 目前仍有任务运行中,后续也请一并确认。

Powered by Nyanpasu with gpt-5.5 xhigh, please check the suggestions carefully.

for attr_name in getattr(compat_module, PUBLIC_ATTR_DECLARATION):
if attr_name.startswith("_"):
continue
compat_attr = getattr(compat_module, attr_name)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 优先级:P1

这里会把每个 paddle.compat.*.__all__ 符号 alias 到真实 paddle.* namespace,因此全局 paddle.enable_compat()paddle.nn.functional.smooth_l1_loss 也会变成 compat wrapper。当前 wrapper 在函数尾部仍直接调用 paddle.nn.functional.smooth_l1_loss(..., delta=beta, is_huber=False);alias 生效后这个名字已经指回 wrapper 自身,ForbidKeywordsDecorator 会拒绝 delta/is_huber(或进入自调用路径),导致迁移代码里的 paddle.nn.functional.smooth_l1_loss(x, y, beta=...) 失败。

请像 unfold/SDPA 一样通过 _native 调回 Paddle 原生实现,并补一个 enable_compat() 下的回归用例覆盖这个别名路径。建议形状:

if beta == 0:
    return _native(paddle.nn.functional, "l1_loss")(
        input, target, reduction=reduction
    )

return _native(paddle.nn.functional, "smooth_l1_loss")(
    input, target, reduction=reduction, delta=beta, is_huber=False
)

@zhwesky2010 zhwesky2010 changed the title [API Compatibility] enhance paddle.enable_compat() -part [API Compatibility] enhance paddle.enable_compat -part Jun 29, 2026
@Manfredss

Copy link
Copy Markdown
Contributor Author

/re-run all-failed

@codecov-commenter

codecov-commenter commented Jun 29, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.92265% with 11 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (develop@9aa3379). Learn more about missing BASE report.

Files with missing lines Patch % Lines
python/paddle/compat/proxy.py 92.99% 11 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             develop   #79391   +/-   ##
==========================================
  Coverage           ?   93.92%           
==========================================
  Files              ?        6           
  Lines              ?      181           
  Branches           ?        0           
==========================================
  Hits               ?      170           
  Misses             ?       11           
  Partials           ?        0           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.



@cache
def _register_compat_override():

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

如果 paddle.compat.* 已经patch到 paddle.* 上了,这里还需要去额外操作吗

# disable_compat() on exit would remove the finder and destroy the
# surrounding scoped enable (the runtime entry point of the migration
# workflow). So restore the finder to exactly how it was found.
if already_has_torch_proxy:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里需要额外操作的原因是

Comment thread python/paddle/compat/proxy.py Outdated
public_attrs = getattr(module, PUBLIC_ATTR_DECLARATION)
torch_module_name = module_info.name.replace(
PADDLE_PREFIX, TORCH_PREFIX, 1
# Use the root-inclusive iterator: pkgutil.walk_packages skips the

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这些注释清理一下吧,如果确需要注释的话,也只是简短意赅的关键1~2行,不用长篇大论的写

_restore_paddle_namespace_aliases()


def enable_compat(

@zhwesky2010 zhwesky2010 Jun 29, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

直接改这个可能有点不兼容,加个level参数吧

@zhwesky2010

zhwesky2010 commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

背景

这个主要是解决当前的paddle框架中无法完全对齐的问题,当前Paddle API的对齐状态分为以下三种:

  1. 可以原位对齐;
  2. 无法原位对齐,但可以新增paddle.compat.*
  3. 既无法原位对齐,也无法新增paddle.compat.*,处于完全未对齐状态,分类2的类方法API普遍为此状态,例如paddle.Tensor.max/min/split

用户在编写paddle时,很难知道哪些是对齐的1,哪些是未对齐的2/3,这使得编写paddle代码成本仍较高。
因此计划引入一个Flag开关,将2/3直接切换为pytorch形态,用户无需再关注1/2/3的名单,统一直接按pytorch思路来写代码即可

开发思路

目前考虑在enable_compat最后面加一个level参数,默认为1,对于:

  • level=1:维持之前的逻辑不变
torch.* -> paddle.*(paddle.compat.*)

paddle.*:部分API无法原位对齐torch
  • level=2:除了当前的torch重定向逻辑,还将进行paddle.compat->paddle、paddle.compat->paddle.Tensor的重定向逻辑,消除上述分类2/3的未对齐状态
torch.* -> paddle.*

paddle.*:完全对齐torch形态

针对 第三方库+存量Paddle代码 的用法:level=1,部分API无法对齐,需手动检查调整,主要为避免存量paddle代码出现不兼容

针对 第三方库+新增Paddle代码 的用法:level=2,完全对齐,统一按torch的形态即可

@SigureMo 看下这样的设计思路怎么样,有没有要调整的地方?比如 不加level直接按2来、加level但默认为2、新增一个其他API来作为这个开关不用放到enable_compat里

@zhwesky2010 zhwesky2010 requested a review from SigureMo June 29, 2026 12:45

@SigureMo SigureMo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

针对 第三方库+存量Paddle代码 的用法:level=1,部分API无法对齐,需手动检查调整,主要为避免存量paddle代码出现不兼容

针对 第三方库+新增Paddle代码 的用法:level=2,完全对齐,统一按torch的形态即可

我比较赞同这里的 level=1/2 的方案,主要是考虑到现存代码可能有部分已经使用 Paddle 原生 API 改写过了,此时一旦自动直接将这些 API 迁移到 paddle.compat API 实现可能会出现一些因为兼容性导致的非预期的问题,直接按 2 来我倒是有些担忧的,不过倒是后面可以考虑把 2 切默认(理想情况)

不过值得注意的是,paddle.enable_compat 这个 API 真正的挑战并不在于实现这些映射功能,而在于当环境中真的安装有 PyTorch 时,如何避免发生冲突(CI 中几乎测不到,只有几个使用 fake torch 模拟的 case),这是真实业务场景中会使用的,因此在实现时候需要特别注意下

return x

return paddle.nn.functional.unfold(
return _native(paddle.nn.functional, "unfold")(

@SigureMo SigureMo Jun 29, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

我有个疑问,既然是 patch,那么一旦开启 compat,所有相关 API 都会变吧?为什么当前改动仅限于 paddle.compat 下?非 compat 下的 API 理论上也会受到影响吧?(只是问题并不是像递归这么明显罢了)

当然不是让你把所有 API 改一遍,而是我在想这样做的可行性

@SigureMo SigureMo Jun 29, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

其实这也很好做,改造一下:

from typing import Callable, TypeVar
from typing_extensions import ParamSpec
from contextlib import contextmanager

T1 = TypeVar("T1")
T2 = TypeVar("T2")
P1 = ParamSpec("P1")
P2 = ParamSpec("P2")

COMPAT_API_ENABLED = False


def is_compat_api_enabled():
    return COMPAT_API_ENABLED


@contextmanager
def compat_api_guard(enable: bool = True):
    global COMPAT_API_ENABLED
    if enable == COMPAT_API_ENABLED:
        yield
        return
    original_value = COMPAT_API_ENABLED
    COMPAT_API_ENABLED = enable
    try:
        yield
    finally:
        COMPAT_API_ENABLED = original_value


# compat/registry.py
def dispatch_compat_api(
    compat_fn: Callable[P1, T1],
) -> Callable[[Callable[P2, T2]], Callable[P2, T2]]:
    def native_fn_decorator(native_fn: Callable[P2, T2]) -> Callable[P2, T2]:
        def maybe_use_compat_api(*args: P2.args, **kwargs: P2.kwargs) -> T2:
            if is_compat_api_enabled():
                # 关键在这里,灵活关掉 compat
                return compat_api_guard(enable=False)(compat_fn)(*args, **kwargs)  # type: ignore
            return native_fn(*args, **kwargs)

        return maybe_use_compat_api

    return native_fn_decorator


def unfold_compat(x, y):
    return unfold(x, y) + 1


@dispatch_compat_api(unfold_compat)
def unfold(x, y):
    return x * y


with compat_api_guard():
    print(unfold(1, 2))  # 3

with compat_api_guard(enable=False):
    print(unfold(1, 2))  # 2

随便搞的原型,仅供参考

Comment on lines +177 to +186
paddle.enable_compat()
try:
self.assertEqual(paddle.min(t).item(), 1.0)
self.assertEqual(paddle.max(t).item(), 3.0)
# dim form returns a namedtuple (values, indices)
r = paddle.min(t, dim=0)
self.assertEqual(r.values.item(), 1.0)
self.assertEqual(r.indices.item(), 1)
finally:
paddle.disable_compat()

@SigureMo SigureMo Jun 29, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这种单测不能直接用 @use_compat_guard 装饰器么?

@zhwesky2010 zhwesky2010 Jun 30, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Manfredss 对于整个框架内部,调用到这些compat系列API的地方(约30个),确实应该都是paddle naive版的,而这一系列API对外呈现的则是torch版的。

这个主要影响到的是 单测 + 部分调用paddle API的组合实现API。

可能得加大一下测试面,比如level=2下,paddle其他的API还能否跑过,除了paddle自身测试外,建议在paconvert里也进行测试(全局设置为level=2),测量全量API的行为是否正常。

@paddle-bot paddle-bot Bot added the contributor External developers label Jun 29, 2026
…d the compat impls) get native, only external user code sees compat -- keeps paddle's own layers working
@Manfredss

Manfredss commented Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

paddle.enable_compat 增加 level 参数:level=2paddle.* 直接呈现 torch 形态

背景:Paddle API 对齐分三类:

  1. 可原位对齐;
  2. 无法原位、但有 paddle.compat.*
  3. 类方法等无法对齐。用户难以区分、编写成本高。目标是用一个开关把 2 和 3 统一切到 torch 形态,用户统一按 torch 思路写代码。

设计:enable_compat(level=...)

  • level=1(默认):维持现状——仅安装 import torch → paddle 代理,不改写 paddle.*,对存量代码零影响(完全向后兼容)。
  • level=2:在此基础上把 torch 对齐的 paddle.compat.* 路由到 paddle.*,用户可直接按 torch 形态写 paddle 代码。

level=2 关键实现

  • 函数用 dispatcher 包装、类直接别名;enable/disable 仅切全局标志,suspendcontextvars
  • 调用方感知:仅当调用方不是 paddle.* 模块时才走 compat —— paddle 内部约 130 处调用(F.softmax/paddle.max 等,传原生 axis=/name=)一律拿原生实现,故 TransformerEncoderLayer、原生 MultiHeadAttention 等组合层在 level=2 下正常运行

测试

  • 新增命名空间别名 + level=2 单测;本 PR 仅改动 paddle/compat/,不触碰 paddle 核心。
  • paconvert 转换后注入 enable_compat(level=2),其现有全量 API 单测即为 level=2 的全量行为测量。

PaddlePaddle-bot

This comment was marked as outdated.

…Co-Authored-By: Claude <noreply@anthropic.com>
PaddlePaddle-bot

This comment was marked as outdated.

PaddlePaddle-bot

This comment was marked as outdated.

@Manfredss

Copy link
Copy Markdown
Contributor Author

/re-run all-failed

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends paddle.enable_compat() with a new compatibility “level=2” mode that (when enabled globally) aliases public paddle.compat.* APIs onto the live paddle / paddle.nn / paddle.nn.functional namespaces, using caller-aware dispatch to keep Paddle internal calls on native implementations. It also adds guards to avoid recursion when compat wrappers call back into aliased paddle.* APIs, and introduces comprehensive tests for aliasing + lifecycle behavior.

Changes:

  • Add a level-based compat mode where global enable_compat(level=2) aliases paddle.* to paddle.compat.* using caller-aware dispatchers/proxies (and restores exactly on disable).
  • Add compat_api_guard(enable=False) usage in compat wrappers to ensure internal calls hit native implementations under aliasing.
  • Add new test coverage (including fake modules) for namespace aliasing, scoped/global lifecycle behavior, and torch root API mapping.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
test/compat/test_compat_namespace_aliased.py New end-to-end tests for level=2 aliasing, restoration, lifecycle correctness, and caller-aware dispatch behavior.
test/compat/fake_modules/torch_proxy_root_api_module.py Fake module used to validate scoped imports + torch root API capture behavior.
python/paddle/compat/proxy.py Implements level=2 namespace aliasing, caller-aware dispatch, compat suspension guard, root-package override coverage, and improved guard restoration logic.
python/paddle/compat/nn/transformer.py Applies compat_api_guard(enable=False) to internal paths that must remain native under aliasing.
python/paddle/compat/nn/functional/sdpa.py Guards compat SDPA implementation so its internals consistently use native implementations.
python/paddle/compat/nn/functional/init.py Guards compat unfold to ensure internal calls remain native under level=2 aliasing.
python/paddle/compat/nn/init.py Guards compat AvgPool* forward; adjusts Softmax.extra_repr (but currently introduces a bug).
python/paddle/compat/init.py Exposes compat dispatch/guard helpers and guards several top-level compat functions to prevent recursion under aliasing.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines 840 to 842
def extra_repr(self) -> str:
return f"dim={self._dim}"
return f"dim={self.dim}"

Comment on lines +418 to +422
# (live module, attr name) -> original value (or ``_MISSING``) for every
# ``paddle.*`` symbol aliased to its ``paddle.compat.*`` counterpart, so the
# paddle namespace can be restored exactly on ``disable_compat()``.
_PADDLE_NAMESPACE_SAVED: dict[tuple[types.ModuleType, str], Any] = {}

@Manfredss

Copy link
Copy Markdown
Contributor Author

/re-run all-failed

PaddlePaddle-bot

This comment was marked as outdated.

@Manfredss

Copy link
Copy Markdown
Contributor Author

/re-run all-failed

1 similar comment
@Manfredss

Copy link
Copy Markdown
Contributor Author

/re-run all-failed

@zhwesky2010

Copy link
Copy Markdown
Contributor

@Manfredss

  1. 在paconvert的全局变量修改为paddle.enable_compat(level=2),全量API的测试能否通过
  2. 在paddle的单测中找一些调用split、max、min...的组合API,比如vsplit、dsplit这些,挑10来个API单测,在首行设置paddle.enable_compat(level=2),确保非compat API行为不会被波及

也就是确保paddle.enable_compat(level=2)仅修改对外的 paddle.* ,而对框架内部API的互相组合引用无任何影响,这点务必充分测试保证,并写死到paddle、paconvert各自API单测里

@liuhao2638 liuhao2638 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已复查当前提交。此前由 risemeup1111 提出的 paddle.compat lazy attribute 和 compat public API 删除问题,在当前代码中看起来已恢复;本轮新增的阻塞问题已放在行级评论里,主要是公开 API 装饰器导致 API spec 签名变化并触发 Static-Check 失败。

P1 优先级:P1
非行级:测试覆盖缺口不能准确附着到单一 diff 行。当前 Paddle 侧新增测试覆盖了几个代表性层和 Tensor 方法,但还没有把 enable_compat(level=2) 写入约 10 个会在内部调用 split/max/min 的组合 API 用例(例如 vsplit/hsplit/dsplit 及相关组合路径),也没有在本 PR 可见范围内给出 PaConvert 全量 API 在 level=2 下通过的结果。请补齐这些回归用例或在 PR 中给出对应测试记录,确保 level=2 只影响外部 paddle.* 调用,不波及 Paddle 内部组合 API。

P3 优先级:P3
非行级:PR 描述不在 changed diff line 上。当前描述仍写 enable_compat()paddle.X 即变为 compat、通过 _native helper 回调 native、以及 flash_attention.py 显式传 F.softmax(product, -1);但当前实现实际是默认 level=1、只有 enable_compat(level=2) 才启用 paddle.* namespace alias,并通过 compat_api_guard/caller-aware dispatch 处理内部调用,flash_attention.py 当前也仍是 F.softmax(product)。建议同步描述,避免后续 reviewer 和用户按旧语义理解这个开关。

Powered by Nyanpasu with gpt-5.5 xhigh, please check the suggestions carefully.

Comment thread python/paddle/compat/proxy.py Outdated
Comment on lines +447 to +456
@contextmanager
def compat_api_guard(enable: bool = True) -> Generator[None, None, None]:
"""Suspend/restore compat dispatch for the current thread / async task only
(task-local). Compat APIs use ``@compat_api_guard(enable=False)`` so their own
``paddle.*`` calls hit native instead of re-dispatching into compat."""
token = _COMPAT_SUSPENDED.set(not enable)
try:
yield
finally:
_COMPAT_SUSPENDED.reset(token)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 优先级:P1

compat_api_guard 现在由 contextlib.contextmanager 生成后直接当公开 API 装饰器使用,会把被装饰函数在 inspect.getfullargspec() 下暴露成 *args/**kwds。当前 Static-Check 已经因此在 paddle.compat.nn.Unfold.forwardpaddle.compat.nn.functional.scaled_dot_product_attention 上报 api_params_diff,这会阻塞 API 兼容检查。请改成同时支持 with 和装饰器、并保留原函数签名的实现,或把这些公开 API 内的 guard 改成函数体里的 with

一个集中修复的形状如下:

Suggested change
@contextmanager
def compat_api_guard(enable: bool = True) -> Generator[None, None, None]:
"""Suspend/restore compat dispatch for the current thread / async task only
(task-local). Compat APIs use ``@compat_api_guard(enable=False)`` so their own
``paddle.*`` calls hit native instead of re-dispatching into compat."""
token = _COMPAT_SUSPENDED.set(not enable)
try:
yield
finally:
_COMPAT_SUSPENDED.reset(token)
class _CompatApiGuard:
def __init__(self, enable: bool = True) -> None:
self.enable = enable
self._token: Any | None = None
def __enter__(self) -> None:
self._token = _COMPAT_SUSPENDED.set(not self.enable)
def __exit__(self, exc_type: Any, exc: Any, tb: Any) -> None:
token = self._token
self._token = None
if token is not None:
_COMPAT_SUSPENDED.reset(token)
def __call__(self, func: Any) -> Any:
@wraps(func)
def wrapper(*args: Any, **kwargs: Any) -> Any:
with type(self)(self.enable):
return func(*args, **kwargs)
wrapper.__signature__ = inspect.signature(func)
return wrapper
def compat_api_guard(enable: bool = True) -> _CompatApiGuard:
"""Suspend/restore compat dispatch for the current thread / async task only
(task-local). Compat APIs use ``@compat_api_guard(enable=False)`` so their own
``paddle.*`` calls hit native instead of re-dispatching into compat."""
return _CompatApiGuard(enable)

…omposite APIs

enable_compat(level=2) only aliases the outward paddle.* surface; composite APIs
(vsplit/chunk/quantile/nan_to_num/nll_loss/embedding/histogram*/GRUCell/LSTMCell/
clip_grad_norm_) that internally call split/sort/max/equal/unique must stay native
via caller-aware dispatch. Verifies zhwesky2010's review request.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PaddlePaddle-bot

This comment was marked as outdated.

@Manfredss

Copy link
Copy Markdown
Contributor Author

/re-run all-failed

1 similar comment
@Manfredss

Copy link
Copy Markdown
Contributor Author

/re-run all-failed

…ghtweight core

The newly-added test hit the added-UT CI gate (--repeat-until-fail 3 --timeout
15/20) on Linux-CPU/XPU: heavy/exotic ops (GRUCell/LSTMCell with unseeded randn,
histogramdd, in-place embedding renorm) caused nondeterministic abort+timeout,
while Windows/Mac (no such gate) passed. Reduce to fixed-input, lightweight core
composites (vsplit/hsplit/dsplit/chunk, quantile, nan_to_num, histogram_bin_edges,
nll_loss) that still verify internal split/sort/min/max/equal stay native.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@risemeup1111 risemeup1111 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已复查新提交。Paddle 侧新增了 test_compat_level2_internal_composites.py,覆盖 vsplit/hsplit/dsplit/chunkquantilenan_to_numhistogram_bin_edgesnll_loss 等在 level=2 下仍需走 native 内部调用的轻量路径,上一轮关于 Paddle 单测覆盖的缺口已部分补齐。本轮没有新增行级评论;此前公开 API 签名变化的问题仍由已有行级评论覆盖。

P2 优先级:P2
非行级:跨仓 PaConvert 验证不在本 PR diff 行内。关联的 PaConvert PR 已改为注入 paddle.enable_compat(level=2),但当前仍未看到全量 API/CI 通过结果,且关联 PR 的 Common API Consistency、Distributed、Modeltest 等检查仍处于失败/未完成状态。请补充通过记录或修复对应失败后再把这里作为完整验证闭环。

P3 优先级:P3
非行级:PR 描述不在 changed diff line 上。当前描述仍按 plain paddle.enable_compat() 即启用 namespace alias、_native helper、以及 flash_attention.py 显式 F.softmax(product, -1) 来表述;当前实现实际是默认 level=1level=2 才启用 alias,并通过 compat_api_guard/caller-aware dispatch 处理内部调用。建议同步描述到当前实现,避免 reviewer 和用户按旧语义理解。

Powered by Nyanpasu with gpt-5.5 xhigh, please check the suggestions carefully.

PaddlePaddle-bot

This comment was marked as outdated.

@Manfredss

Copy link
Copy Markdown
Contributor Author

/re-run all-failed

1 similar comment
@Manfredss

Copy link
Copy Markdown
Contributor Author

/re-run all-failed

@risemeup1111 risemeup1111 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已复查本次测试更新:test_compat_level2_internal_composites.py 补充了 tensor_split 覆盖,Paddle 侧内部组合 API 的轻量回归覆盖进一步收窄;未发现这次测试改动带来新的行级问题。此前最早提出的 paddle.compat lazy attribute 和 compat public API 删除问题在当前代码中仍为已恢复。公开 API 签名问题当前实现未变化,仍由已有行级讨论覆盖,本轮不重复评论。

P2 优先级:P2
非行级:跨仓 PaConvert 验证不在本 PR diff 行内。关联 PaConvert PR 目前已继续调整 enable_compat(level=2) 注入/重置逻辑,但当前 CI UnittestCommon API Consistency CheckModeltest Check 仍失败,Distributed/GPU 也尚未全部完成。请补充全量 API/CI 通过记录,或先修复关联失败后再把这里作为完整验证闭环。

P3 优先级:P3
非行级:PR 描述不在 changed diff line 上。当前描述仍按 plain paddle.enable_compat() 即启用 namespace alias、_native helper、以及 flash_attention.py 显式 F.softmax(product, -1) 来表述;当前实现实际是默认 level=1level=2 才启用 alias,并通过 compat_api_guard/caller-aware dispatch 处理内部调用,flash_attention.py 当前也仍是 F.softmax(product)。建议同步描述到当前实现,避免 reviewer 和用户按旧语义理解。

Powered by Nyanpasu with gpt-5.5 xhigh, please check the suggestions carefully.

@PaddlePaddle-bot

PaddlePaddle-bot commented Jul 7, 2026

Copy link
Copy Markdown

🤖 Paddle-CI-Agent | ci_status_monitor | 2026-07-09 21:34:17 UTC+08:00

CI报告基于以下代码生成(30分钟更新一次):
PR commit: 47414b1 | Merge base: 2235ce4 (branch: develop)


1 Required任务 : 45/48 通过

总执行(rerun次数) 总任务 ✅ 通过 ❌ 失败 ⏳ 运行中 ⏸️ 等待中 跳过
133(53) 80 77 3 0 0 0
任务 错误类型 置信度 日志
Static-Check / Test PR问题:API变更缺少审批 Job
Fleet Unit test (multi-card) 环境问题:PaddleFleet缺少torch依赖 Job
Fleet Unit test (single card) 环境问题:Triton无active driver Job

2 失败详情

🔴 Static-Check / Test — PR问题(置信度: 高)

分析器: 通用分析(fallback)
失败用例: API approval 静态检查

用例 错误摘要
tools/check_api_approvals.sh paddle.enable_compat 签名/文档变更,paddle.use_compat_guard 文档变更,缺少所需审批

关键日志:

Please find RD for approval first, and then find TPM for approval.
You must have one RD approval for API change.
You must have one member of Typing group approval for API annotation change.
There are 2 approved errors.
API Difference is:
- paddle.enable_compat ... kwonlyargs=['scope', 'blocked_modules', 'backend', 'silent']
+ paddle.enable_compat ... kwonlyargs=['scope', 'blocked_modules', 'backend', 'level', 'silent']
- paddle.use_compat_guard ... ('document', '039a648d25b7aa14d1652a68e7023190')
+ paddle.use_compat_guard ... ('document', 'f456ec26b0a80dc51d68fda602c88a80')
  • 根因摘要: API变更未完成审批
    本 PR 修改了 paddle.enable_compat 的公开签名,新增 level 关键字参数,并改变了 paddle.use_compat_guard 的文档签名哈希。Static-Check 的 API approval 规则要求 API 变更获得 RD 审批,类型标注/签名相关变更获得 Typing 组审批,因此以 exit code 6 失败。

修复建议:

  1. 按日志要求补齐 RD 审批和 Typing 组审批;如果 level 不应暴露为公开 API,则回滚或调整签名/文档后重新触发 Static-Check。

关联变更: python/paddle/compat/proxy.pyenable_compat / use_compat_guard 相关签名和文档变更。

🔴 Fleet Unit test (multi-card) — 环境问题(置信度: 高)

分析器: 通用分析(fallback)
失败用例: PaddleFleet MoE 多卡测试

用例 错误摘要
/paddle/tests/multi_card_tests/moe/test_hybrid_ep_fusion.py::TestHybridEPFusion::test_hybrid_ep_moe_fusion_and_overlap_segments PaddleFleet HybridEP 运行时导入依赖失败:No module named 'torch'

关键日志:

Running multi-card test: /paddle/tests/multi_card_tests/moe/test_hybrid_ep_fusion.py with 8 GPUs
ERROR: test_hybrid_ep_moe_fusion_and_overlap_segments
File "/usr/local/lib/python3.12/site-packages/paddlefleet/transformer/moe/fused_a2a.py", line 1026, in hybrid_ep_dispatch
self.runtime = hybrid_ep_cpp.HybridEPBuffer(
ModuleNotFoundError: No module named 'torch'
Test FAILED: /paddle/tests/multi_card_tests/moe/test_hybrid_ep_fusion.py
  • 根因摘要: PaddleFleet依赖torch缺失
    失败发生在安装后的 paddlefleet 包路径中,调用 HybridEP fused A2A 运行时需要 torch 模块,但当前 Fleet 多卡 CI 环境没有该依赖。PR 变更文件集中在 python/paddle/compat/*test/compat/*,未修改该 PaddleFleet MoE 测试或外部依赖安装逻辑,因此更符合 CI 环境/依赖问题。

修复建议:

  1. 环境问题,请 rerun;若持续失败,需检查 Fleet 多卡镜像/安装脚本是否应安装 torch 或调整 PaddleFleet 的可选依赖声明。

关联变更: 未发现与本 PR 修改文件的直接关联。

🔴 Fleet Unit test (single card) — 环境问题(置信度: 高)

分析器: 通用分析(fallback)
失败用例: PaddleFleet 单卡 Triton 扩展测试

用例 错误摘要
test_ai_rr_attn_estimate_triton_op_4.py::TestPrepareStrideMaxminPtrs Triton runtime 检测到 0 个 active driver,kernel 无法启动

关键日志:

Running single card test: tests/single_card_tests/ai_edited_test/extensions/test_ai_rr_attn_estimate_triton_op_4.py
TestPrepareStrideMaxminPtrs.test_mode_1_returns_stride_ptrs
/usr/local/lib/python3.12/site-packages/paddlefleet_ops/_extensions/flashmask/index_utils.py:71: in prepare_maxmin
/usr/local/lib/python3.12/site-packages/triton/runtime/driver.py:20: RuntimeError
RuntimeError: 0 active drivers ([]). There should only be one.
  • 根因摘要: Triton运行时未识别GPU driver
    3 个 _prepare_stride_maxmin_ptrs 用例在调用 paddlefleet_ops 的 Triton kernel 时同样失败,错误来自 Triton driver 选择阶段,提示 active driver 数为 0。PR 未修改该测试、paddlefleet_ops 或 Triton 运行路径,因此判断为 CI 运行环境/驱动识别问题。

修复建议:

  1. 环境问题,请 rerun;若持续失败,需检查该 runner 的 GPU/Triton 后端激活状态和 PaddleFleet 单卡镜像配置。

关联变更: 未发现与本 PR 修改文件的直接关联。

@Manfredss

Copy link
Copy Markdown
Contributor Author

/re-run all-failed

PaddlePaddle-bot

This comment was marked as outdated.

@Manfredss

Copy link
Copy Markdown
Contributor Author

/re-run all-failed

1 similar comment
@Manfredss

Copy link
Copy Markdown
Contributor Author

/re-run all-failed

@risemeup1111 risemeup1111 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已复查最新提交。compat_api_guard 已从 contextmanager 改为同时支持上下文和装饰器的 class,并在装饰器路径保留 __signature__;我用等价最小复现确认 inspect.getfullargspec 不再退化成 *args/**kwargs,此前公开 API 签名变化的 P1 问题看起来已由新 commit 修复。最早提出的 paddle.compat lazy attribute 和 compat public API 删除问题当前也仍为已恢复。本轮没有新增行级评论。

P2 优先级:P2
非行级:跨仓 PaConvert 验证不在本 PR diff 行内。当前关联 PaConvert PR 的 checks 已显示全部通过,我倾向于认为验证证据已补齐;但该项此前作为 P2 跟进意见提出,仍需要作者在本 PR 中回复确认对应验证记录。

处理要求:请针对该评论进行回复(同意并已修改请回复 Done,不同意请说明理由)。

P3 优先级:P3
非行级:PR 描述不在 changed diff line 上。当前描述仍按 plain paddle.enable_compat() 即启用 namespace alias、_native helper、以及 flash_attention.py 显式 F.softmax(product, -1) 来表述;当前实现实际是默认 level=1level=2 才启用 alias,并通过 compat_api_guard/caller-aware dispatch 处理内部调用,flash_attention.py 当前也仍是 F.softmax(product)。请同步描述到当前实现。

处理要求:请针对该评论进行回复(同意并已修改请回复 Done,不同意请说明理由)。

Powered by Nyanpasu with gpt-5.5 xhigh, please check the suggestions carefully.

@PaddlePaddle-bot PaddlePaddle-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Paddle-CI-Agent | pr_review | 2026-07-08 19:48:21

📋 Review 摘要

PR 概述:为 paddle.enable_compat(level=2) 增加 paddle 命名空间到 paddle.compat 的 caller-aware alias,并补充内部 composite 调用保护与测试。
变更范围python/paddle/compatpaddle.compat.nn/functional、compat namespace 回归测试。
影响面 TagUser Experience

问题

未发现阻塞性问题。PR 规范问题在下面章节报,不要在这里重复

历史 Findings 修复情况

Finding 问题 状态
F1 enable_compat() 默认仍是 level=1,与 plain paddle.enable_compat() 迁移入口不一致。 ⚠️ 仍存在
F2 dispatcher 使用 @wraps(native_fn),level=2 后公开 paddle.* 仍暴露 native 元数据/签名。 ⚠️ 仍存在
F3 新测试在模块导入阶段永久追加 fake_modules 到全局 sys.path ⚠️ 仍存在

📝 PR 规范检查

标题 Tag API Compatibility 不在当前 Paddle 模板的 PR Category / PR Types 枚举中。描述包含必填四段,且“是否引起精度变化”为“否”,结构符合模板。

标题建议(可直接复制):

  • [User Experience] enhance paddle.enable_compat level=2 namespace aliasing

总体评价

本轮按 Python 公开 API/兼容性路径检查了 namespace alias、内部 guard、scope lifecycle、Tensor method patch 和新增测试。未新增 inline findings;历史 3 条仍未在当前 diff 中修复,建议后续一并收敛。

@Manfredss

Copy link
Copy Markdown
Contributor Author

/re-run all-failed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants