Request: Powerful use-case for macros

[rebcabin]

Issue #1908 is a plea for CTR in LPython in the distant future. CTR enables macros. However, even a special-purpose, simplified macro system à la macropy would be immediately useful for cases like the following:

    assert a_char == '(', "a_char == '('"

With even very simple macros, this could be shortened to

    assertm a_char == '('

[certik]

Or in this case we can simply improve our backend to print a much better error message for assert a_char == '(' that includes the string itself, line number and a stacktrace.

[rebcabin]

wouldn't that be a language extension, thus violating one of our tenets? Macros can be achieved in ordinary CPython!

[Thirumalai-Shaktivel]

Or in this case we can simply improve our backend to print a much better error message for assert a_char == '(' that includes the string itself, line number and a stacktrace.

Yup, currently we have the following:

$ lpython examples/expr2.py -g --debug-with-line-column
  File "examples/expr2.py", line 2
    assert t == 'c'
AssertionError

We can improve this or give it a try to use macros.

I opened Issue #2009, which makes this use-case for macros go away :) However, the discussion is still valuable, because it clarifies use cases for CTR, in general, from use cases for macros, in particular.

[certik]

The tenet is that if it runs in LPython, it will run in CPython and produce the same result. Our exceptions and failure modes do produce slightly different results (the stacktrace might look slightly differently, depending on how you compile it). I think we have some wiggle room there, and I thought a nicer assert error message would still fit the tenet. But I agree it's borderline and one can argue against doing that.

The issue with macropy is that last commit was 6 years ago and I worry it will take the same route as the goto, that we eventually removed, because a new CPython version broke it.

I feel macropy is actually extending Python, just like goto. It's extending Python with "hacks" that it was not meant to do. It works, until it gets broken by upstream. I would only commit to CPython as released upstream and its features.

We do support somethings like structs and unions, but we do it via ctypes, which are supported upstream, so I think that's fine.

So I would not do macros, since Python doesn't have macros, so it would be against the tenets. But making an error message from assert slightly nicer I think would be fine and within tenets. :)

[rebcabin]

I disagree with your last point (again, very long term). Macropy employed a standard feature of LPython, that the AST is exposed at import time. Yes, macropy is out of maintenance, but that's beside the point. Direct AST rewrites at import time are still part of the Python standard, sfaik. I'm not saying we implement macropy the way we did goto. I am proposing that we embrace AST-rewriting as part of Python and integrate it with all the rest of LPython's unique symbolic capabilities: ASR, reflectively and SymPy / SymEngine integration. Macros are just one exploitation of AST rewriting. Rewriting is MUCH deeper and, again, very long term, should be a powerful differentiator of LPython -- without extending Python! -- from other compilers and implementations.

…

On Wed, Jun 21, 2023 at 7:48 AM Ondřej Čertík ***@***.***> wrote: The tenet is that if it runs in LPython, it will run in CPython and produce the same result. Our exceptions and failure modes do produce slightly different results (the stacktrace might look slightly differently, depending on how you compile it). I think we have some wiggle room there, and I thought a nicer assert error message would still fit the tenet. The issue with macropy is that last commit was 6 years ago and I worry it will take the same route as the goto, that we eventually removed, because a new CPython version broke it. I feel macropy is actually extending Python, just like goto. It's extending Python with "hacks" that it was not meant to do. It works, until it gets broken by upstream. I would only commit to CPython as released upstream and its features. We do support somethings like structs and unions, but we do it via ctypes, which are supported upstream, so I think that's fine. So I would not do macros, since Python doesn't have macros, so it would be against the tenets. But making an error message from assert slightly nicer I think would be fine and within tenets. :) — Reply to this email directly, view it on GitHub <#1972 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABSRR2LWWIMFHGEWRJH6V3XMMCU7ANCNFSM6AAAAAAZNYBGBM> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[certik]

Macros are powerful, but remember that AST changes from CPython version to version, and there are even some very slight differences with Python AST and LPython AST. So if we do it, we must tackle these challenges.

[rebcabin]

I totally agree! That's why macropy went out of maintenance! This is not a small job, and for the far future if ever. I'm NOT selling macros, in particular. I AM selling CTR (Conditional Term Rewriting) as something we should take very seriously to propel LPython into the world's BEST symbolic programming language SOMEDAY :) Beyond Mathematica. My plea is in Issue #1908.

…

On Wed, Jun 21, 2023 at 12:07 PM Ondřej Čertík ***@***.***> wrote: Macros are powerful, but remember that AST changes from CPython version to version, and there are even some very slight differences with Python AST and LPython AST. So if we do it, we must tackle these challenges. — Reply to this email directly, view it on GitHub <#1972 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABSRR2PVNJHJR7U5DR25OTXMNA7VANCNFSM6AAAAAAZNYBGBM> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[certik]

Note that I thought #1908 is at the ASR level, while macropy is at the AST level. Rewriting ASR is powerful. The issue is that it's hard to do in pure Python mode without just calling LPython (but then you can't use debuggers). AST rewriting will work in both, but then it's tied to Python, unrelated to ASR (and unrelated to ASR->ASR passes).

[rebcabin]

yes, I wasn't all that clear about it. Rewriting applied to both AST and ASR, but it's MUCH more interesting at ASR :)

…

On Wed, Jun 21, 2023 at 1:48 PM Ondřej Čertík ***@***.***> wrote: Note that I thought #1908 <#1908> is at the ASR level, while macropy is at the AST level. Rewriting ASR is powerful. The issue is that it's hard to do in pure Python mode without just calling LPython (but then you can't use debuggers). AST rewriting will work in both, but then it's tied to Python, unrelated to ASR. — Reply to this email directly, view it on GitHub <#1972 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABSRRYISVK6JSGC4EWA5CDXMNMZDANCNFSM6AAAAAAZNYBGBM> . You are receiving this because you authored the thread.Message ID: ***@***.***>