feat: allow user-defined string formats (close #1227)

[itchyny]

This commit extends the "def" syntax to define string formats.
By writing def @foo: ...;, @foo string format is available
in the following query. This idea is influenced by jaq. Close #1227.

[itchyny]

Thank you. This is a language change. Any thoughts from other maintainers?

[pkoppstein]

@itchyny wrote:

This is a language change. Any thoughts from other maintainers?

Thanks for the opportunity to ask what backward-compatibility issue this change entails.

My main question (posed purely out of ignorance) is whether the ability to use the @ filters both as ordinary 0-arity filters (... | @tsv) and as "prefix filters" (@tsv "abc \([1,2])") would be compromised.

[myaaaaaaaaa]

This commit extends the "def" syntax to define string formats. By writing def @foo: ...;, @foo string format is available in the following query.

How about instead redefining the @foo syntax to be syntactic sugar for calling a regular foo function? That way, users could do something like this:

$ "world" | @ascii_upcase "hello \(.)"
"hello WORLD"

Note how ascii_upcase is a function that is already defined by jq

[itchyny]

How about instead redefining the @foo syntax to be syntactic sugar for calling a regular foo function?

Since existing string formatters are named to represent the resulting string (e.g. HTML string, URI string, etc.), I think it is better to have a different namespace from the 0-arity filters. Also, they are limited in the type they output and jq doesn't check typing statically, so the syntactic sugar for invoking the 0-arity filter would allow queries that wouldn't actually work.

[myaaaaaaaaa]

Also, they are limited in the type they output and jq doesn't check typing statically, so the syntactic sugar for invoking the 0-arity filter would allow queries that wouldn't actually work.

I don't think this argument holds up - it's just as possible to define a @formatter that wouldn't work, such as def @foo: {};.

And if there were some way to statically analyze a def @foo body to catch such a mistake, is there anything preventing the same techniques from being used on filters invoked by the @ operator?

Since existing string formatters are named to represent the resulting string (e.g. HTML string, URI string, etc.), I think it is better to have a different namespace from the 0-arity filters.

I agree that adding html, uri, ... as regular builtins would cause namespace pollution. On the other hand, I don't think def @foo is the correct solution, given that it essentially introduces a second category of 0-arity filters that are identical in the way they're defined, differing only in the way they're called.

Is this really the best way to solve the namespace pollution problem? For example, one possible alternative would be to have the @ operator fall back on a fmt_ prefix - the query @foo would first attempt to resolve to foo, then to fmt_foo if that doesn't exist. This way, we could add the builtin formatters as fmt_html, fmt_uri, ..., and still use @html, @uri, ...

[itchyny]

@myaaaaaaaaa String formatters can be called without strings (jq "@csv"). How would you like to redefine such syntax?

[myaaaaaaaaa]

I think the most consistent approach would be to redefine @foo to mean something like @foo "\(.)", rather than simply making it an alias for foo. For example, this would allow jq to report an error if foo returns something other than a string.