RFE: %{_system_bindir} and such macro

[ignatenkobrain]

When people build with RPMs as part of flatpaks, they are changing %_bindir to /app/whatsoever which is entirely valid case. But that means, you can't rely on BuildRequires: %{_bindir}/foo. Specifying /usr/bin/foo in BRs is ugly as well (because what if base system is actually installed in different location).

I think we need to add some number of %_system_… macro which will be set during build of RPM and won't change when you change %_bindir` for example.

xref: https://pagure.io/packaging-committee/pull-request/847

[ignatenkobrain]

this would also help with cases where people define their own macro, for example %meson used to be a %{_bindir}/meson, but I had to change it to hardcode /usr/bin because once you redefine %_bindir, it just breaks...

[jasontibbitts]

This just reveals how redefining of %_bindir (or I guess more correctly they're redefining %_prefix) is not an ideal solution to the problem they're facing. I honestly have no problem with specifying a full path in a build dependency in the case that you can't for some reason specify a package dependency, but the problem does go beyond that.

Unfortunately I don't see any general solution besides what's being proposed here: different prefixes and path macros defined from them for different contexts. Adding a "_system" seems as logical as any other.

I just hope that this kind of thing wouldn't get mass-pushed into packages and instead used only where it's actually necessary.

[owtaylor]

I'm not opposed to this proposal, but it does seem perhaps like overcomplicating things.

Do we have examples where the %_system* macros would differ from the expected FHS values and it would matter? %_libdir is probably the most variable thing across different systems, but BuildRequires: %_libdir is not common.

The actual %meson change @ignatenkobrain landed substitutes Meson's bindir into the macro file (https://src.fedoraproject.org/rpms/meson/c/795680bbd10068673580261e134e2f4fbac4072d?branch=master) - which IMO is better tham %_system_bindir.

[ignatenkobrain]

The actual %meson change @ignatenkobrain landed substitutes Meson's bindir into the macro file (https://src.fedoraproject.org/rpms/meson/c/795680bbd10068673580261e134e2f4fbac4072d?branch=master) - which IMO is better tham %_system_bindir.

Yes, and I'm not happy about this because every distribution would have to carry such sed line.

[voxik]

Just FTR, there is already precedent in software collections, where _root_ prefix [1] is used for the same purpose. It would be nice to reuse this prefix instead of coming with new one.

[ignatenkobrain]

agree, %_root_* sounds good.

@ffesti @pmatilai would you like to add some comment here?

[pmatilai]

I do prefer "root" over "system" as it's both more precise and generic.

[keszybz]

An alternative approach: let's accept the fact that %_bindir=/usr/bin is here to stay. Using a macro for "system binaries directory" is a legacy of old times and we can just as well use the fixed path of "/usr/bin" for that everywhere. (Using a macro for "where should I install my binaries", which can be /usr/bin or something else in case of flatpaks, is a different story and is OK.)

Specifying /usr/bin/foo in BRs is ugly as well (because what if base system is actually installed in different location).

I disagree. It's legible and perfectly fine. The system is not going to be installed in a different location.

[dmnks]

I'm leaning more towards what @keszybz suggests above, which is that we need to clarify the semantics of these path-based macros in the first place. Fedora in particular prefaces the relevant section with the following sentence (emphasis mine):

The macros for build system invocations (for example, %configure, %cmake, or %meson) use the values defined by RPM to set installation paths for packages.

From a packager's point of view, there are two kinds of paths:

1. Those that the package's own files should be installed into. Using macros here makes sense as it allows the build system to override the default prefix with a different value, Flatpaks and Software Collections being the most obvious examples.

2. Those that the package expects on the installed (build) system. The build system is not responsible for these, meaning that there's no reason to assume the paths could be overridden to non-default values, meaning that those can/should be used explicitly.

It really is more of a philosophical question. Adding a collection of _root macros then just seems like a "remedy" for bad packaging. We already have enough confusing macros, see e.g. _host_cpu, _build_cpu, _target_cpu and similar. Adding more of that doesn't seem like a particularly good idea...

[voxik]

Adding a collection of _root macros then just seems like a "remedy" for bad packaging.

I don't care that much what the prefix is, but this kind of macros was essential for SCLs, which allowed to have one source package, but the build configuration changed the destination. In this scenario, there really needs to be distinction between files from system or from some alternative destination.

And I believe the same applies to Flatpak.

But if you want to have philosophical discussion, then don't forget that rpmbuild on itself is using this concept, where it is necessary to distinguish between content from system and from %{_builddir} / %{_buildroot}

[dmnks]

I don't care that much what the prefix is, but this kind of macros was essential for SCLs, which allowed to have one source package, but the build configuration changed the destination. In this scenario, there really needs to be distinction between files from system or from some alternative destination.

Which is what I'm saying. The distinction already exists, you either use the macro (e.g. in %files) or you don't (e.g. in BuildRequires). Why have another set of macros for the latter?

The %_prefix macro, which all the other macros inherit from, doesn't say anything useful about the "common prefix" in the system. In fact, there's no such thing. The macro is set at RPM's own configure time, from the cmake install prefix. The distributions can choose to override that default as they see fit.

But if you want to have philosophical discussion, then don't forget that rpmbuild on itself is using this concept, where it is necessary to distinguish between content from system and from %{_builddir} / %{_buildroot}

See above.

[dmnks]

In yet another words, AIUI, the two reasons we have the path macros are:

1. Save the packagers some typing
2. Allow for overriding the installation prefix (SCL, Flatpak, ...)

The only advantage of having such _root macros would be, in my opinion, the former of the above. Which itself doesn't seem like a good reason for another set of macros given that the use case is not so common.

[dmnks]

The distributions can choose to override that default as they see fit.

If this happens, then yes, the (hypothetical) distribution may also want to provide such a set of _root macros so that the packages can refer to those.

Overriding the default %_prefix when building a package (as is the case with SCL or Flatpak) is something that happens "outside" of RPM itself. Which makes me think that maybe these macros should perhaps indeed exist but rather in an external package (e.g. redhat-rpm-config in Fedora).

[dmnks]

Yet another way to put it:

RPM providing a set of %_root* macros would basically mean: "In case you choose to override the default %_prefix, these paths will be based on the original value before the override". By that logic, why stop at the path macros? Shouldn't RPM provide the "pre-override" variants for all the macros? That just sounds backwards.

The reality is, the reason why this is needed is technology-specific, i.e. Flatpak or SCL in this case. Therefore, I think those stacks should also provide the alternative macros (which is what SCL already does).

[voxik]

The thing is that I'll be fine if RPM does not provide any _root prefixed macros. But I'll be sad if Flatpak and RPMs will use different prefix for the same thing. You might leave this to SCL or Flatpak. OTOH, RPM is the common technology for both of them and the place where the technologies could unite and benefit from what RPM provides. I don't think there is any benefit if everybody needs to reinvent this from the ground once again.

BTW I also agree that having available "pre-override" variants of macros might be nice from time to time 😉

[dmnks]

The thing is that I'll be fine if RPM does not provide any _root prefixed macros. But I'll be sad if Flatpak and RPMs will use different prefix for the same thing. You might leave this to SCL or Flatpak. OTOH, RPM is the common technology for both of them and the place where the technologies could unite and benefit from what RPM provides. I don't think there is any benefit if everybody needs to reinvent this from the ground once again.

Agreed on that one. I'm not opposed to the idea of making the packaging job easier and more convenient, of course. In fact that's actually one of our goals on the roadmap (under the rather grand and ambiguous "hands-free packaging" umbrella).

In this case, the discussion is more about the "where do we do this" than it is about "if". RPM needs to stay distro-agnostic but maybe there's some underlying pattern here.

BTW I also agree that having available "pre-override" variants of macros might be nice from time to time 😉

Yup, seems like we've (partially) identified that pattern 😄

[keszybz]

Save the packagers some typing

I generally agree with the rest of the discussion, but I want to reply to this point in particular:
I very much disagree that those macros make things easier. Typing /usr/bin is not any harder than typing %{_bindir}. The original form is shorter and also has characters which are more accessible on the keyboard. For me the process of substituting the macro forms is an annoyance that only slows me down. First I have to figure out what the directory is, so I look in the file system, so I have the path at hand, ready to select&paste, and that'd be it, but then I have to find the right macro. Is it %_datadir or is it %_sharedstatedir or is %_configdir? The names are arbitrary and confusing and introduce obscurity and make it easier to make mistakes.

As far as I'm concerned, such macros to "save typing" are not worth using and should not be recommended at all.

[dmnks]

Actually, now that I really think about it, I must agree with that, too 😄 It might have been one of the motivations originally (I have no facts to back up that claim, though) but indeed, whenever I'm making dummy packages, I just type out the paths by hand, precisely because of you what you said: It's easier to touch-type without reaching for those special chars, and it's easier to not have to consult the table (because I typically forget the macro names too).

[dmnks]

The names are arbitrary

That said, I don't think they're completely arbitrary, the names have (most likely) been copied from this GNU standard:
https://www.gnu.org/prep/standards/html_node/Directory-Variables.html

[pmatilai]

It actually cites the source:

rpm/macros.in

Line 960 in 6714ec7

# Macro(s) slavishly copied from autoconf's config.status.

[pmatilai]

And yeah, the point of those macros is NOT to make it easier to type. They're there for flexibility.
Those of us who lived through the rise of the multilib will remember fixing a thousand specs wrt /usr/lib -> %{_libdir} to accommodate for that. And /usr/doc -> /usr/share/doc change before that. It's not likely that something like /usr/bin gets moved to some different location now, but they still serve the flexibility purpose in allowing for entirely different repackaging with a few simple defines - like Flatpak, SCL and so on.

But, that "few simple defines" doesn't hold in practise because these macros are ambiguous: do they refer to the system or the package being built? Traditionally these are the same thing, but it falls apart with Flatpak, SCL and so on, and changing the prefix can and probably does break all manner of other helpers which just slavishly use the macros as alleged system level paths. And hence we have this ticket.

As with any ambiguity, the fix must start forcing the ambiguity out, and making people choose what they intend. As painful as that sometimes is (looking at you, Python 3)

[ppisar]

The distinction of refer-to-system and refer-to-being-built is simple when describing a single package.

But when we build a package which delivers a directory and then another package which installs into that directory, a path which was refer-to-being-built in the former package cannot be refer-to-system in the latter package. The refer-to-being-built property spreads transitively against a dependency tree.

Where does the spread ends? Where the packaged stacks want. That's the problem: it's specific to a stack. And that makes me feel that no single macro can satisfy them all.