Make it easier to supply custom archive to `packit srpm` #566

TomasTomecek · 2019-10-14T14:20:01Z

I see. Makes sense. But it would be good to have a little bit more easier / straightforward way how to prepare the upstream archive. For fmf I had to add three Makefile targets plus this:
   current_version_command: ["make", "packit-version"]

   actions:
       create-archive:
       - make packit-tarball
       - make packit-path
I believe it should be much simpler/shorter/obvious.

Originally posted by @psss in packit/packit-service#103 (comment)

The text was updated successfully, but these errors were encountered:

jpopelka · 2019-12-05T12:30:59Z

AI:
Write down documentation on how to do this the easiest way.
Deprecate dedicated config options for getting version and creating archive command for sake of actions.

psss · 2019-12-05T15:15:12Z

So there is already a better way, just not yet documented? Or the implementation is also in progress?

lachmanfrantisek · 2019-12-05T15:28:06Z

So there is already a better way, just not yet documented? Or the implementation is also in progress?

We need to document the prefered way for the current state at first. (It's definitely not so clear now.)
We support some values that were replaced by the actions (e.g. current_version_command).
- We want to get rid of those.
After that, we can simplify the config or support more workflows so less people needs to do it.
- Do you have some better way, how it can be configured?

psss · 2019-12-13T13:34:31Z

I think the best way would be just a single command which would either:

show the resulting archive file name in the output (e.g. last line)
or store the archive to the location provided in an environment variable

It should not be necessary to configure the current_version_command.

lachmanfrantisek · 2019-12-13T13:42:35Z

show the resulting archive file name in the output (e.g. last line)

This should work in the current version:
https://github.com/packit-service/packit/blob/master/packit/upstream.py#L376
https://github.com/packit-service/packit/blob/b6d9bf76f021e9053e8c4f52b15408407fd6a923/packit/upstream.py#L377

psss · 2020-01-06T12:15:53Z

Nice, thanks. Trying on psss/edd#10

psss · 2020-01-06T12:36:25Z

Does not seem to be working:

The create-archive action did not output a path to the
generated archive. Please make sure that an absolute path to
the archive is printed.

Here's the output of the make packit-tarball command:

> make packit-tarball
mkdir -p /home/psss/git/edd/tmp/{SOURCES,edd-0.3}
cp -a LICENSE README.rst Makefile edd.spec edd docs /home/psss/git/edd/tmp/edd-0.3
cp docs/man.rst /home/psss/git/edd/tmp
tail -n+16 README.rst >> /home/psss/git/edd/tmp/man.rst
rst2man /home/psss/git/edd/tmp/man.rst | gzip > /home/psss/git/edd/tmp/edd-0.3/docs/edd.1.gz
rst2html README.rst --stylesheet=style.css --link-stylesheet > /home/psss/git/edd/tmp/edd-0.3/docs/index.html
cd /home/psss/git/edd/tmp && tar cfj SOURCES/edd-0.3.tar.bz2 edd-0.3
/home/psss/git/edd/tmp/SOURCES/edd-0.3.tar.bz2

Is there anything wrong with the file path format? Or anything else needs to be done?

lachmanfrantisek · 2020-01-06T12:38:39Z

Is there anything wrong with the file path format? Or anything else needs to be done?

I am just playing with that locally.

lachmanfrantisek · 2020-01-06T12:43:09Z

Using user-defined script for ActionName.create_archive: ['make', 'packit-tarball']
13:37:52.794 utils.py          DEBUG  cmd = 'make packit-tarball'
13:37:52.845 utils.py          DEBUG  b'mkdir -p /home/flachman/Projects/psss/edd/tmp/{SOURCES,edd-0.3}'
13:37:52.848 utils.py          DEBUG  b'cp -a LICENSE README.rst Makefile edd.spec edd docs /home/flachman/Projects/psss/edd/tmp/edd-0.3'
13:37:52.852 utils.py          DEBUG  b'cp docs/man.rst /home/flachman/Projects/psss/edd/tmp'
13:37:52.854 utils.py          DEBUG  b'tail -n+16 README.rst >> /home/flachman/Projects/psss/edd/tmp/man.rst'
13:37:52.858 utils.py          DEBUG  b'rst2man /home/flachman/Projects/psss/edd/tmp/man.rst | gzip > /home/flachman/Projects/psss/edd/tmp/edd-0.3/docs/edd.1.gz'
13:37:53.439 utils.py          DEBUG  b'rst2html README.rst --stylesheet=style.css --link-stylesheet > /home/flachman/Projects/psss/edd/tmp/edd-0.3/docs/index.html'
13:37:54.122 utils.py          DEBUG  b'cd /home/flachman/Projects/psss/edd/tmp && tar cfj SOURCES/edd-0.3.tar.bz2 edd-0.3'
13:37:54.141 utils.py          DEBUG  b'/home/flachman/Projects/psss/edd/tmp/SOURCES/edd-0.3.tar.bz2'
13:37:54.142 base_git.py       DEBUG  Action command output: ['mkdir -p /home/flachman/Projects/psss/edd/tmp/{SOURCES,edd-0.3}\ncp -a LICENSE README.rst Makefile edd.spec edd docs /home/flachman/Projects/psss/edd/tmp/edd-0.3\ncp docs/man.rst /home/flachman/Projects/psss/edd/tmp\ntail -n+16 README.rst >> /home/flachman/Projects/psss/edd/tmp/man.rst\nrst2man /home/flachman/Projects/psss/edd/tmp/man.rst | gzip > /home/flachman/Projects/psss/edd/tmp/edd-0.3/docs/edd.1.gz\nrst2html README.rst --stylesheet=style.css --link-stylesheet > /home/flachman/Projects/psss/edd/tmp/edd-0.3/docs/index.html\ncd /home/flachman/Projects/psss/edd/tmp && tar cfj SOURCES/edd-0.3.tar.bz2 edd-0.3\n/home/flachman/Projects/psss/edd/tmp/SOURCES/edd-0.3.tar.bz2\n']
Created archive: /home/flachman/Projects/psss/edd/tmp/SOURCES/edd-0.3.tar.bz2

It looks like the parsing went well but the build failed, because we expect to have the archive in the current directory:

12:37:54.443 utils.py          DEBUG  cmd = 'rpmbuild -bs --define _sourcedir . --define _specdir /home/flachman/Projects/psss/edd --define _srcrpmdir /home/flachman/Projects/psss/edd --define _topdir /home/flachman/Projects/psss/edd --define _builddir /home/flachman/Projects/psss/edd --define _rpmdir /home/flachman/Projects/psss/edd --define _buildrootdir /home/flachman/Projects/psss/edd edd.spec'
b'error: Bad source: ./edd-0.3.tar.bz2: No such file or directory'
ERROR    Command ['rpmbuild', '-bs', '--define', '_sourcedir .', '--define', '_specdir /home/flachman/Projects/psss/edd', '--define', '_srcrpmdir /home/flachman/Projects/psss/edd', '--define', '_topdir /home/flachman/Projects/psss/edd', '--define', '_builddir /home/flachman/Projects/psss/edd', '--define', '_rpmdir /home/flachman/Projects/psss/edd', '--define', '_buildrootdir /home/flachman/Projects/psss/edd', 'edd.spec'] failed
ERROR    Command ['rpmbuild', '-bs', '--define', '_sourcedir .', '--define', '_specdir /home/flachman/Projects/psss/edd', '--define', '_srcrpmdir /home/flachman/Projects/psss/edd', '--define', '_topdir /home/flachman/Projects/psss/edd', '--define', '_builddir /home/flachman/Projects/psss/edd', '--define', '_rpmdir /home/flachman/Projects/psss/edd', '--define', '_buildrootdir /home/flachman/Projects/psss/edd', 'edd.spec'] failed.
ERROR    Failed to create SRPM: PackitCommandFailedError("Command ['rpmbuild', '-bs', '--define', '_sourcedir .', '--define', '_specdir /home/flachman/Projects/psss/edd', '--define', '_srcrpmdir /home/flachman/Projects/psss/edd', '--define', '_topdir /home/flachman/Projects/psss/edd', '--define', '_builddir /home/flachman/Projects/psss/edd', '--define', '_rpmdir /home/flachman/Projects/psss/edd', '--define', '_buildrootdir /home/flachman/Projects/psss/edd', 'edd.spec'] failed: Command ['rpmbuild', '-bs', '--define', '_sourcedir .', '--define', '_specdir /home/flachman/Projects/psss/edd', '--define', '_srcrpmdir /home/flachman/Projects/psss/edd', '--define', '_topdir /home/flachman/Projects/psss/edd', '--define', '_builddir /home/flachman/Projects/psss/edd', '--define', '_rpmdir /home/flachman/Projects/psss/edd', '--define', '_buildrootdir /home/flachman/Projects/psss/edd', 'edd.spec'] failed.")
ERROR    Failed to create SRPM.

lachmanfrantisek · 2020-01-06T12:45:22Z

What do you/we preffer?

Copy the archive to the root directory.
Edit the rpmbuild command to accept other location?
Anything else what can be done to fix that?

psss · 2020-01-06T12:51:08Z

I think the rpmbuild command should not depend on the archive location. If a full path is provided (and if I understand it correctly this is required by packit) then the archive should be taken from that location. No additional steps (such as moving or copying the archive) should be required. Packit should take care of it.

psss · 2020-01-06T12:55:22Z

Also, while we're looking into this, could we get rid of the current_version_command as well?. Would be nice if the custom tarball could be enabled with just this:

actions:
    create-archive:
    - make packit-tarball

lachmanfrantisek · 2020-01-06T13:06:07Z

I am going to fix the archive path in the specfile to be relative to the spec location. (Currently, we use only the filename.)

Also, while we're looking into this, could we get rid of the current_version_command as well?

It's an old config value from the times before actions, now we have get-current-version action. (We left that because of the backwards compatibility.)
We need to know the version before we run the create_archive command. (e.g. we are forwarding PACKIT_PROJECT_VERSION to the create-archive action)

lachmanfrantisek · 2020-01-06T13:29:30Z

I am going to fix the archive path in the specfile to be relative to the spec location. (Currently, we use only the filename.)

OK, that does not work. We need to:

change the _sourcedir rpmbuild macro or
copy the archive to the root or
use absolute path but the srpm can be built only locally because of the absolute path. (?)

psss · 2020-01-06T13:34:00Z

Changing the _sourcedir macro seems to be the most natural solution to me.

psss · 2020-05-20T17:38:37Z

@lachmanfrantisek, so what is now the shortest way to enable custom archive? Something like this?

actions:
  create-archive:
  - make packit-tarball
  - make packit-path
  get-current-version:
  - make packit-version

Or is it possible now to have only a single create-archive command? The following part of the documentation is not clear to me:

If there are more steps, then one of them has to return the archive name.

Should it return the archive name as the last line? Or?

lachmanfrantisek · 2020-05-20T18:44:27Z

@psss Your example looks good to me.

Or is it possible now to have only a single create-archive command?

Yes, if it contains archive-name in the output.

If there are more steps, then one of them has to return the archive name.

We run the commands one-by-one and then go through the output in reverse order and for each line of the output (also in the reverse order), we check if it is an existing file. If yes, we accept that as a newly created archive. I know, it's a bit complicated but should work for most situations.

psss · 2020-05-21T05:11:08Z

Nice, thanks! Trying with teemtee/tmt#249. Does not work...

Mark recursively searching for a specfile as deprecated The functionality is going to be removed in the future, see packit#1799. Related to packit#1771. Signed-off-by: Hunor Csomortáni [email protected] Reviewed-by: Jiri Popelka <None> Reviewed-by: Tomas Tomecek <[email protected]>

TomasTomecek added kind/feature New feature or a request for enhancement. area/user-experience Usability issue labels Oct 14, 2019

TomasTomecek mentioned this issue Oct 17, 2019

let users provide their own command to generate SRPMs #451

Closed

jpopelka added the triaged This issue was already processed by the team. label Dec 5, 2019

lachmanfrantisek mentioned this issue Jan 7, 2020

Fix archive path #651

Merged

softwarefactory-project-zuul bot closed this as completed in #651 Jan 7, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make it easier to supply custom archive to `packit srpm` #566

Make it easier to supply custom archive to `packit srpm` #566

TomasTomecek commented Oct 14, 2019 •

edited by jpopelka

Loading

jpopelka commented Dec 5, 2019

psss commented Dec 5, 2019

lachmanfrantisek commented Dec 5, 2019 •

edited

Loading

psss commented Dec 13, 2019

lachmanfrantisek commented Dec 13, 2019 •

edited

Loading

psss commented Jan 6, 2020

psss commented Jan 6, 2020

lachmanfrantisek commented Jan 6, 2020 •

edited

Loading

lachmanfrantisek commented Jan 6, 2020 •

edited

Loading

lachmanfrantisek commented Jan 6, 2020

psss commented Jan 6, 2020

psss commented Jan 6, 2020

lachmanfrantisek commented Jan 6, 2020

lachmanfrantisek commented Jan 6, 2020

psss commented Jan 6, 2020

psss commented May 20, 2020

lachmanfrantisek commented May 20, 2020

psss commented May 21, 2020 •

edited

Loading

Make it easier to supply custom archive to packit srpm #566

Make it easier to supply custom archive to packit srpm #566

Comments

TomasTomecek commented Oct 14, 2019 • edited by jpopelka Loading

jpopelka commented Dec 5, 2019

psss commented Dec 5, 2019

lachmanfrantisek commented Dec 5, 2019 • edited Loading

psss commented Dec 13, 2019

lachmanfrantisek commented Dec 13, 2019 • edited Loading

psss commented Jan 6, 2020

psss commented Jan 6, 2020

lachmanfrantisek commented Jan 6, 2020 • edited Loading

lachmanfrantisek commented Jan 6, 2020 • edited Loading

lachmanfrantisek commented Jan 6, 2020

psss commented Jan 6, 2020

psss commented Jan 6, 2020

lachmanfrantisek commented Jan 6, 2020

lachmanfrantisek commented Jan 6, 2020

psss commented Jan 6, 2020

psss commented May 20, 2020

lachmanfrantisek commented May 20, 2020

psss commented May 21, 2020 • edited Loading

Make it easier to supply custom archive to `packit srpm` #566

Make it easier to supply custom archive to `packit srpm` #566

TomasTomecek commented Oct 14, 2019 •

edited by jpopelka

Loading

lachmanfrantisek commented Dec 5, 2019 •

edited

Loading

lachmanfrantisek commented Dec 13, 2019 •

edited

Loading

lachmanfrantisek commented Jan 6, 2020 •

edited

Loading

lachmanfrantisek commented Jan 6, 2020 •

edited

Loading

psss commented May 21, 2020 •

edited

Loading