Add a story for exporting tests into nitrate

[psss]

No description provided.

[thrix]

Looks good to me. I believe there should be also a --url (or --api-url) parameter, shich can maybe specify the nitrate url ot use. I guess we will default to tcms and hopeffuly that will be in some config settings sometime.

[packit-as-a-service]

Congratulations! The build has finished successfully. 🍾

You can install the built RPMs by following these steps:

• sudo yum install -y dnf-plugins-core on RHEL 8
• sudo dnf install -y dnf-plugins-core on Fedora
• dnf copr enable packit/psss-tmt-83
• And now you can install the packages.

Please note that the RPMs should be used only in a testing environment.

[packit-as-a-service]

Congratulations! The build has finished successfully. 🍾

You can install the built RPMs by following these steps:

• sudo yum install -y dnf-plugins-core on RHEL 8
• sudo dnf install -y dnf-plugins-core on Fedora
• dnf copr enable packit/psss-tmt-83
• And now you can install the packages.

Please note that the RPMs should be used only in a testing environment.

[packit-as-a-service]

Congratulations! The build has finished successfully. 🍾

You can install the built RPMs by following these steps:

• sudo yum install -y dnf-plugins-core on RHEL 8
• sudo dnf install -y dnf-plugins-core on Fedora
• dnf copr enable packit/psss-tmt-83
• And now you can install the packages.

Please note that the RPMs should be used only in a testing environment.

[psss]

Based on the feedback from @t184256 I've added mention about adding a warning into the test case notes.

[psss]

I guess we will default to tcms and hopeffuly that will be in some config settings sometime.

Good point with the config file. We will need one sooner or later. I've filed #84 to brainstorm possible ways how to approach this.

[psss]

One Way Sync

In the comments above, the question whether synchronization should be bidirectional or not has been raised several times. So far I see me, @hegerj and @lukaszachy leaning towards one direction only:

After migrating the test metadata to git, fmf should be the primary source of the information which would be synced to nitrate.

Everybody else OK with that?

@hegerj, @leospol, @lukaszachy, @pvalena, @thrix, @t184256, please reivew and 👍 or 👎. Thanks!

Update: This one way sync question is related to the tmt test export functionality only.

[psss]

I've updated the pull request and added a list of fmf attributes and corresponding nitrate fields.

[t184256]

leaning towards one direction only: After migrating the test metadata to git, fmf should be the primary source of the information which would be synced to nitrate.

@psss, I fail to see how your phrasing answers the dilemma as I see it.

Yes, clearly, in the blissful future fmf should be the primary source of the information, (all tests must pass and everyone should live happily ever after). The question is what should be implemented right away for now, isn't it?

And if we are choosing between

1. unidirectional nitrate -> fmf sync only (import) and

2. bidirectional nitrate <-> fmf sync (import & export) for the transition period,

then 'one direction only' (1) is mutually incompatible with 'would be synced to nitrate' (2).

What am I missing here?

[psss]

@t184256, I see, thanks for the question. So the idea is:

• Use tmt test convert once to migrate metadata to fmf
• Run tmt test export to create the connection (add fmf key into structured field)
• Do all future metadata updates directly in git and then sync data to nitrate using tmt test export

Does that make more sense?

[packit-as-a-service]

There was an error while creating SRPM. You can re-trigger copr build by adding a comment (/packit copr-build) into this pull request.

Output:

An unexpected error occurred when creating the SRPM: Command failed (rc=127, reason={"metadata":{},"status":"Failure","message":"command terminated with non-zero exit code: Error executing in Docker Container: 127","reason":"NonZeroExitCode","details":{"causes":[{"reason":"ExitCode","message":"127"}]}})
bash: /sandcastle/script.sh: No such file or directory

[t184256]

@psss, OK, this is what I consider bidirectional sync, and it makes sense to me. But, since I don't see a clearly outlined alternative, I still have to ask: what are we supposed to vote for?

[psss]

@t184256, this one-way question is related to tmt test export only. @hegerj mentioned the problem which would arise if we tried to sync further updates (meaning after migration) in nitrate back to fmf. So we need to clarify we don't want that.

[t184256]

Oh. I see now.

Thumbed up on the original poll comment to signify that I don't want importing to happen on export.

[psss]

/packit build

[psss]

I've updated the spec based on today's discussion. Included also the extra attributes to sync hardware and pepa fields for backward compatibility. Once metadata are migrated into git fmf should be the only source of truth and all further changes (including pepa and hardware) should be done in git and synced to nitrate.

@lukaszachy, @hegerj, @t184256, @thrix, anybody please review. Let's close this soon so that we can start implementing. Thanks.

[packit-as-a-service]

Congratulations! One of the builds has completed. 🍾

You can install the built RPMs by following these steps:

• sudo yum install -y dnf-plugins-core on RHEL 8
• sudo dnf install -y dnf-plugins-core on Fedora
• dnf copr enable packit/psss-tmt-83
• And now you can install the packages.

Please note that the RPMs should be used only in a testing environment.

[lukaszachy]

LGTM

[psss]

Thanks to all for your feedback!

[kkaarreell]

Hi,
sorry for late reply, I have somehow missed email notifications.
Regarding the sync, I vote for the one way sync. Also, regarding the TCMS TC tags/labels, export should probably only add them (not remove). There are some tools (like TIP) that are adding/removing labels in TCMS on their own and the export should not probably be overwriting them. Maybe the tag removal could be handled better but I do not have any specific proposal at hand. Also, I would propose to add also an extra tag (e.g. "__fmf_export") to TC. The reason for that is that such thing could be easily detected in TCMS web UI and we could eventually use a custom script (greasemonkey) to prevent or issue warning in case of manual tag editing in TCMS for such test cases. Structured field note could be handy for other tools but tags might be easier to work with in the UI. Moreover, having such tag would enable TC filtering within the TCMS web UI (e.g. from the plan show me test cases with fmf metadata - structured field cannot be used for that).

UPDATE: I have changed my mind. Export should overwrite all TCMS TC tags so that there is one clear source. If it turns out there are some issues with that approach I can imagine introducing a global option defining flags (list of regexps?) that should not be touched in TCMS.

[psss]

Thanks for the comment, @kkaarreell. I like the idea of adding an extra tag to allow easier filtering using the web interface. That tag, however, should not be included in the fmf metadata, right?

[psss]

Added mention about the special tag in #95, please review.

[kkaarreell]

That tag, however, should not be included in the fmf metadata, right?
Right.