Skip to content

Commit afe1a56

Browse files
authored
Extension of the ReportPortal plugin using API
* option `--merge` to create suite per plan and merge them all in one launch (stored launch uuid in run of the first plan) * option '--attributes' for additional attributes, but mainly for assignment of attributes to the launch with merged plans * option `--uuid` to append new plans to an existing launch * store launch uuid as rp_uuid per merged run, and as launch_uuid per each plan, store launch_url per each plan * rewritten environment variables to the uniform form TMT_PLUGIN_REPORT_REPORTPORTAL_${option} * prepared defect type locator for implementation of continuous update (idle) * prepared rerun for implementation of launch update (retry) * mapping according to options --launch-per-plan and --suite-per-plan * uploading to existing launch/suite with options --upload-to-launch LAUNCH_ID, --upload-to-suite SUITE_ID * option --launch-description and preparation for --launch-attributes (for suite-per-plan mapping) * trial option --launch-rerun * trial option --defect-type * functional --suite-per-plan option that uploads all plans into a launch; + reporting common atributes from all plans, closing the launch after last plan * additional upload of tests/suites into a existing launch * functional idle report and additional report within run id * functional upload to launch * debug based on option combinations * fixed the functonality to report idle tests and additional results
1 parent 97b84ca commit afe1a56

File tree

3 files changed

+445
-129
lines changed

3 files changed

+445
-129
lines changed

spec/plans/report.fmf

Lines changed: 37 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -104,19 +104,47 @@ description:
104104
web page, filter them via context attributes and get links
105105
to detailed test output and other test information.
106106
description:
107-
Fill json with test results and other fmf data per each plan,
108-
and send it to a Report Portal instance via its API.
107+
Provide test results and fmf data per each plan,
108+
and send it to a Report Portal instance via its API
109+
with token, url and project name given.
109110
example:
110111
- |
111-
# Set environment variables with the server url and token
112-
export TMT_REPORT_REPORTPORTAL_URL=<url-to-RP-instance>
113-
export TMT_REPORT_REPORTPORTAL_TOKEN=<token-from-RP-profile>
112+
# Optionally set environment variables according to TMT_PLUGIN_REPORT_REPORTPORTAL_${OPTION}
113+
export TMT_PLUGIN_REPORT_REPORTPORTAL_URL=${url-to-RP-instance}
114+
export TMT_PLUGIN_REPORT_REPORTPORTAL_TOKEN=${token-from-RP-profile}
114115
- |
115-
# Enable ReportPortal report from the command line
116+
# Enable ReportPortal report from the command line depending on the use case:
117+
118+
## Simple upload with all project, url endpoint and user token passed in command line
119+
tmt run --all report --how reportportal --project=baseosqe --url="https://reportportal.xxx.com" --token="abc...789"
120+
121+
## Simple upload with url and token exported in environment variable
116122
tmt run --all report --how reportportal --project=baseosqe
117-
tmt run --all report --how reportportal --project=baseosqe --exclude-variables="^(TMT|PACKIT|TESTING_FARM).*"
118-
tmt run --all report --how reportportal --project=baseosqe --launch=test_plan
119-
tmt run --all report --how reportportal --project=baseosqe --url=... --token=...
123+
124+
## Upload with project name in fmf data, filtering out parameters (environemnt variables) that tend to be unique and break the history aggregation
125+
tmt run --all report --how reportportal --exclude-variables="^(TMT|PACKIT|TESTING_FARM).*"
126+
127+
## Upload all plans as suites into one ReportPortal launch
128+
tmt run --all report --how reportportal --suite-per-plan --launch=Errata --launch-description="..."
129+
130+
## Rerun the launch with suite structure for the test results to be uploaded into the latest launch with the same name as a new 'Retry' tab (mapping based on unique paths)
131+
tmt run --all report --how reportportal --suite-per-plan --launch=Errata --launch-rerun
132+
133+
## Rerun the tmt run and append the new result logs under the previous one uploaded in ReportPortal (precise mapping)
134+
tmt run --id run-012 --all report --how reportportal --again
135+
136+
## Additional upload of new suites into given launch with suite structure
137+
tmt run --all report --how reportportal --suite-per-plan --upload-to-launch=4321
138+
139+
## Additional upload of new tests into given launch with non-suite structure
140+
tmt run --all report --how reportportal --launch-per-plan --upload-to-launch=1234
141+
142+
## Additional upload of new tests into given suite
143+
tmt run --all report --how reportportal --upload-to-suite=123456
144+
145+
## Upload Idle tests, then execute it and add result logs into prepared empty tests
146+
tmt run discover report --how reportportal --defect-type=Idle
147+
tmt run --last --all report --how reportportal --again
120148
- |
121149
# Use ReportPortal as the default report for given plan
122150
report:

tmt/schemas/report/reportportal.yaml

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,27 @@ properties:
3434
launch:
3535
type: string
3636

37+
launch-description:
38+
type: string
39+
40+
launch-per-plan:
41+
type: boolean
42+
43+
suite-per-plan:
44+
type: boolean
45+
46+
upload-to-launch:
47+
type: string
48+
49+
upload-to-suite:
50+
type: string
51+
52+
launch-rerun:
53+
type: boolean
54+
55+
defect_type:
56+
type: string
57+
3758
exclude-variables:
3859
type: string
3960

0 commit comments

Comments
 (0)