Skip to content

v0.70.0-beta

Pre-release
Pre-release
Compare
Choose a tag to compare
@portante portante released this 10 Nov 21:59
· 10 commits to b0.70 since this release
v0.70.0-beta
a7d1f0d

v0.70.0 (Beta) Release Notes

This is a very significant "minor" release of the pbench code base, agent and server.

The "Tool Meister" functionality (PR #1248) is the major piece of functionality delivered with this release. This is a significant change, where the pbench-agent first orchestrates the instantiation of a "Tool Meister" process on all hosts registered with tools, using a Redis instance to coordinate their operation, and the new "Tool Data Sink" handles the collection of data into the pbench run directory hierarchy. We effectively eliminate all remote SSH operations for tools except one per host to orchestrate the creation of the Tool Meister instance.

The Tool Meister work has enabled us to integrate with Prometheus for tool data collection. Two new pbench "tools" have been added, node-exporter and dcgm. If one registers either or both of these new tools (e.g. via pbench-register-tools --name=node-exporter), then the Tool Meister sub-system will run the node_exporter code on the registered hosts, and a local instance of Prometheus to collect the data. The collected Prometheus data is stored in the pbench result directory as a tar ball at: ${pbench_run}/<script>_<config>_YYYY.MM.DDTHH.mm.ss/tools-<group>/prometheus. For the duration of the run, the Prometheus instance is available on localhost:9090 if one desires to review the metrics being collected live.

NOTE WELL: like all the other "tools" the pbench-agent supports, the node-exporter and dcgm tools themselves need to be installed separately.

Along with the delivery of the "Tool Meister" work, the notion of tool registration has changed significantly, where tools are now recorded as registered only on the local host were pbench-register-tool and pbench-register-tool-set are invoked. As a result of this change, the following behavioral changes follow:

  • The process of registering tools on local or remote hosts no longer validates that those tools are available during tool registration
  • All tools registered prior to installing v0.70.0-beta must be re-registered; tools registered locally or remotely on a controller with a version of the pbench-agent prior to v0.70.0 will be ignored until they are re-registered

For both the Pbench Agent and the Server we have removed the use of the Software Collections Library (SCL) in order to provide a Python 3 environment for RHEL 7 & CentOS 7 distributions. We now rely on the Python 3 provided with RHEL 7.7 and later.

We did not bump the "major" release version number with these changes because we still don't consider all the necessary functionality in place for such a major version bump.

Installation

There are no other installation changes in this release: see the Getting Started Guide for how to install or update.

After installation or update, you should have version 0.70.0-9ga7d1f0d6 of the pbench-agent RPM installed.

RPMs are available from Fedora COPR, covering Fedora 31, 32, EPEL 7 & 8.

There are Ansible playbooks available via Ansible Galaxy to install the pbench-agent, and the pieces needed (key and configuration files) to be able to send results to a server. To use the RPMs provided above via COPR with the playbooks, an inventory file needs to include the fedoraproject_username variable set to portante, for example:

...

[servers:vars]
fedoraproject_username: portante

...

Alternatively, one can specify fedoraproject_username on the command line, rather than having it specified in the inventory file:

ansible-playbook -i <inventory> <playbook> -e '{fedoraproject_username: portante}'

NOTE WELL: If the inventory file also has a definition for pbench_repo_url_prefix (which was standard practice before fedoraproject_username was introduced), it needs to be deleted, otherwise it will override the default repo URL and the fedoraproject_username change is not going to take effect.

While we don't include installation instructions for the new node-exporter and dcgm tools in the published documentation, you can find a manual installation procedure for the Prometheus "node_exporter" and references to the Nvidia "DCGM" documentation in the agent/tool-scripts/README.

Container images built using the above RPMs are available in the Pbench organization in the Quay.io container image repository using tags beta, v0.70.0-9, and a7d1f0d6.

Agent

In addition to the major changes described above for this release, the following significant changes for the agent are also worth calling out specifically:

  • The pbench-fio bench script now requires fio-3.21 or later; see 5048a14

    • You can find a Fedora COPR build of fio-3.21-6.pbench for EPEL 7 & 8

  • [DEPRECATED] The pbench-cleanup utility command is deprecated, and will be removed in a subsequent release (see PR #1828)

  • The release takes in the latest code from stockpile, including support for GPU data collection via the Nvidia SMI role

  • The new dcgm tool requires Python 2, an Nvidia based install which might conflict with the Pbench Agent's Python 3 operational requirement in some cases

Server

Initial support for moving results to the pbench-server via HTTP PUT operations.

Web Server

There are no changes and no new web-server RPMs have been produced.

Pbench Dashboard

The development of the dashboard is not tracked in these release notes. The dashboard has been moved into its own git repo.

ChangeLog

This is the list of visible commits since the v0.69.3-agent release:

a7d1f0d Add 'v0.70.0-beta' release notes
46aa2ad Update the development image we use for 'b0.70'
8ae1367 Add missing 'net-tools' req agent RPM
8b7424a Remove colorlog; add Makefile for pbench-devel
e702457 Rework pass-thru API implementation a bit
6dc6e4f Jenkins integration using Fedora 32 container
e927f02 Fix the flaky util-scripts test-51 & test-52
dd1c52d Address common logging between agent and server
a37b40c Fix agent side 'test_move_results'
54b6522 Use '_pbench_' prefixed env var for host names
03c14b4 Make sure we explicitly ask for the full hostname
50c301d Refactor base to extract unit test overrides
1261d6d Address some undesirable tox behaviors
af520f2 Fix misaligned deps in tox.ini
f0f6217 Direct 'black' to ignore '.git' subtree
589caa6 Cleanup 'datalog/prometheus-metrics-datalog'
ea58135 Rejig functional unit tests
aaef93a Fix warning while running unit tests
1496c7c Remove colorlog
1609912 Only do git submodule init at the top of the tree
98f42b7 Remove wayward py3-functional unit test
cb93ab3 Add ability to tag agent images with beta and alpha
f28c7a3 Require pyesbulk 1.0.0 for now
0c05fb8 Remove SCL from the agent side
4eb6d19 Remove SCL reference in the server trampoline code
98d50ce Fix the README to restore working URLs
e3e893f Enhance agent container builds
7dd58f0 Add required package name for vmstat
a431636 Remove use of 'screen' from Tool Meister
5671ea5 Stop invoking screen directly in unit tests
1a1dfb7 Source agent 'base' for remote tool meisters
b37ae97 Record the pip3 command output to a log file
6977291 Add warning for left-over 'id_rsa' file
ad6e9de Encapsulate building the final requirements files
8c7cadf Don't '%ghost' 'id_rsa'; only 'rm' in '%postun'
afa583a Move to rh-python38 for RHEL 7 & use python RPMs
9574bf6 Add redis and python3-redis agent RPM requires
3241acb Add a "common" unit test environment
6d44341 Move md5sum() method to common utils
fa6b25d Don't remove 'id_rsa' and 'pbench-agent.cfg' when uninstalling
e1ca5f3 Remove blanket executable file permissions
b722ecc Use the %config for the agent config file
3612c00 Ensure proper use of rh-python36 only for RHEL 7
c4d6848 Remove pbench-agent RPM dependencies for tools
d72f829 Update minimum supported Fedora to 31
92d0383 Update pbench-agent spec file summary & desc
8458a45 Add use of 'rpmlint' on '.spec' files
5b425b4 DRY out run-unittests, adding one for the server
f908712 Create the notion of a utils directory
d438663 First pass at a container image for the pbench-server
fdda138 Making a server RPM (without SCL)
720e6c3 Initial pbench-agent container image layering
20a307e RPM making for pbench-agent
3aa69d6 General method of fetching git commit ID
59d1989 update agent cli to reflect server upload PUT changes
3526a76 Additional changes to break up testing requirements
80c4b7e Break up agent & server requirements
c29f7ca Resolves logging error in pbench-server-prep-shim
ed39d86 Fix missing space in sosreport --quiet option
f32e0fd write md5sum of a tarfile on disk on server and add upload POST to upload PUT
7d122d2 Fix pbench-clear-tools based on new unit tests
736bd27 Fix handling of expected exit status in unit tests
b03cddd Remove , add periods in pbench-clear-tools
2b8df4d give pbench user access to /run/pbench-server directory
2021484 Deprecate pbench-cleanup & add tests for clear-results
3d0d1f4 Switch legacy unit tests to use C.UTF-8 locale
79086b4 Updated benchmarks wit init/end
f72e16e Add unit tests for filesize_bytes
52e0f97 Reformat the upload api test and fix stored md5sum of log.tar.xz
c32596f Fix util-scripts test-53
33ee319 Post-Merge Review Changes (#1839)
8bca118 Prevent server unit test 'test-25' from failing
ae03d28 Fix generation of sosreport command
a27edb0 Ensure Tool Data Sink internal server object exists
9663330 Tell pbr to stop creating AUTHORS & ChangeLog
276df1a Fix bad printf statement in server/test-find-behavior
d10ab00 Add missing quote
f66e8e5 Early prep for the 0.70 release
4775f1d Add gunicorn wsgi server configuration
9c29bc0 Added DCGM Tool to pbench-agent (keshavm02)
cb3434e Prometheus/Node_Exporter Full V1 Integration Commit
a6c5890 Sort modules in *requirements.txt
637c4f3 Lock in use of Perl 5.30+ for Travis CI
0e7f706 Setup base class
20bfd2b Pbench-server API improvements
30d2246 Pbench Server API
c0e9380 Clean up the sosreport version number checking
ed6d804 Record the sosreport command that was used
4f5e6ee Fix bench/util-scripts unit tests env overrides
ac69d66 Perhaps useful script for updating gold files
d32a091 Switch indexer.py to pyesbulk package
64bcdbb Minimal attempt to remove tool meister commands
01d17fc Only emit a warning message on missing debugfs
4ee7dd9 Fix tool argument handling
61da5dc Tweak tool meister tests to use multiple tool opts
94e42e5 Ansible role for systemd service
b5a4871 Fixes to the systemd service file plus a service file for RHEL7
ad9eb15 Make shell.py into a package that exports a main().
4222d0f A few more cleanups
d4c6a44 Make all non-environment variables local to functions named _pbench*
23cc097 First pass implementation of the "Tool Meister"
d8f835d Prepare roles for Ansible Galaxy
991ad99 Set backup cron job frequency to every minute
1fae071 Make spelling of commit_id consistent
1643205 Fixes issue #1668
d382a8d Lock versions of flake8 and black to known-good (#1768)
80c8d8e Give default values to most of the variables.
5048a14 Require fio 3.21 and later
98beb10 Fix uperf "rr" (round-robin) test XML
2561f44 Rename pbench-agent.cfg.example to pbench-agent.cfg
b8cb2b8 Avoid hardcoded tools location on bench-scripts
6815743 pbench-server-prep-shim-002 into python program
dbfb7f7 Fix project name to Pbench.
c3002f1 Fix tabs vs spaces in agent/base
ff58abb Fix formatting of contribution guide
0180aad Use Python 3's pathlib where possible
bf382c2 Fix datalog-cpuacct directory order
47edfa8 Sort the output of pbench-cull-unpacked-tarballs.py
500a737 Change ToC 'parent' field to "keyword"
8a00fe1 Get rid of sudo requirement
4a06bd4 Refactoring to introduce PbenchAgentConfig class
5b3874a Implement PbenchConfig base class
f9ae8a8 Move server side logging to common
c2661dc Run agent side test directories individually
d142d16 Remove explicit sub-classing of 'object'
2ee00fc Change pbench-dbench to stop using /tmp
9d428af Add py3-functional to run-unittests
b07c678 Restructure pbench namespace
127eb7c Add config options and debug options for agent
77fa536 Refactor pbench-cli
d44f0b5 Centralize agent results classes
b8958d3 Centralize click options
3e9b9c6 Search for agent configuration
a18fc93 Add unittests
32d3f28 Remove getconf.py for server
59ce3bc Remove getconf.py for agent
e0e6570 Add pbench-config cli
c16ada5 Refactored pbench server and agent code
af4feec Formatted scripts using f-strings
c1b4e13 Added script specific and default logging level
1c91c89 feat: pbench-server REST API
6b082d4 Setup pbench namespace

Assets 2
Loading