v25.0

As announced before, this is the first release using gRPC instead of crossbar/autobahn for communication between client/exporter and coordinator.

Crossbar/autobahn are unfortunately not very well maintained anymore. In labgrid, the crossbar component was moved to its own virtualenv to cope with the high number of conflicts caused by outdated dependencies. Also, Python 3.13 is not supported in an official crossbar release.

That's why labgrid moves to gRPC with this release. gRPC is a well maintained RPC framework with a lot of users. As a side effect, the message transfer is more performant and the import times are shorter.

This is a breaking change. Version 25.0 exporters / coordinators / clients can not communicate with Version 24.0 and earlier exporters / coordinators / clients. All exporters, coordinators and clients in a deployment have to be updated at the same time. See the "Breaking changes in 25.0" section below for more information.

New Features in 25.0

• All components can be installed into the same virtualenv again.
• Add support for Python 3.13.
• The QEMUDriver now supports setting the display option to qemu-default, which will neither set the QEMU -display option or pass along -nographic.
• The amount of buffering in GStreamer video pipelines was reduced to improve latency for USBVideoDriver network streaming.
• The RawNetworkInterfaceDriver now supports live streaming of captured packets, setting/getting various configuration options and reporting statistics.
• Add support for LogiLink UA0379 / Microdia cameras to the USBVideoDriver.
• Add more backends to the NetworkPowerDriver:
  • Gude 87-1210-18
  • Digital Loggers PDUs via the REST API
  • Ubiquity mFi mPower power strips
• Add support for sigrok's channel-group parameter.
• Make USB hubs exportable for easier monitoring of large bus topologies.
• Add support for Digital Loggers PDUs via the REST API.
• Add support for custom paths for ssh, scp, sshfs and rsync in the SSHDriver.
• Make the QEMUDriver configuration more flexible and improve startup error reporting.
• Support labgrid-client power via GPIOs.
• The Getting Started chapter in the docs was improved.
• The systemd service for the labgrid-exporter is now configured with Restart=always.
• Support listing only unused/released places with labgrid-client places.
• Replace deprecated SmartStrip class with IotStrip in the tplink power backend.
• Add a erase() method to the AndroidFastbootDriver.

Bug fixes in 25.0

• Fix ShellDriver to correctly match network interfaces names with dots and/or dashes in get_default_interface_device_name().
• Fix reservations for multiple matching tags or values starting with a number.
• Fix concurrent access to USB HID relays.
• Fix support for networks without SSID in the agent used by the NetworkInterfaceDriver.
• Add new stlink USB IDs for the USBDebugger resource.
• Fix code coverage reporting and submit test results to codecov.
• Fix waiting for NetworkService created by the DockerDaemon.
• Fix error handling for missing interfaces in the NetworkInterfaceDriver's agent.
• Fix pip install for archives auto-generated by GitHub.

Breaking changes in 25.0

Maintaining support for both crossbar/autobahn as well as gRPC in labgrid would be a lot of effort due to their different architectures. Therefore, hard migration to gRPC is required.

Previously, the coordinator was running as a component started by the crossbar. Now, it is a standalone service which exposes a gRPC API directly.

Due to the gRPC migration, 25.0 includes the following breaking changes:

• The labgrid environment config option crossbar_url was replaced by coordinator_address. The environment variable LG_CROSSBAR was replaced by LG_COORDINATOR.
• The labgrid environment config option crossbar_realm is now obsolete as well as the environment variable LG_CROSSBAR_REALM.
• The coordinator is available as labgrid-coordinator (instead of crossbar start). No additional configuration file is required.
• The systemd services in contrib/systemd/ were updated.

To update your deployment to 25.0, you'll need to:

• Update the coordinator, all clients and all exporters to the new release.
• Start labgrid-coordinator directly (instead of via the crossbar configuration file). If needed, you can set the listening address using the --listen option.
• Update any existing client and exporter configuration by replacing the crossbar URL with the coordinator address.

Other breaking changes include:

• Support for Python 3.8 was dropped.
• The siglent power backend is deprecated because it uses the no longer maintained vxi11 module which again uses the deprecated (and in Python 3.13 removed) xdrlib. See issue #1507.

Known issues in 25.0

• gRPC sometimes prints a confusing error message during shutdown. We've worked around this by closing stderr on shutdown. See issue #1544 and PR #1605.

New Contributors

• @enkiusz made their first contribution in #1491
• @tretter made their first contribution in #1494
• @nvincent-vossloh made their first contribution in #1518
• @QSchulz made their first contribution in #1543
• @moto-timo made their first contribution in #1512
• @idan-arm made their first contribution in #1591
• @pslldq made their first contribution in #1562
• @alextercete made their first contribution in #1558
• @pmelange made their first contribution in #1611
• @lemoer made their first contribution in #1619
• @istepic made their first contribution in #1581
• @hundeboll made their first contribution in #1638

Full Changelog: v24.0...v25.0

v24.0.3

Bug fixes in 24.0.3

• Fix support for networks without SSID in the agent used by the NetworkInterfaceDriver.
• Fix concurrent access to USB HID relays.
• Fix some actions workflows.

Full Changelog: v24.0.2...v24.0.3

v24.0.2

Bug fixes in 24.0.2

• Limit pysnmp's pyasn1 dependency to <0.6.0 in pyproject.toml to work around pyasn1.compat.octets ModuleNotFoundError

v24.0.1

Bug fixes in 24.0.1

• Add "six" to crossbar-requirements.txt as a stop-gap solution for implicit six usage of txtorcon (a crossbar dependency). Fixes errors such as: ModuleNotFoundError: No module named 'six'

v24.0

New Features in 24.0

• When invoking tests with pytest, the --log-(cli|file)-(level|format)
  command line arguments and their corresponding pytest.ini configure options
  are now respected (making it possible to have different format and logging
  levels in the log file than then console).
• A new log level called CONSOLE has been added between the default
  INFO and DEBUG levels. This level will show all reads and writes made
  to the serial console during testing.
• The docker support was extended to support buildx, allowing the build of arm64
  container images.
• The tool lookup function has been extended to return the original name in case
  the path can't be found. This makes specification of the qemu binary easier to
  use.
• The bindings base class has been extended, allowing the user to retrieve
  all resources used by a driver.
• Support for STLink V2 was added.
• UBootStrategy was extended with a force() function.
• labgrid was switched from pysnmp to pysnmp-lexstudio.
• Support for Segger J-Link was added.
• Place tags are now exposed by the RemotePlace.
• The sync-places contrib script has gained support for named matches.
• Remote support for YKush Devices was added.
• Support for sigrok DMMs was added.
• Support for Digital Outputs switched via HTTP was added.
• The QEMUDriver has a new get_qemu_base_args() function which can be used to
  extract the arguments passed to qemu.
• The SSHDriver has gained support to forward unix sockets.
• The exporter has gained an --fqdn argument to set the hostname to the
  fully qualified domain name instead of the hostname.
• The QEMUDriver now has an additional disk_opts property which can be
  used to pass additional options for the disk directly to QEMU
• All drivers now inherit a logger from the Driver base class and many
  drivers were changed to use this logger.
• The new poe_mib backend allows switching of power over Ethernet-capable
  ports on switches that use the corresponding SNMP MIB.
• The RawNetworkInterfaceDriver allows the replay and recording of network
  packets on ethernet interfaces.
• The i.MX93 usb loader USB ID has been added to the IMXUSBLoader resource.
• Support for udev matched GPIOs has been added.
• labgrid-client now has a write-files subcommand to copy files onto mass
  storage devices.
• The NetworkPowerPort supports a new backend ubus. It controls PoE
  switches running OpenWrt using the ubus interface.
• The pyproject.toml gained a config for ruff <https://github.com/astral-sh/ruff>_.
• setuptools_scm is now used to generate a version file.
• labgrid-client console will fallback to telnet if microcom is not available.
• A power backend for tinycontrol.eu IP Power Socket 6G10A v2 was added.
• Labgrid now publishes arm64 docker images.
• Labgrid's YAML parser will now warn when mapping keys are duplicated and thus
  overwritten.
• LC USB Relais are now supported.

Bug fixes in 24.0

• The pypi release now uses the labgrid pyserial fork in the form of the
  pyserial-labgrid package. This fixes installation with newer versions
  of pip.
• Several tests have gained an importorskip() call to skip them if the
  module is not available.
• labgrid now uses its own pyserial fork from pypi since installation from
  github as an egg is no longer properly supported.
• The build-and-release workflow supports building wheels.
• Fix named SSH lookups in conjunction with an environment file in
  labgrid-client.
• The crossbar virtual-environment now needs to be separate from the labgrid
  environment, for more information please consult the current documentation <https://labgrid.readthedocs.io/en/latest/getting_started.html#coordinator>_.
• The markers now are restricted to patterns which won't match WARN,
  ERROR, INFO and similar log notifiers.
• A race inside the SSHDriver cleanup has been fixed.
• The labgrid-client monitor command now outputs the full resource identifier.
• Many of the USB loader commands e.g. imx-usb-loader will now print to the
  console when logging is not enabled.
• An UnboundLocalError inside the atomic_replace code which is used inside the
  coordinator was fixed.
• Resources of different classes can now have the same name.
• A bug within the pytest logging setup was fixed.
• The QemuDriver correctly handles the different command lines for virgl
  enablement.
• A bug was fixed where resource names were ignored during lookup of the correct
  power driver.
• ManagedFile was fixed to work with the stat command on Darwin.
• Instead of using a private member on the pytest config, the labgrid plugin now
  uses the pytest config stash.
• The ShellDriver was fixed to set the correct status attribute.
• The USBNetworkInterface now warns if the interface name is set, as it will be
  overwritten by the ResourceManager to assign the correct interface name.
• Fix sftp option issue in SSH driver that caused sftp to only work once per
  test run.
• ManagedFile NFS detection heuristic now does symlink resolution on the
  local host.
• XModem support within the Shelldriver was fixed by removing the newline from
  the marker.
• A typo in the NFSProviderDriver class was fixed. Documentation was already
  correct, however the classname contained an additional P.
• The --loop argument for labgrid-client console was fixed.
• The password for the ShellDriver can now be an empty string.
• The default crossbar configuration now enables auto-fragmentation to handle
  bigger labs where the payload size can be bigger than 1 megabyte.
• The SSHDriver redirects /dev/null to stdin of commands run via SSH.
  This prevents unexpected input, especially when using the
  ManualPowerDriver or a REPL.
• The ser2net version check for YAML configurations in the exporter was
  fixed.
• The exporter forces ser2net TCP connections for versions >=4.2.0.
• The retrieval of the DTR status for SerialPortDigitalOutputDriver was
  fixed.
• The SSHDriver keepalive is now correctly stopped when using existing
  connections.
• The power backend for raritan devices now supports devices with more than 16
  outlets.
• The ExternalConsoleDriver now correctly sets the bufsize to zero to
  prevent buffering.

Breaking changes in 24.0

• Support for Python 3.7 was dropped.

• Support for the legacy ticket authentication was dropped: If the coordinator
  logs ModuleNotFoundError on startup, switch the crossbar config to anonymous
  authentication (see .crossbar/config-anonymous.yaml for an example).

• The Debian package (debian/) no longer contains crossbar. Use the
  coordinator container <https://hub.docker.com/r/labgrid/coordinator>_ or
  install it into a separate local venv as described in the
  documentation <https://labgrid.readthedocs.io/en/latest/getting_started.html#coordinator>_.
  If you see WARNING: Ticket authentication is deprecated. Please update your coordinator. on the client when running an updated coordinator, your
  coordinator configuration may set ticket instead of anonymous auth.

• The StepReporter API has been changed. To start step reporting, you must
  now call StepReporter.start() instead of StepReporter(), and set up
  logging via labgrid.logging.basicConfig().

• Logging output when running pytest is no longer sent to stderr by default,
  since this is both chatty and also unnecessary with the improved logging
  flexibility. It it recommended to use the --log-cli-level=INFO command
  line option, or log_cli_level = INFO option in pytest.ini, but if you
  want to restore the old behavior add the following to your conftest.py
  file (note that doing so may affect the ability to use some more advanced
  logging features)::

  def pytest_configure(config):
  import logging
  import sys

     logging.basicConfig(
         level=logging.INFO,
         format='%(levelname)8s: %(message)s',
         stream=sys.stderr,
     )
  

• The interpretation of the -v command line argument to pytest has changed
  slightly. -vv is now an alias for --log-cli-level=INFO (effectively
  unchanged), -vvv is an alias for --log-cli-level=CONSOLE, and
  -vvvv is an alias for --log-cli-level=DEBUG.

• The BareboxDriver now remembers the log level, sets it to 0 on initial
  activation/reset and recovers it on boot(). During
  run()/run_check() the initially detected log level is used.

• The NFSProviderDriver now returns mount and path information on stage()
  instead of the path to be used on the target. The previous return value did
  not fit the NFS mount use case.

• The NFSProvider and RemoteNFSProvider resources no longer expect the
  internal and external arguments as they do not fit the NFS mount use
  case.

Known issues in 24.0

• Some client commands return 0 even if the command failed.

v23.0.6

Bug fixes in 23.0.6

• In USBVideoDriver, use the playbin3 element instead of playbin to fix decoding via VA-API for certain webcams on AMD graphic cards.
• Let the SSHDriver redirect /dev/null to stdin on run() to prevent unexpected consumption of stdin of the remotely started process.
• Cover more failure scenarios in the exporter and coordinator systemd services, fix the service startup order, do not buffer journal logs.

v23.0.5

Bug fixes in 23.0.5

• Fix readthedocs build by specifying Python version and OS.
• Fix several incompatibilities with doc sphinxcontrib-* dependencies having dropped their explicit Sphinx dependencies, which prevented generation of labgrid's docs.

v23.0.4

Bug fixes in 23.0.4

• Fix dockerfiles syntax error that became fatal in a recent docker release.
• Fix ShellDriver's xmodem functionality.
• Pin pylint to prevent incompatibility with pinned pytest-pylint.
• Fix labgrid-client console --loop on disappearing serial ports (such as on-board FTDIs).

v23.0.3

Bug fixes in 23.0.3

• Update to PyYAML 6.0.1 to prevent install errors with Cython>=3.0, see:
  yaml/pyyaml#601
  yaml/pyyaml#726 (comment)

v23.0.2

Bug fixes in 23.0.2

• Move SSHDriver's control socket tmpdir clean up after the the SSH process
  has terminated. Ignore errors on cleanup since it's best effort.
• Add missing class name in labgrid-client monitor resource output.
• Print USB loader process output if log level does not cover logging it.
• Fix UnboundLocalError in atomic_replace() used by the coordinator and
  labgrid-client export to write config files.
• Let Config's get_tool() return the requested tool if it is not found in
  the config. Return the resolved path if it exists, otherwise return the value
  as is. Also drop the now obsolete tool fallbacks from the drivers and add
  tests.
• Fix USBSDMuxDevice/USBSDWireDevice udev race condition leading to
  outdated control/disk paths.
• Fix SSHDriver's explicit_sftp_mode option to allow calls to put()
  and get() multiple times. Also make scp() respect this option.
• Add compatibility with QEMU >= 6.1.0 to QEMUDriver's display argument
  for the egl-headless option.