Skip to content

feature: IMDReader Integration #4923

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 13 commits into
base: develop
Choose a base branch
from
Draft
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/actions/setup-deps/action.yaml
Original file line number Diff line number Diff line change
@@ -82,6 +82,8 @@ inputs:
default: 'seaborn>=0.7.0'
tidynamics:
default: 'tidynamics>=1.0.0'
imdclient:
default: 'imdclient'
Comment on lines +85 to +86
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's going to take time to get imdclient to a stage where the imdclient package does not actually affect MDAnalysis.

Is there a way that we could temporarily (for initial CI testing) install imdclient from a branch or tarball, e.g., in a pip section? Then we could fairly rapidly create a preliminary (unpublished) imdclient package without IMDReader.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By initial CI testing, do you mean "in this PR"?

There's a pip section just below, which should work if you put in the git location for pip install, but also you can just temporarily modify the CI script to do an additional pip install if it's for "testing within the PR itself".

If it's "after merge", this would require more discussion.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for right now to bootstrap the PR.

I don't want to merge without a solid conda-forge imdclient package in place.

# pip-installed min dependencies
coverage:
default: 'coverage'
@@ -137,6 +139,7 @@ runs:
${{ inputs.distopia }}
${{ inputs.gsd }}
${{ inputs.h5py }}
${{ inputs.imdclient }}
${{ inputs.hole2 }}
${{ inputs.joblib }}
${{ inputs.netcdf4 }}
1 change: 1 addition & 0 deletions azure-pipelines.yml
Original file line number Diff line number Diff line change
@@ -77,6 +77,7 @@ jobs:
displayName: 'Install tools'
- script: >-
python -m pip install --only-binary=scipy,h5py
imdclient
cython
hypothesis
h5py>=2.10
1 change: 1 addition & 0 deletions maintainer/conda/environment.yml
Original file line number Diff line number Diff line change
@@ -30,6 +30,7 @@ dependencies:
- sphinxcontrib-bibtex
- mdaencore
- waterdynamics
- imdclient
- pip:
- mdahole2
- pathsimanalysis
215 changes: 215 additions & 0 deletions package/MDAnalysis/coordinates/IMD.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,215 @@
"""
IMDReader --- :mod:`MDAnalysis.coordinates.IMD`
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Read and analyze simulation data interactively using `IMDClient`_.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A bit more detail here for the documentation would be good.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need a complete example people can follow

.. _IMDClient: https://github.com/Becksteinlab/imdclient
Units
-----
The units in IMDv3 are fixed.
.. list-table::
:widths: 10 10
:header-rows: 1
* - Measurement
- Unit
* - Length
- angstrom
* - Velocity
- angstrom/picosecond
* - Force
- kilojoules/(mol*angstrom)
* - Time
- picosecond
* - Energy
- kilojoules/mol
Classes
-------
.. autoclass:: IMDReader
:members:
:inherited-members:
"""

import numpy as np
import logging
import warnings

from MDAnalysis.coordinates import core
from MDAnalysis.lib.util import store_init_arguments
from MDAnalysis.coordinates.base import StreamReaderBase


from packaging.version import Version

MIN_IMDCLIENT_VERSION = Version("0.1.4")

try:
import imdclient
from imdclient.IMDClient import IMDClient
except ImportError:
HAS_IMDCLIENT = False
imdclient_version = Version("0.0.0")

# Allow building documentation without imdclient
import types

class MockIMDClient:
pass
imdclient = types.ModuleType("imdclient")
imdclient.IMDClient = MockIMDClient
imdclient.__version__ = "0.0.0"

else:
HAS_IMDCLIENT = True
imdclient_version = Version(imdclient.__version__)

# Check for compatibility: currently needs to be >=0.1.4
if imdclient_version < MIN_IMDCLIENT_VERSION:
warnings.warn(

Check warning on line 74 in package/MDAnalysis/coordinates/IMD.py

Codecov / codecov/patch

package/MDAnalysis/coordinates/IMD.py#L74

Added line #L74 was not covered by tests
f"imdclient version {imdclient_version} is too old; "
f"need at least {imdclient_version}, Your installed version of "
"imdclient will NOT be used.",
category=RuntimeWarning,
)
HAS_IMDCLIENT = False

Check warning on line 80 in package/MDAnalysis/coordinates/IMD.py

Codecov / codecov/patch

package/MDAnalysis/coordinates/IMD.py#L80

Added line #L80 was not covered by tests

logger = logging.getLogger("MDAnalysis.coordinates.IMDReader")


class IMDReader(StreamReaderBase):
Copy link
Member

@IAlibay IAlibay Feb 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'm getting confused about which PR is for which thing. @orbeckst given our discussion earlier this week, and your comment above which I take to be "IMDClient is still in flux", does it not make sense for the IMDReader to exist upstream and then just import it here? (edit: here my intent is "well then you could make releases and it wouldn't be limited to MDA release frequency").

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have to split IMDReader from imdclient and make a version of imdclient without IMDReader (which is in the works Becksteinlab/imdclient#54 ). At the same time we are moving what was split off into coordinates.IMD.

Amru is working on both at the moment.

The way IMDReader depends on imdclient is not the problem, and imdclient itself is also pretty stable, it's just that the tests for imdclient have made use of a lot of MDAnalysis/IMDReader for convenience, and we now have to rewrite some of these tests to use bare-bones python.

"""
Reader for IMD protocol packets.
Parameters
----------
filename : a string of the form "imd://host:port" where host is the hostname
or IP address of the listening GROMACS server and port
is the port number.
n_atoms : int (optional)
number of atoms in the system. defaults to number of atoms
in the topology. Don't set this unless you know what you're doing.
kwargs : dict (optional)
keyword arguments passed to the constructed :class:`IMDClient`
"""

format = "IMD"
one_pass = True

@store_init_arguments
def __init__(
self,
filename,
convert_units=True,
n_atoms=None,
**kwargs,
):
if not HAS_IMDCLIENT:
raise ImportError(

Check warning on line 113 in package/MDAnalysis/coordinates/IMD.py

Codecov / codecov/patch

package/MDAnalysis/coordinates/IMD.py#L113

Added line #L113 was not covered by tests
"IMDReader requires the imdclient package. "
"Please install it with 'pip install imdclient'."
)

super(IMDReader, self).__init__(filename, **kwargs)

Check warning on line 118 in package/MDAnalysis/coordinates/IMD.py

Codecov / codecov/patch

package/MDAnalysis/coordinates/IMD.py#L118

Added line #L118 was not covered by tests

self._imdclient = None
logger.debug("IMDReader initializing")

Check warning on line 121 in package/MDAnalysis/coordinates/IMD.py

Codecov / codecov/patch

package/MDAnalysis/coordinates/IMD.py#L120-L121

Added lines #L120 - L121 were not covered by tests

if n_atoms is None:
raise ValueError("IMDReader: n_atoms must be specified")
self.n_atoms = n_atoms

Check warning on line 125 in package/MDAnalysis/coordinates/IMD.py

Codecov / codecov/patch

package/MDAnalysis/coordinates/IMD.py#L124-L125

Added lines #L124 - L125 were not covered by tests

host, port = parse_host_port(filename)

Check warning on line 127 in package/MDAnalysis/coordinates/IMD.py

Codecov / codecov/patch

package/MDAnalysis/coordinates/IMD.py#L127

Added line #L127 was not covered by tests

# This starts the simulation
self._imdclient = IMDClient(host, port, n_atoms, **kwargs)

Check warning on line 130 in package/MDAnalysis/coordinates/IMD.py

Codecov / codecov/patch

package/MDAnalysis/coordinates/IMD.py#L130

Added line #L130 was not covered by tests

imdsinfo = self._imdclient.get_imdsessioninfo()

Check warning on line 132 in package/MDAnalysis/coordinates/IMD.py

Codecov / codecov/patch

package/MDAnalysis/coordinates/IMD.py#L132

Added line #L132 was not covered by tests
# NOTE: after testing phase, fail out on IMDv2

self.ts = self._Timestep(

Check warning on line 135 in package/MDAnalysis/coordinates/IMD.py

Codecov / codecov/patch

package/MDAnalysis/coordinates/IMD.py#L135

Added line #L135 was not covered by tests
self.n_atoms,
positions=imdsinfo.positions,
velocities=imdsinfo.velocities,
forces=imdsinfo.forces,
**self._ts_kwargs,
)

self._frame = -1

Check warning on line 143 in package/MDAnalysis/coordinates/IMD.py

Codecov / codecov/patch

package/MDAnalysis/coordinates/IMD.py#L143

Added line #L143 was not covered by tests

try:
self._read_next_timestep()
except StopIteration as e:
raise RuntimeError("IMDReader: No data found in stream") from e

Check warning on line 148 in package/MDAnalysis/coordinates/IMD.py

Codecov / codecov/patch

package/MDAnalysis/coordinates/IMD.py#L145-L148

Added lines #L145 - L148 were not covered by tests

def _read_frame(self, frame):

try:
imdf = self._imdclient.get_imdframe()
except EOFError as e:
raise e

Check warning on line 155 in package/MDAnalysis/coordinates/IMD.py

Codecov / codecov/patch

package/MDAnalysis/coordinates/IMD.py#L152-L155

Added lines #L152 - L155 were not covered by tests

self._frame = frame
self._load_imdframe_into_ts(imdf)

Check warning on line 158 in package/MDAnalysis/coordinates/IMD.py

Codecov / codecov/patch

package/MDAnalysis/coordinates/IMD.py#L157-L158

Added lines #L157 - L158 were not covered by tests

logger.debug("IMDReader: Loaded frame %d", self._frame)
return self.ts

Check warning on line 161 in package/MDAnalysis/coordinates/IMD.py

Codecov / codecov/patch

package/MDAnalysis/coordinates/IMD.py#L160-L161

Added lines #L160 - L161 were not covered by tests

def _load_imdframe_into_ts(self, imdf):
self.ts.frame = self._frame

Check warning on line 164 in package/MDAnalysis/coordinates/IMD.py

Codecov / codecov/patch

package/MDAnalysis/coordinates/IMD.py#L164

Added line #L164 was not covered by tests
if imdf.time is not None:
self.ts.time = imdf.time

Check warning on line 166 in package/MDAnalysis/coordinates/IMD.py

Codecov / codecov/patch

package/MDAnalysis/coordinates/IMD.py#L166

Added line #L166 was not covered by tests
# NOTE: timestep.pyx "dt" method is suspicious bc it uses "new" keyword for a float
self.ts.data["dt"] = imdf.dt
self.ts.data["step"] = imdf.step

Check warning on line 169 in package/MDAnalysis/coordinates/IMD.py

Codecov / codecov/patch

package/MDAnalysis/coordinates/IMD.py#L168-L169

Added lines #L168 - L169 were not covered by tests
if imdf.energies is not None:
self.ts.data.update(

Check warning on line 171 in package/MDAnalysis/coordinates/IMD.py

Codecov / codecov/patch

package/MDAnalysis/coordinates/IMD.py#L171

Added line #L171 was not covered by tests
{k: v for k, v in imdf.energies.items() if k != "step"}
)
if imdf.box is not None:
self.ts.dimensions = core.triclinic_box(*imdf.box)

Check warning on line 175 in package/MDAnalysis/coordinates/IMD.py

Codecov / codecov/patch

package/MDAnalysis/coordinates/IMD.py#L175

Added line #L175 was not covered by tests
if imdf.positions is not None:
# must call copy because reference is expected to reset
# see 'test_frame_collect_all_same' in MDAnalysisTests.coordinates.base
np.copyto(self.ts.positions, imdf.positions)

Check warning on line 179 in package/MDAnalysis/coordinates/IMD.py

Codecov / codecov/patch

package/MDAnalysis/coordinates/IMD.py#L179

Added line #L179 was not covered by tests
if imdf.velocities is not None:
np.copyto(self.ts.velocities, imdf.velocities)

Check warning on line 181 in package/MDAnalysis/coordinates/IMD.py

Codecov / codecov/patch

package/MDAnalysis/coordinates/IMD.py#L181

Added line #L181 was not covered by tests
if imdf.forces is not None:
np.copyto(self.ts.forces, imdf.forces)

Check warning on line 183 in package/MDAnalysis/coordinates/IMD.py

Codecov / codecov/patch

package/MDAnalysis/coordinates/IMD.py#L183

Added line #L183 was not covered by tests

@staticmethod
def _format_hint(thing):
try:
parse_host_port(thing)
except:
return False
return HAS_IMDCLIENT and True

Check warning on line 191 in package/MDAnalysis/coordinates/IMD.py

Codecov / codecov/patch

package/MDAnalysis/coordinates/IMD.py#L191

Added line #L191 was not covered by tests

def close(self):
"""Gracefully shut down the reader. Stops the producer thread."""
logger.debug("IMDReader close() called")

Check warning on line 195 in package/MDAnalysis/coordinates/IMD.py

Codecov / codecov/patch

package/MDAnalysis/coordinates/IMD.py#L195

Added line #L195 was not covered by tests
if self._imdclient is not None:
self._imdclient.stop()

Check warning on line 197 in package/MDAnalysis/coordinates/IMD.py

Codecov / codecov/patch

package/MDAnalysis/coordinates/IMD.py#L197

Added line #L197 was not covered by tests
# NOTE: removeme after testing
logger.debug("IMDReader shut down gracefully.")

Check warning on line 199 in package/MDAnalysis/coordinates/IMD.py

Codecov / codecov/patch

package/MDAnalysis/coordinates/IMD.py#L199

Added line #L199 was not covered by tests

# NOTE: think of other edge cases as well- should be robust
def parse_host_port(filename):
if not filename.startswith("imd://"):
raise ValueError("IMDReader: URL must be in the format 'imd://host:port'")
# Check if the format is correct
parts = filename.split("imd://")[1].split(":")

Check warning on line 206 in package/MDAnalysis/coordinates/IMD.py

Codecov / codecov/patch

package/MDAnalysis/coordinates/IMD.py#L206

Added line #L206 was not covered by tests
if len(parts) == 2:
host = parts[0]
try:
port = int(parts[1])
return (host, port)
except ValueError as e:
raise ValueError("IMDReader: Port must be an integer") from e

Check warning on line 213 in package/MDAnalysis/coordinates/IMD.py

Codecov / codecov/patch

package/MDAnalysis/coordinates/IMD.py#L208-L213

Added lines #L208 - L213 were not covered by tests
else:
raise ValueError("IMDReader: URL must be in the format 'imd://host:port'")

Check warning on line 215 in package/MDAnalysis/coordinates/IMD.py

Codecov / codecov/patch

package/MDAnalysis/coordinates/IMD.py#L215

Added line #L215 was not covered by tests
6 changes: 6 additions & 0 deletions package/MDAnalysis/coordinates/__init__.py
Original file line number Diff line number Diff line change
@@ -274,6 +274,11 @@ class can choose an appropriate reader automatically.
| library | | | file formats`_ and |
| | | | :mod:`MDAnalysis.coordinates.chemfiles` |
+---------------+-----------+-------+------------------------------------------------------+
| IMD | IP address| r/w | Receive simulation trajectory data using interactive |
| | and port | | molecular dynamics version 3 (IMDv3) by configuring |
| | number | | a socket address to a NAMD, GROMACS, or LAMMPS |
| | | | simulation. |
+---------------+-----------+-------+------------------------------------------------------+
.. [#a] This format can also be used to provide basic *topology*
information (i.e. the list of atoms); it is possible to create a
@@ -770,6 +775,7 @@ class can choose an appropriate reader automatically.
from . import DMS
from . import GMS
from . import GRO
from . import IMD
from . import INPCRD
from . import LAMMPS
from . import MOL2
Loading