Catalog the python lab automation landscape

[bilderbuchi]

To consolidate efforts, it's important to get an overview of the different available packages. These should be analysed w.r.t. various parameters (to be defined, but things like maintenance status, activity level, maturity, design goals, scope of features, number of instrument drivers, automated test availability, ...).

Edit: Spreadsheet can be found here!
(Ethercalc has moved, link fixed 2021-01-09; site seems to be slow)

Since this is quite the large task, I have, as a first step, compiled a list of any packages mentioned in the discussion in pymeasure/pymeasure#53, as well as some from my notes:

• https://github.com/ralph-group/pymeasure
• https://www.github.com/Galvant/InstrumentKit
• https://github.com/LabPy/lantz
• https://github.com/pyvisa/pyvisa
• https://github.com/python-ivi/python-ivi
• https://github.com/BBN-Q/Auspex
• https://github.com/dursobr/Pythics and https://github.com/avelo/Pythics
• https://github.com/Exopy/i3py and https://github.com/Exopy/exopy
• https://github.com/QCoDeS/Qcodes
• https://github.com/heeres/qtlab (Inspiration for QCodes)
• https://github.com/pazzarpj/Fixate
• https://github.com/pyinstruments/pyinstruments
• https://github.com/masonlab/labdrivers
• https://github.com/Ulm-IQO/qudi
• https://github.com/LaboratoireMecaniqueLille/crappy
• https://github.com/mabuchilab/Instrumental
• https://github.com/PALab/place
• https://github.com/uetke/experimentor
• https://github.com/MatthieuDartiailh/HQCMeas (superceded by Exopy)
• https://itom.bitbucket.io/
• https://github.com/p3trus/slave
• https://github.com/wking/pycomedi
• https://pythonhosted.org/PyDAQmx/
• https://github.com/ni/nidaqmx-python
• https://github.com/NSLS-II/bluesky
• https://github.com/NSLS-II/ophyd

Preliminary list of things to note (i.e. columns in a spreadsheet) when testing packages

• Name
• One-sentence/paragraph description (e.g. from Github project description or README)
• Homepage
• Documentation URL
• nr. of commits/age, e.g. 5542/4y
• Date of latest commit
• Test system (approx. number of tests), e.g. pytest(700)
• scope: instrument/hardware communication?
• scope: live control of instruments?
• scope: running predefined procedures?
• scope: GUI/presentation layer?
• number of instrument drivers available
• types of compatible instruments (text based, register map, vendor DLL, COM protocol,...)
• unit support? (library used), e.g. yes(pint)
• License
• ... please suggest additional useful things to know.

[MatthieuDartiailh]

Please note that HQCMeas is the ancestor of Exopy and anything useful it was doing can be done in Exopy (with the exopy_hqc_legacy package installed).

[bilderbuchi]

Thanks, noted!

[Bruyant]

Hi,
you missed:
https://github.com/p3trus/slave

On the data acquisition side (DAQ cards) we have also:
https://github.com/wking/pycomedi
https://pythonhosted.org/PyDAQmx/
and a lot of vendors specifics packages

Why not putting all these projects in a spreadsheet?
I tested 3 or 4 of them I would be happy to put a feedback.

As I'm working a lot for my lab in python with the willingness to increase, it I can contributes some ressources (Engineers fro others French research team and trainee and maybe €€€) to have a long term supported framework.

[vascotenner]

This project also contains many drivers:
https://github.com/heeres/qtlab

[bilderbuchi]

Thank you, I have updated the first post.

Why not putting all these projects in a spreadsheet?
I tested 3 or 4 of them I would be happy to put a feedback.

Yes, that's a good plan. Let's start on developing a list of criteria to judge when testing packages (in the first post) to get a good, structured overview.
I don't know how to best host this - I guess markdown/wiki tables will become crazy at the size of table I am expecting. Google spreadsheet could be a good alternative, but I don't know if everybody is fine with that.

As I'm working a lot for my lab in python with the willingness to increase, it I can contributes some ressources (Engineers fro others French research team and trainee and maybe €€€) to have a long term supported framework.

This is very good to hear! I think (and others have agreed) that steady commitments will be essential.
I think at first if you can spare time to evaluate/test the different packages, and participate in discussions that would be great.

I think we should avoid reinventing the wheel, so if later on it turns out that there exists active project that is well-usable for general needs (maybe QCodes, exopy, lantz,..) that folks should just join and adapt/enhance (e.g. by porting existing drivers from other packages), that would be fine, too. I am guessing concentrating the available drivers would be a valuable contribution, but the first task is to come up with a good spec first, see #22, #9.

[Bruyant]

Spreadsheet started:
https://ethercalc.org/1anmq248ktu6

[MatthieuDartiailh]

Just a note Exopy is unlikely to ever contains instrument drivers, I3py will deal with instrument drivers. For the time being I3py does not contain any driver but the first ones are under way. The documentation about developing new drivers is quite complete, if anybody take time to actually read I will be happy to address any comment you may have.

[MatthieuDartiailh]

By the way I completed some info about I3py in the spreadsheet. Out of curiosity what do you have in mind with 'Transport layers abstraction' ?

[MatthieuDartiailh]

I would also mention that PyVisa does not have the ambitions to host standard drivers (the pyvisa-drivers is explicitely not that) however it provides a nice way to abstract over the transport layer for VISA based instrument (and with a bit more love pyvisa-py could be a good alternative to manually writing transport protocols).

[bilderbuchi]

@Bruyant great work! Did you notice the list of characteristics/columns that I put in the first post, that would probably be good to include/adapt.
edit: I added that and filled in pymeasure and instrumentkit entries

@MatthieuDartiailh Of course, having instrument drivers in a separate packages makes lots of sense (for reuse etc). Probably we should have two separate entries for exopy and i3py. I tried to be inclusive in the "scope" entries in the list in the OP, so that we can track also GUI solutions
"Transport layers abstraction": Maybe they meant which "protocols"(?) are supported in the background, what you called text based, register map, vendor DLL, COM protocol in the wiki document.
Yeah, the list I put together above also includes things that are more low level, like pyvisa, python-ivi, pycomedi.

[pbnjeff89]

I started filling out the record for masonlab/labdrivers, but I'd like to clarify what this field means:

running predefined procedures

Everything else is, as far as I'm aware, accurate. On the spreadsheet it looks like the labdrivers record has a discrepancy for the number of drivers, but to explain: we have a total of 9 drivers for various instruments, though only 2 could be categorized under the ones listed on the spreadsheet. We have drivers for a QuantumDesign dynacool (which we were able to write using pythonnet), a few Oxford instruments, Stanford Research Systems lock-ins, and a LakeShore temperature controller.

[bilderbuchi]

Thanks!

running predefined procedures

What I mean to convey is that the software in question has some built-in ability to define a measurement/acquisition sequence (e.g. loop five times: increase instrument_a voltage by 10 V, acquire instrument_b signal for 5 s, average, save results b,c,d) that can be called at will. I am open to suggestions of a better formulation.

Number of drivers: The primary measure is the overall number of drivers (I clarified the column name), I thought it would be useful to include the most popular brands so that people can get a better feel of the variety of devices supported.

[bklebel]

I was so free to add Cryostat-GUI and measureSequences to the spreadsheet.

I have to admit, when I looked at possibilities to control the devices we set up in our lab, I was discouraged to add new drivers to any of the mentioned frameworks (like pymeasure/InstrumentKit) by the rather complex architecture, and everything one has to learn about those packages and the way new drivers may be added, before being able to use 'my' instruments in this framework.
Only now that this project runs for a while, and I learned a lot about setting up something like this, I can appreciate these systems - before, I had never written object oriented python code, or much looked at code like this at all anyways. I think it will be very important, when attracting researchers to such projects, to educate them about the benefits of using and contributing to such a system from the start, compared to a 'quick and dirty' solution, where everything moves fast at first, and becomes quite painful just so late that it will be quite painful to switch at that point - especially if a considerable amount of effort invested would go to waste, as driver architectures are just too different.

regarding measureSequences:
it might not fall completely into this category here, as it only focuses on an abstract layer of running sequences parsed from Quantum Design sequence files (currently just from resistivity-PPMS). It is partially aimed at providing a basis for a fruitful collaboration between coding and non-coding researchers, as for example the pymeasure procedures seem rather hard to stomach for non-coding researchers, who might just want to do small changes.

[JAnderson419]

I have to admit, when I looked at possibilities to control the devices we set up in our lab, I was discouraged to add new drivers to any of the mentioned frameworks (like pymeasure/InstrumentKit) by the rather complex architecture, and everything one has to learn about those packages and the way new drivers may be added, before being able to use 'my' instruments in this framework.

I think this sums up the challenge with building a critical mass of engineers/physicists interesting in contributing to a unified package. Most people automating test equipment will take the path of least resistance, which is often writing their own [poor] code from scratch if their instrument is not implemented in a package. InstrumentKit has the right idea with abstract instruments that attempt to emulate common functionality of equipment, but there is still a fair bit of complexity for a beginning programmer to wrap their head around. I think the key for any package is having great descriptive tutorials for non-programmers (or beginning programmers). - I imagine that most users can write some basic scripts in matlab/mathematica/c and understand things like conditional statements and looping but might not have a solid background in test driven development, object creation/inheritance, decorators, etc.

I added my own personal take on RF measurement drivers to the csv. As an aside, if anyone happens to be at SciPy19 and wants to meet up to discuss further let me know.

[bilderbuchi]

@bklebel I appreciate your input! Do you think it would help to improve the "getting started" documentation of pymeasure/InstrumentKit? If so, would you mind opening issues at the respective repos about this, with some details about your experience and where it could be improved?

I fear the point about quick&dirty self-made making it painful to do it "properly" later is something that is hard to avoid. Especially because it's easy to cook up your own stuff that is "just right" for what you need right then, instead of going with something pre-existing which does not 100% fit your requirements/taste/preferences. Smoother "onboarding" for an existing solution would help, though! This is partly why this initiative exists, to reduce the barriers for people to contribute to existing solutions instead of DIYing it.

Btw, are you in the Freihaus? I'm a TPH alumnus, myself :-)

[bilderbuchi]

@JAnderson419 yeah I agree mostly. However, do you think it feasible to structure a project such that the code is generic, elegant and powerful, while still being grokable enough for a non-/new-programmer, who lacks background e.g. in object orientation, to contribute new code to?

I think much of the pain point also stems from the fragmentation in the community/package landscape. Without a unifying drive, you end up with N>>1 packages which are bound to have 0.1-1 maintainers, so should you need help the probability is high that you will wait a long time. Also, projects end up dead/unmaintained if the maintainer loses interest.
Compare e.g. to numpy - could you imagine the situation if there were 50 small numerics subpackages, each maintained by a single person somewhere?

[bklebel]

@bilderbuchi Yes, I think it would help a lot, especially since this section (to me) seems to be aimed at somehow experienced programmers - I think it could help a lot if a description of "how can I make stuff quick but not too dirty, using this framework" could be placed rather prominent. Thinking about the mindset people are in, and taking them further from there.
Yes, I will open such issues, hopefully within due time.

regarding your comment to @JAnderson419, I think this is certainly possible, if you can "collect" people just when they start, give them easy recipes to follow, they will become engaged with a much higher probability at some later stage.

Regarding TU, I will be at Freihaus again from October on, currently on a research stay abroad. However the setup I am building this GUI for, is in Freihaus, IFP