How to give subclasses of Instrument similar behavior as find_or_create_instrument

[rsokolewicz]

I'm one of the developers of Quantify and we have a few classes that inherit from Instrument, but they don't always correspond to physical Instruments. They are more used as containers for qcodes parameters and submodules. So we would like to be able to be able to run multiple constructor calls without running into "Instrument with already exists" type of errors.

I am aware of the find_or_create_instrument function, but I'd like to have a similar behavior on the subclasses themselves. I've tried a few things but nothing seems to work out.

For example,

from qcodes.instrument import Instrument, find_or_create_instrument
Instrument.close_all()

class Foo(Instrument):
    def __new__(cls, name):
        instrument_exists = Instrument.exist("foo") and Instrument.find_instrument("foo").__class__ is cls
        if instrument_exists:
            return Instrument.find_instrument("foo")
        else:
            return cls(name)
    
    def __init__(self, name):
        super().__init__(name)
        self.hello = "hello"

    def get_idn(self):
        return {
            "vendor": None,
            "model": None,
            "serial": None,
            "firmware": None,
        }

foo = Foo("foo")
foo2 = Foo("foo")

assert foo is foo2

gives me the following error trace:

---

{
	"name": "RecursionError",
	"message": "maximum recursion depth exceeded while getting the str of an object",
	"stack": "---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
File ~/.anaconda3/envs/dev/lib/python3.9/site-packages/qcodes/instrument/instrument.py:323, in Instrument.exist(name, instrument_class)
    322 try:
--> 323     _ = Instrument.find_instrument(name, instrument_class=instrument_class)
    325 except KeyError as exception:

File ~/.anaconda3/envs/dev/lib/python3.9/site-packages/qcodes/instrument/instrument.py:296, in Instrument.find_instrument(cls, name, instrument_class)
    295 if name not in cls._all_instruments:
--> 296     raise KeyError(f\"Instrument with name {name} does not exist\")
    297 ins = cls._all_instruments[name]

KeyError: 'Instrument with name foo does not exist'

During handling of the above exception, another exception occurred:

RecursionError                            Traceback (most recent call last)
Cell In[22], line 24
     16     def get_idn(self):
     17         return {
     18             \"vendor\": None,
     19             \"model\": None,
     20             \"serial\": None,
     21             \"firmware\": None,
     22         }
---> 24 foo = Foo(\"foo\")
     25 foo2 = Foo(\"foo\")
     27 # foo2 = find_or_create_instrument(Foo, \"foo\")

File ~/.anaconda3/envs/dev/lib/python3.9/site-packages/qcodes/instrument/instrument_meta.py:36, in InstrumentMeta.__call__(cls, *args, **kwargs)
     31 def __call__(cls, *args: Any, **kwargs: Any) -> Any:
     32     \"\"\"
     33     Overloads `type.__call__` to add code that runs only if __init__ completes
     34     successfully.
     35     \"\"\"
---> 36     new_inst = super().__call__(*args, **kwargs)
     37     is_abstract = new_inst._is_abstract()
     38     if is_abstract:

Cell In[22], line 10, in Foo.__new__(cls, name)
      8     return Instrument.find_instrument(\"foo\")
      9 else:
---> 10     return cls(name)

File ~/.anaconda3/envs/dev/lib/python3.9/site-packages/qcodes/instrument/instrument_meta.py:36, in InstrumentMeta.__call__(cls, *args, **kwargs)
     31 def __call__(cls, *args: Any, **kwargs: Any) -> Any:
     32     \"\"\"
     33     Overloads `type.__call__` to add code that runs only if __init__ completes
     34     successfully.
     35     \"\"\"
---> 36     new_inst = super().__call__(*args, **kwargs)
     37     is_abstract = new_inst._is_abstract()
     38     if is_abstract:

Cell In[22], line 10, in Foo.__new__(cls, name)
      8     return Instrument.find_instrument(\"foo\")
      9 else:
---> 10     return cls(name)

    [... skipping similar frames: InstrumentMeta.__call__ at line 36 (739 times), Foo.__new__ at line 10 (738 times)]

Cell In[22], line 10, in Foo.__new__(cls, name)
      8     return Instrument.find_instrument(\"foo\")
      9 else:
---> 10     return cls(name)

File ~/.anaconda3/envs/dev/lib/python3.9/site-packages/qcodes/instrument/instrument_meta.py:36, in InstrumentMeta.__call__(cls, *args, **kwargs)
     31 def __call__(cls, *args: Any, **kwargs: Any) -> Any:
     32     \"\"\"
     33     Overloads `type.__call__` to add code that runs only if __init__ completes
     34     successfully.
     35     \"\"\"
---> 36     new_inst = super().__call__(*args, **kwargs)
     37     is_abstract = new_inst._is_abstract()
     38     if is_abstract:

Cell In[22], line 6, in Foo.__new__(cls, name)
      5 def __new__(cls, name):
----> 6     instrument_exists = Instrument.exist(\"foo\") and Instrument.find_instrument(\"foo\").__class__ is cls
      7     if instrument_exists:
      8         return Instrument.find_instrument(\"foo\")

File ~/.anaconda3/envs/dev/lib/python3.9/site-packages/qcodes/instrument/instrument.py:326, in Instrument.exist(name, instrument_class)
    323     _ = Instrument.find_instrument(name, instrument_class=instrument_class)
    325 except KeyError as exception:
--> 326     instrument_is_not_found = any(
    327         str_ in str(exception) for str_ in [name, \"has been removed\"]
    328     )
    330     if instrument_is_not_found:
    331         instrument_exists = False

File ~/.anaconda3/envs/dev/lib/python3.9/site-packages/qcodes/instrument/instrument.py:327, in <genexpr>(.0)
    323     _ = Instrument.find_instrument(name, instrument_class=instrument_class)
    325 except KeyError as exception:
    326     instrument_is_not_found = any(
--> 327         str_ in str(exception) for str_ in [name, \"has been removed\"]
    328     )
    330     if instrument_is_not_found:
    331         instrument_exists = False

RecursionError: maximum recursion depth exceeded while getting the str of an object"
}

I also tried overloading __call__ and using find_or_create_instrument inside __new__ but no success yet. Do you have any ideas how to get this done?

thank you :)

[jenshnielsen]

I'm one of the developers of Quantify and we have a few classes that inherit from Instrument, but they don't always correspond to physical Instruments. They are more used as containers for qcodes parameters and submodules. So we would like to be able to be able to run multiple constructor calls without running into "Instrument with already exists" type of errors.

I actually think we should implement this for all instruments which I was toying with in #4038 but it not something that I have had the time to pursue and will not have for some time.

I would however be happy to review a pr that implements that in general. To make that work I think we need to implement some kind of the same instrument as interface that can check the arguments to init to check if they match.

E.g. for visa I would:

• return the existing instrument iff name and adresse is the same.
• If name matches but address is different error out
• If name does not exist create a new one.
• Could also potentially extend to check if the address is already in use and error out.

As for why your specific code does not work:

1. When you construct a class you call __call__ on the metaclass
2. This calls __new__ and then __init__
3. But inside your implementation of __new__ you do cls() which is equivalent to calling call on the metaclass which gets you back to one so you have an infinite loop which eventually triggers a RecursionError.

In other words doing cls() from within new is wrong. I think the correct solution would be to trigger new from the super class with. super().new()