refid KeyError when adding SNAP "Collocate" node to workflow

[pbrotoisworo]

Hello, I get an error when trying to create an XML file with the SNAP "Collocate" operator. I haven't encountered this before and the code should be right.

• which operating system are you using?
  Windows 8
• which environment is pyroSAR running in?
  Anaconda virtual environment
• which version of pyroSAR are you using?
  Via conda
• which function of pyroSAR did you call with which parameters?

# Code that I'm using

workflow = parse_recipe('blank')

read1 = parse_node('Read')
read1.parameters['file'] = dim
workflow.insert_node(read1)

read2 = parse_node('Read')
read2.parameters['file'] = clip_water_mask
read2.parameters['formatName'] = 'GeoTIFF'
workflow.insert_node(read2)

collocate = parse_node('Collocate')
collocate.parameters['masterProductName'] = os.path.basename(dim).rstrip('.dim')
collocate.parameters['renameMasterComponents'] = False
collocate.parameters['renameSlaveComponents'] = False

# This line below raises the error
workflow.insert_node(collocate, before=[read1.id, read2.id])

• if applicable, which version of SNAP or GAMMA are you using in pyroSAR?
  SNAP
• the full error message

  File "workflows.py", line 186, in add_water_mask
    workflow.insert_node(collocate, before=[read1.id, read2.id])
  File "C:\Users\Angelo\Miniconda3\envs\pyrosar\lib\site-packages\pyroSAR\snap\auxil.py", line 892, in insert_node
    self.__reset_successor_source(newnode.id)
  File "C:\Users\Angelo\Miniconda3\envs\pyrosar\lib\site-packages\pyroSAR\snap\auxil.py", line 788, in __reset_successor_source
    reset(id, self[id].source)
  File "C:\Users\Angelo\Miniconda3\envs\pyrosar\lib\site-packages\pyroSAR\snap\auxil.py", line 1124, in source
    sources.append(element.attrib['refid'])
KeyError: 'refid'

I've confirmed that this graph should work in SNAP. I tested it in the GUI. Here is an XML that I created using the graph tool in SNAP GUI.

Open XML from SNAP GUI

<graph id="Graph">
  <version>1.0</version>
  <node id="Read">
    <operator>Read</operator>
    <sources/>
    <parameters class="com.bc.ceres.binding.dom.XppDomElement">
      <file>C:\Users\Angelo\Documents\PANJI\Projects\sar_disaster_assessment\flood\seasonality_120E_20Nv1_3_2020.tif</file>
    </parameters>
  </node>
  <node id="Collocate">
    <operator>Collocate</operator>
    <sources>
      <sourceProduct refid="Read"/>
      <sourceProduct.1 refid="Read(2)"/>
    </sources>
    <parameters class="com.bc.ceres.binding.dom.XppDomElement">
      <sourceProductPaths/>
      <masterProductName>stack</masterProductName>
      <targetProductName>_collocated</targetProductName>
      <targetProductType>COLLOCATED</targetProductType>
      <renameMasterComponents>false</renameMasterComponents>
      <renameSlaveComponents>false</renameSlaveComponents>
      <masterComponentPattern>${ORIGINAL_NAME}_M</masterComponentPattern>
      <slaveComponentPattern>${ORIGINAL_NAME}</slaveComponentPattern>
      <resamplingType>NEAREST_NEIGHBOUR</resamplingType>
    </parameters>
  </node>
  <node id="Read(2)">
    <operator>Read</operator>
    <sources/>
    <parameters class="com.bc.ceres.binding.dom.XppDomElement">
      <file>C:\Users\Angelo\Documents\PANJI\Projects\sar_disaster_assessment\flood\output\stack.dim</file>
    </parameters>
  </node>
  <node id="Write">
    <operator>Write</operator>
    <sources>
      <sourceProduct refid="Collocate"/>
    </sources>
    <parameters class="com.bc.ceres.binding.dom.XppDomElement">
      <file>C:\Users\Angelo\Documents\PANJI\Projects\sar_disaster_assessment\flood\output\_collocated.dim</file>
      <formatName>BEAM-DIMAP</formatName>
    </parameters>
  </node>
  <applicationData id="Presentation">
    <Description/>
    <node id="Read">
            <displayPosition x="37.0" y="134.0"/>
    </node>
    <node id="Collocate">
      <displayPosition x="246.0" y="133.0"/>
    </node>
    <node id="Read(2)">
      <displayPosition x="27.0" y="186.0"/>
    </node>
    <node id="Write">
            <displayPosition x="453.0" y="136.0"/>
    </node>
  </applicationData>
</graph>

[johntruckenbrodt]

Hi @pbrotoisworo I have never used the Collocate node and apparently pyroSAR is not treating it correctly.
The XMl template looks like this:

<?xml version="1.0" ?><node id="Collocate">	
      	<operator>Collocate</operator>	
      	<sources>		
        		<sourceProduct refid="Read"/>		
        		<slave/>		
        		<sourceProducts/>		
      	</sources>	
      	<parameters class="com.bc.ceres.binding.dom.XppDomElement">		
        		<sourceProductPaths/>		
        		<masterProductName/>		
        		<targetProductType>COLLOCATED</targetProductType>		
        		<renameMasterComponents>true</renameMasterComponents>		
        		<renameSlaveComponents>true</renameSlaveComponents>		
        		<masterComponentPattern/>		
        		<slaveComponentPattern/>		
        		<resamplingType>NEAREST_NEIGHBOUR</resamplingType>		
      	</parameters>	
    </node>

passing multiple IDs as source to a node was developed for SliceAssembly, whose template looks like this:

<?xml version="1.0" ?><node id="SliceAssembly">	
      	<operator>SliceAssembly</operator>	
      	<sources>		
        		<sourceProduct refid="Read"/>		
      		        <sourceProduct.1 refid="Read (2)"/>	
        </sources>	
      	<parameters>		
        		<selectedPolarisations/>		
      	</parameters>	
    </node>

I am sure it is not so hard to adjust, but it will take me some days to find the time for it. If you are faster then a PR is of course always appreciated 😉.

[pbrotoisworo]

Hmm, I found the relevant XML file in the .pyrosar folder. But I'm not sure where is the code that creates the data in .pyrosar\snap\nodes. Can you point me to where it is @johntruckenbrodt ? I think I can try and sort this out.

[johntruckenbrodt]

This is all done in function pyroSAR.snap.auxil.parse_node. It executes gpt Collocate -h, reads the XML representation, and stores it in the .pyrosar/snap/nodes folder.

Setting the node's source(s) is done here. Node.source has a get method (property) and a set method (@source.setter).

So the first question is whether parse_node has to be modified so that the XML representation is better or whether this looks alright and just the source.setter has to be adjusted to properly fill in the values of the list of node IDs you want to set as source.

[pbrotoisworo]

Ah I see. Thanks for the pointers I'll take a look at it.

[johntruckenbrodt]

Awesome! Thanks @pbrotoisworo