ValueError: could not broadcast input array from shape

[inofechm]

Hi I am trying to run enact but have the following error:

Processing overlapping bins: 100%|##############################################################################################| 12152/12152 [14:40<00:00, 13.80bin/s]
  0%|                                                                                                                                                         | 0/24564 [00:00<?, ?it/s]
  0%|                                                                                                                                           | 0/40 [16:23<?, ?it/s] [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/.mounts/labs/abelsonlab/public/Ido/u20_python_3.9/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/.mounts/labs/abelsonlab/public/Ido/u20_python_3.9/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/.mounts/labs/abelsonlab/public/Ido/u20_python_3.9_venv/enact-pipeline/src/enact/pipeline.py", line 1070, in <module>
    so_hd.run_enact()
  File "/.mounts/labs/abelsonlab/public/Ido/u20_python_3.9_venv/enact-pipeline/src/enact/pipeline.py", line 1035, in run_enact
    self.assign_bins_to_cells(bins_adata)
  File "/.mounts/labs/abelsonlab/public/Ido/u20_python_3.9_venv/enact-pipeline/src/enact/pipeline.py", line 677, in assign_bins_to_cells
    result_spatial_join, expanded_adata = weight_by_cluster_assignment(
  File "/.mounts/labs/abelsonlab/public/Ido/u20_python_3.9_venv/enact-pipeline/src/enact/assignment_methods/weight_by_gene.py", line 150, in weight_by_cluster_assignment
    expanded_adata = apply_weights_to_adata_counts(expanded_adata, weights_df)
  File "/.mounts/labs/abelsonlab/public/Ido/u20_python_3.9_venv/enact-pipeline/src/enact/assignment_methods/weight_by_gene.py", line 33, in apply_weights_to_adata_counts
    weights_matrix[idx] = weights.iloc[0].tolist()
ValueError: could not broadcast input array from shape (18127,) into shape (18085,)

Can you please provide any advice on how to fix this, perhaps it is related to issues in #13?

here is the full log if that helps :

2024-12-09 12:01:19,416 - ENACT - INFO - <initiate_instance_variables> ENACT running with the following configurations:
 analysis_name: CRC_P1
 run_synthetic: False
 cache_dir: /.mounts/labs/abelsonlab/public/Ido/visium_hd/outs/CRC_P1/enact/
 wsi_path: /.mounts/labs/abelsonlab/public/Ido/visium_hd/data/CRC_P1/Visium_HD_Human_Colon_Cancer_P1_tissue_image.btf
 visiumhd_h5_path: /.mounts/labs/abelsonlab/public/Ido/visium_hd/data/CRC_P1/binned_outputs/square_002um/filtered_feature_bc_matrix.h5
 tissue_positions_path: /.mounts/labs/abelsonlab/public/Ido/visium_hd/data/CRC_P1/binned_outputs/square_002um/spatial/tissue_positions.parquet
 segmentation: True
 bin_to_geodataframes: True
 bin_to_cell_assignment: True
 cell_type_annotation: False
 seg_method: stardist
 patch_size: 4000
 bin_representation: polygon
 bin_to_cell_method: weighted_by_cluster
 cell_annotation_method: no annotation
 cell_typist_model:
 use_hvg: False
 n_hvg: 1000
 n_clusters: 4
 chunks_to_run: []
 cell_markers: {}

pipeline.py (274):
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
pipeline.py (277):
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
2024-12-09 12:01:43,658 - ENACT - INFO - <load_image> Successfully loaded image!
2024-12-09 12:02:14,069 - ENACT - INFO - <normalize_image> Successfully normalized image!
Found model '2D_versatile_he' for 'StarDist2D'.
2024-12-09 12:02:14.282188: W tensorflow/core/common_runtime/gpu/gpu_device.cc:2343] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are instal
led properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
Loading network weights from 'weights_best.h5'.
Loading thresholds from 'thresholds.json'.
Using default values: prob_thresh=0.692478, nms_thresh=0.3.
effective: block_size=(4096, 4096, 3), min_overlap=(128, 128, 0), context=(128, 128, 0)
functional.py (238): The structure of `inputs` doesn't match the expected structure.
Expected: ['input']
Received: inputs=Tensor(shape=(1, 256, 256, 3))
  0%|                                                                                                                                           | 0/49 [00:00<?, ?it/s]functional.py (23
8): The structure of `inputs` doesn't match the expected structure.
Expected: ['input']
Received: inputs=Tensor(shape=(1, None, None, 3))
100%|##################################################################################################################################| 49/49 [16:01<00:00, 19.62s/it]
2024-12-09 12:18:17,229 - ENACT - INFO - <run_segmentation> Successfully segmented cells!
2024-12-09 12:19:02,141 - ENACT - INFO - <split_df_to_chunks> Splitting into chunks. output_dir: /.mounts/labs/abelsonlab/public/Ido/visium_hd/outs/CRC_P1/enact/CRC_P1/chunks/cells_gdf
100%|##################################################################################################################################| 40/40 [00:20<00:00,  1.98it/s]
anndata.py (1820): Variable names are not unique. To make them unique, call `.var_names_make_unique`.
anndata.py (1820): Variable names are not unique. To make them unique, call `.var_names_make_unique`.
pipeline.py (449): Series.__getitem__ treating keys as positions is deprecated. In a future version, integer keys will always be treated as labels (consistent with DataFrame behavior).
 To access a value by position, use `ser.iloc[pos]`
2024-12-09 12:20:01,299 - ENACT - INFO - <generate_bin_polys> Generating bin polygons. num_bins: 8115564
100%|#####################################################################################################################| 8115564/8115564 [03:19<00:00, 40694.21it/s]
2024-12-09 12:23:34,013 - ENACT - INFO - <split_df_to_chunks> Splitting into chunks. output_dir: /.mounts/labs/abelsonlab/public/Ido/visium_hd/outs/CRC_P1/enact/CRC_P1/chunks/bins_gdf
100%|##################################################################################################################################| 38/38 [03:58<00:00,  6.28s/it]
anndata.py (1820): Variable names are not unique. To make them unique, call `.var_names_make_unique`.
anndata.py (1820): Variable names are not unique. To make them unique, call `.var_names_make_unique`.
pipeline.py (449): Series.__getitem__ treating keys as positions is deprecated. In a future version, integer keys will always be treated as labels (consistent with DataFrame behavior).
 To access a value by position, use `ser.iloc[pos]`
2024-12-09 12:28:11,468 - ENACT - INFO - <assign_bins_to_cells> Assigning bins to cells using weighted_by_cluster method
  0%|                                                                                                                                           | 0/40 [00:00<?, ?it/s]pipeline.py (641)
: Trying to modify attribute `.obs` of view, initializing view as actual.
anndata.py (1818): Observation names are not unique. To make them unique, call `.obs_names_make_unique`.
anndata.py (1820): Variable names are not unique. To make them unique, call `.var_names_make_unique`.
anndata.py (1818): Observation names are not unique. To make them unique, call `.obs_names_make_unique`.
anndata.py (1820): Variable names are not unique. To make them unique, call `.var_names_make_unique`.
pipeline.py (648): Trying to modify attribute `.obs` of view, initializing view as actual.
anndata.py (1820): Variable names are not unique. To make them unique, call `.var_names_make_unique`.
anndata.py (1820): Variable names are not unique. To make them unique, call `.var_names_make_unique`.
2024-12-09 12:28:21,028 - ENACT - INFO - <assign_bins_to_cells> done spatial join
anndata.py (1820): Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Processing overlapping bins: 100%|##############################################################################################| 12152/12152 [14:40<00:00, 13.80bin/s]
  0%|                                                                                                                                                         | 0/24564 [00:00<?, ?it/s]
  0%|                                                                                                                                           | 0/40 [16:23<?, ?it/s] [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/.mounts/labs/abelsonlab/public/Ido/u20_python_3.9/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/.mounts/labs/abelsonlab/public/Ido/u20_python_3.9/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/.mounts/labs/abelsonlab/public/Ido/u20_python_3.9_venv/enact-pipeline/src/enact/pipeline.py", line 1070, in <module>
    so_hd.run_enact()
  File "/.mounts/labs/abelsonlab/public/Ido/u20_python_3.9_venv/enact-pipeline/src/enact/pipeline.py", line 1035, in run_enact
    self.assign_bins_to_cells(bins_adata)
  File "/.mounts/labs/abelsonlab/public/Ido/u20_python_3.9_venv/enact-pipeline/src/enact/pipeline.py", line 677, in assign_bins_to_cells
    result_spatial_join, expanded_adata = weight_by_cluster_assignment(
  File "/.mounts/labs/abelsonlab/public/Ido/u20_python_3.9_venv/enact-pipeline/src/enact/assignment_methods/weight_by_gene.py", line 150, in weight_by_cluster_assignment
    expanded_adata = apply_weights_to_adata_counts(expanded_adata, weights_df)
  File "/.mounts/labs/abelsonlab/public/Ido/u20_python_3.9_venv/enact-pipeline/src/enact/assignment_methods/weight_by_gene.py", line 33, in apply_weights_to_adata_counts
    weights_matrix[idx] = weights.iloc[0].tolist()
ValueError: could not broadcast input array from shape (18127,) into shape (18085,)

[inofechm]

Hi following up I was able to solve this issue by adding:

bins_adata.obs_names_make_unique()
bins_adata.var_names_make_unique()

everytime after an adata object is loaded in such as
bins_adata, bin_size = self.load_visiumhd_dataset(crop_bounds)