Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error running ICA-AROMA #51

Open
StephDocTUM opened this issue Oct 26, 2020 · 22 comments
Open

Error running ICA-AROMA #51

StephDocTUM opened this issue Oct 26, 2020 · 22 comments

Comments

@StephDocTUM
Copy link

Dear all,

I ran ICA-AROMA for several subjects and everything worked well.
For some subjects I encounter the following error, although they are very similar to the ones where ICA-AROMA works fine:

------------------------------- RUNNING ICA-AROMA -------------------------------
--------------- 'ICA-based Automatic Removal Of Motion Artifacts' ---------------

Step 1) MELODIC
libc++abi.dylib: terminating with uncaught exception of type std::logic_error: subtraction: incompatible matrix dimensions: 0x0 and 1x0
Traceback (most recent call last):
File "/Users/neurolab/Desktop/Stephan/Programmes/ICA-AROMA-master/ICA_AROMA.py", line 204, in
aromafunc.runICA(fslDir, inFile, outDir, melDir, mask, dim, TR)
File "/Users/neurolab/Desktop/Stephan/Programmes/ICA-AROMA-master/ICA_AROMA_functions.py", line 94, in runICA
nrICs = int(float(subprocess.getoutput(cmd)))
ValueError: could not convert string to float: Image Exception : #63 :: No image files match: /Users/neurolab/Desktop/Drmed/CART_Code_rearranged_concat_prep/derivatives/participant/fmriprep/sub-10/ses-5/func/ICA_AROMA/melodic.ica/melodic_IC
No ima

I couldn't find out why this is happening,

Best,

@maartenmennes
Copy link
Owner

Hi Stephan,
it looks that for these subject the melodic run failed? Perhaps it ran out of memory and you did not notice.

@StephDocTUM
Copy link
Author

Thanks a lot for your reply.

That error looks different:

------------------------------- RUNNING ICA-AROMA -------------------------------
--------------- 'ICA-based Automatic Removal Of Motion Artifacts' ---------------

Step 1) MELODIC
Step 2) Automatic classification of the components

  • registering the spatial maps to MNI
  • extracting the CSF & Edge fraction features
  • extracting the Maximum RP correlation feature
  • extracting the High-frequency content feature
  • classification
    Found 1 file(s)
    Found 20 head motion-related components in a total of 20 components.
    Traceback (most recent call last):
    File "/Users/neurolab/Desktop/Stephan/Programmes/ICA-AROMA-master/ICA_AROMA.py", line 229, in
    outDir)
    File "/Users/neurolab/Desktop/Stephan/Programmes/ICA-AROMA-master/classification_plots.py", line 172, in classification_plot
    hist_kws={'alpha': 0.2})
    File "/Users/neurolab/Library/Python/2.7/lib/python/site-packages/seaborn/distributions.py", line 231, in distplot
    kdeplot(a, vertical=vertical, ax=ax, color=kde_color, **kde_kws)
    File "/Users/neurolab/Library/Python/2.7/lib/python/site-packages/seaborn/distributions.py", line 691, in kdeplot
    cumulative=cumulative, **kwargs)
    File "/Users/neurolab/Library/Python/2.7/lib/python/site-packages/seaborn/distributions.py", line 294, in _univariate_kdeplot
    x, y = _scipy_univariate_kde(data, bw, gridsize, cut, clip)
    File "/Users/neurolab/Library/Python/2.7/lib/python/site-packages/seaborn/distributions.py", line 366, in _scipy_univariate_kde
    kde = stats.gaussian_kde(data, bw_method=bw)
    File "/Users/neurolab/Library/Python/2.7/lib/python/site-packages/scipy/stats/kde.py", line 208, in init
    self.set_bandwidth(bw_method=bw_method)
    File "/Users/neurolab/Library/Python/2.7/lib/python/site-packages/scipy/stats/kde.py", line 540, in set_bandwidth
    self._compute_covariance()
    File "/Users/neurolab/Library/Python/2.7/lib/python/site-packages/scipy/stats/kde.py", line 552, in _compute_covariance
    self._data_inv_cov = linalg.inv(self._data_covariance)
    File "/Users/neurolab/Library/Python/2.7/lib/python/site-packages/scipy/linalg/basic.py", line 974, in inv
    raise LinAlgError("singular matrix")
    numpy.linalg.linalg.LinAlgError: singular matrix

@maartenmennes
Copy link
Owner

yes: "Found 20 head motion-related components in a total of 20 components."

i.e., all of your components are marked to be removed from the data, nothing would be left... Something is likely off, either in your data itself (e.g., a severe artifact) or in the analysis path (e.g., registration failed so your brain data is not on the standard brain)

@StephDocTUM
Copy link
Author

or because I set the criteria to -dim 20?

@maartenmennes
Copy link
Owner

potentially, but unlikely. A good indication could be to see how many components melodic returns if you don't set the dimensionality to 20. Is there a particular reason why you did this? We typically recommend to not set the dimensionality for denoising purposes (unless you have a huge number of volumes e.g., in the case of multiband data).

@StephDocTUM
Copy link
Author

without the command it works:

------------------------------- RUNNING ICA-AROMA -------------------------------
--------------- 'ICA-based Automatic Removal Of Motion Artifacts' ---------------

Step 1) MELODIC
Step 2) Automatic classification of the components

  • registering the spatial maps to MNI
  • extracting the CSF & Edge fraction features
  • extracting the Maximum RP correlation feature
  • extracting the High-frequency content feature
  • classification
    Found 1 file(s)
    Found 49 head motion-related components in a total of 51 components.
    Step 3) Data denoising

----------------------------------- Finished -----------------------------------

@StephDocTUM
Copy link
Author

But I am still having the error from above:
runs for about 5 seconds and then brings up this error (available memory 64GB and same command runs fine for other subjects)

--------------- 'ICA-based Automatic Removal Of Motion Artifacts' ---------------

Step 1) MELODIC
libc++abi.dylib: terminating with uncaught exception of type std::logic_error: subtraction: incompatible matrix dimensions: 0x0 and 1x0
Traceback (most recent call last):
File "/Users/neurolab/Desktop/Stephan/Programmes/ICA-AROMA-master/ICA_AROMA.py", line 204, in
aromafunc.runICA(fslDir, inFile, outDir, melDir, mask, dim, TR)
File "/Users/neurolab/Desktop/Stephan/Programmes/ICA-AROMA-master/ICA_AROMA_functions.py", line 94, in runICA
nrICs = int(float(subprocess.getoutput(cmd)))
ValueError: could not convert string to float: Image Exception : #63 :: No image files match: /Users/neurolab/Desktop/Drmed/CART_Code_rearranged_concat_prep/derivatives/participant/fmriprep/sub-1/ses-1/func/ICA_AROMA_auto/melodic.ica/melodic_IC
No

and it is not a memory error.

@maartenmennes
Copy link
Owner

This error is related to FSL's melodic and not ICA-AROMA. I suspect something is off with your input data or the mask that is being calculated. Have you looked at the raw data? Have you tried running melodic directly on those data?

@StephDocTUM
Copy link
Author

The input data looks fine. And running directly with melodic works.

@maartenmennes
Copy link
Owner

Do the components look sensible? You can now also try to run AROMA where you provide the melodic directory you just created. If you run melodic as part of FEAT, AROMA will detect the .ica directory and not rerun melodic.

@StephDocTUM
Copy link
Author

How would I provide the output directory to AROMA?

for ((i = 1; i < 2; i++));for ((ses = 1; ses < 8; ses++));
python2.7 /Users/neurolab/Desktop/Stephan/Programmes/ICA-AROMA-master/ICA_AROMA.py -in /Users/neurolab/Desktop/Drmed/CART_Code_rearranged_concat_prep/derivatives/participant/fmriprep/sub-${i}/ses-${ses}/func/sub-${i}_ses-${ses}_task-rest_run-3_space-MNI152NLin2009cAsym_desc-preproc_bold_skipped_5_smoothed_6.nii.gz -out /Users/neurolab/Desktop/Drmed/CART_Code_rearranged_concat_prep/derivatives/participant/fmriprep/sub-${i}/ses-${ses}/func/sub-1_ses-1_task-rest_run-3_space-MNI152NLin2009cAsym_desc-preproc_bold_skipped_5_smoothed_6.ica -mc /Users/neurolab/Desktop/Drmed/CART_Code_rearranged_concat_prep/sub-${i}/ses-${ses}/func/sub-${i}_ses-${ses}_task-rest_run-3_bold_mcf.par

sub-1_ses-1_task-rest_run-3_space-MNI152NLin2009cAsym_desc-preproc_bold_skipped_5_smoothed_6.ica is the created output directory.

@maartenmennes
Copy link
Owner

use the -md option. The -out option is for the directory where you want AROMA to save its results...

python2.7 /Users/neurolab/Desktop/Stephan/Programmes/ICA-AROMA-master/ICA_AROMA.py --help

gives your more info...

@StephDocTUM
Copy link
Author

ah, great. Could not find that command in the manual.
Coming back to my post above:
...Found 49 head motion-related components in a total of 51 components.. means only 2 components are left in the end?
Isn't that too little?
I was putting dim to 20 because, I had the feeling ICA-AROMA is denoting out too many components. In the end my correlation for my bold signal seemed to be very high.

What would be your thoughts on that?

@StephDocTUM
Copy link
Author

running ICA-AROMA after melodic gives the following error:

------------------------------- RUNNING ICA-AROMA -------------------------------
--------------- 'ICA-based Automatic Removal Of Motion Artifacts' ---------------

Step 1) MELODIC

  • The existing/specified MELODIC directory will be used.
  • The MELODIC directory does not contain the required 'stats' folder. Mixture modeling on the Z-statistical maps will be run.
    Step 2) Automatic classification of the components
  • registering the spatial maps to MNI
  • extracting the CSF & Edge fraction features
    • The spatial map of component 1 is empty. Please check!
    • The spatial map of component 2 is empty. Please check!
    • The spatial map of component 3 is empty. Please check!
    • The spatial map of component 4 is empty. Please check!
    • The spatial map of component 5 is empty. Please check!
    • The spatial map of component 6 is empty. Please check!
    • The spatial map of component 7 is empty. Please check!
    • The spatial map of component 8 is empty. Please check!
    • The spatial map of component 9 is empty. Please check!
    • The spatial map of component 10 is empty. Please check!
    • The spatial map of component 11 is empty. Please check!
    • The spatial map of component 12 is empty. Please check!
    • The spatial map of component 13 is empty. Please check!
    • The spatial map of component 14 is empty. Please check!
    • The spatial map of component 15 is empty. Please check!
    • The spatial map of component 16 is empty. Please check!
    • The spatial map of component 17 is empty. Please check!
    • The spatial map of component 18 is empty. Please check!
    • The spatial map of component 19 is empty. Please check!
    • The spatial map of component 20 is empty. Please check!
    • The spatial map of component 21 is empty. Please check!
    • The spatial map of component 22 is empty. Please check!
    • The spatial map of component 23 is empty. Please check!
    • The spatial map of component 24 is empty. Please check!
    • The spatial map of component 25 is empty. Please check!
    • The spatial map of component 26 is empty. Please check!
    • The spatial map of component 27 is empty. Please check!
    • The spatial map of component 28 is empty. Please check!
    • The spatial map of component 29 is empty. Please check!
    • The spatial map of component 30 is empty. Please check!
    • The spatial map of component 31 is empty. Please check!
    • The spatial map of component 32 is empty. Please check!
    • The spatial map of component 33 is empty. Please check!
    • The spatial map of component 34 is empty. Please check!
    • The spatial map of component 35 is empty. Please check!
    • The spatial map of component 36 is empty. Please check!
    • The spatial map of component 37 is empty. Please check!
    • The spatial map of component 38 is empty. Please check!
    • The spatial map of component 39 is empty. Please check!
    • The spatial map of component 40 is empty. Please check!
    • The spatial map of component 41 is empty. Please check!
    • The spatial map of component 42 is empty. Please check!
    • The spatial map of component 43 is empty. Please check!
    • The spatial map of component 44 is empty. Please check!
    • The spatial map of component 45 is empty. Please check!
    • The spatial map of component 46 is empty. Please check!
    • The spatial map of component 47 is empty. Please check!
    • The spatial map of component 48 is empty. Please check!
    • The spatial map of component 49 is empty. Please check!
    • The spatial map of component 50 is empty. Please check!
    • The spatial map of component 51 is empty. Please check!
    • The spatial map of component 52 is empty. Please check!
    • The spatial map of component 53 is empty. Please check!
    • The spatial map of component 54 is empty. Please check!
    • The spatial map of component 55 is empty. Please check!
    • The spatial map of component 56 is empty. Please check!
    • The spatial map of component 57 is empty. Please check!
    • The spatial map of component 58 is empty. Please check!
    • The spatial map of component 59 is empty. Please check!
    • The spatial map of component 60 is empty. Please check!
    • The spatial map of component 61 is empty. Please check!
    • The spatial map of component 62 is empty. Please check!
    • The spatial map of component 63 is empty. Please check!
    • The spatial map of component 64 is empty. Please check!
  • extracting the Maximum RP correlation feature
  • extracting the High-frequency content feature
  • classification
    Found 1 file(s)
    Found 28 head motion-related components in a total of 64 components.
    Traceback (most recent call last):
    File "/Users/neurolab/Desktop/Stephan/Programmes/ICA-AROMA-master/ICA_AROMA.py", line 229, in
    outDir)
    File "/Users/neurolab/Desktop/Stephan/Programmes/ICA-AROMA-master/classification_plots.py", line 180, in classification_plot
    hist_kws={'alpha': 0.2})
    File "/Users/neurolab/Library/Python/2.7/lib/python/site-packages/seaborn/distributions.py", line 231, in distplot
    kdeplot(a, vertical=vertical, ax=ax, color=kde_color, **kde_kws)
    File "/Users/neurolab/Library/Python/2.7/lib/python/site-packages/seaborn/distributions.py", line 691, in kdeplot
    cumulative=cumulative, **kwargs)
    File "/Users/neurolab/Library/Python/2.7/lib/python/site-packages/seaborn/distributions.py", line 294, in _univariate_kdeplot
    x, y = _scipy_univariate_kde(data, bw, gridsize, cut, clip)
    File "/Users/neurolab/Library/Python/2.7/lib/python/site-packages/seaborn/distributions.py", line 366, in _scipy_univariate_kde
    kde = stats.gaussian_kde(data, bw_method=bw)
    File "/Users/neurolab/Library/Python/2.7/lib/python/site-packages/scipy/stats/kde.py", line 208, in init
    self.set_bandwidth(bw_method=bw_method)
    File "/Users/neurolab/Library/Python/2.7/lib/python/site-packages/scipy/stats/kde.py", line 540, in set_bandwidth
    self._compute_covariance()
    File "/Users/neurolab/Library/Python/2.7/lib/python/site-packages/scipy/stats/kde.py", line 552, in _compute_covariance
    self._data_inv_cov = linalg.inv(self._data_covariance)
    File "/Users/neurolab/Library/Python/2.7/lib/python/site-packages/scipy/linalg/basic.py", line 974, in inv
    raise LinAlgError("singular matrix")
    numpy.linalg.linalg.LinAlgError: singular matrix

@maartenmennes
Copy link
Owner

something is off either with your registration parameters or with the mixture modelling thresholding. Did you run melodic with the --Oalll and the --Ostats option?

@StephDocTUM
Copy link
Author

Yes, I tried that as well. The registration looks fine of the functional bold image.

@maartenmennes
Copy link
Owner

based on this: "The MELODIC directory does not contain the required 'stats' folder. Mixture modeling on the Z-statistical maps will be run." it looks like you're not providing the melodic run with the --Oall or the --Ostats option?

@StephDocTUM
Copy link
Author

With that option it changed to:

------------------------------- RUNNING ICA-AROMA -------------------------------
--------------- 'ICA-based Automatic Removal Of Motion Artifacts' ---------------

Step 1) MELODIC

  • The existing/specified MELODIC directory will be used.
    Step 2) Automatic classification of the components
  • registering the spatial maps to MNI
  • extracting the CSF & Edge fraction features
    • The spatial map of component 1 is empty. Please check!
    • The spatial map of component 2 is empty. Please check!
    • The spatial map of component 3 is empty. Please check!
    • The spatial map of component 4 is empty. Please check!
    • The spatial map of component 5 is empty. Please check!
    • The spatial map of component 6 is empty. Please check!
    • The spatial map of component 7 is empty. Please check!
    • The spatial map of component 8 is empty. Please check!
    • The spatial map of component 9 is empty. Please check!
    • The spatial map of component 10 is empty. Please check!
    • The spatial map of component 11 is empty. Please check!
    • The spatial map of component 12 is empty. Please check!
    • The spatial map of component 13 is empty. Please check!
    • The spatial map of component 14 is empty. Please check!
    • The spatial map of component 15 is empty. Please check!
    • The spatial map of component 16 is empty. Please check!
    • The spatial map of component 17 is empty. Please check!
    • The spatial map of component 18 is empty. Please check!
    • The spatial map of component 19 is empty. Please check!
    • The spatial map of component 20 is empty. Please check!
  • extracting the Maximum RP correlation feature
  • extracting the High-frequency content feature
  • classification
    Found 1 file(s)
    Found 19 head motion-related components in a total of 20 components.
    Traceback (most recent call last):
    File "/Users/neurolab/Desktop/Stephan/Programmes/ICA-AROMA-master/ICA_AROMA.py", line 229, in
    outDir)
    File "/Users/neurolab/Desktop/Stephan/Programmes/ICA-AROMA-master/classification_plots.py", line 180, in classification_plot
    hist_kws={'alpha': 0.2})
    File "/Users/neurolab/Library/Python/2.7/lib/python/site-packages/seaborn/distributions.py", line 231, in distplot
    kdeplot(a, vertical=vertical, ax=ax, color=kde_color, **kde_kws)
    File "/Users/neurolab/Library/Python/2.7/lib/python/site-packages/seaborn/distributions.py", line 691, in kdeplot
    cumulative=cumulative, **kwargs)
    File "/Users/neurolab/Library/Python/2.7/lib/python/site-packages/seaborn/distributions.py", line 294, in _univariate_kdeplot
    x, y = _scipy_univariate_kde(data, bw, gridsize, cut, clip)
    File "/Users/neurolab/Library/Python/2.7/lib/python/site-packages/seaborn/distributions.py", line 366, in _scipy_univariate_kde
    kde = stats.gaussian_kde(data, bw_method=bw)
    File "/Users/neurolab/Library/Python/2.7/lib/python/site-packages/scipy/stats/kde.py", line 208, in init
    self.set_bandwidth(bw_method=bw_method)
    File "/Users/neurolab/Library/Python/2.7/lib/python/site-packages/scipy/stats/kde.py", line 540, in set_bandwidth
    self._compute_covariance()
    File "/Users/neurolab/Library/Python/2.7/lib/python/site-packages/scipy/stats/kde.py", line 552, in _compute_covariance
    self._data_inv_cov = linalg.inv(self._data_covariance)
    File "/Users/neurolab/Library/Python/2.7/lib/python/site-packages/scipy/linalg/basic.py", line 974, in inv
    raise LinAlgError("singular matrix")
    numpy.linalg.linalg.LinAlgError: singular matrix

@maartenmennes
Copy link
Owner

is there a way for you to send me a copy of this data? Preferably the preprocessed data with everything tarred up into a single archive.

@StephDocTUM
Copy link
Author

I could post screenshots if that would help?

@maartenmennes
Copy link
Owner

no, I would want to take a look and run a couple of things myself. If you can host on dropbox or something alike - password protected. Shoot me an email on m dot mennes at donders dot ru dot nl

@StephDocTUM
Copy link
Author

StephDocTUM commented Nov 6, 2020

did u receive it?
It works now. I masked the input data for ICA-AROMA. But one question:
ICA-AROMA often finds 90% of the ICs as head motion and regresses them out. (e.g. 18 out of 20). Aren't there too little ICs left in the of m bold signal?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants