Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add caching of ranks for computing RSA with Spearman #34

Merged
merged 6 commits into from
Oct 15, 2024

Conversation

gydis
Copy link
Contributor

@gydis gydis commented Oct 9, 2024

I have added a cache_ranks flag to rsa_gen and rsa_array to enable caching of ranks for model RDMs when computing RSA with Spearman. For now, the ignore_nan=True case is ignored and an error is raised when cache_ranks is used with it.

Copy link
Owner

@wmvanvliet wmvanvliet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a really neat little speedup!

mne_rsa/rsa.py Show resolved Hide resolved
mne_rsa/rsa.py Outdated Show resolved Hide resolved
mne_rsa/rsa.py Outdated Show resolved Hide resolved
@wmvanvliet
Copy link
Owner

wmvanvliet commented Oct 9, 2024

would this also word for the tau-a metric?

@wmvanvliet wmvanvliet marked this pull request as ready for review October 15, 2024 06:52
@wmvanvliet
Copy link
Owner

To give an indication of the speedup of this PR, here is a speed test:

import mne_rsa
import numpy as np
from tqdm import trange

rng = np.random.default_rng(1)

# 10 model RDMs
model_rdms = [rng.random(1_000) for _ in range(10)]


# 100_000 data RDMs
def gen_data_rdms(n=100_000):
    """Generate the same RDM over and over again."""
    data_rdm = rng.random(1_000)
    for _ in trange(n):
        yield data_rdm


# RSA the 10 model against the 100_000 data RDMs
for _ in mne_rsa.rsa_gen(gen_data_rdms(), model_rdms):
    pass

On my analysis machine, using a single core, before this PR, this took about 13 minutes to complete. After this PR it takes a single minute.

@wmvanvliet
Copy link
Owner

thanks @gydis!

@wmvanvliet wmvanvliet merged commit ed7cfd4 into wmvanvliet:main Oct 15, 2024
2 checks passed
@gydis gydis deleted the rank_caching branch December 19, 2024 11:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants