Exposing Cf amplitude in st[:,3] #844
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hi! I've been working on some quality control metrics for KS4 and made a small change that I thought others might benefit from. I explain below:
A commonly used quality control metric for previous versions of Kilosort has been to quantify the truncation of the amplitude distribution as an estimate of the percentage of spikes missing in a given cluster (i.e. prevalence of type 2 errors). Such methods are implemented in bombcell and the IBL spike sorting pipeline. In the current version of Kilosort4, amplitudes.npy always returns an approximately gaussian distribution for amplitudes, making it impossible to estimate the degree of truncation. The amplitudes column in st[:,2] often shows a truncation, but occasionally appears to superimpose two truncated distributions with different truncation thresholds points (I believe due to normalization and merges, but I am not certain). This PR makes a small change to
template_matching.pyto append an extra column tost. This column contains the raw value ofCffor each spike, which is the value used to determine spike inclusion. This is the value that is directly compared toTh_learned, and always shows the truncation clearly. This makes the spike inclusion criteria clear to the user, and allows previously developed QC methods to be used with KS4.I'm also including a figure below to help demonstrate this point. The top row is the a cluster identified by KS2.5, while the bottom 3 rows show the same cluster (high % of coincident spikes) identified by KS4. The top row plots amplitudes.npy from KS2.5 with a clear truncation after the red line (amplitude histograms in the right column). The second row shows amplitudes.npy from KS4 with no clear truncation. The third row shows st[:,2], which matches KS2.5 amplitudes.npy more closely, but seemly has two superimposed truncated distributions. The final row shows the proposed st[:,3] with a clearly truncated distribution that matches (roughly) the KS2.5 distribution. The truncation is also at the
Th_learned = 8value, which makes it clear to the user that this is the amplitude value used for spike inclusion.Thanks for your time!
Ryan Ressmeyer