-
-
Notifications
You must be signed in to change notification settings - Fork 36
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added AMX acceleration #39
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is really cool thanks for the contribution! Do you have any sense of the speedup gain from these changes?
Sources/SimilaritySearchKit/Core/Embeddings/Metrics/DistanceMetrics.swift
Outdated
Show resolved
Hide resolved
Sources/SimilaritySearchKit/Core/Embeddings/Metrics/DistanceMetrics.swift
Outdated
Show resolved
Hide resolved
Sources/SimilaritySearchKit/Core/Embeddings/Metrics/DistanceMetrics.swift
Outdated
Show resolved
Hide resolved
I was searching through a substantial index, and the time required went from 0.35 seconds to 0.05 seconds; so a 7x increase in speed. Note that since the new code uses hardware acceleration, the speed would differ on machines. For context, the 7x increase in speed was recorded on my M2 Max MacBook, which has 2 AMX acceleration units. An A17 Pro would have only 1 of these AMX acceleration units, so expect only a ~3-4x increase in speed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good to merge! This should be a nice speedup 🙏
Hashable
andIdentifiable
conformance toSimilarityIndex