You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'm currently exploring this library as a first level filter for external model repository, and found that one of the core functionality hasnt been fully supported: the need to scan models saved using huggingface's transformer library.
As far as I understand and tested, models save as .h5 using huggingface's TF library have a different way of storing model configuration compared to tensorflow keras, and thus are not able to be loaded using keras' models.load_model() directly.
Example of how .h5 models saved using transformers:
from transformers import TFAutoModelForImageClassification
tf_model = TFAutoModelForImageClassification.from_pretrained("my_model", from_pt=True)
tf_model.save_pretrained("my_model_tf")
I also noticed that prior to v4.0.0, there were initial support explicitly for huggingface, but opted to move towards a extendable, modular codebase. However, as part of the security evaluation an external open source solution, we want to evaluate if modelscan has intention to support capability to scan most huggingface models over in the long term, and can be adopted.
I'd like to enquire and understand further the team's position on this, and if this is something I can suggest for?
Thanks for your time !
The text was updated successfully, but these errors were encountered:
I'm currently exploring this library as a first level filter for external model repository, and found that one of the core functionality hasnt been fully supported: the need to scan models saved using huggingface's transformer library.
As far as I understand and tested, models save as
.h5
using huggingface's TF library have a different way of storing model configuration compared to tensorflow keras, and thus are not able to be loaded using keras'models.load_model()
directly.Example of how .h5 models saved using transformers:
I also noticed that prior to v4.0.0, there were initial support explicitly for huggingface, but opted to move towards a extendable, modular codebase. However, as part of the security evaluation an external open source solution, we want to evaluate if
modelscan
has intention to support capability to scan most huggingface models over in the long term, and can be adopted.I'd like to enquire and understand further the team's position on this, and if this is something I can suggest for?
Thanks for your time !
The text was updated successfully, but these errors were encountered: