fmeval is incompatible to properly install on (new) SageMaker Studio Python kernel

[athewsey]

The (newly launched at re:Invent) SageMaker Studio standard JupyterLab Python 3 kernel currently includes:

• amazon-sagemaker-jupyter-scheduler 3.0.4 which requires aiobotocore==2.7.*
  • That aiobotocore enforces botocore>=1.31.16,<1.31.65
  • fmeval requires sagemaker = "^2.199.0" which I think depends on boto3>=1.33.3,<2.0
• autogluon-multimodal 0.8.2 which requires transformers[sentencepiece]<4.32.0,>=4.31.0
  • fmeval pins transformers = "4.22.1"

...And so while kind of I'm able to %pip install fmeval on it, I haven't been able to find any configuration that doesn't introduce dependency conflicts somewhere.

Can we relax the constraint to allow some newer versions of transformers? Does fmeval really need that new a version of sagemaker?

[keerthanvasist]

Thanks for bringing it up. We don't need that version of Transformers, but were forced to use it because of detoxify. We will see if there are alternative approaches.

[keerthanvasist]

This particular issue has been solved in #180. Please feel free to re-open if there are any related issues.