Gemma capping

[ArthurZucker]

What does this PR do?

Adds capping for gemma2, fixes #32877

[Cyrilvallez]

There are a lot of edge cases in imports which are very hard to deal with with the proposed approach. I think a simpler and more general approach is to do it the other way around:

• dump all imports from the modular_xxx.py as is
• dump all imports from the dependency files as is (this is currently the case)
• Then, in the PostModularConverterCleaner, clean the imports (may even only clean the protected imports, and let ruff remove the other unused, non-protected imports)

This approach is much easier and versatile because in the Cleaner, we have access to the final source code, which is not the case when visiting the modular_xxx.py file (we only see the modular + the dependencies, and it is hard to check imports relative to only the part of the dependency files that we copy in the final file). Thus, it would ensure that all needed imports are present (i.e. we will never reach a weird edge-case when trying to match the imports as we do currently), and we can correctly remove imports that were wrongly added from the dependency files (i.e. see duplicate import in Glm due to Phi3 dependency).
This would greatly simplify the code complexity as well in my opinion.

[vasqu]

Isn't it a bit misleading to use flex attn when we have attn_implementation="sdpa"? My concerns would be

• People that previously used sdpa (forced or not) will suddenly have different torch requirements
• Sdpa != Flexattn imo, it's a different API, name, and potentially slightly different behaviour
• Are the slow tests still passing? We should ensure that it's still behaving the same ish in comparison to eager

Wdyt about making another attn implementation option for flex attn specifically? Not sure if this goes over the goal but control over the specific implementation is always appreciated.

Overall excited to see this, great work!

[ArthurZucker]

SDPA version of gemma never "worked" TBH!
I'll probably add a new class for flex attention, this was simpler for testing

[ArthurZucker]

Okay @Cyrilvallez good point regarding cleaning! Makes more sense indeed, will update to fix 😉

[Cyrilvallez]

Very nice approach! Much simpler IMO 🤗 just added some nits for clarity

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[Cyrilvallez]

LGTM, I actually love it, I think it's much better to use different attention functions instead of different attention classes (clearer, less duplicated code, and we can easily switch between implementations even after the model has been instantiated)