Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add --use_kv_cache to image-to-text pipeline #1292

Merged

Conversation

KimBioInfoStudio
Copy link
Contributor

@KimBioInfoStudio KimBioInfoStudio commented Aug 23, 2024

What does this PR do?

  • add --use_kv_cahe to image-to-text pipeline
  • add two new models to supported list
  • replace hard code prompt with transformers.prcessor

Fixes # (issue)

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests?

@KimBioInfoStudio
Copy link
Contributor Author

llava-hf/llama3-llava-next-8b-hf
image

@KimBioInfoStudio
Copy link
Contributor Author

llava-hf/llava-v1.6-34b-hf
image

@KimBioInfoStudio
Copy link
Contributor Author

@yafshar should me split this pr into 3 small one to speed up the merge process?

  1. kv-cache
  2. Processor
  3. Profiling

@yafshar
Copy link
Contributor

yafshar commented Sep 5, 2024

@yafshar should me split this pr into 3 small one to speed up the merge process?

1. kv-cache

2. Processor

3. Profiling

No need to do that. If you need time to workout profiling, then remove it for now. The rest are working nicely. My comments above were just suggestions

Copy link
Contributor

@yafshar yafshar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@regisss please check this PR.

@libinta libinta added the run-test Run CI for PRs from external contributors label Sep 11, 2024
Copy link

The code quality check failed, please run make style.

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@KimBioInfoStudio
Copy link
Contributor Author

@yafshar @libinta the failed cases not related on this pr

@regisss regisss merged commit 520c875 into huggingface:main Sep 14, 2024
3 of 4 checks passed
skaulintel added a commit that referenced this pull request Sep 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
run-test Run CI for PRs from external contributors
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants