Skip to content

Commit

Permalink
Update tests/models/idefics3/test_processing_idefics3.py
Browse files Browse the repository at this point in the history
Co-authored-by: amyeroberts <[email protected]>
  • Loading branch information
andimarafioti and amyeroberts committed Sep 18, 2024
1 parent d707652 commit 7ee2ec8
Showing 1 changed file with 2 additions and 3 deletions.
5 changes: 2 additions & 3 deletions tests/models/idefics3/test_processing_idefics3.py
Original file line number Diff line number Diff line change
Expand Up @@ -92,9 +92,8 @@ def get_splitted_image_expected_tokens(self, processor, image_rows, image_cols):
)
text_split_images += processor.tokenizer("\n", add_special_tokens=False)["input_ids"]
text_split_images = text_split_images[:-1] # remove last newline
text_split_images += processor.tokenizer("\n\n", add_special_tokens=False)[
"input_ids"
] # add double newline, as it gets its own token
# add double newline, as it gets its own token
text_split_images += processor.tokenizer("\n\n", add_special_tokens=False)["input_ids"]
text_split_images += (
[self.fake_image_token_id]
+ self.global_img_tokens_id
Expand Down

0 comments on commit 7ee2ec8

Please sign in to comment.