Issue in finding the correct rotation for rotated image

[mkustererenbw]

Hello together,

I am currently investigating an issue in my application where I the extracted image from a document is rotated.

In short, the problem is that I extract a rotation of 90° while it should be -90° (more information following).

Following these discussions here

#1297
#385
#1896

and the docs (https://pymupdf.readthedocs.io/en/latest/app3.html#imagetransformation)

I tried the following:

1. Extract the image and its transformation matrix via page.get_text("dict") as bytes
2. Use the transformation matrix in order to retrieve the applied rotation

    def _get_image_rotation(matrix: fitz.Matrix) -> int | None:
        if min(matrix.a, matrix.d) > 0 and matrix.b == matrix.c == 0:
            rot = 0
        elif matrix.a == matrix.d == 0:
            if matrix.b > 0 and matrix.c < 0:
                rot = 90
            elif matrix.b < 0 and matrix.c > 0:
                rot = -90
            else:
                rot = None
        elif min(matrix.a, matrix.d) < 0 and matrix.b == matrix.c == 0:
            rot = 180
        else:
            rot = None
        return rot

I used the rot-detector-script from here for this: #385
3. Use the rotation information in order to transform the image:

    def _rotate_image(img_bytes: bytes, transform_matrix: fitz.Matrix) -> bytes:
        my_img = Image.open(io.BytesIO(img_bytes))
        matrix = fitz.Matrix(list(np.array(transform_matrix).round(4)))  # I am rounding since Matrix.a == -7*10e-06
        rotation = PDFProcessor._get_image_rotation(matrix)
        if rotation is not None:
            img_byte_arr = io.BytesIO()
            my_img.rotate(rotation, expand=True).save(img_byte_arr, format=my_img.format)
            new_img_bytes = img_byte_arr.getvalue()
        else:
            new_img_bytes = img_bytes
        return new_img_bytes

After that, the image gets rotated, however, the rotation should be -90° while the extracted rotation is 90°.

Since it might be interesting, the values for transform and shrink and shrink*transform

width = block["width"]
height = block["height"]
transform = block["transform"]
shrink = pymupdf.Matrix(1 / width, 0, 0, 1 / height, 0, 0)

shrink
Matrix(0.001034126163391934, 0.0, 0.0, 0.001379310344827586, 0.0, 0.0)

transform
Matrix(-7.6289998105494305e-06, 315.95001220703125, -237.08999633789062, -0.0, 311.64398193359375, 151.53997802734375)

shrink * transform
Matrix(-7.889347841683048e-09, 0.32673215866088867, -0.32702070474624634, 0.0, 311.64398193359375, 151.53997802734375)

Here are the images for better reference

Desired (-90°)

How it's extracted with my code (90°)

The original image from bytes

The screenshot in the pdf (-90°)

Am I missing something? I do not need to take the inverse, right? Thank you in advance.

[mkustererenbw]

I will try to bump it one time as I have still failed to solve this. Maybe someone has an idea or can hint to what I'm missing. Thanks :)

[JorjMcKie]

You're only sharing pictures here - nothing we could look up in the PDF or image itself. So we have to take a shot in the dark:
Depending on the image, there may exist EXIF information.
Also consider that MuPDF's coordinate system counts the y-direction from top to bottom.