How to draw certain parts of image?

[tmssngr]

We have created a lot of images in our applications that contains of rows and columns of subimages. Depending on some state we want to draw the right subimage. Code to reproduce:

import org.eclipse.swt.SWT;
import org.eclipse.swt.graphics.Image;
import org.eclipse.swt.graphics.ImageData;
import org.eclipse.swt.graphics.ImageDataProvider;
import org.eclipse.swt.widgets.Display;
import org.eclipse.swt.widgets.Monitor;
import org.eclipse.swt.widgets.Shell;

import java.io.FileInputStream;
import java.io.IOException;

public class LinuxImageSizeProblem {

    public static void main(String[] args) throws IOException {
        Display display = new Display();
        for (Monitor monitor : display.getMonitors()) {
            System.out.println(monitor.getBounds());
        }

        final ImageData id100;
        final ImageData id200;
        try (FileInputStream stream = new FileInputStream("files.png")) {
             id100 = new ImageData(stream);
        }
        System.out.println(id100.width + "x" + id100.height);
        try (FileInputStream stream = new FileInputStream("files-2.png")) {
             id200 = new ImageData(stream);
        }
        System.out.println(id200.width + "x" + id200.height);

        final Image image = new Image(display, new ImageDataProvider() {
            @Override
            public ImageData getImageData(int zoom) {
                if (zoom < 150) {
                    return id100;
                }
                return id200;
            }
        });
        System.out.println(image.getBounds());// expected: 352x192

        Shell shell = new Shell(display);
        shell.addListener(SWT.Paint, event -> {
            final int size = 16;
            event.gc.drawImage(image, 0, 0, size, size, 10, 10, size, size);
            event.gc.drawImage(image, 21 * size, 0, size, size, 100, 10, size, size);
            event.gc.drawImage(image, 0, size, size, size, 10, 10 + size, size, size);
            event.gc.drawImage(image, 21 * size, size, size, size, 100, 10 + size, size, size);
        });
        shell.setSize(400, 300);
        shell.open();

        while (!shell.isDisposed()) {
            if (!display.readAndDispatch()) {
                display.sleep();
            }
        }

        display.dispose();
    }
}

Images: files.zip
When running on Linux (Linux Mint, Gnome 48, 2 monitors: 4k and FullHD) with 200% zoom and 100% zoom I get following results:

Rectangle {0, 0, 1920, 1080}
Rectangle {111, 1080, 1920, 1080}
352x192
704x384
Rectangle {0, 0, 352, 192}

When I change the screen zooming to 225% and 125% and launch again, I'm getting:

Rectangle {0, 0, 1744, 981}
Rectangle {111, 981, 1536, 864}
352x192
704x384
Rectangle {0, 0, 235, 128}

In both cases the window was shown on the 4k monitor and I did not move the window.

How to correctly draw pixel-exactly defined parts of the original image to the screen? Note, that the the *-2.png is just a high-resolution variant of the other.

[laeubi]

I'm not 100% sure if it is applicable to you use-case but if you want a certain "pixel perfect" part of your image, why don't you extract the required part at the ImageDataProvider already?
Currently you provide two different sized pixel source but always using the same dimension (16) to draw only a part. So either you need to use the actual image size and e.g. divide it by some factor (e.g. if your tiles are always 8x8 items you can divide height/width by 8).

[HeikoKlare]

Just to be sure that we do not miss anything right before the release: with "become useless/unreliable", do you mean this is a recent regression?

[HeikoKlare]

That should not be necessary as all passed sizes are in logical points. The drawImage calls basically means that a region of logical width and height size is taken from the image and printed into an area of logical width and height size to the target. Both these logical sizes should then be transformed to the according pixel coordinate system of the image/GC taking into account the respective zoom.

The printed values show that Cairo's fractional scaling mechanism is applied to the image as well. That scaling fakes all positions/sizes with a applied fictive scaling zoom so that you can render as if you were at 100%/200% zoom while actually filling an appropriate area of, e.g., 225% zoom (that's the reason for, e.g., the "strange" monitor bounds on 225%). It is somehow reasonable that this is also applied to the used images to make them conforming in size, but any environment-independent assumption that you make on the logical size of the image (which usually conforms to the 100% pixel size) becomes wrong with that. That's also the reason why just taking a the size of 16 (the pixel size at 100% of the image extracts) is wrong.
This whole behavior appears a bit unexpected to me, but might actually be the consistent implementation on Cairo (though I did not test and further investigate yet).

Currently you provide two different sized pixel source but always using the same dimension (16) to draw only a part. So either you need to use the actual image size and e.g. divide it by some factor (e.g. if your tiles are always 8x8 items you can divide height/width by 8).

That would probably be the proper way to deal with this, though I am surprised this is necessary as I have never expected the SWT API to behave like that (but Cairo fractional scaling seems to make it necessary).

[laeubi]

I just wanted to mention that for me, the fractional scaling is marked with a hint that it is experimental and even does not work well for all native applications e.g. the nautilus explorer or gedit.

And as it seem to work on the 100% and 200% standard sizes it might be simply a bug. What you can also try is to just set a clipping area or generally draw a box of 16x16 px so one can better see here the actual over painting happens.

[tmssngr]

When debugging I noticed that the 225% zoom was internally treated as 300% by SWT.

[HeikoKlare]

When debugging I noticed that the 225% zoom was internally treated as 300% by SWT.

That's not true. Cairo uses 300% and, as explained above, fakes different sizes of controls, images and the like so that it can internally scale a 300% rendering to 225% having the requested sizes. That's how fractional scaling works with Cairo. SWT cannot do anything about and the good thing is that SWT autoscaling capabilities should actually work quite well with that way of autoscaling.
Just note that on fractional scales on Linux you will always have blurry results to a certain degree due to the mentioned internal scaling of Cairo. That's an essential difference to the fractional scaling on Windows where the UI is rendered at the fractional scale and not at a multiple of 100 that is then scaled down to the desired zoom (in a blurry way).

[HeikoKlare]

Turns out that there is a bug in the snippet which causes the problem. The implementation of the ImageDataProvider violates its contract. It delivers, e.g., a 200% image when requesting for a zoom of 175%. A contract-conforming implementation would be like this:

final Image image = new Image(display, new ImageDataProvider() {
@Override
public ImageData getImageData(int zoom) {
    if (zoom == 100) {
        return id100;
    } else if (zoom == 200) {
        return id200;
    }
    return null;
}

Then at 225% you get the expected result:

And please note that there are build-in ways to load different images sources at 100% and 200%, so that you do not need to load different image data on your own and wrap them into a data provider.

[tmssngr]

Thanks, a

if (zoom > 200) {
  return null;
}

fixes the problem for me. As we do not want autoscaled images between >100% and <200% zoom, is returning either the 100%-zoom variant or the 200% one correct?

[HeikoKlare]

If you want non-autoscaled images between 100% and 200%, you have to provide specific images for those zoom levels. Simply returning the 100% version at 125% violates the contract and results in the same behavior as you have shown in the initial screenshot.
Note that you wont experience issues on Linux and MacOS as they do fractional scaling differently. The images are not requested on fractional scaled, but on Linux the images will be autoscaled anyway when using fractional scaling, there is no way around it.
On Windows, you will experience issues with the images as they are actually retrieved in an autoscaled version for fractional scaled (but autoscaled by SWT instead of Cairo as done on Linux).

[tmssngr]

New snippet:

import java.io.*;
import java.nio.file.*;
import java.nio.file.Path;

import org.eclipse.swt.*;
import org.eclipse.swt.graphics.*;
import org.eclipse.swt.widgets.*;

public class AutoscaleImageTest {

	public static void main(String[] args) throws IOException {
		final Display display = new Display();

		final ImageData imageData = getImageDataNotNull("image.png");
		System.out.println(imageData.width + "x" + imageData.height);
		final ImageData imageData2x = getImageDataNotNull("image-2.png");
		final Image image = new Image(display, (ImageDataProvider)zoom -> {
			if (zoom == 100) {
				return imageData;
			}
			if (zoom == 200) {
				return imageData2x;
			}
			return null;
		});

		final Shell shell = new Shell(display);
		shell.addListener(SWT.Paint, event -> {
			event.gc.drawImage(image,
			                   0, 0, 24, 24,
			                   10, 10, 24, 24);
			event.gc.drawImage(image, 100, 10);
		});

		shell.setSize(200, 100);
		shell.open();

		while (!shell.isDisposed()) {
			if (!display.readAndDispatch()) {
				display.sleep();
			}
		}

		image.dispose();

		display.dispose();
	}

	private static ImageData getImageDataNotNull(String path) throws IOException {
		try (InputStream inputStream = Files.newInputStream(Path.of(path))) {
			return new ImageData(inputStream);
		}
	}
}

images.zip
It looks like the 9-argument drawImage() variant produces much uglier results than drawing it directly (at 175% zoom):

How can I tell the 9-argument drawImage() to produce a result as good as the 3-argument drawImage()?

[HeikoKlare]

You don't have anti-aliasing enabled for the GC, which is why the 9-argument method renders the image without anti-aliasing. You can fix that with event.gc.setAntialias(SWT.ON);.

In contrast, the 3-argument method always creates an image handle at the appropriate zoom (using smooth scaling == anti-aliasing), which is not possible for the other methods that allow scaling of the image, as you would otherwise create handles for pre-rendered images at arbitrary sizes. That would introduce a severe risk of performance issues.