Problem 2: Bot controls mouse and keyboard. It's inconvenient

[9-9-9-9]

This bot currently using java.awt.Robot class to send mouse click and press keyboard and it's inconvenient.

I would like to migrate to use PostMessage or similar APIs so users can use their device while doing bot.

If anyone have interested in developing this bot and can help me to resolve/implement this feature, please go on.
Targets: Windows (7+) & Linux (or at least Ubuntu)

[9-9-9-9]

Bad news:
I already tested to send PostMessage to Steam client (BitHeroes.exe) on Windows but it doesn't work while the window is inactive. Only effective when window has been set to foreground window

[ilpersi]

What you can do on Windows is to use native User32 SendMessage API via JNA.

Of course this would be windows specific and on Linux you should probably use something else. This will work even if window is running in the background. The only caveat is that the windows you want to send messages to is not minimized.

I used to do this in a proof of concept for my old BHBot and you can get a sample of the code here (clickInGame method).

[9-9-9-9]

What you can do on Windows is to use native User32 SendMessage API via JNA.

Of course this would be windows specific and on Linux you should probably use something else. This will work even if window is running in the background. The only caveat is that the windows you want to send messages to is not minimized.

I used to do this in a proof of concept for my old BHBot and you can get a sample of the code here (clickInGame method).

I have tried it and wrote before your comment

Bad news:
I already tested to send PostMessage to Steam client (BitHeroes.exe) on Windows but it doesn't work while the window is inactive. Only effective when window has been set to foreground window

As long as BH Client need to be foreground window, all are useless. What I want to achieve is user can interact with their PC while bot still running

[daredoes]

I'm thinking of trying to put together an MVP that uses Selenium Mouse/Keyboard Actions to do this:

https://www.selenium.dev/documentation/webdriver/actions_api/mouse/#alternate-button-clicks

[daredoes]

Nope. Taking screenshots with Selenium, or the web browser, seems to require focus. I can get mouse clicks to work without focus, but that's not enough

[jimmydiesel]

If you have Windows Pro, you can run RDP and have it logged in on the VM