[BUG] Clifp doesn't work with Flashpoint 13

[TheDoomCider]

When I launch clifp using flashpoint 13, It can't find the php executable. (See attached log).

Steps to reproduce the behavior:

1. Install Linux Version of Flashpoint 13
2. Download the latest version of clifp
3. Use command play -t "Ben 10 Ultimate Crisis"
4. Error: "Failed to find executable. php"

Expected behavior
Ben 10 Ultimate Crisis opens.

Log file
clifp.log

Version:

• CLIFp v0.9.10 - [CLIFp_v0.9.10_Linux_Static_x64.clang++-12.zip]
• Flashpoint 13 (Linux Version)

[oblivioncth]

Thanks for the report. It's taking me extra long to get to the compatibility update this time around, but it will come.

[TheDoomCider]

Anything I can do to help?

[oblivioncth]

Likely not really. Just needs some dev time.

I don't want to get any hopes up too much, but I'm just barely starting to poke at this. I suffered an additional setback during my hiatus in that a file server I have running that a lot of my CI/CD relies on has run into some issues that will take some time to resolve, so I'll have to work around those to get a release out. It's still going to take some time as I currently only can get to this in small pieces, but it's better than nothing :)

[TheDoomCider]

No worries, thanks for the update.

[oblivioncth]

Not doing top-to-bottom testing, but at the moment I have things working for Infinity/Ultimate on Windows as far as I can tell, now I need to tackle Linux.

[TheDoomCider]

Good to hear! Thanks for the update.

[oblivioncth]

1. Send HTTP request to FlashpointGameServer on Windows. No Problems
2. Send same exact HTTP request to FlashpointGameServer on Linux. "Illegal path access" HTTP 200 bad request
3. Send same exact HTTP request to FlashpointGameServer on Linux AFTER letting the vanilla Launcher start the game server instead of CLIFp (i.e. companion mode). HTTP 200 OK....

😠

Must have to do with something that changed between FP 12 and 13 with the exact way/environment in which FPGS is started which is causing a permission issue. I compared the launch commands and environment variables between both CLIFp and the Launcher starting FGPS though and found nothing of note AKAICT.

Time to install a Go debugger and step through the damn thing...

[oblivioncth]

Seems like someone isn't handling absolute paths correctly 😆. Probably a safety check thrown in that wasn't accounting for them.

[TheDoomCider]

Oh wow lol. Weird that this wasn't the case for the windows version...

[oblivioncth]

Although it's not explicitly stated anywhere, the source of FlashpointGameServer implies (ok, nearly explicitly states) that all paths passed via the cmdline must be relative, and in the end seemingly random/arbitrary differences between it's proxySettings.json file in the Linux and Windows packages (and presumably earlier version of the file in the FP12 Linux packages) have made it so that this shortcoming wasn't tripped up on until now.

I've been sending the paths as absolute paths, but before now the only path sent via by the command line due to the parameters in services.json was "-rootPath" which is the only parameter that CAN be an absolute path, so that didn't break anything. It seems now that the Linux packages expect "-gameRootPath" as well, which must be relative and I was sending it as absolute, which of course didn't affect the Windows build since it's configuration seems to remained how it was in FP12.

The differences are likely just due to how large of a project FP is and the haste with which it is developed, and further the fact that a lot of it is by one person.

I'll change CLIFp to pass the paths as relative but also likely make a PR for the FlashpointGameServer repo that allows absolute paths to work as well.

EDIT:
https://github.com/FlashpointProject/launcher/blob/9e768b23ed1bb025e6342863841d263e74127879/docs/services.md?plain=1#L5

Me thinks someone doesn't like to follow their own rules. On both Windows and Linux that path stated above is set as "../" in config.json, and yet unlike what that states the default launcher does not convert it into an absolute path and instead just uses it verbatim, which... funny enough causes the correct relative path to be sent for "-gameRootPath" to FPGS, but almost as a coincidence, since FPGS happens to be at the same directory depth as the Launcher (i.e. one folder down from the root of the install). so it ends up sending "..//Data/Games". That is the right relative path from the FlashpointGameServer folder, but the way it was derived was... a bit jank, lol. This is technically an uncaught bug that just happened to work out on their part, or at least, it was intentional but the related documentation, comments, and variable names weren't really updated to follow suit.

[oblivioncth]

@TheDoomCider Alright, when you get a chance, let me know if this build works for you (download the artifact of your choice at the bottom).

As an FYI, I'm unable to get the particular title you used in your report working even with the standard Launcher, so I believe that one simply isn't supported. Of course, it gets past the CLIFp specific error you noted before now.

https://github.com/oblivioncth/CLIFp/actions/runs/9815619782

[TheDoomCider]

Thanks! So, I tested it out on the steam deck (using CLIFp_v0.9.10-16-gdf98abf_Linux_Static_x64 [clang++-12]) and I noticed an issue with downloading games. If I haven't played a specific game between. I will get a download error. However, if I open the game in the official launcher and then try it again in clifp, it opens without issue. I tested this with color fill and Doeo!.

As for Ben 10 Ultimate Crisis. It does open in clifp and the official launcher but it is unplayable. This is interesting to me because I was able to get it working in Flashpoint 12 by combining the windows and linux versions (without overwriting anything in the linux version). Then downloading the windows version of clifp, opening the official linux launcher for flashpoint 12 and then launching clifp. I figured it would just work in 13 because they seemed to have added Unity Web Player Support. Before it wouldn't even load but now it at least loads, even if it is hot garbage right. I'll see about presenting my findings to the flashpoint discord server about that. Note: This workaround doesn't work in 13, most likely due to the differences you noted.

EDIT:
By accident, I was able to get the game working using my original files and having 13 open instead of 12. I'm going to try downloading the original windows files for flashpoint 12.1 and combining them with flashpoint 13 to see if that works.

UPDATE:
That didn't work and running either of the windows versions on their own in proton does nothing. I guess there is something to that combination I discovered.

[oblivioncth]

Yea it's the same thing for me, it runs but with tons of errors, CLIFp or stock launcher. That's interesting that a very particular combo can get it working, though parkerlreed was doing that for a bit before CLIFp had native Linux support. I imagine if support for the plugin on Linux has been added, they'll get it working eventually.

I noticed an issue with downloading games. If I haven't played a specific game between. I will get a download error. However, if I open the game in the official launcher and then try it again in clifp, it opens without issue. I tested this with color fill and Doeo!.

Crap, the problem that never dies. For a while I would seemingly have issues where after making certain unrelated change sto the rest of the program in question, my general purpose downloader class would regress into having this issue where the download would be stuck at 0% and timeout after 30 seconds. I tweaked the class multiple times, but it would always come back; however, I think it was due to a fundamental, albeit niche handling of the event loop I had to employee so I heavily tweaked the class again to account for what I had realized might be the issue and that seemed to have solved it once and for all as I personally haven't had issues with it since.

So, I'm hesitant to believe it's back and hope it's a different, unrelated problem, but god that would be awful. I had so thoroughly thought over the design and any potential pitfalls last time, I'm not really sure what I can do if it is that problem. What I've done for now is update the version of Qt used (on a whim to hope that affects anything) and also improved the verbosity of the error message shown when a download fails. So if you want to try the most recent artifacts, maybe we get lucky and the downloads just work, or at the very least you can get more info about the failure: https://github.com/oblivioncth/CLIFp/actions/runs/9869987929

I also hope it's not some kind of issue with the fact that Arch, and therefore SteamOS tend to be very bleeding edge in terms of libraries and that there isn't some kind of compatibility issue with the network stack between the OS and the builds of this program, like with OpenSSL.

Of course, if you feel like it, you can give the different Linux builds a try if the first one still can't download anything.

[TheDoomCider]

I had my fingers crossed but no dice. I get the same error on the version I mentioned and the others fail to even run.

I think it is as you feared. Even the old version I had in my flashpoint 12 folder doesn't work any more. Same error...

I also tried with Ethernet but the result was the same.

[TheDoomCider]

clifp.log

[oblivioncth]

I think it is as you feared. Even the old version I had in my flashpoint 12 folder doesn't work any more. Same error...

Wait, so you're saying that the build you were using for FP12, that's been sitting there untouched and was able to download games, now can't anymore? That is disconcerting that some kind of system update might have changed things.

However, are you sure you ran the most recent build I last linked? Or at least did you upload the correct log? The one you just sent shows that it's based on 'df98abf', but it should be based on c66dd4f, hence why you didn't get the error with details, and I noticed things in the log immediately that were corrected since that build :).

I'd also be curious to see any errors shown with the other builds that don't run, unless the issue is they just silently crash, because the clang-12 builds are done on Ubuntu 20.04 which is significantly older than Ubuntu 22.0, on which the clang-14 and gcc-12 builds are made, so I'd assume those two would do a better job on the Steam Deck funny enough.

I'll also try these builds out on my Arch VM and see if it's the same for me.

[TheDoomCider]

I guess that log is little polluted. Here is a log from flashpoint 12:
clifp.log

Here is a fresh one from flashpoint 13:
clifp.log

This was part of the error message:
Specific:

• [https://download.unstable.life/gib-roms/Games/5efea4aa-a316-41eb-91dc-787d556e125c-1651457111891.zip] Error: TLS initialization failed

[oblivioncth]

Ah sorry. I was viewing the logs on my phone where the text editor's scrollbar is practically invisible so I didn't realize there were multiple entries.

So my hunch was correct, this is a library compatibility issue. Given SteamOS runs on arch it's libraries are very up to date and is already using OpenSSL 3.x, which isn't compatible with applications looking for OpenSSL 1.1.x, you can see that right in the log.

I build Qt with the older OpenSSL because its still the most commonly used version outside of bleeding edge installs like Arch. I think Ubuntu 24.04 uses OpenSSL 3.0 and there is a beta image for that with Github Actions so I guess I can throw that into the CI/CD chain... too many linux builds, lol, but eventually I'll be able to drop 20.04 so for now whatever.

While get the builds going for that, if you add OpenSSL 1.1 to your system the current builds should just work:

sudo pacman -S openssl-1.1

Still curious to know what exactly happens when you try to run the gcc12/clang14 builds.

[TheDoomCider]

Here are the results:

Linux_Shared_x64 [clang++-12], Linux_Shared_x64 [clang++-14], Linux_Shared_x64 [g++-12]: (No logs generated, got this when I ran each one in the terminal)

Linux_Static_x64 [clang++-14], Linux_Static_x64 [g++-12]: (No logs generated, got this when I ran each one in the terminal)

[TheDoomCider]

Also, I tried installing Openssh 1.1 but I have been having an issue with pacman installs lately. I keep getting this error and have to trial and error different versions of packages until one works. No clue why though...

[oblivioncth]

Here are the results:

Interesting. The first issue set is due to some kind of issue with my builds I guess, that file it says is missing is one that's supposed to be part of the build automatically so I'll have to work that out. The second is the same thing that's going on with OpenSSL. The build depends on a certain version of libtiff that is older than what Arch has. I might be able to do something about but that I'm not sure. I wish Arch was just an image offered on GitHub runners, but I imagine they're hesitant since having specific images of Arch goes a bit against it's rolling release model.

For the open SSL issue, it seems like something is wrong with your keyring. You can kind of reset it by doing the following, though idk if this is more risky on SteamOS vs pure Arch:

# Ensure keyring binary is up to date
sudo pacman -S archlinux-keyring

# Remove all keys
rm -r /etc/pacman.d/gnupg

# Recreate structure
pacman-key --init

# Reacquire keys
pacman-key --populate

https://wiki.archlinux.org/title/Pacman/Package_signing#Resetting_all_the_keys

[TheDoomCider]

Thank you! That fixed the issue and after installing openssh 1.1, I can download games in clifp now. However, the next big steam os update will remove that package. It resets packages to default after an os update.

[oblivioncth]

Thank you! That fixed the issue and after installing openssh 1.1, I can download games in clifp now. However, the next big steam os update will remove that package. It resets packages to default after an os update.

I figured it was something like that, a lot of more "appliance" like operating systems handle things that way. It was more of a temporary workaround, I'm gonna spin up that CI/CD workflow that will hopefully get a build that can work out of the box on arch going.

Absolute worse case scenario I'd have to add an advisory to the repo to instruct people to make sure they have OpenSSL1.1 installed.

[oblivioncth]

@TheDoomCider Three things.

1. The reason you got the libqx-network.so error in all of the shared builds is because you have to copy the entire directory structure into a "clifp" folder within Flashpoint with those, as they depend on libraries in the "lib" folder. This is the difference between the shared and static builds. There's a good note about and the recommended build to use this that I added to the readme under "Release Builds" that you can see in the dev branch, and soon master.

2. Turns out that Ubuntu 22.04, which I already have builds produced from, uses OpenSSL 3.x by default, so I just had to solve the libtiff dependency issue and now those builds are suitable for Arch/the Steam Deck.

3. Everything on my end at this point looks good, so I'm working to formalize everything for release. Let me know if after the fact you run into any other issues.

[oblivioncth]

Fixed with #85

[oblivioncth]

Part of v0.9.11

[TheDoomCider]

Thank you!!