Some effort to compile on windows #7
Conversation
There was a problem hiding this comment.
This is pretty cool, but I have to wonder why you're even working on this :) STFL is an extremely niche library. To my knowledge (backed by apt-cache rdepends libstfl0), it's only used by Newsboat, but we plan to move away from it (newsboat/newsboat#232).
I'll merge this once the iconv thing is fixed, and I'm not dissuading you from further working on this; just want to know your motivation.
|
Also, what Perl issues did you run into? Perhaps I'd be able to help. |
I guess mainly telling windows perl that using bash.exe is okay I assume the file that it wants it's 'C:\Users\kreij\scoop\apps\msys2\2022-01-28\mingw64\lib\perl5\core_perl\Config.pm' but because of path conversion shenanigans it's hard to see :(, as for the reason for doing this, it's because it was a dependency of newsboat, and wanted to compile it for windows, and yeah, I'm excited about the new tool/library that is going to replace sftl |
Wow! Okay, let's work on this then :) The output here on Debian stable looks somewhat differently: I can run the Perl script manually and examine the Makefile: In it, I see this line: Something's up with and finally:
This is on Linux. On Windows, you might be running into ramifications of this one: https://stackoverflow.com/a/9455881/2350060 You're getting a different error, it looks like for you However, if we take a step back and re-examine the situation, then maybe you did enough already. Newsboat doesn't use Perl bindings to STFL. All it needs is libstf.so library and stfl.h header. You got that building already, so mission accomplished I guess? (Edited once I realized that of course for me the separator is a slash -- I'm on Linux!) |
|
my DIRFILESEP is |
| VERSION := 0.24 | ||
| ifeq ($(detected_OS),Windows) | ||
| SHARED_LIB_NAME := libstfl.$(VERSION).$(SHARED_LIB_EXT) |
There was a problem hiding this comment.
SHARED_LIB_EXT is only used here. How about hard-coding it to dll here and removing the variable?
| install -m 644 libstfl.$(VERSION).dll $(DESTDIR)$(prefix)/$(libdir) | ||
| cp $(DESTDIR)$(prefix)/$(libdir)/libstfl.$(VERSION).dll $(DESTDIR)$(prefix)/$(libdir)/libstfl.dll | ||
| else | ||
| install -m 644 libstfl.so.$(VERSION) $(DESTDIR)$(prefix)/$(libdir) | ||
| ln -fs libstfl.so.$(VERSION) $(DESTDIR)$(prefix)/$(libdir)/libstfl.so | ||
|
|
||
| endif |
There was a problem hiding this comment.
I think it'd be a good idea to use SHARED_LIB_NAME here, and also check if there's other places where it can be used. That way, we confine the knowledge to a single variable, and it'd be harder to accidentally break the build in the future by hard-coding a specific name.
| wchar_t codePageStr[10]; | ||
| swprintf(codePageStr, 10, L"%u", codePage); | ||
| ipool = stfl_ipool_create(codePageStr); |
There was a problem hiding this comment.
After looking at this again, I wonder how it even compiles. Here, we're passing wchar_t* into stfl_ipool_create() which expects char* — why isn't it a compilation error to do so?
Assuming wchar_t* simply gets casted to char* somehow, this code is broken: wchar_t for a digit would contain zeroes (digits are part of ASCII, many encodings are extensions of ASCII, including all of Unicode encodings). Thus stfl_ipool_create() will only use the first digit of the codepage, if even that (zeroes might be at the very start of the string).
Here's a test program to verify that a string can be converted from UTF-8 to WCHAR_T and then to UTF-32:
#include <iconv.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <wchar.h>
// "Hello" in Ukrainian, in UTF-8
char INPUT[] = {
0xd0, 0x9f, 0xd1, 0x80, 0xd0, 0xb8, 0xd0, 0xb2, 0xd1, 0x96, 0xd1, 0x82, 0x0 };
// Same string in UTF-32
char EXPECTED_UTF32_OUTPUT[] = {
0xff, 0xfe, 0x00, 0x00, 0x1f, 0x04, 0x00, 0x00, 0x40, 0x04, 0x00, 0x00, 0x38, 0x04,
0x00, 0x00, 0x32, 0x04, 0x00, 0x00, 0x56, 0x04, 0x00, 0x00, 0x42, 0x04, 0x00, 0x00
};
int main() {
const char input_code[] = "UTF-8";
iconv_t conv = iconv_open("WCHAR_T", input_code);
if(conv == (iconv_t)-1) {
perror("opening iconv for conversion from UTF-8 to WCHAR_T failed");
exit(1);
}
char* input = INPUT;
char* input_ptr = input;
size_t inputbytesleft = strlen(input_ptr);
const size_t OUT_BUFFER_SIZE = 4096; // should be enough for any encoding under the Sun
char* wc_string = malloc(OUT_BUFFER_SIZE);
if(wc_string == NULL) {
perror("allocating a buffer for the wide string failed");
exit(1);
}
char* output_ptr = wc_string;
size_t outbytesleft = OUT_BUFFER_SIZE;
size_t chars_converted =
iconv(
conv,
&input_ptr,
&inputbytesleft,
&output_ptr,
&outbytesleft);
if(chars_converted == (size_t)-1) {
free(wc_string);
perror("conversion from UTF-8 to WCHAR_T failed");
exit(1);
}
if(wcslen((wchar_t*)wc_string) == 0) {
free(wc_string);
printf("Expected WCHAR_T string to contain some bytes, but it is empty\n");
exit(1);
}
if(iconv_close(conv) == -1) {
free(wc_string);
perror("closing iconv for conversion from UTF-8 to WCHAR_T failed");
exit(1);
}
const char output_code[] = "UTF-32";
conv = iconv_open(output_code, "WCHAR_T");
if(conv == (iconv_t)-1) {
free(wc_string);
perror("opening iconv for conversion from WCHAR_T to UTF-32 failed");
exit(1);
}
input_ptr = wc_string;
inputbytesleft = OUT_BUFFER_SIZE - outbytesleft;
char* utf32_string = malloc(OUT_BUFFER_SIZE);
if(utf32_string == NULL) {
free(wc_string);
perror("allocating a buffer for the UTF-32 string failed");
exit(1);
}
output_ptr = utf32_string;
outbytesleft = OUT_BUFFER_SIZE;
chars_converted =
iconv(
conv,
&input_ptr,
&inputbytesleft,
&output_ptr,
&outbytesleft);
if(chars_converted == (size_t)-1) {
free(utf32_string);
free(wc_string);
perror("conversion from WCHAR_T to UTF-32 failed");
exit(1);
}
if(iconv_close(conv) == -1) {
free(utf32_string);
free(wc_string);
perror("closing iconv for conversion from WCHAR_T to UTF-32 failed");
exit(1);
}
const size_t utf32_string_len = OUT_BUFFER_SIZE - outbytesleft;
const size_t EXPECTED_UTF32_STRING_LEN = 28;
if(utf32_string_len != EXPECTED_UTF32_STRING_LEN) {
printf("Expected UTF-32 string to contain %lu bytes, but it contains %lu\n",
EXPECTED_UTF32_STRING_LEN,
utf32_string_len);
free(utf32_string);
free(wc_string);
exit(1);
}
for(size_t i = 0; i < EXPECTED_UTF32_STRING_LEN; ++i) {
if(utf32_string[i] != EXPECTED_UTF32_OUTPUT[i]) {
printf("UTF-32 output differs from expected at position %lu: expected %u, found %u\n",
i,
EXPECTED_UTF32_OUTPUT[i],
utf32_string[i]);
}
}
free(utf32_string);
free(wc_string);
}Can you please modify it to run on Windows? In the process you'll figure out all the kinks of how to initialize iconv, and then you can just copy that code here.
This is some effort, complete compilation fails due to perl shenanigans and because I have no idea how perl works, but at least the example.c is compiled.