Non-ASCII Character Not Showing Up As Expected

[SimonMaracine]

I'm loading my own font from disk and I want to render non-ASCII characters. The code works for Linux, but not on Windows. I specifically want to render "ă", but on Windows it shows up as a question mark ("?").

The version of ImGUI is 1.84.

Here is the code that runs as part of the setup:

ImGui::CreateContext();

ImGuiIO& io = ImGui::GetIO();

// ...

ImFontGlyphRangesBuilder builder;
builder.AddRanges(io.Fonts->GetGlyphRangesDefault());
builder.AddText(u8"ă");

ImVector<ImWchar> ranges;
builder.BuildRanges(&ranges);

io.FontDefault = io.Fonts->AddFontFromFileTTF("data/fonts/OpenSans-Semibold.ttf", 20.0f,
        nullptr, ranges.Data);
io.Fonts->Build();

// ...

ImGui_ImplOpenGL3_Init("#version 430 core");
ImGui_ImplGlfw_InitForOpenGL(app->window->get_handle(), false);

Does anybody know what may be the problem?

I also tried moving ImGui_ImplOpenGL3_Init and ImGui_ImplGlfw_InitForOpenGL before building the font atlas, right after context creation, but it didn't solve anything.

[mnesarco]

Maybe not relevant, but have you tried this:

builder.AddText("\u0103");

// or this:

builder.AddChar(0x0103); 

[PathogenDavid]

but on Windows it shows up as a question mark ("?").

Make sure you're using the u8 prefix when you submit UI elements too.

For example, this code:

static void GH4561()
{
    ImGui::Begin("GH-4561");
    ImGui::Text("ăăăă");
    ImGui::Text(u8"ăăăă");
    ImGui::End();
}

results in this UI:

Assuming you're using a modern version of MSVC, you should get a warning for the first Text call, you might consider configuring that warning to be an error:

warning C4566: character represented by universal-character-name '\u0103' cannot be represented in the current code page (1252)

[SimonMaracine]

Right now I tried adding the u8 prefix when submitting the text. It changed something, but it is still not right.

This is the code for submitting the UI elements:

if (ImGui::BeginPopupModal("About Nine Morris 3D", nullptr, ImGuiWindowFlags_AlwaysAutoResize)) {
    ImGui::Text("A 3D implementation of the board game Nine Men's Morris");
    ImGui::Text("Version %d.%d.%d", VERSION_MAJOR, VERSION_MINOR, VERSION_PATCH);
    ImGui::Separator();
    ImGui::Text("All programming by:");
    // ImGui::Text(u8"Simon Teodor Mărăcine - [email protected]");
    ImGui::Text(u8"ăăă");

    if (ImGui::Button("Ok", ImVec2(430, 0))) {
        ImGui::CloseCurrentPopup();
    }

    ImGui::EndPopup();
}

And this is how it gets rendered:

[PathogenDavid]

It looks like your source file is encoded without a signature. If you interpret the UTF8 bytes for ăăă as Windows 1252 you get ăăă.

I get the same if I save my source file as UTF8 without a signature:

This happens because MSVC falls back on the system codepage when the file doesn't contain any signature.

You can either:

1. Encode your source files as UTF-8 with a BOM. (In Visual Studio go to File > Save As... and click the arrow next to the save button. In VSCode use the Change File Encoding command or click the encoding in the status bar. Other editors are similar.)
2. Pass the /utf-8 option to MSVC. (This actually fixes the non-u8 prefixed code too.)
3. Pass /source-charset:utf-8 to MSVC. (This makes sure the source is interpreted as UTF-8 but doesn't change the non-u8 prefix behavior.)

Option 1 avoids ambiguous encoding issues like this. Option 2 is "screw it it's 2021 UTF-8 for everything". Option 3 is "UTF-8 everything but I want to be pedantic about requiring the u8 prefix."

(I don't think any option is the most correct, I tend to use option 1 out of habit but option 2 is certainly attractive.)

[SimonMaracine]

Thank you! It works now. I chose option 2, because it seemed the easiest one. I'm glad that I can learn these things.