Cosmic Text for font rendering

[ElhamAryanpur]

As discussed in #1016 , I have done some testing on cosmic-text.

First of all, they have some docs that I've been exploring. And I was testing their example. They do have no_std and wasm support too according to the Cargo.toml they have.

I've noticed that the way it renders, is by drawing rectangles. Which is something I've seen for the first time to be honest. You can check it here. It gives a x,y,width,height, and color which is usually just color predefined and alpha channel being different for aliasing and stuff.

There is also another method if rectangles aren't possible: Swash Image which basically returns an image bytes to be rendered instead of individually creating rectangles. It requires some things I couldn't implement myself to be honest.

I was able to sort of hack the rectangle method in my engine

It had... not good results to be honest

Although maybe that's on me for having some issues with the engine as their examples do work and work very well. But yeah, that's my findings so far on cosmic text.

[thomaskrause]

I also experimented with cosmic text but went with hacking some completely inefficient and not really generic code in the tessellate_text function of the epaint Tessellator.

pub fn tessellate_text(&mut self, text_shape: &TextShape, out: &mut Mesh) {
        let TextShape {
            pos: galley_pos,
            galley,
            underline,
            override_text_color,
            angle,
        } = text_shape;

        if galley.is_empty() {
            return;
        }

        if galley.pixels_per_point != self.pixels_per_point {
            eprintln!("epaint: WARNING: pixels_per_point (dpi scale) have changed between text layout and tessellation. \
                       You must recreate your text shapes if pixels_per_point changes.");
        }

        for row in &galley.rows {
            let metrics =
                cosmic_text::Metrics::new(galley.job.sections[0].format.font_id.size, row.height());
            let mut buffer = cosmic_text::Buffer::new(&mut self.font_system, metrics);
            let mut buffer = buffer.borrow_with(&mut self.font_system);
            buffer.set_size(row.rect.width(), row.rect.height());

            let attrs = cosmic_text::Attrs::new();

            buffer.set_text(&row.text(), attrs, cosmic_text::Shaping::Advanced);
            buffer.shape_until_scroll();
            let text_color = override_text_color
                .map(|c| cosmic_text::Color::rgba(c.r(), c.g(), c.b(), c.a()))
                .unwrap_or_else(|| cosmic_text::Color::rgb(0xFF, 0xFF, 0xFF));

            buffer.draw(&mut self.swash_cache, text_color, |x, y, w, h, color| {
                let min_pos = *galley_pos + row.rect.min.to_vec2() + vec2(x as f32, y as f32);
                let size = vec2(w as f32, h as f32);
                let output_rect = Rect::from_min_size(min_pos, size);
                let ecolor =
                    Color32::from_rgba_unmultiplied(color.r(), color.g(), color.b(), color.a());
                out.add_colored_rect(output_rect, ecolor);
            });
        }
    }

Using cosmic text would mean to replace the whole pipeline of text layout and not just rendering, which seems to be quite a change, even possible in the public facing API. But it is definitely possible.

[ElhamAryanpur]

Interesting work! Can you type in some RTL text? e.g. (سلامم)

And yes I agree, it'll be a big change. User facing API's could stay the same still in my opinion, but the pipeline will need some change.

[thomaskrause]

Since the text layout part is still original, the example RTL text is placed weirdly and the font rendering is still quite blurry.

[emilk]

Very cool!

It seems like if we switch to cosmic text we need to switch from using a glyph atlas to a string atlas.

egui currently uses a glyph atlas, which is a big image/texture where each glyph (character) is rendered once and then stored so it can be referenced at rendering. Text layout is about generating UV-rects that point into the individual characters in the glyph atlas.
The glyph-atlas in egui can be found in Backend -> Inspection -> Font texture of egui.rs:

Cosmic Text seems to work by rendering a full string of text as a bitmap (the rectangles in buffer.draw are patches of pixels). This improves kerning and anti-aliasing of the text, but is more complex. We need to cache the results of the rendering. The easiest way would be to cache each text string in its own bitmap and stream that to the GPU each frame. That would mean a lot of texture switching in he render backend, which I believe would be quite slow for something like the WebGL2 backend.

15 years or go so I worked on a text renderer for Phun/Algodoo which used a string atlas for this: I combined all the small string-textures into one a few big texture atlases. Back then I did a lot of book-keeping in order to only update the atlas when a new piece of text was added and removed. There is a lot of complexity here, like atlas fragmentation, and handling of text too large to fit in an atlas. Perhaps these days GPU streaming is fast enough that we can construct the atlases once at the start of each frame and stream it to the GPU every frame. That would remove all the book-keeping from the problem.

[ElhamAryanpur]

I do not know font rendering, but can't there be a GPU cache of the bitmaps and be reused without having to create a new texture? that should be fast enough. although problem would be the issue in different sizes, could quickly grow in memory size.. hmm

I'd say GPUs nowadays are fast enough, and given WebGPU is right around the corner, this could be possible. I wonder how did the ICED achieved it.

[ElhamAryanpur]

Since the text layout part is still original, the example RTL text is placed weirdly and the font rendering is still quite blurry.

Oh interesting! ICED had similar issue, they have opened an issue on cosmic-text to define bounding sizes for fonts to solve it. I'm not sure how it works tho-

[emilk]

Oh interesting! ICED had similar issue, they have opened an issue on cosmic-text to define bounding sizes for fonts to solve it. I'm not sure how it works tho-

Do you have a link for that issue?

[ElhamAryanpur]

yes of course! pop-os/cosmic-text#70

[ElhamAryanpur]

related issue that explains further: iced-rs/iced#1877

[dignifiedquire]

maybe this helps as direction for how to solve the caching? pop-os/cosmic-text#26 (comment)

[mikeandmore]

I'm digging (trying...) into this. It looks like we need to draw textures and not polygons... buffer.draw() is terribly slow. It renders pixel by pixel...

[mikeandmore]

Here are some notes I have while reading the code.

1. layout_section() in text_layout.rs calls FontImpl::font_impl_and_glyph_info to look for glyph info for each (unicode?) character.
2. If the glyph isn't found, it'll render via FontImpl::allocate_glyph(). This function first allocates the texture with TextureAtlas::allocate(), and it draws on the FontImage. Interestingly, glyph.draw() is a similar API to the buffer.draw() in cosmic_text, which renders the glyph pixel-by-pixel.
3. TextureAtlas::allocate() returns a position in the FontImage. If not enough, resize the FontImage.
4. FontImpl keeps a reference of TextureAtlas, which owns a FontImage. In update_texture() in render.rs, it'll update upload the actual texture into the GPU.

[ElhamAryanpur]

interesting, I can hardly understand most of them 😅

[emilk]

This might be useful for reference: https://github.com/grovesNL/glyphon

[StratusFearMe21]

For people who need this right now, feel free to use this crate I just created https://github.com/StratusFearMe21/egui-glyphon

[crumblingstatue]

It might also be worth checking out parley.
The authors of xilem considered using cosmic-text, but they decided that they want to do things differently, so they are going to be using parley for their text rendering needs.

Both cosmic-text and parley are using swash under the hood.

[emilk]

I had a talk to the people behind Parley the other day, and I think it is exactly what we want for egui. It is not quite ready yet (lacking docs/examples), but according to the fine folks at linebender, it will be ready for testing in a month or so.

Parley promises to solve:

• layout (including RTL)
• font fallback
• text editing (e.g. "move cursor to the next word")
• glyph rasterization

…and with a minimal amount of dependencies.

This is very exciting!

[jackpot51]

Could someone summarize the issues with cosmic-text that would be blocking its adoption by egui?

[torokati44]

according to the fine folks at linebender, it will be ready for testing in a month or so.

Which is right about now... 👀

[jackpot51]

I fixed pop-os/cosmic-text#70 recently. I'll be doing a new release of cosmic-text soon that includes this change.

[ElhamAryanpur]

🚀 🚀 🚀

[crumblingstatue]

For reference, Bevy is working to decide between parley and cosmic-text right now.
Here is a document they are working on to compare the two: https://hackmd.io/-0nNajS9QaGNu9FWg41ziA

[jackpot51]

I released a new version of cosmic-text, 0.12.0, with numerous fixes for use by bevy. Please let me know if there is anything I need to do to support egui.

[torokati44]

FWIW, Bevy has just merged Cosmic Text support: bevyengine/bevy#10193
Are you still partial to Parley, @emilk?

[emilk]

@crumblingstatue thanks for linking that Bevy document! I think the reasoning and conclusions in there is sound: Parley is very promising, but not yet ready, while Cosmic Text is ready for production today.

I therefor support switching egui to Cosmic Text, if someone coulenteers to do the actual work 😆

The above linked Bevy PR should be a very helpful guide for migrating from ab_glyph to Cosmic Text. I suggest we do this is the least invasive way possible as a first step: only use cosmic text for rasterization, and as much as possible just hot-swap out ab_glyph, keeping the current glyph atlas etc.

---

One thing that worries me is the added dependencies. ab_glyph is very minimal, leading to fast compiles and small binaries (important for .wasm bundle size).

❯ cargo tree -p ab_glyph
ab_glyph v0.2.21
├── ab_glyph_rasterizer v0.1.8
└── owned_ttf_parser v0.19.0

By comparison, we have:

❯ cargo tree -p cosmic-text --no-default-features       
cosmic-text v0.12.0 (/Users/emilk/code/forks/cosmic-text)
├── bitflags v2.6.0
├── fontdb v0.16.2
│   ├── log v0.4.22
│   ├── slotmap v1.0.7
│   │   [build-dependencies]
│   │   └── version_check v0.9.4
│   ├── tinyvec v1.7.0
│   │   └── tinyvec_macros v0.1.1
│   └── ttf-parser v0.20.0
├── log v0.4.22
├── rangemap v1.5.1
├── rustc-hash v1.1.0
├── rustybuzz v0.14.1
│   ├── bitflags v2.6.0
│   ├── bytemuck v1.16.1
│   ├── libm v0.2.8
│   ├── smallvec v1.13.2
│   ├── ttf-parser v0.21.1
│   ├── unicode-bidi-mirroring v0.2.0
│   ├── unicode-ccc v0.2.0
│   ├── unicode-properties v0.1.1
│   └── unicode-script v0.5.6
├── self_cell v1.0.4
├── ttf-parser v0.21.1
├── unicode-bidi v0.3.15
├── unicode-linebreak v0.1.5
├── unicode-script v0.5.6
└── unicode-segmentation v1.11.0

The build.rs in there is especially annoying.

Still, the build time is only 2x, for quite a lot more features:

cargo build -p ab_glyph --quiet  5.39s user 0.22s system 405% cpu 1.384 total
cargo build -p cosmic-text --no-default-features -F std --quiet  10.51s user 1.20s system 492% cpu 2.379 total

So I say as long as the .wasm size doesn't balloon (and I doubt it will), let's go for it 🚀

[parasyte]

One thing that worries me is the added dependencies. ab_glyph is very minimal, leading to fast compiles and small binaries (important for .wasm bundle size).

This is a concern for me, as well. On the other hand, cosmic-text does a lot more than ab_glyph, and they are all things that are needed for proper text rendering. That tradeoff may be worth it.

The other consideration is that cosmic-text might be open to compile-time and binary-size optimizations. (@jackpot51 what do you say?) cargo build --timings points out that the top 5 slowest crates to build on my machine ¹ are:

Crate	Version	Self-time
cosmic-text	0.12.0	2.4s
ttf-parser	0.21.1	2.1s
ttf-parser	0.20.0	1.9s
rusty-buzz	0.14.1	1.8s
rayon	1.10.0	1.6s

ttf-parser has two versions that build in parallel, but they both push fontdb and rustybuzz out by about 2 seconds. Meaning cosmic-text doesn't even start building until at least 3.2s into the build. The total cumulative time is about 5.6s on this machine. It's certainly reasonable on my hardware, but more than 10 seconds on other machines is really pushing it, IMHO.

Footnotes

1. The machine in question has a 12-core/24-thread Ryzen 5900X. Building cosmic-text and all of its dependencies on a 16-core M3-Max takes just 2.4s total! Build times are highly dependent on silicon architecture age. ↩

[jackpot51]

Yes, I am always open to optimizations, and I am tracking some upstream crate issues that cause the duplicate ttf-parser issue.

[barries]

How much is this change likely to improve the kerning, and perhaps the vertical alignment of text from two different fonts? (We're evaluating a change from Electron.js to Tauri or egui, resulting in a bit of a beauty context for this sort of thing here).

For reference, here's the relevant Rust code, the HTML is a styled <button ...>START <...icon...></button>

let mut format = TextFormat {                                                       
    font_id: egui::FontId::new(orig_text_height_px, egui::FontFamily::Proportional),
    color:   Color32::BLACK,                                                        
    valign:  Align::Center,                                                         
    ..Default::default()                                                            
};                                                                                  
                                                                                    
let mut job = LayoutJob::default();                                                 
job.append(text, 0.0, format.clone());                                              
job.append(" ",  0.0, format.clone());                                              
format.font_id.family = egui::FontFamily::Name("icons".into());                     
job.append(icon, 0.0, format);                                              
                                                                                    
let galley = ui.painter().layout_job(job);                                          
let galley_size = galley.size();                                                    
let galley_pos = rect.min + padding + (text_size - galley_size) / 2.0;              
ui.painter().galley(galley_pos, galley, Color32::WHITE);                            

[jb55]

For people who need this right now, feel free to use this crate I just created https://github.com/StratusFearMe21/egui-glyphon

I updated this for 0.30.0 if anyone wants to start hacking on this:

• Update to egui 0.30.0 and glyphon 0.7.0 StratusFearMe21/egui-glyphon#3

works great!