Skip to content

Reserve two grid cells for East Asian wide glyphs (CJK) - #88

Merged
goossens merged 1 commit into
goossens:masterfrom
chemPolonium:wide-glyph-cjk
Sep 13, 2026
Merged

Reserve two grid cells for East Asian wide glyphs (CJK)#88
goossens merged 1 commit into
goossens:masterfrom
chemPolonium:wide-glyph-cjk

Conversation

@chemPolonium

Copy link
Copy Markdown

Problem

With a CJK fallback font merged into the atlas (the standard way to display
Chinese/Japanese/Korean comments), every wide glyph overprints its
neighbours. The layout grid advances exactly one cell per codepoint, with the
cell width measured from #. A CJK glyph comes from the fallback font at
roughly two cells wide, so each new glyph starts before the previous one ends
— the text becomes unreadable. (The old BalazsJako editor treated non-ASCII
as double-width; the 2024 rewrite dropped that.)

Fix

Add a glyphColumns() helper keyed on the Unicode East Asian Width
wide/fullwidth
ranges (Hangul Jamo, CJK radicals through Yi, Hangul
syllables, CJK compatibility ideographs/forms, fullwidth forms and signs,
emoji, CJK extension planes), and apply it at every column-advance site:

  • renderSquiggles() and renderText() glyph walks
  • the color-section builder
  • TypeSetter::wrapLine() and TypeSetter::updateLine() column totals
  • TypeSetter::docPos2VisPos() (both wrapped and unwrapped branches)
  • TypeSetter::visPos2DocPos() and TypeSetter::screenPos2DocPos()

Because cursor rendering, selection boxes, bracket matching, mouse
hit-testing, and the minimap all derive from the same column model, keeping
the advance consistent in one helper fixes all of them together. Wide glyphs
still render at their natural width inside their two-cell slot (about 1.8
cells for CJK at equal font size); ambiguous-width characters stay one cell;
tabs and ASCII layout are unchanged.

Testing

  • Compiles cleanly with MSVC (/std:c++20, ImGui 1.92 headers).

  • Used in production by roxy-can
    (a CAN bus analysis tool whose script editor is cimCTE via the
    dear-imgui-cte Rust bindings),
    with Chinese comments as the primary editing language; before/after:

    before after
    characters stacked at half-cell advance clean two-cell advance, cursor and selection aligned

Happy to adjust the wide-range table (e.g. if you'd rather gate it behind a
config flag or use a fuller EAW implementation).

Comment thread TextEditor.cpp
// neighbours. Ranges follow Unicode East Asian Width wide/fullwidth.
//

static size_t glyphColumns(uint32_t codepoint) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if this is called for every glyph, could we at the minimum optimize it a bit to not do all this code point tests for more regular characters? If I am counting right, for a regular ASCII characters that is 9 comparisons...

For example, starting with this code would eliminate all these tests:

// short circuit
if(codepoint < 0x1100) return 1;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose another solution would be to introduce a macro that needs to be enabled for full width support... something like:

#ifndef IMGUI_COLOR_TEXT_EDIT_FULL_GLYPH_SUPPORT
static constexpr size_t glyphColumns(uint32_t codepoint) { return 1; }
#else
// the original code
#endif

which would mean there is 0 test and at compilation time it will result in the exact same current code since the compiler will obviously replace the constexpr call with "1"

@goossens

goossens commented Sep 13, 2026

Copy link
Copy Markdown
Owner

@chemPolonium

Thanks for raising the issue and providing a solution!!! Later today, I will accept your suggestion so you get credit but I will make some changes so it fits better in the current architecture.

  • I will move glyphColumns to TextEditor::CodePoint::getGlyphWidth.
  • Most of the code in CodePoint is automatically generated from the unicode database as will this function. That also adds more fidelity.
  • TextEditor can currently be compiled for 16 or 32 bit codepoints. This will also be added.

@ypujante

  • I will also optimize the code since checking widths (potentially multiple times) for each visible glyph per frame adds up quickly.

@goossens
goossens merged commit 4bbc074 into goossens:master Sep 13, 2026
goossens added a commit that referenced this pull request Sep 13, 2026
@goossens

Copy link
Copy Markdown
Owner

I did everything I stated above and I also adjusted the following to handle wide glyphs:

  • Any function that handles insertSpacesOnTabs.
  • tabsToSpaces and spacesToTabs.
  • Updated the TextDiff widget to also handle wide glyphs.

As my knowledge of East Asian languages is non-existent, I would appreciate a good test and feedback. I did add a Simplified Chinese font to the example application and that seems to work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants