feat: Support overriding language ID of the text emitted - #1837
Open
cqjjjzr wants to merge 1 commit into
Open
Conversation
There was a problem hiding this comment.
Pull request overview
This PR adds a mechanism to override the TSF language ID applied to committed / composing text, so apps can use the correct font/spellcheck language even when the active keyboard layout would otherwise force a different LANGID.
Changes:
- Introduces a
commit_langidconfig value transported over IPC and stored per session. - Applies
GUID_PROP_LANGIDon TSF ranges (composition start, inline preedit updates, and committed text insertion). - Removes reliance on the previously-unused
Config::inline_preeditfield and uses UI style instead.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| WeaselTSF/WeaselTSF.h | Adds _SetRangeLanguage API and _textLangId storage for TSF language override. |
| WeaselTSF/EditSession.cpp | Reads commit_langid from IPC response and switches inline-preedit decision to style. |
| WeaselTSF/DisplayAttribute.cpp | Implements setting GUID_PROP_LANGID on a TSF range. |
| WeaselTSF/Composition.cpp | Applies the language override to composition/preedit/commit ranges. |
| WeaselIPC/Configurator.cpp | Parses config.commit_langid from IPC messages. |
| RimeWithWeasel/RimeWithWeasel.cpp | Loads locale-based override from configs and emits config.commit_langid over IPC. |
| include/WeaselIPCData.h | Updates IPC Config struct to carry commit_langid. |
| include/RimeWithWeasel.h | Extends session status to store commit_langid and adds loader method declaration. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
18
to
21
| if (ok) { | ||
| bool inline_preedit = _cand->style().inline_preedit; | ||
| _textLangId = static_cast<LANGID>(config.commit_langid); | ||
| if (!commit.empty()) { |
There was a problem hiding this comment.
_textLangId is stored as a mutable WeaselTSF member, but the actual language assignment happens later in separately requested (potentially async) edit sessions (_StartComposition/_InsertText/_ShowInlinePreedit). If multiple edit sessions are queued, _textLangId can be overwritten before those sessions run, causing the wrong LANGID to be applied to the range. Consider capturing the langid per edit-session instance (store it in the edit session object) or applying the GUID_PROP_LANGID value within the same edit session that sets the text.
|
|
||
| if (ok) { | ||
| bool inline_preedit = _cand->style().inline_preedit; | ||
| _textLangId = static_cast<LANGID>(config.commit_langid); |
There was a problem hiding this comment.
Casting config.commit_langid (int) to LANGID will silently truncate out-of-range values. Please validate/clamp to the valid LANGID range (0..0xFFFF) and treat invalid values as “no override” (or log) to avoid setting an unintended language ID.
Suggested change
| _textLangId = static_cast<LANGID>(config.commit_langid); | |
| if (config.commit_langid >= 0 && config.commit_langid <= 0xFFFF) { | |
| _textLangId = static_cast<LANGID>(config.commit_langid); | |
| } |
Comment on lines
+64
to
+69
| if (SUCCEEDED(pContext->GetProperty(GUID_PROP_LANGID, &pLangIdProperty))) { | ||
| VARIANT var; | ||
| var.vt = VT_I4; | ||
| var.lVal = _textLangId; | ||
| hr = pLangIdProperty->SetValue(ec, pRange, &var); | ||
| pLangIdProperty->Release(); |
There was a problem hiding this comment.
VARIANT var is used without VariantInit(). Even though only VT_I4 is assigned, initializing the VARIANT avoids uninitialized padding/fields and aligns with COM best practices (same pattern also exists in _SetCompositionDisplayAttributes).
Comment on lines
15
to
21
| void Configurator::Store(Deserializer::KeyType const& key, | ||
| std::wstring const& value) { | ||
| if (!m_pTarget->p_context || key.size() < 2) | ||
| if (!m_pTarget->p_config || key.size() < 2) | ||
| return; | ||
| bool bool_value = (!value.empty() && value != L"0"); | ||
| if (key[1] == L"inline_preedit") { | ||
| m_pTarget->p_config->inline_preedit = bool_value; | ||
| if (key[1] == L"commit_langid") { | ||
| m_pTarget->p_config->commit_langid = _wtoi(value.c_str()); | ||
| } |
There was a problem hiding this comment.
Parsing commit_langid with _wtoi provides no error reporting and accepts negative / out-of-range values, which then flow into TSF language property setting. Prefer a checked parse (e.g., wcstol with endptr) and clamp/reject values outside the valid LANGID range to keep IPC robust against malformed inputs.
| if (!lcid) | ||
| return false; | ||
|
|
||
| *commit_langid = LANGIDFROMLCID(lcid); |
There was a problem hiding this comment.
TryGetLangIdFromConfig writes the resulting LANGID into an int without any bounds/type enforcement. Since this value is later serialized over IPC and cast to LANGID, it’s safer to explicitly constrain it to 0..0xFFFF (and treat 0 as “not set”) before storing/sending.
Suggested change
| *commit_langid = LANGIDFROMLCID(lcid); | |
| const unsigned long langid = static_cast<unsigned long>(LANGIDFROMLCID(lcid)); | |
| if (langid == 0 || langid > 0xFFFFul) | |
| return false; | |
| *commit_langid = static_cast<int>(langid); |
Member
|
我來參加，最終還是看梁老師決策～ 配置中的 |
Contributor
Author
是可行的，不过语言代码 <=> 微软的 LANGID 并非一一对应... |
Member
|
哦……行吧。那就讓用家自己查。 我在想，要不要註冊輸入法的時候給這個參數，免得每次往輸入片段上設置。安裝程序還可以提供多種語言選項。 |
Contributor
Author
那样就对全局生效了，但如果全局例如设置成日语，那中文输入又会出问题了，所以至少需要每个 schema 设置，因为输入法本身选择哪个 schema 并不会体现在全局状态上（我并没找到“临时切换一个输入法语言”的 API）。 
|
Contributor
其实我有做一个类似的事情，只是没有pr还 |
Contributor
|
再想下，其实这样也是不完整的，万一方案中有混合输出的内容，有中文有英文有日文的，如何应对？ 研究下微软拼音的简繁的切换的实现，我猜可能是有API可以切换状态的 |
Contributor
Author
|
这样就要求 librime 本身具备能标记一次编辑生成的文本中各个段的语言了，恐怕有点做不完。「同时支持多种语言的方案」这一需求是否真的存在也留待调查（至少这个 PR 提供了临时修改一次编辑的语言的基础设施）我后面看看MS拼音的实现，但我目前的确没搜出来具体的“临时切换输入法 LANGID 而不每次修改ITfRange语言”的做法。 |
Contributor
|
ITfInputProcessorProfiles::ChangeCurrentLanguage |
Contributor
|
其实可以再简单一点，只改tsf加菜单选择语言只对当前应用有效 |
Contributor
Author
这会切换整个输入法，类比按了 Alt+Shift (即 Windows 7 下的 Ctrl+Space)
没太看懂 |
Contributor
你现在的语言期望从方案或weasel.yaml传来，其实可以再简单一点， 在语言栏按钮的右键菜单里做一个层级菜单，里面有若干你要用的语言可选，选择之后对当前应用就按新的语言来标识。这样不用动服务不用动ipc，只要给tsf改就行。 |
Contributor
Author
一个方案基本只会有一个语言，假设我只有中文和日文方案，那么在 Word 写作时基本每次切换方案都需要一起切换语言，不是特别方便。我感觉还是想办法联动一下 |
Member
|
我看現在這個 PR 也是全局生效的對麼；在運行時標註語言，但配置仍是全局的；那就不如註冊輸入法的時候聲明語言。 之前有人提過，想要把輸入法註冊到其他的語言/地區。 |
Contributor
Author
并不是，我是在 因为是跟着 schema 走，切换方案就会变更输入语言。
这是另一个问题了，目前用注册表 trick 就能解决 |
Contributor
|
我的想法是，如果不能根本解决问题，更好的办法是更小耦合的方案先用。直接上了一个未必成熟的方案后面要改会break很麻烦 |
Contributor
Author
这里的核心问题是“LANGID 需要跟着方案切换而一并切换”，因此除非完全引入新的配置源，否则总需要有一种方法把 schema 里面的配置送到 TSF 前端去，可以放在 schema 配置，可以放在 weasel 配置，但 IPC 恐怕是不得不改的（除非直接在 TSF 端去读文件）。 不过这个因为引入了新的配置项，的确需要更多讨论。不如先看看另外几个正确性bug。
考虑这个问题在 Discussion 里面也多次有人提过，而在这里用中文提问的人自然大概也会用 RIME 打中文，而如果他们要用 RIME 打别的什么语言遇到问题想注册到其它语言上并用了注册表之类的各种 trick，那这种“多语言输入”（在不同 schema 上打不同语言）应该是很自然的需求。 |
Contributor
|
有个不成熟的想法，根据候选自动切换langid, 测试Demo输出如下，性能应该是够用的,其他语言应该有扩展的机会 编译命令
|
Contributor
Author
中文日文有大量写法完全一致的字词，例如我中文打五笔，而日文也以单词输入为主，那么识别错误率会特别高。重要的是这样在 MS Word 等软件里，即使我全程打中文，也可能被误判，或打日文时出现汉字词中文字体，假名部分日文字体的行为。因此在输入端进行语言检查我认为是完全不可行的。 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
参照：
发现
weasel::Config里面的字段其实是没有被用到的（可以被 style 替代），因此去掉并换成了 LANGID，主要是利用 TSF 在输入ITfRange时可以能通过GUID_PROP_LANGID指定这段文本的语言的特性，覆盖掉由于当前键盘设置导致的使用 RIME 输入的其它语言文本被指定为中文而引起字体、拼写检查的错误。增加了新的配置项，可能需要 document。
另外 pre-edit 文本闪烁的问题也解决了，如下图（注意到日文自动切换到了 Yu Mincho，而中文使用默认的等线）