fix(plugin): decode diff Git paths as UTF-8 - #543
Open
sylvesterkaczmarek wants to merge 3 commits into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Decode Git filename lists explicitly as UTF-8 in diff ranking instead of using Python's process-locale decoder.
Fixes #533.
Reproduction / evidence
Current upstream
mainat37bf87a692fc72d41f7312cc48808d699d204fbahas two direct text-mode Git subprocesses ingenerate_rank_input.pywithout an explicit encoding:git diff --name-status -zfor committed, staged, and unstaged changed paths;git ls-files --others --exclude-standard -zfor working-tree untracked paths.With
text=True, Python chooses the process locale encoding. On Windows with a non-UTF-8 ANSI code page, Git's UTF-8 filename output can therefore decode incorrectly or fail before ranking begins. The repository path itself can be ASCII; a changed filename containing non-ASCII characters is sufficient.This exact call site was independently identified in the earlier closed #293 investigation while addressing #192. That PR documented failures for non-ASCII Git filenames and explicitly noted this helper as a second call site. The current helper still contained both locale-dependent subprocesses.
The regression in this PR instruments
subprocess.runinside the actual Python helper and refuses every Git text invocation unlessencoding == "utf-8". It exercises both the changed-path and untracked-path flows using synthetic測試.pyand新規.pynames. On the unfixed helper the assertion fails becauseencodingis absent.Root cause
The diff-ranking helper launches Git directly rather than routing these path-list operations through the already-hardened workbench Git helper. Its text-mode subprocesses therefore inherited Python's locale decoder.
Fix
Add
encoding="utf-8"to both text-mode Git filename-list subprocesses:No error handler or replacement decoding is added. Invalid UTF-8 still fails rather than silently changing a filename.
Tests / validation
Added
diff-rank-git-encoding.test.ts. The test imports the real bundled Python helper, replaces only its subprocess boundary, and verifies:The branch is based directly on current upstream
mainat37bf87a692fc72d41f7312cc48808d699d204fbaand is not behind it. Production change: exactly 2 additions, 0 deletions. The remainder is focused regression coverage.Full repository tests cannot be run in this execution environment because the repository cannot be cloned here. Pushed-head CI remains the authoritative full-suite validation.
Risk
Low. Git filename decoding becomes deterministic UTF-8 instead of locale-dependent. ASCII paths are unchanged, and binary Git operations are untouched.