Skip to content

Upgrade yapf to v0.43.0 to fix EOFError failure in CI - #8574

Merged
delock merged 1 commit into
deepspeedai:masterfrom
jinyouzhi:yapf
Sep 18, 2026
Merged

delock merged 1 commit into
deepspeedai:masterfrom
jinyouzhi:yapf

Conversation

@jinyouzhi

Copy link
Copy Markdown
Contributor

Fix the bug(google/yapf#1164) which randomly block Formatting CI

https://github.com/deepspeedai/DeepSpeed/actions/runs/35191823983/job/105163117287?pr=8309

Run pip show pre-commit clang-format
Name: pre_commit
Version: 4.6.2
Summary: A framework for managing and maintaining multi-language pre-commit hooks.
Home-page: https://github.com/pre-commit/pre-commit
Author: Anthony Sottile
Author-email: asottile@umich.edu
License: MIT
Location: /home/runner/.local/lib/python3.10/site-packages
Requires: cfgv, identify, nodeenv, pyyaml, virtualenv
Required-by: 
---
Name: clang-format
Version: 18.1.3
Summary: Clang-Format is an LLVM-based code formatting tool
Home-page: 
Author: 
Author-email: Dominic Kempf <ssc@iwr.uni-heidelberg.de>
License: Apache 2.0
Location: /home/runner/.local/lib/python3.10/site-packages
Requires: 
Required-by: 
[INFO] Initializing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Initializing environment for https://github.com/google/yapf.
[INFO] Initializing environment for https://gitlab.com/daverona/pre-commit/cpp.
[INFO] Initializing environment for local.
[INFO] Initializing environment for https://github.com/codespell-project/codespell.
[INFO] Initializing environment for https://github.com/pycqa/flake8.
[INFO] Installing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://github.com/google/yapf.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for local.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://github.com/codespell-project/codespell.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://github.com/pycqa/flake8.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
Check hooks apply to the repository......................................Passed
Check for useless excludes...............................................Passed
check for case conflicts.................................................Passed
check json...............................................................Passed
check for broken symlinks................................................Passed
check yaml...............................................................Passed
detect destroyed symlinks................................................Passed
fix end of files.........................................................Passed
fix utf-8 byte order marker..............................................Passed
fix python encoding pragma (deprecated)..................................Passed
mixed line ending........................................................Passed
fix requirements.txt.....................................................Passed
trim trailing whitespace.................................................Passed
yapf.....................................................................Failed
- hook id: yapf
- exit code: 1

Traceback (most recent call last):
  File "/home/runner/.cache/pre-commit/repos_508syp/py_env-python3.10/bin/yapf", line 3, in <module>
    from yapf import run_main
  File "/home/runner/.cache/pre-commit/repos_508syp/py_env-python3.10/lib/python3.10/site-packages/yapf/__init__.py", line 41, in <module>
    from yapf.yapflib import yapf_api
  File "/home/runner/.cache/pre-commit/repos_508syp/py_env-python3.10/lib/python3.10/site-packages/yapf/yapflib/yapf_api.py", line 39, in <module>
    from yapf.pyparser import pyparser
  File "/home/runner/.cache/pre-commit/repos_508syp/py_env-python3.10/lib/python3.10/site-packages/yapf/pyparser/pyparser.py", line 44, in <module>
    from yapf.yapflib import format_token
  File "/home/runner/.cache/pre-commit/repos_508syp/py_env-python3.10/lib/python3.10/site-packages/yapf/yapflib/format_token.py", line 23, in <module>
    from yapf.pytree import pytree_utils
  File "/home/runner/.cache/pre-commit/repos_508syp/py_env-python3.10/lib/python3.10/site-packages/yapf/pytree/pytree_utils.py", line 30, in <module>
    from yapf_third_party._ylib2to3 import pygram
  File "/home/runner/.cache/pre-commit/repos_508syp/py_env-python3.10/lib/python3.10/site-packages/yapf_third_party/_ylib2to3/pygram.py", line 39, in <module>
    pattern_grammar = driver.load_grammar(_PATTERN_GRAMMAR_FILE)
  File "/home/runner/.cache/pre-commit/repos_508syp/py_env-python3.10/lib/python3.10/site-packages/yapf_third_party/_ylib2to3/pgen2/driver.py", line 252, in load_grammar
    g.load(gp)
  File "/home/runner/.cache/pre-commit/repos_508syp/py_env-python3.10/lib/python3.10/site-packages/yapf_third_party/_ylib2to3/pgen2/grammar.py", line 95, in load
    d = pickle.load(f)
EOFError: Ran out of input

clang-format.............................................................Passed
check-torchdist..........................................................Passed
check-license............................................................Passed
codespell................................................................Passed
flake8...................................................................Passed
check-torchcuda..........................................................Passed
check-extraindexurl......................................................Passed

Signed-off-by: Jin, Youzhi <youzhi.jin@intel.com>
@jinyouzhi jinyouzhi changed the title Upgrade to v0.43.0 to fix the race condition fail in CI Upgrade yapf to v0.43.0 to fix EOFError failure in CI Sep 17, 2026
@delock
delock self-requested a review September 18, 2026 01:05

@delock delock left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Locally verified: reproduced the EOFError race (30 parallel yapf runs, 1/30 crashed with yapf 0.40.0, 0/30 with 0.43.0; root cause matches google/yapf#1164, fixed by atomic pickle write in google/yapf#1243). Also confirmed yapf 0.43.0 produces zero reformatting diffs against the repo, so the one-line bump is safe. Approving — please add to merge queue.

@delock
delock added this pull request to the merge queue Sep 18, 2026
Merged via the queue into deepspeedai:master with commit 9ce505d Sep 18, 2026
13 checks passed
@jinyouzhi
jinyouzhi deleted the yapf branch September 18, 2026 08:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants