test: isolate unit tests from user state

[luozhixiong01]

Summary

Default unit tests contained several side effects that depended on or modified developer state: two opt-in-worthy skills integration tests invoked the real npx skills CLI, ordinary update tests could fall through to npx, multiple packages wrote lark-cli cache/log/event state, registry users started background remote-metadata refreshes, and one API test saved a binary into the repository working directory.

Changes

• Gate the two live skills integration tests behind LARKSUITE_CLI_RUN_LIVE_SKILLS_TESTS=1 and redirect all relevant user and agent directories to a temporary root.
• Add make live-skills-test as the documented explicit entrypoint.
• Fully mock skills discovery and sync in ordinary cmd/update tests so they cannot invoke npx.
• Isolate config, data, log, and event state for packages that intentionally exercise those write paths.
• Disable remote metadata refresh in unit-test packages that only exercise embedded registry/catalog behavior.
• Run the binary auto-save API test from a temporary working directory.

Validation

• make unit-test with an isolated HOME and blocked network: passed.
• Dynamic default-test scan across 116 cmd, internal, shortcuts, and extension packages: passed.
• Asserted no .lark-cli, .agents, .npm, or .claude state remained under the isolated HOME.
• Asserted cmd/api/download.bin was not created in the repository.
• Default cmd/update runtime dropped from about 80 seconds with real npx fallthrough to about 2 seconds with mocks.
• make -n live-skills-test: passed.

The branch is rebased on current origin/main at 1c36744.

Summary by CodeRabbit

• Tests
  • Added opt-in controls for live skills integration tests (disabled by default).
  • Improved test isolation by running with temporary filesystem directories and redirected config/log locations.
  • Expanded live-update coverage to verify skill availability and state synchronization, with longer timeouts where needed.
  • Standardized test startup to prevent remote metadata refresh during test runs.
• Documentation
  • Updated build/test instructions to include make live-skills-test.
• Chores
  • Added a live-skills-test Makefile target that runs the selected live tests with the opt-in flag.

[coderabbitai]

📝 Walkthrough

Walkthrough

Live skills integration tests now require explicit opt-in, isolate user and configuration directories in temporary locations, and share preparation logic across synchronization scenarios. Test packages also gain isolated harnesses, while a dedicated Make target documents and runs the live tests.

Changes

Test isolation and live skills integration

Layer / File(s)	Summary
Package test harness isolation cmd/*/testmain_test.go, internal/*/testmain_test.go	Package-level test entry points now isolate configuration, data, and log directories or disable remote metadata refreshes.
Update synchronization test control cmd/update/update_test.go	Adds mocked skills synchronization setup, opt-in live-test gating, temporary environment redirection, and shared preparation for live synchronization scenarios.
Command test workspace isolation cmd/api/api_test.go	Runs the binary-response autosave test from a temporary working directory.
Live test command and documentation Makefile, AGENTS.md	Adds and documents the live-skills-test target for the selected live update tests.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

• larksuite/cli#858: Adds related isolation for CLI skills configuration state.
• larksuite/cli#1042: Introduces related real skills-state synchronization tests.

Suggested labels: feature

Suggested reviewers: maxhuang22

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 16.67% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title is concise and accurately reflects the main goal of isolating tests from user state.
Description check	✅ Passed	The description covers the required summary, changes, and validation, but it should use the template's Test Plan and Related Issues sections.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/isolate-live-skills-tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@cmd/update/update_test.go`:
- Around line 1696-1701: Update TestUpdateCommand_SkillsSyncColdStart to create
and defer-cancel a 90-second context, then use it for the skills CLI commands.
Before collecting global skills with “skills ls -g”, seed the isolated
environment by running the same “skills add” command for lark-calendar with
global and noninteractive flags, skipping with descriptive errors if listing or
seeding fails.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 745563ab-abbf-4e2d-be84-6a235eb2a292

📥 Commits

Reviewing files that changed from the base of the PR and between 37d490a and 09679cc.

📒 Files selected for processing (1)

• cmd/update/update_test.go

[github-actions]

🚀 PR Preview Install Guide

🧰 CLI update

npm i -g https://pkg.pr.new/larksuite/cli/@larksuite/cli@05345c22566e1aedb7de4fce5525ef59d955e79d

🧩 Skill update

npx skills add larksuite/cli#fix/isolate-live-skills-tests -y -g

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.67%. Comparing base (1c36744) to head (05345c2).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1883   +/-   ##
=======================================
  Coverage   74.66%   74.67%           
=======================================
  Files         877      877           
  Lines       91731    91731           
=======================================
+ Hits        68494    68496    +2     
+ Misses      17926    17925    -1     
+ Partials     5311     5310    -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

PR Quality Summary

CI did not complete successfully. Use the failed check links below to decide whether this PR needs a code change or a rerun.

Failed checks

• e2e-live — failure — details
• results — failure — details