ci: build, test and Plugin Verifier on every push and PR - #13
Merged
Conversation
The repo had no CI. Contributor PRs were landing with nothing checking that
they compiled, let alone that the plugin still loaded on the IDE range the
manifest claims.
Two jobs:
build — runs :core:test (the parser suite; :plugin has no tests by design, so
`gradlew test` alone is green either way), then build, the two configuration
checks, and buildPlugin, uploading the zip.
verify — runs the IntelliJ Plugin Verifier against 2024.2.5 and 2024.3.5. The
first is the oldest release in the 242 line sinceBuild claims, which is where
an API the plugin uses might not exist yet; the compile target cannot surface
that. untilBuild = 261.* stays unverified: no release exists to check against.
Two traps worth recording, both of which produce a green run that verified
nothing:
- pluginVerification.ides accepts ide(IntelliJPlatformType, String) and then
schedules no verification at all. Only the ide("IC-2024.2.5") string form
works. The task still succeeds, having checked one IDE instead of two.
- verifyPlugin needs pluginVerifier() in the dependencies block; without it
the task fails outright, which is the friendlier of the two failures.
The verify job therefore asserts that the number of report directories matches
the number of declared IDEs, so a silently-dropped entry fails the build
instead of passing quietly.
…g step The first run failed both jobs with exit 126 — gradlew was committed 100644, so Linux runners could not execute it. Nobody noticed because the repo had no CI and every build until now was on Windows, where the mode bit is not consulted. The report step then said "no test results — :core:test produced nothing", which is true and misleading: the test step had already failed. It now only treats missing results as a finding when the test step reported success, which is the case it exists for — a suite that quietly stops running.
This was referenced Aug 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The repo had no CI at all — intellij#12 arrived with
no checks reported, so nothing verified it compiled, let alone that the plugin still loads across the IDE range the manifest claims.Jobs
build —
:core:test(the parser suite), thenbuild,verifyPluginProjectConfiguration,verifyPluginStructure,buildPlugin, uploading the zip as an artifact.:core:testrather thantest, because:plugindisables its test task by design —gradlew testreports success whether or not any test exists.verify — the IntelliJ Plugin Verifier against 2024.2.5 and 2024.3.5. The first is the oldest release in the 242 line
sinceBuildclaims, which is where an API the plugin uses might not exist yet; the compile target can't surface that.untilBuild = 261.*stays unverified — there's no release to check against, so it remains a forward-looking promise.Two ways this passes while checking nothing
Both found by running it rather than reading docs:
pluginVerification.idesacceptside(IntelliJPlatformType.IntellijIdeaCommunity, "2024.2.5"), compiles, runs — and schedules no verification for it. Only theide("IC-2024.2.5")string form works. The task still succeeds, having checked one IDE instead of two.verifyPluginneedspluginVerifier()in the dependencies block. Without it the task fails outright, which is the friendlier failure of the two.So the verify job asserts the number of report directories equals the number of declared IDEs. A silently-dropped entry fails the build instead of passing quietly.
Verified locally
The assertion was also run against a deliberately wrong declared count to confirm it fails.