Skip to content

Move Unreal/pak knowledge out of core and behind the MergeCompiler seam (before a second DeployCompile game exists) #256

Description

@dyoung522

Summary

internal/core knows what an Unreal .pak is. The source.MergeCompiler seam that abstracts compile-mode games is otherwise clean — a two-method interface (internal/source/source.go:153-168) dispatched by registry type-assertion (mergeCompilerSourceForGame, internal/core/service.go:214-233), with the whole Icarus implementation self-contained in 1630 non-test lines and exactly one registration site (cmd/lmm/root.go:369).

But five specific pieces of Unreal/Icarus knowledge sit outside that seam, in core and in the shared source package. Those are what make a second DeployCompile game a surgical change rather than an additive one.

This is deliberate, documented deferral, not an oversight — resolveBasePak's own comment says "v1 scope: Icarus only, one known pak filename pattern — extend this if a second DeployCompile-using game is ever added rather than generalizing speculatively now." That was the right call. This issue is to pay it down now, while there is still only one implementation to migrate, so game #2 is a new package plus a registration line.

Explicitly not a plugin system. Dynamic Go plugins would force -buildmode=plugin → CGO + dynamic linking, which trades away the no-CGO posture (modernc.org/sqlite was chosen for exactly that), the single stripped static binary (Makefile:9, -s -w), cross-compilation, and go install, and pins users' plugin .so files to an exact Go toolchain. The out-of-process route stays open for free: MergeCompile already takes and returns only file paths, never bytes, so nothing about this refactor forecloses it.

The five leaks

# Leak Location
1 resolveBasePak hardcodes <InstallPath>/Icarus/Content/Data/data.pak internal/core/service.go:1078-1084
2 basePakIndexHash imports unrealpak — core reads pak footers internal/core/service.go:1091-1098
3 isConvertEligiblePakFile tests for a .pak suffix in core internal/core/service.go:1060-1063
4 MergeSourcePak / MergeSourceExmodz format constants live in the shared source package internal/source/source.go:172-175
5 mergedPakFileName = "zzz_LMM_Merged_P.pak" encodes UE's filename-sort mount-order convention (zzz = load last, _P = override suffix) in core internal/core/merged_pak.go:31-41

Leak 4 is the most telling: Icarus-specific vocabulary has already leaked into the shared abstraction from a single implementation. That is the empirical case for fixing the seam before a second implementation exists — abstracting from one example bakes in that example's assumptions without anyone noticing.

Suggested shape

Push each leak behind the interface that already exists, so core asks the source rather than knowing:

  • 1 + 2 → the compile-capable source resolves and fingerprints its own base artifact. Something like ResolveBaseArtifact(game *domain.Game) (string, error) and FingerprintBase(path string) (string, error) on MergeCompiler (or a sibling interface if keeping MergeCompiler at two methods is worth preserving). Core then holds a path and an opaque hash string, and imports no format library.
  • 3 → the source decides what a convert-eligible file is. Core currently combines a DeployCompile/ConvertPaks policy check (which genuinely is core's business) with a format test (which is not) — split those.
  • 4 → move the kind constants into the Icarus package, or generalize them to opaque strings the source defines and core only round-trips. Core already treats Kind as pass-through data everywhere except mergeSourceKind.
  • 5 → the source names its own output artifact, since the "sorts last so it wins" trick is entirely a UE mount-order property.

Naming is a cheaper, optional follow-on: merged_pak.go, MergedFingerprint, MergedPakOutcomes, SyncMergedPak, PurgeMergedPak, PakNeedsReingest, ModHasPakMergeSource, mergedPakModID. These are cosmetic — do them only if the rename lands cleanly, and don't let them expand the diff. Renaming public core methods is API churn for callers in cmd/lmm and internal/tui.

Non-goals

  • No plugin system, dynamic loading, or subprocess protocol. See above.
  • No second game implementation. This issue only removes the obstacles to one.
  • No declarative/YAML extension for compile games. The custom-sources mechanism (epic Epic: User-defined mod sources #45) expresses HTTP shapes; pak merging is arbitrary binary-format logic (Unreal pak index parsing, JSON table diffing, field-level row upserts) and there is no YAML for that. Recorded so it isn't proposed later.

Acceptance

  • internal/core imports no format library — rg -l unrealpak internal/core/ returns nothing outside tests.
  • internal/core non-test sources contain no .pak string literal and no Icarus path segment.
  • The Icarus package remains the only place that knows the base-pak location, the pak format, the .pak extension, the merge-source kinds, and the merged artifact's filename.
  • Behavior is unchanged: existing Icarus compile tests pass untouched, in particular internal/core/service_icarus_compile_test.go, internal/core/pak_convert_e2e_test.go, and the merged_pak_*_test.go set.
  • A reviewer can point at the interface and say what a second DeployCompile game would have to implement, without reading core.

Context

Filed out of an architecture review prompted by "what happens when we add a second custom-modding game (Satisfactory)?" The conclusion was that the seam is already right and a plugin system would be a bad trade at this scale — but that these five leaks are real, cheap to fix now, and get more expensive after a second implementation exists. Rule of three applies to the plugin boundary, not to the format leaks: those are wrong today regardless of how many games ever land.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:coreinternal/core - flows, install/update/deploy orchestrationarea:sourcesinternal/source - NexusMods, CurseForge, custom sourcesenhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions