Skip to content

Fix excessive memory use with large external-level projects - #1246

Open
GameGuyy wants to merge 1 commit into
deepnight:masterfrom
GameGuyy:fix/level-json-disk-cache
Open

Fix excessive memory use with large external-level projects#1246
GameGuyy wants to merge 1 commit into
deepnight:masterfrom
GameGuyy:fix/level-json-disk-cache

Conversation

@GameGuyy

Copy link
Copy Markdown

Fixes #1172

Large projects with external levels were crashing the renderer because Electron's V8 heap is capped around 4GB (pointer compression), and every level was kept in memory 3 times: the live data, a parsed LevelJson object, and the pretty-printed JSON string. With the crasher project from the issue this goes over the cap before loading even finishes.

Changes:

  • For external-level projects, the level JSON cache no longer keeps the string/object in RAM. It just points at the level's own .ldtkl file on disk, and reads it back only when actually needed.
  • Saving doesn't wipe and rewrite the whole levels folder anymore. Unchanged levels are skipped entirely, renamed levels get their file copied, and only dirty levels are re-serialized — one at a time, so the peak memory during a save is a single level. Orphaned .ldtkl files are cleaned up after the save succeeds.
  • The main project JSON is built with layer instances skipped instead of serializing everything and stripping fields afterwards.
  • Crash backups go through the same one-level-at-a-time path.

Single-file projects keep the old in-memory cache, nothing changes for them.

Tested with the crasher project from the issue (55 levels, ~276MB): loads and saves fine now, renderer stays under 1GB where it used to hit 4GB and die. Also checked: saving without edits touches only the main .ldtk file, editing one level rewrites only that level's file, level rename/delete, save as to another folder, 0.9.3 project migration, and close/reopen round-trips (multi-world sample included).

@GameGuyy

GameGuyy commented Sep 1, 2026

Copy link
Copy Markdown
Author

Just wanted to bump this. It fixes a renderer OOM crash on large external-level projects (#1172). Tested against the crasher project from that issue: renderer memory stays under 1GB now, where it previously blew past the ~4GB V8 heap cap and crashed before load even finished. Happy to address any review feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Any way to support >4GB RAM allocations or save/load with a smaller mem footprint?

1 participant