Skip to content

Implement Hazel3D-style dual-manager asset system with async loading and specialized serializers#25

Merged
sheazywi merged 10 commits intodevfrom
copilot/implement-hazel-asset-system
Apr 11, 2026
Merged

Implement Hazel3D-style dual-manager asset system with async loading and specialized serializers#25
sheazywi merged 10 commits intodevfrom
copilot/implement-hazel-asset-system

Conversation

Copy link
Copy Markdown

Copilot AI commented Apr 7, 2026

  • Inspect current EditorLayer ImGui flow and identify high-impact cleanup/perf points
  • Run existing build entrypoint before edits (blocked by missing premake5/Vulkan SDK in environment)
  • Refactor dockspace setup in EditorLayer::OnImGuiRender to reduce per-frame clutter and make lifecycle cleaner
  • Add lightweight UI-state controls to avoid unnecessary heavy debug drawing in stats panel
  • Reduce repeated panel lookups in EditorLayer by caching panel refs where safe
  • Run available validation/build command after edits (build still blocked by missing premake5/Vulkan SDK; CodeQL timed out in validation)
  • Commit and push changes
Original prompt

Implement the Hazel Engine's asset system in LuxEngine. The new asset system should include the dual manager architecture (EditorAssetManager and RuntimeAssetManager), specialized serializers (like MeshSerializer, TextureSerializer), asynchronous loading threads (EditorAssetSystem and RuntimeAssetSystem), and YAML-based metadata tracking, mirroring the provided Hazel3D implementation.

The following is the prior conversation context from the user's chat exploration (may be truncated):

User: Can you remake my asset system, exactly the same as the Hazel3D engine,

Assistant: I can certainly help you understand the architecture of the asset system you've shared so you can implement these concepts in your own engine, though I cannot rewrite or reproduce the exact engine source code line-by-line.

Based on the files provided, here is a high-level summary of how this asset system is structured:

  • Core Asset Representation: Everything revolves around an Asset base class that contains a UUID (AssetHandle) and an AssetType enum.
  • Dual Manager Architecture: The system uses a virtual AssetManagerBase with two distinct implementations:
    • EditorAssetManager: Designed for development. It relies on an AssetRegistry to map UUIDs to file paths, reads metadata from disk, supports YAML-based serialization, handles hot-reloading, and tracks file modifications.
    • RuntimeAssetManager: Optimized for the shipped game. It bypasses individual files and instead loads packed, binary asset data from an AssetPack.
  • Specialized Serializers: A factory pattern (AssetImporter) routes different asset types (meshes, materials, audio configs, etc.) to specific serializers inherited from AssetSerializer. These handle both YAML (editor) and binary stream (runtime) parsing.
  • Asynchronous Loading: Dedicated background threads (EditorAssetSystem and RuntimeAssetSystem) manage a thread-safe queue of loading requests (AssetMetadata or RuntimeAssetLoadRequest). Once an asset is loaded in the background, it is synced back to the main thread's loaded asset map.

We can explore how to build out your own asset system using these design patterns. Would you like to dive deeper into how the asynchronous loading thread is implemented, or focus on how the EditorAssetManager tracks file dependencies and metadata?

User: Let's just make the asset system the same as the Hazel Engine

Assistant: I've initiated the process to implement the new asset system, and I'm currently waiting for your approval on the dialog to proceed.

This pull request was created from Copilot chat.

Copilot AI and others added 2 commits April 7, 2026 04:36
Copilot AI changed the title [WIP] Add dual manager architecture and serializers for asset system Implement Hazel3D-style dual-manager asset system with async loading and specialized serializers Apr 7, 2026
Copilot AI requested a review from sheazywi April 7, 2026 04:42
- Update GitHub Actions to use MSBuild v3 and target VS2026.
- Add post-build commands in Premake to copy Assimp DLLs to the output directory on Windows.
- Standardize ImGuiEx include paths across Editor panels.
…stem' into copilot/implement-hazel-asset-system
@sheazywi sheazywi marked this pull request as ready for review April 11, 2026 05:58
Copilot AI review requested due to automatic review settings April 11, 2026 05:58
@sheazywi sheazywi merged commit d5449e4 into dev Apr 11, 2026
2 checks passed
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a Hazel3D-style asset pipeline (serializer-based importing + async loading + dual editor/runtime managers) and updates the editor UI/panel plumbing to better support project/scene lifecycle changes.

Changes:

  • Added serializer-based asset importing (mesh/material/texture) and new Assimp-based mesh source importer.
  • Implemented async background asset loading systems for editor and runtime, plus a new RuntimeAssetManager.
  • Refactored editor panels to integrate with PanelManager + project-change notifications, and updated build/CI scripts.

Reviewed changes

Copilot reviewed 31 out of 31 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
Editor/Source/Panels/TextEditorPanel.cpp Update ImGuiEx include path + property grid usage.
Editor/Source/Panels/SceneRendererPanel.cpp Update ImGuiEx include path + property grid usage.
Editor/Source/Panels/SceneHierarchyPanel.h Convert panel to EditorPanel interface + new render signature.
Editor/Source/Panels/SceneHierarchyPanel.cpp Panel open-state handling and ImGui window lifecycle updates.
Editor/Source/Panels/ContentBrowserPanel.h Convert to EditorPanel + add project-change handling hooks.
Editor/Source/Panels/ContentBrowserPanel.cpp Initialize/reset based on active project; remove reliance on global active project.
Editor/Source/Panels/ConsolePanel.cpp Update ImGuiEx include path + property grid usage.
Editor/Source/EditorLayer.h Switch some panels to Ref<> and reduce direct panel ownership.
Editor/Source/EditorLayer.cpp Dockspace/UI cleanup, panel-manager wiring for scene/project changes, selection guards.
Editor/premake5.lua Add Windows post-build steps to copy Assimp DLLs per config.
Core/Source/Lux/Asset/TextureSerializer.h Add texture serializer interface.
Core/Source/Lux/Asset/TextureSerializer.cpp Implement texture load via TextureImporter.
Core/Source/Lux/Asset/RuntimeAssetSystem.h Add runtime async loading system + request/callback types.
Core/Source/Lux/Asset/RuntimeAssetSystem.cpp Implement runtime load worker thread + sync step.
Core/Source/Lux/Asset/RuntimeAssetManager.h Add runtime asset manager using registry + async loader.
Core/Source/Lux/Asset/RuntimeAssetManager.cpp Implement runtime asset retrieval and async load orchestration.
Core/Source/Lux/Asset/MeshSerializer.h Add mesh source/mesh/static-mesh serializers.
Core/Source/Lux/Asset/MeshSerializer.cpp Implement YAML mesh/static-mesh serialization + mesh source import.
Core/Source/Lux/Asset/MaterialSerializer.h Add material YAML serializer.
Core/Source/Lux/Asset/MaterialSerializer.cpp Implement YAML material serialization/deserialization.
Core/Source/Lux/Asset/EditorAssetSystem.h Add editor async loading system.
Core/Source/Lux/Asset/EditorAssetSystem.cpp Implement editor load worker thread + sync step.
Core/Source/Lux/Asset/EditorAssetManager.h Add async API and embed EditorAssetSystem.
Core/Source/Lux/Asset/EditorAssetManager.cpp Extend asset type map + async queueing + metadata handle field.
Core/Source/Lux/Asset/AssimpMeshImporter.h Add Assimp-based mesh importer API.
Core/Source/Lux/Asset/AssimpMeshImporter.cpp Implement Assimp mesh import into MeshSource (buffers, nodes, AABBs).
Core/Source/Lux/Asset/AssetSerializer.h Introduce common serializer interface.
Core/Source/Lux/Asset/AssetMetadata.h Add Handle into metadata.
Core/Source/Lux/Asset/AssetImporter.h Document serializer-based routing.
Core/Source/Lux/Asset/AssetImporter.cpp Implement serializer dispatch table + legacy importer fallback.
.github/workflows/main.yml Update MSBuild action version + setup script invocation.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +47 to +51
if (!ImGui::Begin("Scene Hierarchy", &isOpen))
{
ImGui::End();
return;
}
Copy link

Copilot AI Apr 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OnImGuiRender returns early when ImGui::Begin("Scene Hierarchy", &isOpen) returns false (collapsed window). In ImGui, Begin returning false only means the window is collapsed; it is still open. This early return prevents the "Properties" window from rendering whenever the hierarchy window is collapsed, which is a behavioral regression.

Consider restructuring so you always call ImGui::End() for the hierarchy window, but only skip drawing its contents when Begin returns false, and then continue to render the Properties window independently of the hierarchy window's collapsed state.

Copilot uses AI. Check for mistakes.
Comment on lines +175 to +176
// Insert sentinel root so every real node has a valid parentIndex
meshSource->m_Nodes.emplace_back(); // root placeholder
Copy link

Copilot AI Apr 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The node hierarchy construction inserts a root placeholder at m_Nodes[0], but then calls TraverseNodes(..., parentIndex = s_InvalidParentIndex), so the real imported root ends up at index 1 and MeshSource::GetRootNode() returns the empty placeholder instead of the actual root.

Either populate the placeholder with the imported root node (and use parentIndex 0 for its children), or remove the placeholder and update GetRootNode()/root assumptions accordingly.

Suggested change
// Insert sentinel root so every real node has a valid parentIndex
meshSource->m_Nodes.emplace_back(); // root placeholder
// Store the imported scene root at m_Nodes[0] so GetRootNode() returns
// the actual root node rather than an empty placeholder.

Copilot uses AI. Check for mistakes.
Comment on lines +29 to +39
std::scoped_lock lock(m_FinishedQueueMutex);
while (!m_FinishedQueue.empty())
{
auto& entry = m_FinishedQueue.front();
if (entry.LoadedAsset)
{
loadedAssets[entry.Handle] = entry.LoadedAsset;
if (entry.CallbackFn != nullptr)
entry.CallbackFn(entry.Handle, entry.LoadedAsset);
}
m_FinishedQueue.pop();
Copy link

Copilot AI Apr 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SyncLoadedAssets holds m_FinishedQueueMutex while invoking entry.CallbackFn(...). If a callback calls back into the asset system/manager (or otherwise tries to sync assets), this can deadlock or create long stalls while the mutex is held.

A safer pattern is to move finished entries into a local list/queue under the mutex, release the lock, then update loadedAssets and invoke callbacks without holding m_FinishedQueueMutex.

Suggested change
std::scoped_lock lock(m_FinishedQueueMutex);
while (!m_FinishedQueue.empty())
{
auto& entry = m_FinishedQueue.front();
if (entry.LoadedAsset)
{
loadedAssets[entry.Handle] = entry.LoadedAsset;
if (entry.CallbackFn != nullptr)
entry.CallbackFn(entry.Handle, entry.LoadedAsset);
}
m_FinishedQueue.pop();
decltype(m_FinishedQueue) finishedEntries;
{
std::scoped_lock lock(m_FinishedQueueMutex);
finishedEntries.swap(m_FinishedQueue);
}
while (!finishedEntries.empty())
{
auto& entry = finishedEntries.front();
if (entry.LoadedAsset)
{
loadedAssets[entry.Handle] = entry.LoadedAsset;
if (entry.CallbackFn != nullptr)
entry.CallbackFn(entry.Handle, entry.LoadedAsset);
}
finishedEntries.pop();

Copilot uses AI. Check for mistakes.
Comment on lines +55 to +63
uses: microsoft/setup-msbuild@v3
with:
msbuild-architecture: 'x64'

- name: Setup
working-directory: ${{env.GITHUB_WORKSPACE}}
run: |
cd scripts
python Setup.py
python Setup.py vs2026
Copy link

Copilot AI Apr 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR title describes implementing a Hazel3D-style asset system, but the PR description checklist focuses on EditorLayer ImGui cleanup/perf work (and doesn’t mention the new asset-manager/serializer/async-loading changes). Please update the PR description to reflect the asset system work (or split the editor UI refactor into a separate PR) so reviewers can scope risk appropriately.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants