Skip to content

Refactor/skeleton frontend#151

Merged
afonsobspinto merged 22 commits intofeature/edit-modefrom
refactor/skeleton-frontend
May 8, 2026
Merged

Refactor/skeleton frontend#151
afonsobspinto merged 22 commits intofeature/edit-modefrom
refactor/skeleton-frontend

Conversation

@seankmartin
Copy link
Copy Markdown

@seankmartin seankmartin commented May 7, 2026

PR Summary

Refactor: Align skeleton render layer with segmentation layer patterns

This PR refactors src/skeleton/frontend.ts and related files to bring the skeleton rendering architecture in line with the more mature segmentation render layer patterns — cleaner interface boundaries, shared GPU utilities, reactive shader parameters, and elimination of duplicated pick-handling and rendering logic.

What changed

Interface modernization — Old mixed-concern interfaces (SkeletonLayerInterface, SkeletonChunkInterface) replaced with clearly separated types:

  • SkeletonShaderContext — rendering context (gl, displayState, shader parameters)
  • SkeletonShaderParameters — shader feature flags as a WatchableValue (replaces scattered booleans)
  • SkeletonGPUGeometry — pure GPU geometry interface
  • PackedSkeletonGeometry — CPU-side vertex/index data

GPU upload consolidationgpu_upload_utils.ts deleted; its core logic was already in texture_access.ts. Added updateOneDimensionalTextureElement() for in-place single-element texture updates, and extracted uploadSkeletonChunkToGPU() as a shared helper for both chunk types.

SkeletonOverlayChunk promoted to full class — Now implements SkeletonGPUGeometry, owns its GPU resources (textures, buffers, picking data), and exposes updateSelectedNode() which patches a single texture element in-place instead of rebuilding all GPU state when only node selection changes.

Overlay geometry memory optimization — Introduced a reusable scratch buffer for arrays that are only needed during GPU upload (segmentIds, selected, edgeIndices, edgeSegIds), avoiding repeated allocation. Arrays needed for CPU-side picking (positions, nodeIds, pickSegmentIds) are still freshly allocated and owned by the chunk.

Two-level overlay caching — Geometry key excludes selectedNodeId, so selection-only changes trigger the fast updateSelectedNode() path. Geometry changes trigger a full rebuild. A frame-number guard prevents redundant work across multi-pass rendering.

Shader parameter managementskeletonShaderParameters as a WatchableValue<SkeletonShaderParameters> replaces scattered booleans. Reactive update registration listens to segmentStatedColors and segmentDefaultColor, decoupling shader feature flags from layer state.

Picking logic deduplicated — Both PerspectiveViewSpatiallyIndexedSkeletonLayer and SliceViewPanelSpatiallyIndexedSkeletonLayer had verbatim copies (~75 lines each) of transformPickedValue and updateMouseState. Extracted as free functions transformSpatiallyIndexedSkeletonPickedValue and updateSpatiallyIndexedSkeletonMouseState; each class method now delegates in 1–4 lines.

Cast cleanupas SkeletonLayerDisplayState & SpatialSkeletonDisplayState intersection casts removed throughout draw(), isReady(), and constructors. chunkGeometryRenderLayerInterface renamed to browsePassLayerView.

Naming and removalsSkeletonRenderMode enum deleted (render_mode.ts removed); SpatiallyIndexedSkeletonParameterAccess union type removed; DEBUG_SPATIAL_SKELETON_CHUNKS set to false.


What's Left to Refactor

High priority

Segment ID precision bug (frontend.ts ~L625)
The existing TODO captures this: segmentAttribute is Uint32 but segments are Uint64. Fixing this also unblocks the structural cleanup below — once the attribute is known to be internal, it can be removed from vertexAttributes entirely.

Internal attributes mixed into vertexAttributes (frontend.ts ~L625, ~L630)
segmentAttribute and selectedNodeAttribute live in the same vertexAttributes array as user-defined attributes but are skipped by index check in both shader getters:

if (i === this.segmentAttributeIndex || i === this.selectedNodeAttributeIndex) continue;

These are internal implementation details, not user-defined. They should be stored separately so the user attribute loop needs no special-casing and the boundary is explicit. This also makes the code clearer about what a user can address in the shader vs. what is managed internally.

Medium priority

Edge and node shader getters share ~80% of their body (frontend.ts L516–860)
Both edgeShaderGetter and nodeShaderGetter call parameterizedEmitterDependentShaderGetter with identical config (parameters, fallback, extraParameters, shaderError), then inside defineShader:

  • Same preamble: defineCommonShader, defineAttributeAccess, defineDynamicSegmentAppearance
  • Same three-way fragment color branch (dynamic appearance / legacy uniform / per-vertex attribute)
  • Same segmentColor() definition logic
  • Verbatim identical user attribute varying loop at the end

The only genuine differences are the primitive-specific parts: edge uses defineLineShader + paired index vertex logic; node uses defineCircleShader + gl_InstanceID + selected-node outline. The shared preamble, color branch, and user attribute loop could be extracted into helpers — e.g. defineShaderPreamble(), defineSegmentColorFragment(), defineUserAttributeVaryings() — so each getter only contains the primitive-specific parts.

attach() bodies are identical (frontend.ts ~L3229, ~L3500)
Both view classes have ~30 lines of identical attach() logic; the only difference is "3d" vs "2d" passed to getSources(). Same situation as updateMouseState was before this PR — can become a shared free function attachSpatiallyIndexedSkeletonLayer(base, self, attachment, view: "2d" | "3d").

Low priority / cleanup

  • data: any in updateMouseState — Both view classes still have data: any; should be unknown to match the direction of removing casts.
  • getValueAt unused param conventionposition should be _position to match the _pickedValue convention already in the same methods.
  • registerDisposer(base.redrawNeeded...) placement — Registered near the top of the 3d constructor, at the bottom of the 2d constructor; should be consistent.
  • PackedSkeletonGeometry.lod field (~L152) — Unused, should be deleted.
  • Scratch buffer shrinking (overlay_geometry.ts) — gpuScratchBuffer grows monotonically; needs a trim/clear strategy under sustained use.

seankmartin added 22 commits May 7, 2026 16:02
Considering to have another pass and also updating the overlay pass. Having what's common extracted is useful for this
this is in case multiple places, e.g. panels, all ask to update the chunk
This refactor aims to remove some casting to types and instead infer the types. And also aims to change the meaning of the interfaces. The interfaces previously often indicated where they were used, as opposed to what they represented. As an example SkeletonChunkInterface is a bit confusing, what is it an interface for. It's not really for a SkeletonChunk, as it's not complete enough.
The overlay memory had a bit of churn in how often we were creating memory to then
immediately send it to the GPU and recreate all that memory again on the next build.
Instead we build a monotically increasing scratch buffer. This buffer can be reused
across sources also because it's not running in parallel. We later might want to look at
removing elements from the scratch, but as a first step it should help.

For uploading the textures, we packed them, then unpacked them. So introduces new
utility to upload these unpacked versions to avoid this.

For rebuilding the overlay, we don't keep "selected" as part of the cache key anymore.
So now we just cache the geoemtry, and handle the selection updating as a separate
update to the geometry.
also revert back to master branch style on these
Replace the loose `dynamicSegmentAppearance` boolean on `SkeletonShaderContext`
with a typed `SkeletonShaderParameters` interface passed as `extraParameters` to
`parameterizedEmitterDependentShaderGetter`. `hasSegmentStatedColors` and
`hasSegmentDefaultColor` now participate in shader memoization rather than being
toggled via runtime uniforms (`uUseSegmentStatedColors`, `uUseSegmentDefaultColor`),
so shaders are only recompiled when color state changes and the branches are resolved
at compile time.

 GPU hash tables for visible segments, excluded segments, and stated colors are now
 cached on `RenderHelper` and recreated only when the underlying hash table changes,
 matching the pattern in `segmentation_render_layer.ts` and fixing repeated GPU
 allocations on every draw call. `RenderHelper.disposed()` is extended to cover all
 three tables (previously only stated colors were disposed), and
 `endLayerSeenTextureUnits`
 is promoted to a reused field to avoid a per-call Set allocation.

 `SkeletonLayer` explicitly implements `SkeletonShaderContext` with a static
 `skeletonShaderParameters` value (all false), since non-spatial skeletons resolve
 color upstream and pass it via uniforms.
these were posing a bit like the custom user attrs, but they are not. Could use a
further refactor in the future along with the seg ID fix, but another card about that
we made a general change which doesn't need to be in place. For our own attrs, we know
the type and don't need to handle them in the general loop. For user types, they are
fixed to being float only
@seankmartin seankmartin marked this pull request as ready for review May 8, 2026 16:27
@seankmartin seankmartin requested a review from afonsobspinto May 8, 2026 16:27
@afonsobspinto afonsobspinto merged commit 3915ef8 into feature/edit-mode May 8, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants