diff --git a/CHANGELOG.md b/CHANGELOG.md index e0697ded6..b8a6e7ff4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,57 @@ All notable changes to Stability Matrix will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning 2.0](https://semver.org/spec/v2.0.0.html). +## v2.16.2 +> Full technical notes for this release: [docs.lykos.ai/stability-matrix/release-notes/2.16.2](https://docs.lykos.ai/stability-matrix/release-notes/2.16.2) +### Added +- 📚 **New documentation site** at [docs.lykos.ai](https://docs.lykos.ai/stability-matrix/) — getting started and installation guides, package manager and Inference walkthroughs, environment variable and advanced configuration references, a terminology glossary, and troubleshooting for common issues. Written by @NeuralFault! +- Added **OneTrainer** to the native **Windows ROCm (AMD GPU)** helper — new OneTrainer installs on supported AMD hardware get the ROCm PyTorch build, ROCm-aware bitsandbytes and triton dependencies, and the right launch environment applied automatically. New OneTrainer installs also default to Python 3.12 on all platforms - thanks to @NeuralFault! +### Changed +- **Generate now auto-resumes after launching ComfyUI.** If you press Generate in Inference while ComfyUI isn't connected and choose to launch it from the connection prompt, the queued generation now waits for startup and runs on its own once connected - no need to press Generate a second time +- Updated the **Windows ROCm helper**'s bundled bitsandbytes wheel to a build compatible with ROCm 7.13–7.15, so it keeps working as AMD's ROCm Technical Preview builds update - thanks to @NeuralFault! +- CivitAI downloads now pick their destination folder from the file's declared type (**Diffusion Model**/**UNet** → DiffusionModels, **Text Encoder** → TextEncoders, **CLIP Vision** → ClipVision, **ControlNet** → ControlNet, **Upscaler** → upscalers) instead of guessing from the model's name. Name-based guessing remains only for files typed plain "Model", and now recognizes **Krea 2** checkpoints as UNet-only +### Fixed +- Fixed a class of random crashes in the Inference **mask editor** and **image annotation editor** — undo/redo, layer operations, exporting, or closing the editor could free graphics resources that were still being drawn with, occasionally crashing mid-stroke or while saving. Canvas rendering has been restructured so this can't happen +- Fixed **pen pressure** re-widening the whole stroke instead of following the pen while drawing, and mouse-drawn strokes coming back ~25% thicker after saving and reopening a project. Existing project files load exactly as before +- Fixed the mask editor and image annotation editor leaking graphics memory — the paint canvas wasn't released on close, and paint-bucket fills were never freed +- Fixed fast brush strokes occasionally failing with a "collection was modified" error while the stroke was still being drawn +- Fixed opening older projects whose masks contained stroke points outside the canvas failing with an overflow error +- Fixed a potential crash when the paint canvas rendered before its size was set +- Fixed CivitAI models showing an empty Files section with no download links when their files use CivitAI's newer type labels (e.g. **Krea 2 Turbo** and **Z-Image**, typed **Diffusion Model** or **Text Encoder**). All current CivitAI file types are now recognized across the browser, details page, version dialog, bulk download, and installed/update detection +- Fixed CivitAI "Download with Stability Matrix" links saving UNet-only checkpoints (Flux, Wan Video, Hunyuan, Krea 2) into the **StableDiffusion** folder — external links now use the same destination logic as the in-app browser +- Fixed [#1666](https://github.com/LykosAI/StabilityMatrix/issues/1666) - AppImage builds creating a broken `.desktop` entry that never appeared in the application launcher and reverted manual edits on the next launch. AppImage runs now install a proper launcher entry with the app icon, and the running window shows the correct icon in the dock/taskbar (deb/rpm/flatpak installs keep their package-managed entries) +- Fixed `stabilitymatrix://` deep links (e.g. CivitAI "Download with Stability Matrix" buttons) being ignored on Linux AppImage builds +- Fixed [#1668](https://github.com/LykosAI/StabilityMatrix/issues/1668) - the **Output Browser** crashing on open when an output folder contained a broken junction or symlink (a folder whose target no longer exists); those entries are now skipped instead of taking the page down +- Fixed [#1681](https://github.com/LykosAI/StabilityMatrix/issues/1681) - Inference generation with **FaceDetailer** failing instantly with "An item with the same key has already been added" when two installed custom node folders share the same git remote +- Fixed [#1679](https://github.com/LykosAI/StabilityMatrix/issues/1679) - model details pages never showing the **Installed** label, delete button, or version checkmark for files whose type displays as "Unknown". Installed detection now goes by file hash, so it works no matter what type CivitAI reports +- Fixed [#1667](https://github.com/LykosAI/StabilityMatrix/issues/1667) - importing an existing package folder silently pre-selecting **Forge** as the package type, misclassifying re-imported packages. The type is now auto-detected from the folder's git remote +- Fixed [#1669](https://github.com/LykosAI/StabilityMatrix/issues/1669) - **Stable Diffusion WebUI reForge** installing the CUDA build of PyTorch on Linux AMD systems even with ROCm selected +- Fixed [#1672](https://github.com/LykosAI/StabilityMatrix/issues/1672) - **Image Lab** failing with a generic "ComfyUI rejected the workflow" error when generating with a GGUF model while the **ComfyUI-GGUF** extension isn't installed — it now offers the same one-click **install and restart** prompt as Inference +- Fixed GGUF-quantized **text encoders** failing with a ComfyUI error when selected — they now load through the ComfyUI-GGUF CLIP loaders in all Inference workflows and Image Lab (mixed GGUF + safetensors selections work too), with the install prompt appearing if the extension is missing +- Fixed **Chroma** and other single-encoder models being impossible to configure in Inference UNet workflows — the encoder **Type** dropdown was missing `chroma` and other single-encoder types, and the closest option (`flux`) demanded a second encoder these models don't use. The missing types are now listed, and single-encoder types get a single encoder slot +- Fixed GGUF text encoders in the shared **TextEncoders** folder not appearing in the Inference Text Encoder dropdowns while connected to ComfyUI +- Fixed [#1682](https://github.com/LykosAI/StabilityMatrix/issues/1682) - the Linux **AppImage** failing to start on modern distros (Ubuntu 24.04+, Fedora 40+) that no longer ship `libfuse2`. The AppImage now uses a FUSE3-based runtime, and falls back to extract-and-run when FUSE isn't available at all - thanks to @NeuralFault! +- Fixed the Model Browser failing with "CivitAI can't be reached right now (OK: OK)" — CivitAI recently started returning `null` for some model statistics (download counts, ratings), which broke loading the whole page of results. Missing stats are now read as 0 +- Fixed [#1695](https://github.com/LykosAI/StabilityMatrix/issues/1695) - models whose CivitAI page has since been deleted showing an error dialog on every click, with no way to break the link: + - The model details page now shows locally cached info instead of a dead page when the CivitAI page is gone + - **Next/Previous** on the details page skip over deleted models instead of getting stuck on an error + - New right-click **Disconnect from Source** action on Checkpoint Manager cards severs the link to the deleted page while keeping the local metadata (name, description, thumbnail, trigger words) — after disconnecting, clicking the card selects it like any other local model +- Fixed **OneTrainer** failing to launch with current upstream versions after its UI script was renamed (`train_ui.py` → `train_ui_ctk.py`) - thanks to @NeuralFault! +### Performance +- Dragging an image layer with the **Move** tool in the layered mask editor now re-renders only the dragged layer instead of re-compositing every layer on each pointer move, so dragging stays smooth in multi-layer projects +- Faster color-mask extraction for regional prompting — a 1024×1024 canvas with six regions previously did over six million dictionary lookups per extraction +- Fixed a small native memory leak while drawing with the mouse (a path object per frame was never freed), and removed a redundant full-canvas scan after every paint-bucket fill +- The **Checkpoints** and **Outputs** galleries now load thumbnails at display size instead of full resolution, using far less memory and scrolling much more smoothly with large libraries +- The **CivitAI** and **OpenModelDB** browsers now keep card images in memory, so scrolling back over models you've already seen no longer re-loads them from disk +- Lightened the CivitAI model cards so they render faster while scrolling +### Security +- Bundled **ADetailer** model downloads now point at a fixed Hugging Face revision instead of the repository's moving `main` branch, so an upstream re-upload can't change what you get - thanks to @ungrav! +### Supporters +#### 🌟 Visionaries +There's not much glamour in a release like this one: crashes hunted down in the mask editor, downloads finally landing in the folders they belong in, Linux installs that just start. Our Visionaries are the reason we can hand a whole cycle to that kind of work, and to writing the documentation that now lives inside the app. Thank you **Waterclouds**, **MrMxyzptlk12836**, **Psilocyfer18731**, **bluepopsicle**, **Ibixat**, **Droolguy**, **KalAbaddon**, **LG**, **snotty**, **whudunit**, **cusalapapen1481**, and **moon_milky2843** for backing the unglamorous parts as generously as the exciting ones. And a warm welcome to **tarekk071223**, **CC**, and **SnooSnooEternal**, the newest names on this list; it's genuinely good to have you here. 💛 +#### 🚀 Pioneers +Our Pioneers hold up the other half of this, and what a steady crew you are: **Szir777**, **[USA]TechDude**, **SinthCore**, **Jisuren**, **Tigon**, **jweg79**, **rwx14662**, **Hurbie53**, **ahnhj.al**, **drew.lukas**, **Tuskaruho**, **Cjloha**, **Alligator1907**, **Bitti**, **Ghislain G**, **CommissarGiygas16050**, **qob97515211**, **bastardofbethlehem**, and **Zombop**, thank you for sticking around release after release. A big hello as well to **Silerae**, **joshsciascia72**, and **rad64741317**, who join the Pioneers this time; we're really happy you're with us. Every fix in this changelog has a bit of all of you behind it. 💛 + ## v2.16.1 ### Added - Added **automatic text encoder and VAE selection** to the Inference Model card. Selecting a model now fills any empty encoder slots and the default VAE with the matching local files for the detected workflow, so you don't need to know which files pair with which architecture (e.g. `qwen_3_4b` or `qwen_3_8b` + Flux.2 VAE for Flux.2 Klein, `clip_l` + `t5xxl` for Flux, `qwen_3_06b` + `qwen_image_vae` for Anima). Anything you pick manually is never overridden diff --git a/Jenkinsfile b/Jenkinsfile index 0ac34a468..8cd19deae 100644 --- a/Jenkinsfile +++ b/Jenkinsfile @@ -39,7 +39,7 @@ node("Diligence") { // - pupnet 1.9.1 globally installed sh "rm -rf StabilityMatrix.Avalonia/bin/*" sh "rm -rf StabilityMatrix.Avalonia/obj/*" - sh "/home/jenkins/.dotnet/tools/pupnet -r linux-x64 -c Release --kind appimage --app-version $RELEASE_VERSION --clean" + sh "/home/jenkins/.dotnet/tools/pupnet -r linux-x64 -c Release --kind appimage --app-version ${version} --clean -y" } } } finally { diff --git a/StabilityMatrix.Avalonia/App.axaml.cs b/StabilityMatrix.Avalonia/App.axaml.cs index 79a07ee91..3dde9483c 100644 --- a/StabilityMatrix.Avalonia/App.axaml.cs +++ b/StabilityMatrix.Avalonia/App.axaml.cs @@ -366,6 +366,12 @@ private void Setup() // Setup uri handler for `stabilitymatrix://` protocol Program.UriHandler.RegisterUriScheme(); + // Write a correct .desktop entry for AppImage runs so the app appears in the launcher + if (Compat.IsLinux) + { + LinuxDesktopIntegration.CreateDesktopFile(); + } + // Setup activation protocol handlers (uri handler on macOS) if (Compat.IsMacOS && this.TryGetFeature() is { } activatableLifetime) { diff --git a/StabilityMatrix.Avalonia/Assets/sitecustomize.py b/StabilityMatrix.Avalonia/Assets/sitecustomize.py index 7f9278f8d..1394220ad 100644 --- a/StabilityMatrix.Avalonia/Assets/sitecustomize.py +++ b/StabilityMatrix.Avalonia/Assets/sitecustomize.py @@ -65,8 +65,8 @@ def is_terminal(self) -> bool: except ImportError: pass except Exception as e: - print("[sitecustomize error]:", e) - + print("[sitecustomize error]:", e, file=sys.stderr) + try: from pip._vendor.rich import console @@ -79,19 +79,19 @@ def is_terminal(self) -> bool: except ImportError: pass except Exception as e: - print("[sitecustomize error]:", e) + print("[sitecustomize error]:", e, file=sys.stderr) # Patch tqdm to use stdout instead of stderr def _patch_tqdm(): try: import sys from tqdm import std - + sys.stderr = sys.stdout except ImportError: pass except Exception as e: - print("[sitecustomize error]:", e) + print("[sitecustomize error]:", e, file=sys.stderr) # Run startup customizations. Each is isolated so that a failure in one (or an # unusual host environment, e.g. an interpreter probe with no real stdio) can @@ -101,7 +101,7 @@ def _run_safely(func): func() except Exception as e: try: - print("[sitecustomize error]:", e) + print("[sitecustomize error]:", e, file=sys.stderr) except Exception: pass diff --git a/StabilityMatrix.Avalonia/Controls/ImageLoaders.cs b/StabilityMatrix.Avalonia/Controls/ImageLoaders.cs index 804c71fe6..37aa9c8d0 100644 --- a/StabilityMatrix.Avalonia/Controls/ImageLoaders.cs +++ b/StabilityMatrix.Avalonia/Controls/ImageLoaders.cs @@ -15,35 +15,35 @@ internal static class ImageLoaders { private static string BaseFileCachePath => Path.Combine(Path.GetTempPath(), "StabilityMatrix", "Cache"); - private static readonly Lazy OutputsPageImageCacheLazy = - new( - () => new MemoryImageCache { MaxMemoryCacheCount = 64 }, - LazyThreadSafetyMode.ExecutionAndPublication - ); + private static readonly Lazy OutputsPageImageCacheLazy = new( + () => new MemoryImageCache { MaxMemoryCacheCount = 64 }, + LazyThreadSafetyMode.ExecutionAndPublication + ); public static IImageCache OutputsPageImageCache => OutputsPageImageCacheLazy.Value; - private static readonly Lazy OpenModelDbImageCacheLazy = - new( - () => - new ImageCache( - new CacheOptions + private static readonly Lazy OpenModelDbImageCacheLazy = new( + () => + new ImageCache( + new CacheOptions + { + BaseCachePath = BaseFileCachePath, + CacheFolderName = "OpenModelDbImageCache", + CacheDuration = TimeSpan.FromDays(1), + // Keep decoded bitmaps in memory so scrolling doesn't re-decode from disk each time. + MaxMemoryCacheCount = 96, + HttpClient = new HttpClient(NetCache.Background) { - BaseCachePath = BaseFileCachePath, - CacheFolderName = "OpenModelDbImageCache", - CacheDuration = TimeSpan.FromDays(1), - HttpClient = new HttpClient(NetCache.Background) + DefaultRequestHeaders = { - DefaultRequestHeaders = - { - UserAgent = { new ProductInfoHeaderValue("StabilityMatrix", "2.0") }, - Referrer = new Uri("https://openmodelsdb.info/"), - } - } - } - ), - LazyThreadSafetyMode.ExecutionAndPublication - ); + UserAgent = { new ProductInfoHeaderValue("StabilityMatrix", "2.0") }, + Referrer = new Uri("https://openmodelsdb.info/"), + }, + }, + } + ), + LazyThreadSafetyMode.ExecutionAndPublication + ); public static IImageCache OpenModelDbImageCache => OpenModelDbImageCacheLazy.Value; } diff --git a/StabilityMatrix.Avalonia/Controls/Models/LiveStroke.cs b/StabilityMatrix.Avalonia/Controls/Models/LiveStroke.cs new file mode 100644 index 000000000..14dfeaaeb --- /dev/null +++ b/StabilityMatrix.Avalonia/Controls/Models/LiveStroke.cs @@ -0,0 +1,70 @@ +using System; +using System.Runtime.InteropServices; +using System.Threading; + +namespace StabilityMatrix.Avalonia.Controls.Models; + +/// +/// An in-progress stroke being drawn by the user. Unlike a finalized , +/// a live stroke is written by the UI thread (pointer events appending points) while the +/// compositor render thread concurrently reads it every frame. +/// +/// +/// Thread safety comes from copy-on-append publication rather than locks: points live in an +/// array that is never mutated after it becomes visible to readers. Appending copies the +/// current array into a larger one and publishes it with a single +/// , so a reader always observes a fully written, +/// stable snapshot — there is no window where a resize or in-place write can tear a read. +/// Writes must come from a single thread (the UI thread); reads may come from any thread. +/// +public sealed class LiveStroke +{ + private PenPoint[] points = []; + + /// + /// Immutable stroke metadata (color, radius, feathering, shape bounds, etc.). + /// The template's own list is unused and stays empty; + /// live points are tracked by this class instead. + /// + public required PenPath Template { get; init; } + + /// + /// Returns a stable snapshot of the points recorded so far. The returned array is + /// never mutated — callers may iterate it freely on any thread. + /// + public PenPoint[] GetPointsSnapshot() => Volatile.Read(ref points); + + /// + /// Appends points to the stroke. Must only be called from the single writer (UI) thread. + /// + public void AddPoints(ReadOnlySpan newPoints) + { + if (newPoints.IsEmpty) + return; + + // Single-writer: no other thread mutates `points`, so a plain read is sufficient here. + var current = points; + var next = new PenPoint[current.Length + newPoints.Length]; + current.AsSpan().CopyTo(next); + newPoints.CopyTo(next.AsSpan(current.Length)); + + // Release-publish: the array contents above are guaranteed visible to any thread + // that observes the new reference. + Volatile.Write(ref points, next); + } + + /// + /// Snapshots this live stroke into a finalized, fully independent . + /// + public PenPath ToPenPath() => Template with { Points = [.. GetPointsSnapshot()] }; + + /// + /// Creates a live stroke from a previously serialized . + /// + public static LiveStroke FromPenPath(PenPath path) + { + var stroke = new LiveStroke { Template = path with { Points = [] } }; + stroke.AddPoints(CollectionsMarshal.AsSpan(path.Points)); + return stroke; + } +} diff --git a/StabilityMatrix.Avalonia/Controls/Models/PenPath.cs b/StabilityMatrix.Avalonia/Controls/Models/PenPath.cs index 877ded502..494880545 100644 --- a/StabilityMatrix.Avalonia/Controls/Models/PenPath.cs +++ b/StabilityMatrix.Avalonia/Controls/Models/PenPath.cs @@ -261,7 +261,11 @@ public SKPath ToSKPath() { var skPath = new SKPath(); - if (Points.Count <= 0) + // In-progress strokes live in LiveStroke (never in a PenPath), so by the time a PenPath + // exists its Points list is frozen and safe to iterate from any thread. + var count = Points.Count; + + if (count <= 0) { return skPath; } @@ -270,7 +274,7 @@ public SKPath ToSKPath() skPath.MoveTo(Points[0].X, Points[0].Y); // Add the rest of the points - for (var i = 1; i < Points.Count; i++) + for (var i = 1; i < count; i++) { skPath.LineTo(Points[i].X, Points[i].Y); } @@ -313,14 +317,20 @@ public float GetEffectiveRadius() BitConverter.TryWriteBytes(buffer.AsSpan(offset), points.Count); offset += 4; - // Write each point as 3 floats + // Write each point as 3 floats. Mouse points (no pressure, not a pen) are written with + // a -1 pressure sentinel so they round-trip as mouse points instead of being promoted + // to full-pressure pen points on reload (which rendered them ~25% thicker). Old readers + // treat out-of-range pressure as null, matching their existing reload behavior. foreach (var point in points) { BitConverter.TryWriteBytes(buffer.AsSpan(offset), (float)point.X); offset += 4; BitConverter.TryWriteBytes(buffer.AsSpan(offset), (float)point.Y); offset += 4; - BitConverter.TryWriteBytes(buffer.AsSpan(offset), (float)(point.Pressure ?? 1.0)); + BitConverter.TryWriteBytes( + buffer.AsSpan(offset), + (float)(point.Pressure ?? (point.IsPen ? 1.0 : -1.0)) + ); offset += 4; } @@ -382,13 +392,12 @@ public float GetEffectiveRadius() var pressure = BitConverter.ToSingle(buffer, offset); offset += 4; - points.Add( - new PenPoint(x, y) - { - Pressure = pressure >= 0 && pressure <= 1 ? pressure : null, - IsPen = true, // Mark as pen point so it renders correctly - } - ); + // Pressure in [0, 1] means a pen point; anything else (notably the -1 mouse + // sentinel written by CompressPointsPublic, or legacy garbage) is a mouse point, + // so mouse strokes keep their plain polyline rendering across save/load. + var isPen = pressure >= 0 && pressure <= 1; + + points.Add(new PenPoint(x, y) { Pressure = isPen ? pressure : null, IsPen = isPen }); } return points; diff --git a/StabilityMatrix.Avalonia/Controls/Models/PenPoint.cs b/StabilityMatrix.Avalonia/Controls/Models/PenPoint.cs index de6ad27bc..b82e96c90 100644 --- a/StabilityMatrix.Avalonia/Controls/Models/PenPoint.cs +++ b/StabilityMatrix.Avalonia/Controls/Models/PenPoint.cs @@ -115,11 +115,13 @@ public override void Write(Utf8JsonWriter writer, PenPoint value, JsonSerializer [JsonConverter(typeof(PenPointJsonConverter))] public readonly record struct PenPoint(ulong X, ulong Y) { + // Clamp negatives to 0: Convert.ToUInt64 throws OverflowException on negative + // coordinates, which can occur in legacy serialized paths or off-canvas points public PenPoint(double x, double y) - : this(Convert.ToUInt64(x), Convert.ToUInt64(y)) { } + : this(Convert.ToUInt64(Math.Max(0, x)), Convert.ToUInt64(Math.Max(0, y))) { } public PenPoint(SKPoint skPoint) - : this(Convert.ToUInt64(skPoint.X), Convert.ToUInt64(skPoint.Y)) { } + : this(Convert.ToUInt64(Math.Max(0, skPoint.X)), Convert.ToUInt64(Math.Max(0, skPoint.Y))) { } /// /// Radius of the point. diff --git a/StabilityMatrix.Avalonia/Controls/Painting/PaintCanvas.axaml.cs b/StabilityMatrix.Avalonia/Controls/Painting/PaintCanvas.axaml.cs index a902bbff8..661ffd300 100644 --- a/StabilityMatrix.Avalonia/Controls/Painting/PaintCanvas.axaml.cs +++ b/StabilityMatrix.Avalonia/Controls/Painting/PaintCanvas.axaml.cs @@ -26,7 +26,7 @@ namespace StabilityMatrix.Avalonia.Controls; public class PaintCanvas : TemplatedControlBase { - private ConcurrentDictionary TemporaryPaths => ViewModel!.TemporaryPaths; + private ConcurrentDictionary TemporaryPaths => ViewModel!.TemporaryPaths; private ImmutableList Paths { @@ -129,19 +129,19 @@ protected override void OnPropertyChanged(AvaloniaPropertyChangedEventArgs chang { base.OnPropertyChanged(change); - if (change.Property == IsEnabledProperty) + if (change.Property == IsEnabledProperty && ViewModel is { } vm) { var newIsEnabled = change.GetNewValue(); - if (!newIsEnabled && ViewModel is { } vm) + if (!newIsEnabled) { vm.IsPenDown = false; } - // On any enabled change, flush temporary paths + // On any enabled change, flush in-progress strokes into finalized paths if (!TemporaryPaths.IsEmpty) { - Paths = Paths.AddRange(TemporaryPaths.Values); + Paths = Paths.AddRange(TemporaryPaths.Values.Select(stroke => stroke.ToPenPath())); TemporaryPaths.Clear(); } } @@ -240,9 +240,10 @@ private void HandlePointerEvent(PointerEventArgs e) vm.IsPenDown = false; } - if (!vm.IsShapeTool && TemporaryPaths.TryGetValue(e.Pointer.Id, out var path)) + if (!vm.IsShapeTool && TemporaryPaths.TryGetValue(e.Pointer.Id, out var stroke)) { - Paths = Paths.Add(path); + // Snapshot the live stroke into a fully independent finalized path + Paths = Paths.Add(stroke.ToPenPath()); vm.ClearRedoStack(); // New path added, clear redo history } @@ -334,27 +335,32 @@ private void HandlePointerMoved(PointerEventArgs e) viewModel.CurrentPenPressure = points.FirstOrDefault().Properties.Pressure; - // Get or create a temp path - if (!TemporaryPaths.TryGetValue(e.Pointer.Id, out var penPath)) + // Get or create a live stroke for this pointer + if (!TemporaryPaths.TryGetValue(e.Pointer.Id, out var stroke)) { - penPath = new PenPath + stroke = new LiveStroke { - FillColor = viewModel.PaintBrushSKColor.WithAlpha((byte)(viewModel.PaintBrushAlpha * 255)), - IsErase = viewModel.SelectedTool == PaintCanvasTool.Eraser, - Radius = (float)viewModel.PaintBrushSize, - Feathering = (float)viewModel.PaintBrushFeathering, + Template = new PenPath + { + FillColor = viewModel.PaintBrushSKColor.WithAlpha( + (byte)(viewModel.PaintBrushAlpha * 255) + ), + IsErase = viewModel.SelectedTool == PaintCanvasTool.Eraser, + Radius = (float)viewModel.PaintBrushSize, + Feathering = (float)viewModel.PaintBrushFeathering, + }, }; - TemporaryPaths[e.Pointer.Id] = penPath; + TemporaryPaths[e.Pointer.Id] = stroke; } - // Add line for path - // var cursorPosition = e.GetPosition(MainCanvas); - // penPath.Path.LineTo(cursorPosition.ToSKPoint()); - // Get bounds for discarding invalid points var canvasBounds = new Rect(0, 0, MainCanvas?.Bounds.Width ?? 0, MainCanvas?.Bounds.Height ?? 0); - // Add points + // Collect valid points, then publish them to the stroke in one batch so the render + // thread sees a single stable snapshot per pointer event + Span newPoints = points.Count <= 64 ? stackalloc PenPoint[64] : new PenPoint[points.Count]; + var newPointCount = 0; + foreach (var point in points) { // Discard invalid points @@ -363,15 +369,15 @@ private void HandlePointerMoved(PointerEventArgs e) continue; } - var penPoint = new PenPoint(point.Position.X, point.Position.Y) + newPoints[newPointCount++] = new PenPoint(point.Position.X, point.Position.Y) { Pressure = point.Pointer.Type == PointerType.Mouse ? null : point.Properties.Pressure, Radius = viewModel.PaintBrushSize, IsPen = point.Pointer.Type == PointerType.Pen, }; - - penPath.Points.Add(penPoint); } + + stroke.AddPoints(newPoints[..newPointCount]); } /// @@ -559,9 +565,12 @@ or PaintCanvasTool.PaintBucket { if (lastCanvasCursorTool != selectedTool) { - lastCanvasCursor?.Dispose(); + // Assign the new cursor before disposing the old one, which may still be active + var oldCursor = lastCanvasCursor; lastCanvasCursor = new Cursor(StandardCursorType.Cross); lastCanvasCursorTool = selectedTool; + canvas.Cursor = lastCanvasCursor; + oldCursor?.Dispose(); } canvas.Cursor = lastCanvasCursor; return; @@ -572,9 +581,11 @@ or PaintCanvasTool.PaintBucket { if (lastCanvasCursorTool != selectedTool) { - lastCanvasCursor?.Dispose(); + var oldCursor = lastCanvasCursor; lastCanvasCursor = new Cursor(StandardCursorType.SizeAll); lastCanvasCursorTool = selectedTool; + canvas.Cursor = lastCanvasCursor; + oldCursor?.Dispose(); } canvas.Cursor = lastCanvasCursor; return; @@ -644,7 +655,7 @@ private void MainCanvas_OnPointerExited(object? sender, PointerEventArgs e) { if (sender is SkiaCustomCanvas canvas) { - canvas.Cursor = new Cursor(StandardCursorType.Arrow); + canvas.Cursor = Cursor.Default; } } diff --git a/StabilityMatrix.Avalonia/Controls/SelectableImageCard/SelectableImageButton.axaml b/StabilityMatrix.Avalonia/Controls/SelectableImageCard/SelectableImageButton.axaml index ea08be458..a4bd30e54 100644 --- a/StabilityMatrix.Avalonia/Controls/SelectableImageCard/SelectableImageButton.axaml +++ b/StabilityMatrix.Avalonia/Controls/SelectableImageCard/SelectableImageButton.axaml @@ -42,6 +42,7 @@ diff --git a/StabilityMatrix.Avalonia/Controls/SelectableImageCard/SelectableImageButton.cs b/StabilityMatrix.Avalonia/Controls/SelectableImageCard/SelectableImageButton.cs index 3921f17b5..9f4733f59 100644 --- a/StabilityMatrix.Avalonia/Controls/SelectableImageCard/SelectableImageButton.cs +++ b/StabilityMatrix.Avalonia/Controls/SelectableImageCard/SelectableImageButton.cs @@ -30,6 +30,11 @@ public class SelectableImageButton : Button bool >("IsVideo", false); + public static readonly StyledProperty DecodeWidthProperty = AvaloniaProperty.Register< + SelectableImageButton, + int + >("DecodeWidth"); + static SelectableImageButton() { AffectsRender(ImageWidthProperty, ImageHeightProperty); @@ -65,4 +70,14 @@ public bool IsVideo get => GetValue(IsVideoProperty); set => SetValue(IsVideoProperty, value); } + + /// + /// Width (px) to decode the image at, or 0 for source resolution. See + /// . + /// + public int DecodeWidth + { + get => GetValue(DecodeWidthProperty); + set => SetValue(DecodeWidthProperty, value); + } } diff --git a/StabilityMatrix.Avalonia/Controls/VendorLabs/AsyncImage/BetterAsyncImage.Properties.cs b/StabilityMatrix.Avalonia/Controls/VendorLabs/AsyncImage/BetterAsyncImage.Properties.cs index 015ff8fd6..962f4d4c4 100644 --- a/StabilityMatrix.Avalonia/Controls/VendorLabs/AsyncImage/BetterAsyncImage.Properties.cs +++ b/StabilityMatrix.Avalonia/Controls/VendorLabs/AsyncImage/BetterAsyncImage.Properties.cs @@ -26,6 +26,14 @@ public partial class BetterAsyncImage Uri? >(nameof(Source)); + /// + /// Defines the property. + /// + public static readonly StyledProperty DecodeWidthProperty = AvaloniaProperty.Register< + BetterAsyncImage, + int + >(nameof(DecodeWidth)); + /// /// Defines the property. /// @@ -90,6 +98,17 @@ public Uri? Source set => SetValue(SourceProperty, value); } + /// + /// Gets or sets the width (in pixels) to decode the image at. When greater than 0, images wider than this + /// are downscaled at decode time, which avoids keeping full-resolution bitmaps in memory for thumbnail-sized + /// displays. A value of 0 (the default) decodes at the source resolution. + /// + public int DecodeWidth + { + get => GetValue(DecodeWidthProperty); + set => SetValue(DecodeWidthProperty, value); + } + /// /// Gets or sets a value controlling how the image will be stretched. /// diff --git a/StabilityMatrix.Avalonia/Controls/VendorLabs/AsyncImage/BetterAsyncImage.cs b/StabilityMatrix.Avalonia/Controls/VendorLabs/AsyncImage/BetterAsyncImage.cs index 218a50ac9..fcf717180 100644 --- a/StabilityMatrix.Avalonia/Controls/VendorLabs/AsyncImage/BetterAsyncImage.cs +++ b/StabilityMatrix.Avalonia/Controls/VendorLabs/AsyncImage/BetterAsyncImage.cs @@ -13,6 +13,7 @@ using Avalonia.Platform; using Avalonia.Threading; using StabilityMatrix.Avalonia.Controls.VendorLabs.Cache; +using StabilityMatrix.Avalonia.Extensions; namespace StabilityMatrix.Avalonia.Controls.VendorLabs; @@ -121,6 +122,9 @@ private async void SetSource(object? source) var uri = Source; + // Read styled property on the UI thread before dispatching to the background decode. + var decodeWidth = DecodeWidth; + if (!uri.IsAbsoluteUri) { State = AsyncImageState.Failed; @@ -145,17 +149,19 @@ private async void SetSource(object? source) if (uri.Scheme is "http" or "https") { - return await LoadImageAsync(uri, newTokenSource.Token); + return await LoadImageAsync(uri, decodeWidth, newTokenSource.Token); } if (uri.Scheme == "file" && File.Exists(uri.LocalPath)) { if (!IsCacheEnabled) { - return new Bitmap(uri.LocalPath); + return decodeWidth > 0 + ? SkiaExtensions.DecodeFileToAvaloniaImageScaled(uri.LocalPath, decodeWidth) + : new Bitmap(uri.LocalPath); } - return await LoadImageAsync(uri, newTokenSource.Token); + return await LoadImageAsync(uri, decodeWidth, newTokenSource.Token); } if (uri.Scheme == "avares") @@ -194,6 +200,11 @@ private async void SetSource(object? source) private void AttachSource(IImage? image, CancellationToken cancellationToken) { + // NOTE: Do NOT dispose the previous ImagePart.Source here. Our AvaloniaImage is an + // ICustomDrawOperation that holds the SKBitmap and is rendered asynchronously on the compositor + // render thread; disposing it on source swap frees the bitmap mid-frame and causes a native + // use-after-free crash (sk_image_new_from_bitmap) during fast scrolling. Bitmap lifetime is left + // to the GC. A render-safe disposal scheme would be needed to reclaim native memory sooner. if (ImagePart != null) { ImagePart.Source = image; @@ -222,7 +233,7 @@ private void AttachSource(IImage? image, CancellationToken cancellationToken) } } - private async Task LoadImageAsync(Uri url, CancellationToken cancellationToken) + private async Task LoadImageAsync(Uri url, int decodeWidth, CancellationToken cancellationToken) { // Get specific cache for this control or use the default one var cache = InstanceImageCache; @@ -233,10 +244,10 @@ private void AttachSource(IImage? image, CancellationToken cancellationToken) if (IsCacheEnabled) { - return await cache.GetWithCacheAsync(url, cancellationToken); + return await cache.GetWithCacheAsync(url, decodeWidth, cancellationToken); } - return await cache.GetAsync(url, cancellationToken); + return await cache.GetAsync(url, decodeWidth, cancellationToken); } protected override void OnPropertyChanged(AvaloniaPropertyChangedEventArgs change) diff --git a/StabilityMatrix.Avalonia/Controls/VendorLabs/AsyncImage/BetterAsyncImageCacheProvider.cs b/StabilityMatrix.Avalonia/Controls/VendorLabs/AsyncImage/BetterAsyncImageCacheProvider.cs index 53a96836c..f41ae0d85 100644 --- a/StabilityMatrix.Avalonia/Controls/VendorLabs/AsyncImage/BetterAsyncImageCacheProvider.cs +++ b/StabilityMatrix.Avalonia/Controls/VendorLabs/AsyncImage/BetterAsyncImageCacheProvider.cs @@ -9,24 +9,26 @@ namespace StabilityMatrix.Avalonia.Controls.VendorLabs; public static class BetterAsyncImageCacheProvider { - private static readonly Lazy DefaultCacheLazy = - new( - () => - new ImageCache( - new CacheOptions - { - // ReSharper disable twice LocalizableElement - BaseCachePath = - Assembly.GetExecutingAssembly().FullName is { } assemblyName - && !string.IsNullOrEmpty(assemblyName) - ? Path.Combine(Path.GetTempPath(), assemblyName, "Cache") - : Path.Combine(Path.GetTempPath(), "Cache"), - CacheDuration = TimeSpan.FromDays(1), - HttpMessageHandler = NetCache.UserInitiated - } - ), - LazyThreadSafetyMode.ExecutionAndPublication - ); + private static readonly Lazy DefaultCacheLazy = new( + () => + new ImageCache( + new CacheOptions + { + // ReSharper disable twice LocalizableElement + BaseCachePath = + Assembly.GetExecutingAssembly().FullName is { } assemblyName + && !string.IsNullOrEmpty(assemblyName) + ? Path.Combine(Path.GetTempPath(), assemblyName, "Cache") + : Path.Combine(Path.GetTempPath(), "Cache"), + CacheDuration = TimeSpan.FromDays(1), + // Retain decoded bitmaps in memory so scrolling (which recycles item containers and + // re-requests images) doesn't re-decode from disk every time. 0 = disabled. + MaxMemoryCacheCount = 96, + HttpMessageHandler = NetCache.UserInitiated, + } + ), + LazyThreadSafetyMode.ExecutionAndPublication + ); private static IImageCache? _defaultCache; diff --git a/StabilityMatrix.Avalonia/Controls/VendorLabs/Cache/IImageCache.cs b/StabilityMatrix.Avalonia/Controls/VendorLabs/Cache/IImageCache.cs index ed727a570..28ccedaa2 100644 --- a/StabilityMatrix.Avalonia/Controls/VendorLabs/Cache/IImageCache.cs +++ b/StabilityMatrix.Avalonia/Controls/VendorLabs/Cache/IImageCache.cs @@ -19,18 +19,24 @@ public interface IImageCache /// Retrieves item represented by Uri locally or by downloading, does not cache the item. /// /// Uri of the item. + /// Width (px) to downscale the image to at decode time, or 0 for source resolution. /// instance of /// an instance of Generic type - Task GetAsync(Uri uri, CancellationToken cancellationToken = default); + Task GetAsync(Uri uri, int decodeWidth = 0, CancellationToken cancellationToken = default); /// /// Retrieves item represented by Uri from the cache. /// If the item is not found in the cache, it downloads and saves before returning it to the caller. /// /// Uri of the item. + /// Width (px) to downscale the image to at decode time, or 0 for source resolution. /// instance of /// an instance of Generic type - Task GetWithCacheAsync(Uri uri, CancellationToken cancellationToken = default); + Task GetWithCacheAsync( + Uri uri, + int decodeWidth = 0, + CancellationToken cancellationToken = default + ); int ClearMemoryCache(); diff --git a/StabilityMatrix.Avalonia/Controls/VendorLabs/Cache/ImageCache.cs b/StabilityMatrix.Avalonia/Controls/VendorLabs/Cache/ImageCache.cs index f60dddb0c..1dd62419d 100644 --- a/StabilityMatrix.Avalonia/Controls/VendorLabs/Cache/ImageCache.cs +++ b/StabilityMatrix.Avalonia/Controls/VendorLabs/Cache/ImageCache.cs @@ -4,6 +4,8 @@ using System.Threading.Tasks; using Avalonia.Media; using Avalonia.Media.Imaging; +using SkiaSharp; +using StabilityMatrix.Avalonia.Extensions; namespace StabilityMatrix.Avalonia.Controls.VendorLabs.Cache; @@ -12,6 +14,11 @@ namespace StabilityMatrix.Avalonia.Controls.VendorLabs.Cache; /// internal class ImageCache(CacheOptions? options = null) : CacheBase(options), IImageCache { + // Carries the requested decode width into ConvertFromAsync without threading it through the generic + // CacheBase (which can't pass extra arguments to the decode step). The cache key itself is made + // width-aware via WithDecodeWidthKey so different sizes of the same Uri never collide. + private static readonly AsyncLocal CurrentDecodeWidth = new(); + /// /// Creates a bitmap from a stream /// @@ -24,7 +31,45 @@ protected override async Task ConvertFromAsync(Stream stream) throw new FileNotFoundException(); } - return new Bitmap(stream); + return DecodeBitmap(stream, CurrentDecodeWidth.Value); + } + + /// + /// Decodes a stream into a bitmap, downscaling to (px) if it is wider. + /// + private static Bitmap DecodeBitmap(Stream stream, int decodeWidth) + { + if (decodeWidth <= 0) + { + return new Bitmap(stream); + } + + var original = stream.ToSKBitmap(); + if (original is null) + { + stream.Position = 0; + return new Bitmap(stream); + } + + using (original) + { + if (original.Width <= decodeWidth) + { + return original.ToAvaloniaBitmap(); + } + + var targetHeight = Math.Max( + 1, + (int)Math.Round(original.Height * ((double)decodeWidth / original.Width)) + ); + + using var resized = original.Resize( + new SKImageInfo(decodeWidth, targetHeight), + new SKSamplingOptions(SKFilterMode.Linear, SKMipmapMode.Linear) + ); + + return (resized ?? original).ToAvaloniaBitmap(); + } } /// @@ -34,10 +79,8 @@ protected override async Task ConvertFromAsync(Stream stream) /// awaitable task protected override async Task ConvertFromAsync(string baseFile) { - using (var stream = File.OpenRead(baseFile)) - { - return await ConvertFromAsync(stream).ConfigureAwait(false); - } + await using var stream = File.OpenRead(baseFile); + return await ConvertFromAsync(stream).ConfigureAwait(false); } /// @@ -70,14 +113,37 @@ public Task PreCacheAsync(Uri uri, CancellationToken cancellationToken = default return PreCacheAsync(uri, true, true, cancellationToken); } - public async Task GetAsync(Uri uri, CancellationToken cancellationToken = default) + public async Task GetAsync( + Uri uri, + int decodeWidth = 0, + CancellationToken cancellationToken = default + ) { - return await GetFromCacheAsync(uri, false, cancellationToken).ConfigureAwait(false); + CurrentDecodeWidth.Value = decodeWidth; + return await GetFromCacheAsync(WithDecodeWidthKey(uri, decodeWidth), false, cancellationToken) + .ConfigureAwait(false); } - public async Task GetWithCacheAsync(Uri uri, CancellationToken cancellationToken = default) + public async Task GetWithCacheAsync( + Uri uri, + int decodeWidth = 0, + CancellationToken cancellationToken = default + ) + { + CurrentDecodeWidth.Value = decodeWidth; + return await GetFromCacheAsync(WithDecodeWidthKey(uri, decodeWidth), true, cancellationToken) + .ConfigureAwait(false); + } + + /// + /// Returns a Uri that includes the decode width as a fragment so the underlying cache (keyed by Uri) + /// keeps a separate entry per size — e.g. a 450px thumbnail decode can't be served for a later + /// full-resolution request of the same image. Uri fragments are not sent in HTTP requests, so the + /// actual download is unaffected. + /// + private static Uri WithDecodeWidthKey(Uri uri, int decodeWidth) { - return await GetFromCacheAsync(uri, true, cancellationToken).ConfigureAwait(false); + return decodeWidth <= 0 ? uri : new UriBuilder(uri) { Fragment = $"sm-decode={decodeWidth}" }.Uri; } public int ClearMemoryCache() diff --git a/StabilityMatrix.Avalonia/Controls/VendorLabs/Cache/MemoryImageCache.cs b/StabilityMatrix.Avalonia/Controls/VendorLabs/Cache/MemoryImageCache.cs index be3f17530..fd551d5ab 100644 --- a/StabilityMatrix.Avalonia/Controls/VendorLabs/Cache/MemoryImageCache.cs +++ b/StabilityMatrix.Avalonia/Controls/VendorLabs/Cache/MemoryImageCache.cs @@ -132,7 +132,7 @@ public void Remove(IEnumerable uriForCachedItems) /// Awaitable Task public Task PreCacheAsync(Uri uri, CancellationToken cancellationToken = default) { - return GetWithCacheAsync(uri, cancellationToken); + return GetWithCacheAsync(uri, cancellationToken: cancellationToken); } /// @@ -174,7 +174,11 @@ private static ulong CreateHash64(string str) return value; } - public async Task GetAsync(Uri uri, CancellationToken cancellationToken) + public async Task GetAsync( + Uri uri, + int decodeWidth = 0, + CancellationToken cancellationToken = default + ) { IImage? instance = null; @@ -185,12 +189,13 @@ private static ulong CreateHash64(string str) // Local if (File.Exists(uri.LocalPath)) { - instance = LoadLocalImageWithSkia(uri); + instance = LoadLocalImageWithSkia(uri, decodeWidth); } // Remote else { - instance = await DownloadImageAsync(uri, cancellationToken).ConfigureAwait(false); + instance = await DownloadImageAsync(uri, decodeWidth, cancellationToken) + .ConfigureAwait(false); } if (instance != null) @@ -204,11 +209,16 @@ private static ulong CreateHash64(string str) return instance; } - public async Task GetWithCacheAsync(Uri uri, CancellationToken cancellationToken) + public async Task GetWithCacheAsync( + Uri uri, + int decodeWidth = 0, + CancellationToken cancellationToken = default + ) { IImage? instance = null; - var fileName = GetCacheFileName(uri); + // Include the decode width in the cache key so requests at different sizes don't collide. + var fileName = $"{GetCacheFileName(uri)}_{decodeWidth}"; _concurrentTasks.TryGetValue(fileName, out var request); if (request != null) @@ -222,7 +232,7 @@ private static ulong CreateHash64(string str) { request = new ConcurrentRequest { - Task = GetItemWithCacheAsync(uri, fileName, cancellationToken), + Task = GetItemWithCacheAsync(uri, fileName, decodeWidth, cancellationToken), }; _concurrentTasks[fileName] = request; @@ -270,6 +280,7 @@ public int ClearMemoryCache(DateTime olderThan) private async Task GetItemWithCacheAsync( Uri uri, string cacheKey, + int decodeWidth, CancellationToken cancellationToken ) { @@ -297,11 +308,12 @@ CancellationToken cancellationToken { if (isLocal) { - instance = LoadLocalImageWithSkia(uri); + instance = LoadLocalImageWithSkia(uri, decodeWidth); } else { - instance = await DownloadImageAsync(uri, cancellationToken).ConfigureAwait(false); + instance = await DownloadImageAsync(uri, decodeWidth, cancellationToken) + .ConfigureAwait(false); } if (instance != null) @@ -338,14 +350,18 @@ CancellationToken cancellationToken return image; } - private static IImage? LoadLocalImageWithSkia(Uri uri) + private static IImage? LoadLocalImageWithSkia(Uri uri, int decodeWidth) { using var skFileStream = new SKFileStream(uri.LocalPath); - return SKBitmap.Decode(skFileStream).ToAvaloniaImage(); + return SKBitmap.Decode(skFileStream).ToAvaloniaImageScaled(decodeWidth); } - private async Task DownloadImageAsync(Uri uri, CancellationToken cancellationToken) + private async Task DownloadImageAsync( + Uri uri, + int decodeWidth, + CancellationToken cancellationToken + ) { using var ms = new MemoryStream(); @@ -357,7 +373,6 @@ CancellationToken cancellationToken ms.Position = 0; - var image = new Bitmap(ms); - return image; + return Extensions.SkiaExtensions.DecodeToAvaloniaImageScaled(ms, decodeWidth); } } diff --git a/StabilityMatrix.Avalonia/Extensions/SkiaExtensions.cs b/StabilityMatrix.Avalonia/Extensions/SkiaExtensions.cs index 09504217e..4fb49c41b 100644 --- a/StabilityMatrix.Avalonia/Extensions/SkiaExtensions.cs +++ b/StabilityMatrix.Avalonia/Extensions/SkiaExtensions.cs @@ -98,6 +98,60 @@ public void Draw(DrawingContext context, Rect sourceRect, Rect destRect) return default; } + /// + /// Converts the to an Avalonia image, downscaling it to + /// (preserving aspect ratio) if it is wider. Use for thumbnails so full-resolution bitmaps aren't kept in + /// memory or uploaded to the GPU. When is 0 or the source is already + /// smaller, the bitmap is returned unscaled. Ownership of the bitmap (or its scaled replacement) is + /// transferred to the returned image. + /// + public static IImage? ToAvaloniaImageScaled(this SKBitmap? bitmap, int decodeWidth) + { + if (bitmap is null) + { + return null; + } + + if (decodeWidth <= 0 || bitmap.Width <= decodeWidth) + { + return bitmap.ToAvaloniaImage(); + } + + var targetHeight = Math.Max(1, (int)Math.Round(bitmap.Height * ((double)decodeWidth / bitmap.Width))); + + var resized = bitmap.Resize( + new SKImageInfo(decodeWidth, targetHeight), + new SKSamplingOptions(SKFilterMode.Linear, SKMipmapMode.Linear) + ); + + // If the resize failed, fall back to handing off the original bitmap unscaled. + if (resized is null) + { + return bitmap.ToAvaloniaImage(); + } + + bitmap.Dispose(); + return resized.ToAvaloniaImage(); + } + + /// + /// Decodes an image stream into an Avalonia image, downscaling to if wider. + /// Does not dispose the supplied stream. + /// + public static IImage? DecodeToAvaloniaImageScaled(System.IO.Stream? stream, int decodeWidth) + { + return stream.ToSKBitmap().ToAvaloniaImageScaled(decodeWidth); + } + + /// + /// Decodes an image file into an Avalonia image, downscaling to if wider. + /// + public static IImage? DecodeFileToAvaloniaImageScaled(string path, int decodeWidth) + { + using var stream = new SKFileStream(path); + return SKBitmap.Decode(stream).ToAvaloniaImageScaled(decodeWidth); + } + public static Bitmap ToAvaloniaBitmap(this SKBitmap bitmap) { return ToAvaloniaBitmap(bitmap, new Vector(96, 96)); diff --git a/StabilityMatrix.Avalonia/Helpers/ComfyExtensionInstallHelper.cs b/StabilityMatrix.Avalonia/Helpers/ComfyExtensionInstallHelper.cs new file mode 100644 index 000000000..03058e587 --- /dev/null +++ b/StabilityMatrix.Avalonia/Helpers/ComfyExtensionInstallHelper.cs @@ -0,0 +1,135 @@ +using System; +using System.Collections.Generic; +using System.Linq; +using System.Threading.Tasks; +using AsyncAwaitBestPractices; +using Avalonia.Threading; +using FluentAvalonia.UI.Controls; +using NLog; +using StabilityMatrix.Avalonia.Languages; +using StabilityMatrix.Avalonia.Services; +using StabilityMatrix.Core.Exceptions; +using StabilityMatrix.Core.Extensions; +using StabilityMatrix.Core.Helper; +using StabilityMatrix.Core.Models; +using StabilityMatrix.Core.Models.PackageModification; +using StabilityMatrix.Core.Models.Packages.Extensions; + +namespace StabilityMatrix.Avalonia.Helpers; + +/// +/// Shared flow for prompting the user to install missing / out-of-date ComfyUI extensions +/// required by a workflow, then installing them and restarting the package. +/// Used by both Inference and Image Lab before queueing a prompt. +/// +public static class ComfyExtensionInstallHelper +{ + private static readonly Logger Logger = LogManager.GetCurrentClassLogger(); + + /// + /// Shows a confirmation dialog listing the required extensions and, if accepted, installs + /// them via the package modification runner and restarts the package. Must be called from + /// the UI thread. + /// + /// True if the user accepted and the install was started. + public static async Task PromptInstallAndRestartAsync( + IPackageExtensionManager manager, + PackagePair localPackagePair, + IReadOnlyList missingExtensions, + IReadOnlyList<( + ExtensionSpecifier Specifier, + InstalledPackageExtension Installed + )> outOfDateExtensions, + RunningPackageService runningPackageService, + INotificationService notificationService + ) + { + var dialog = DialogHelper.CreateMarkdownDialog( + $"#### The following extensions are required for this workflow:\n" + + $"{string.Join("\n- ", missingExtensions.Select(ext => ext.Name))}" + + $"{string.Join("\n- ", outOfDateExtensions.Select(pair => $"{pair.Specifier.Name} {pair.Specifier.Constraint} {pair.Specifier.Version} (Current Version: {pair.Installed.Version?.Tag})"))}", + "Install Required Extensions?" + ); + + dialog.IsPrimaryButtonEnabled = true; + dialog.DefaultButton = ContentDialogButton.Primary; + dialog.PrimaryButtonText = + $"{Resources.Action_Install} ({localPackagePair.InstalledPackage.DisplayName.ToRepr()} will restart)"; + dialog.CloseButtonText = Resources.Action_Cancel; + + if (await dialog.ShowAsync() != ContentDialogResult.Primary) + { + return false; + } + + var manifestExtensionsMap = await manager.GetManifestExtensionsMapAsync( + manager.GetManifests(localPackagePair.InstalledPackage) + ); + + var steps = new List(); + + // Add install for missing extensions + foreach (var missingExtension in missingExtensions) + { + if (!manifestExtensionsMap.TryGetValue(missingExtension.Name, out var extension)) + { + Logger.Warn("Extension {MissingExtensionUrl} not found in manifests", missingExtension.Name); + continue; + } + + steps.Add(new InstallExtensionStep(manager, localPackagePair.InstalledPackage, extension)); + } + + // Add update for out of date extensions + foreach (var (specifier, installed) in outOfDateExtensions) + { + if (!manifestExtensionsMap.TryGetValue(specifier.Name, out _)) + { + Logger.Warn("Extension {MissingExtensionUrl} not found in manifests", specifier.Name); + continue; + } + + steps.Add(new UpdateExtensionStep(manager, localPackagePair.InstalledPackage, installed)); + } + + var runner = new PackageModificationRunner + { + ShowDialogOnStart = true, + ModificationCompleteTitle = "Extensions Installed", + ModificationCompleteMessage = "Finished installing required extensions", + }; + EventManager.Instance.OnPackageInstallProgressAdded(runner); + + runner + .ExecuteSteps(steps) + .ContinueWith(async _ => + { + if (runner.Failed) + return; + + // Restart Package + try + { + await Dispatcher.UIThread.InvokeAsync(async () => + { + await runningPackageService.StopPackage(localPackagePair.InstalledPackage.Id); + await runningPackageService.StartPackage(localPackagePair.InstalledPackage); + }); + } + catch (Exception e) + { + Logger.Error(e, "Error while restarting package"); + + notificationService.ShowPersistent( + new AppException( + "Could not restart package", + "Please manually restart the package for extension changes to take effect" + ) + ); + } + }) + .SafeFireAndForget(); + + return true; + } +} diff --git a/StabilityMatrix.Avalonia/Helpers/LinuxDesktopIntegration.cs b/StabilityMatrix.Avalonia/Helpers/LinuxDesktopIntegration.cs new file mode 100644 index 000000000..7e0a43a41 --- /dev/null +++ b/StabilityMatrix.Avalonia/Helpers/LinuxDesktopIntegration.cs @@ -0,0 +1,164 @@ +using System; +using System.IO; +using System.Runtime.Versioning; +using System.Text; +using NLog; +using StabilityMatrix.Core.Helper; + +namespace StabilityMatrix.Avalonia.Helpers; + +/// +/// Handles Linux desktop integration, including creating .desktop files. +/// Only relevant for AppImage runs - other Linux installs (deb/rpm/flatpak/AUR) +/// ship their own package-managed .desktop entries which we must not touch. +/// +public static class LinuxDesktopIntegration +{ + private static readonly Logger Logger = LogManager.GetCurrentClassLogger(); + + private const string DesktopFileTemplate = """ +[Desktop Entry] +Name=Stability Matrix +Exec="{0}" %u +Type=Application +NoDisplay=false +Categories=Utility +Icon={1} +StartupWMClass=stabilitymatrix +"""; + + /// + /// Gets the current desktop environment name + /// + /// The name of the current desktop environment (e.g., "KDE", "GNOME", "XFCE") or null if not detected + [SupportedOSPlatform("linux")] + private static string? GetCurrentDesktopEnvironment() + { + try + { + // XDG_CURRENT_DESKTOP can contain multiple colon-separated values (e.g. "ubuntu:GNOME"), + // so match by substring rather than taking only the first element. DESKTOP_SESSION is a + // fallback for environments that don't set XDG_CURRENT_DESKTOP. + foreach (var envVar in new[] { "XDG_CURRENT_DESKTOP", "DESKTOP_SESSION" }) + { + var value = Environment.GetEnvironmentVariable(envVar); + if (string.IsNullOrEmpty(value)) + { + continue; + } + + var upper = value.ToUpperInvariant(); + if (upper.Contains("KDE")) + return "KDE"; + if (upper.Contains("GNOME")) + return "GNOME"; + if (upper.Contains("XFCE")) + return "XFCE"; + } + + return null; + } + catch + { + return null; + } + } + + /// + /// Writes a correct .desktop entry (and extracts the icon) for AppImage runs so the app + /// shows up in the application launcher. No-op when not running as an AppImage, since other + /// install types manage their own desktop entries. + /// + [SupportedOSPlatform("linux")] + public static void CreateDesktopFile() + { + if (!Compat.IsLinux) + { + return; + } + + // Only self-integrate when running as an AppImage. Other installs (deb/rpm/flatpak/AUR) + // have package-managed .desktop files, and Compat.AppCurrentPath throws off-AppImage. + if (string.IsNullOrEmpty(Environment.GetEnvironmentVariable("APPIMAGE"))) + { + return; + } + + try + { + // Respect XDG_DATA_HOME per the XDG Base Directory Specification, falling back to + // ~/.local/share when unset. + var xdgDataHome = Environment.GetEnvironmentVariable("XDG_DATA_HOME"); + var dataHome = !string.IsNullOrEmpty(xdgDataHome) + ? xdgDataHome + : Path.Combine( + Environment.GetFolderPath(Environment.SpecialFolder.UserProfile), + ".local/share" + ); + + var desktopFilePath = Path.Combine(dataHome, "applications/stabilitymatrix-app.desktop"); + + var iconPath = Path.Combine(dataHome, "icons/hicolor/256x256/apps/stabilitymatrix.png"); + + // Ensure directories exist + Directory.CreateDirectory(Path.GetDirectoryName(desktopFilePath)!); + Directory.CreateDirectory(Path.GetDirectoryName(iconPath)!); + + // Extract icon (must be a real PNG - launchers can't render an .ico) + using (var iconStream = Assets.AppIconPng.Open()) + using (var iconFileStream = File.Create(iconPath)) + { + iconStream.CopyTo(iconFileStream); + } + + // Create desktop file with additional desktop environment specific entries + var desktopFileBuilder = new StringBuilder( + string.Format(DesktopFileTemplate, Compat.AppCurrentPath.FullPath, iconPath) + ); + + // Add desktop environment specific entries + var desktopEnv = GetCurrentDesktopEnvironment(); + if (!string.IsNullOrEmpty(desktopEnv)) + { + // The base template's last line has no trailing newline (raw string literals drop + // the final newline before the closing quotes), so add one here - otherwise the + // entry below would be concatenated onto the StartupWMClass line. + desktopFileBuilder.AppendLine(); + switch (desktopEnv) + { + case "KDE": + desktopFileBuilder.AppendLine("X-KDE-StartupNotify=true"); + break; + case "GNOME": + desktopFileBuilder.AppendLine("X-GNOME-UsesNotifications=true"); + break; + case "XFCE": + desktopFileBuilder.AppendLine("X-XFCE-StartupNotify=true"); + break; + } + } + + // UTF-8 without BOM - some .desktop parsers choke on a leading BOM + File.WriteAllText(desktopFilePath, desktopFileBuilder.ToString(), new UTF8Encoding(false)); + + // Make executable + var unixFileMode = + UnixFileMode.UserRead + | UnixFileMode.UserWrite + | UnixFileMode.UserExecute + | UnixFileMode.GroupRead + | UnixFileMode.GroupExecute + | UnixFileMode.OtherRead + | UnixFileMode.OtherExecute; + + File.SetUnixFileMode(desktopFilePath, unixFileMode); + + Logger.Info("Created Linux desktop entry at {DesktopFilePath}", desktopFilePath); + } + catch (Exception e) + { + // Desktop integration is best-effort; never let it block startup + Logger.Warn(e, "Failed to create Linux desktop entry"); + } + } +} diff --git a/StabilityMatrix.Avalonia/Models/HuggingFace/HuggingFaceFolderInference.cs b/StabilityMatrix.Avalonia/Models/HuggingFace/HuggingFaceFolderInference.cs new file mode 100644 index 000000000..36a1fb9e2 --- /dev/null +++ b/StabilityMatrix.Avalonia/Models/HuggingFace/HuggingFaceFolderInference.cs @@ -0,0 +1,181 @@ +using System; +using System.Collections.Generic; +using StabilityMatrix.Core.Models; + +namespace StabilityMatrix.Avalonia.Models.HuggingFace; + +/// +/// Helpers for the live HuggingFace browser: guessing a destination +/// from a file path, and parsing repo ids from URLs. +/// +public static class HuggingFaceFolderInference +{ + /// + /// The destination folders offered in the per-file destination dropdown, + /// ordered roughly by how common they are for downloaded models. + /// + public static IReadOnlyList SelectableFolders { get; } = + new[] + { + SharedFolderType.StableDiffusion, + SharedFolderType.DiffusionModels, + SharedFolderType.Lora, + SharedFolderType.VAE, + SharedFolderType.TextEncoders, + SharedFolderType.ClipVision, + SharedFolderType.ControlNet, + SharedFolderType.IpAdapter, + SharedFolderType.T2IAdapter, + SharedFolderType.StyleModels, + SharedFolderType.Embeddings, + SharedFolderType.Ultralytics, + SharedFolderType.Sams, + SharedFolderType.AudioEncoders, + SharedFolderType.ESRGAN, + }; + + /// + /// Guess the most appropriate destination folder for a file based on its path/name. + /// Falls back to for generic checkpoints. + /// + public static SharedFolderType Infer(string? path) + { + if (string.IsNullOrEmpty(path)) + return SharedFolderType.StableDiffusion; + + var p = path.Replace('\\', '/').ToLowerInvariant(); + var name = p.Contains('/') ? p[(p.LastIndexOf('/') + 1)..] : p; + + // Order matters: more specific matches first. + if (Contains(p, "clip_vision", "clip-vision", "clipvision")) + return SharedFolderType.ClipVision; + + if (Contains(p, "controlnet", "control_net", "control-net", "control_v", "control-v")) + return SharedFolderType.ControlNet; + + if (Contains(p, "t2i", "t2i_adapter", "t2i-adapter")) + return SharedFolderType.T2IAdapter; + + if (Contains(p, "ip-adapter", "ip_adapter", "ipadapter")) + return SharedFolderType.IpAdapter; + + if ( + Contains(p, "text_encoder", "text_encoders", "text-encoder", "/clip/") + || Contains(name, "enconly") + || StartsWith( + name, + "clip_", + "clip-", + "t5", + "umt5", + "byt5", + "mt5", + "llava", + "llama", + "gemma", + "qwen_3", + "qwen2" + ) + ) + return SharedFolderType.TextEncoders; + + if (Contains(p, "vae") || StartsWith(name, "ae.")) + return SharedFolderType.VAE; + + if (Contains(p, "lora", "loras")) + return SharedFolderType.Lora; + + if (Contains(p, "embedding", "textual_inversion")) + return SharedFolderType.Embeddings; + + if (Contains(p, "style_model", "style_models", "redux")) + return SharedFolderType.StyleModels; + + if (Contains(p, "audio_encoder", "audio-encoder")) + return SharedFolderType.AudioEncoders; + + if (Contains(p, "upscal", "esrgan")) + return SharedFolderType.ESRGAN; + + if ( + name.EndsWith(".gguf") + || Contains(p, "unet", "diffusion_model", "diffusion_models", "transformer") + ) + return SharedFolderType.DiffusionModels; + + return SharedFolderType.StableDiffusion; + } + + /// + /// Attempt to parse a HuggingFace repo id (owner/name) from raw user input, + /// accepting bare ids, huggingface.co URLs, and /tree/ or /blob/ links. + /// + public static bool TryParseRepoId(string? input, out string repoId) + { + repoId = string.Empty; + if (string.IsNullOrWhiteSpace(input)) + return false; + + var value = input.Trim(); + + // If it looks like a URL, validate the host is really huggingface.co and take the path. + // (A naive Contains check would accept e.g. not-huggingface.co or huggingface.co.evil.com.) + if (value.Contains("://") || value.Contains("huggingface.co", StringComparison.OrdinalIgnoreCase)) + { + var urlToParse = value.Contains("://") ? value : $"https://{value}"; + if ( + !Uri.TryCreate(urlToParse, UriKind.Absolute, out var uri) + || !( + uri.Host.Equals("huggingface.co", StringComparison.OrdinalIgnoreCase) + || uri.Host.EndsWith(".huggingface.co", StringComparison.OrdinalIgnoreCase) + ) + ) + return false; + + value = uri.AbsolutePath; + } + + // Drop any query/fragment. + foreach (var sep in new[] { '?', '#' }) + { + var qi = value.IndexOf(sep); + if (qi >= 0) + value = value[..qi]; + } + + var segments = value.Trim('/').Split('/', StringSplitOptions.RemoveEmptyEntries); + if (segments.Length < 2) + return false; + + // Take the first two segments as owner/name; ignore /tree/main, /blob/..., etc. + var owner = segments[0]; + var name = segments[1]; + + // Reject obvious non-repo first segments (route prefixes). + if (owner is "models" or "datasets" or "spaces" or "api") + return false; + + repoId = $"{owner}/{name}"; + return true; + } + + private static bool Contains(string haystack, params string[] needles) + { + foreach (var n in needles) + { + if (haystack.Contains(n, StringComparison.Ordinal)) + return true; + } + return false; + } + + private static bool StartsWith(string value, params string[] prefixes) + { + foreach (var prefix in prefixes) + { + if (value.StartsWith(prefix, StringComparison.Ordinal)) + return true; + } + return false; + } +} diff --git a/StabilityMatrix.Avalonia/Program.cs b/StabilityMatrix.Avalonia/Program.cs index 2853ae527..25b17cf31 100644 --- a/StabilityMatrix.Avalonia/Program.cs +++ b/StabilityMatrix.Avalonia/Program.cs @@ -94,8 +94,16 @@ x.Tag is ErrorType.HelpRequestedError or ErrorType.VersionRequestedError } // Launched for custom URI scheme, handle and - // on macOS we use activation events so ignore this - if (!Compat.IsMacOS && Args.Uri is { } uriArg) + // on macOS we use activation events so ignore this. + // Windows registers the scheme handler with --uri, but on Linux the .desktop handler + // invokes us with the URI as a bare positional argument (%u), so accept that form too. + var uriArg = + Args.Uri + ?? args.FirstOrDefault(a => + a.StartsWith($"{UriHandler.Scheme}://", StringComparison.OrdinalIgnoreCase) + ); + + if (!Compat.IsMacOS && uriArg is not null) { if (Uri.TryCreate(uriArg, UriKind.Absolute, out var uri)) { @@ -230,7 +238,7 @@ public static AppBuilder BuildAvaloniaApp() if (Compat.IsLinux) { - app = app.With(new X11PlatformOptions { OverlayPopups = true }); + app = app.With(new X11PlatformOptions { OverlayPopups = true, WmClass = "stabilitymatrix" }); } else if (Compat.IsMacOS) { @@ -249,14 +257,26 @@ public static AppBuilder BuildAvaloniaApp() if (Args.UseVulkanRendering) { - app = app.With(new X11PlatformOptions { RenderingMode = [X11RenderingMode.Vulkan] }) + app = app.With( + new X11PlatformOptions + { + RenderingMode = [X11RenderingMode.Vulkan], + WmClass = "stabilitymatrix", + } + ) .With(new Win32PlatformOptions { RenderingMode = [Win32RenderingMode.Vulkan] }); } if (Args.DisableGpuRendering) { app = app.With(new Win32PlatformOptions { RenderingMode = new[] { Win32RenderingMode.Software } }) - .With(new X11PlatformOptions { RenderingMode = new[] { X11RenderingMode.Software } }) + .With( + new X11PlatformOptions + { + RenderingMode = new[] { X11RenderingMode.Software }, + WmClass = "stabilitymatrix", + } + ) .With( new AvaloniaNativePlatformOptions { diff --git a/StabilityMatrix.Avalonia/Services/ComfyImageGenerationProviderBase.cs b/StabilityMatrix.Avalonia/Services/ComfyImageGenerationProviderBase.cs index fa94d66cb..3ea9971be 100644 --- a/StabilityMatrix.Avalonia/Services/ComfyImageGenerationProviderBase.cs +++ b/StabilityMatrix.Avalonia/Services/ComfyImageGenerationProviderBase.cs @@ -1,4 +1,5 @@ using AsyncAwaitBestPractices; +using Avalonia.Threading; using Microsoft.Extensions.Logging; using Refit; using StabilityMatrix.Avalonia.Helpers; @@ -8,6 +9,7 @@ using StabilityMatrix.Core.Models; using StabilityMatrix.Core.Models.Api.Comfy; using StabilityMatrix.Core.Models.Api.Comfy.Nodes; +using StabilityMatrix.Core.Models.Packages.Extensions; using StabilityMatrix.Core.Services.ImageGeneration; namespace StabilityMatrix.Avalonia.Services; @@ -19,11 +21,17 @@ namespace StabilityMatrix.Avalonia.Services; /// interrupt, output download, and error handling — so subclasses supply only the /// provider-specific model requirements and workflow node graph. /// -public abstract class ComfyImageGenerationProviderBase(ILogger logger, IInferenceClientManager clientManager) - : IImageGenerationProvider +public abstract class ComfyImageGenerationProviderBase( + ILogger logger, + IInferenceClientManager clientManager, + RunningPackageService runningPackageService, + INotificationService notificationService +) : IImageGenerationProvider { protected ILogger Logger { get; } = logger; protected IInferenceClientManager ClientManager { get; } = clientManager; + protected RunningPackageService RunningPackageService { get; } = runningPackageService; + protected INotificationService NotificationService { get; } = notificationService; public abstract string ProviderId { get; } public abstract string ProviderName { get; } @@ -95,6 +103,15 @@ await ComfyImageUploadHelper.UploadImagesAsync( Logger.LogInformation("Building {Provider} workflow", LogName); var nodes = BuildWorkflow(request); + if ( + nodes is NodeDictionary nodeDictionary + && await GetMissingExtensionsAsync(nodeDictionary, cancellationToken) + is { Count: > 0 } missing + ) + { + return await HandleMissingExtensionsAsync(missing); + } + Logger.LogInformation("Queuing prompt to ComfyUI"); var task = await ClientManager.Client.QueuePromptAsync(nodes, cancellationToken); @@ -171,6 +188,101 @@ await ComfyImageUploadHelper.UploadImagesAsync( } } + /// + /// Returns the required extensions declared by the workflow's typed nodes (e.g. + /// ComfyUI-GGUF for UnetLoaderGGUF) that are not installed in the locally-managed ComfyUI, + /// so we can offer an install instead of a generic queue-time 400 rejection. + /// Returns an empty list when nothing is missing or the installed extensions cannot be + /// determined (e.g. a remote / unmanaged server). + /// + private async Task> GetMissingExtensionsAsync( + NodeDictionary nodes, + CancellationToken cancellationToken + ) + { + var requiredExtensions = nodes.RequiredExtensions.DistinctBy(ext => ext.Name).ToList(); + if (requiredExtensions.Count == 0) + return []; + + try + { + if ( + ClientManager.Client?.LocalServerPackage is not { } localPackagePair + || localPackagePair.BasePackage.ExtensionManager + is not GitPackageExtensionManager extensionManager + ) + { + return []; + } + + var installedUrls = ( + await extensionManager.GetInstalledExtensionsLiteAsync( + localPackagePair.InstalledPackage, + cancellationToken + ) + ) + .Select(ext => ext.GitRepositoryUrl) + .OfType() + .ToHashSet(StringComparer.OrdinalIgnoreCase); + + return requiredExtensions.Where(specifier => !installedUrls.Contains(specifier.Name)).ToList(); + } + catch (Exception e) + { + Logger.LogWarning(e, "Failed to check required extensions, proceeding with generation"); + return []; + } + } + + /// + /// Offers the same install-and-restart flow Inference uses for missing extensions, and + /// returns the error response for the chat: either "installing, retry once reconnected" + /// or manual install instructions if the prompt was declined / unavailable. + /// + private async Task HandleMissingExtensionsAsync( + IReadOnlyList missingExtensions + ) + { + var extensionList = string.Join(", ", missingExtensions.Select(ext => ext.Name)); + Logger.LogWarning("Required ComfyUI extensions not installed: {Extensions}", extensionList); + + if ( + ClientManager.Client?.LocalServerPackage is { } localPackagePair + && localPackagePair.BasePackage.ExtensionManager is { } extensionManager + ) + { + var installStarted = await Dispatcher.UIThread.InvokeAsync(() => + ComfyExtensionInstallHelper.PromptInstallAndRestartAsync( + extensionManager, + localPackagePair, + missingExtensions, + [], + RunningPackageService, + NotificationService + ) + ); + + if (installStarted) + { + return new ImageGenerationResponse + { + IsSuccess = false, + ErrorMessage = + $"Installing required ComfyUI extensions: {extensionList}. " + + "ComfyUI will restart - please try again once it has reconnected.", + }; + } + } + + return new ImageGenerationResponse + { + IsSuccess = false, + ErrorMessage = + $"This workflow requires ComfyUI extensions that are not installed: {extensionList}. " + + "Install them from the ComfyUI package's Extensions section, then restart ComfyUI.", + }; + } + /// /// Registers a cancellation callback that interrupts the running ComfyUI prompt. /// diff --git a/StabilityMatrix.Avalonia/Services/Flux2KleinProvider.cs b/StabilityMatrix.Avalonia/Services/Flux2KleinProvider.cs index b58e0dbd8..8ede9e95c 100644 --- a/StabilityMatrix.Avalonia/Services/Flux2KleinProvider.cs +++ b/StabilityMatrix.Avalonia/Services/Flux2KleinProvider.cs @@ -10,8 +10,12 @@ namespace StabilityMatrix.Avalonia.Services; /// Klein 4B is Apache 2.0 licensed; the distilled variant runs at 4 steps with CFG=1 /// making it well-suited to conversational, iterative editing. /// -public class Flux2KleinProvider(ILogger logger, IInferenceClientManager clientManager) - : ComfyImageGenerationProviderBase(logger, clientManager) +public class Flux2KleinProvider( + ILogger logger, + IInferenceClientManager clientManager, + RunningPackageService runningPackageService, + INotificationService notificationService +) : ComfyImageGenerationProviderBase(logger, clientManager, runningPackageService, notificationService) { public override string ProviderId => BananaVisionProviderIds.Flux2Klein; public override string ProviderName => "Flux.2 Klein (Local)"; diff --git a/StabilityMatrix.Avalonia/Services/Flux2KleinWorkflowBuilder.cs b/StabilityMatrix.Avalonia/Services/Flux2KleinWorkflowBuilder.cs index 656e42df9..dfcacb813 100644 --- a/StabilityMatrix.Avalonia/Services/Flux2KleinWorkflowBuilder.cs +++ b/StabilityMatrix.Avalonia/Services/Flux2KleinWorkflowBuilder.cs @@ -120,15 +120,28 @@ public static Dictionary Build( } // Single CLIPLoader with type="flux2" — Klein uses a single Qwen3 text encoder, - // not the Flux.1-style dual CLIP-L + T5. - var clipLoader = nodes.AddTypedNode( - new ComfyNodeBuilder.CLIPLoader - { - Name = nodes.GetUniqueName(nameof(ComfyNodeBuilder.CLIPLoader)), - ClipName = selectedModels.ClipModel.RelativePath, - Type = DefaultClipType, - } - ); + // not the Flux.1-style dual CLIP-L + T5. GGUF encoders route through CLIPLoaderGGUF. + var clipOutput = selectedModels.ClipModel.IsGguf + ? nodes + .AddTypedNode( + new ComfyNodeBuilder.CLIPLoaderGGUF + { + Name = nodes.GetUniqueName(nameof(ComfyNodeBuilder.CLIPLoaderGGUF)), + ClipName = selectedModels.ClipModel.RelativePath, + Type = DefaultClipType, + } + ) + .Output + : nodes + .AddTypedNode( + new ComfyNodeBuilder.CLIPLoader + { + Name = nodes.GetUniqueName(nameof(ComfyNodeBuilder.CLIPLoader)), + ClipName = selectedModels.ClipModel.RelativePath, + Type = DefaultClipType, + } + ) + .Output; var vaeLoader = nodes.AddTypedNode( new ComfyNodeBuilder.VAELoader @@ -143,7 +156,7 @@ public static Dictionary Build( nodes, loras, unetOutput, - clipLoader.Output + clipOutput ); // 2. Encode the positive prompt diff --git a/StabilityMatrix.Avalonia/Services/FluxKontextProvider.cs b/StabilityMatrix.Avalonia/Services/FluxKontextProvider.cs index 0ee87e744..e5106bfbb 100644 --- a/StabilityMatrix.Avalonia/Services/FluxKontextProvider.cs +++ b/StabilityMatrix.Avalonia/Services/FluxKontextProvider.cs @@ -8,8 +8,12 @@ namespace StabilityMatrix.Avalonia.Services; /// /// Image generation provider for Flux Kontext using local ComfyUI backend /// -public class FluxKontextProvider(ILogger logger, IInferenceClientManager clientManager) - : ComfyImageGenerationProviderBase(logger, clientManager) +public class FluxKontextProvider( + ILogger logger, + IInferenceClientManager clientManager, + RunningPackageService runningPackageService, + INotificationService notificationService +) : ComfyImageGenerationProviderBase(logger, clientManager, runningPackageService, notificationService) { public override string ProviderId => BananaVisionProviderIds.FluxKontext; public override string ProviderName => "Flux Kontext (Local)"; diff --git a/StabilityMatrix.Avalonia/Services/FluxKontextWorkflowBuilder.cs b/StabilityMatrix.Avalonia/Services/FluxKontextWorkflowBuilder.cs index 1491142cc..89771ffe3 100644 --- a/StabilityMatrix.Avalonia/Services/FluxKontextWorkflowBuilder.cs +++ b/StabilityMatrix.Avalonia/Services/FluxKontextWorkflowBuilder.cs @@ -78,23 +78,38 @@ public static Dictionary Build( } ); - // DualCLIPLoader for Flux - var clipLoader = nodes.AddTypedNode( - new ComfyNodeBuilder.DualCLIPLoader - { - Name = nodes.GetUniqueName(nameof(ComfyNodeBuilder.DualCLIPLoader)), - ClipName1 = selectedModels.Clip1Model.RelativePath, - ClipName2 = selectedModels.Clip2Model.RelativePath, - Type = "flux", - } - ); + // DualCLIPLoader for Flux (GGUF variant can also load .safetensors, so any gguf pick routes there) + var clipOutput = + selectedModels.Clip1Model.IsGguf || selectedModels.Clip2Model.IsGguf + ? nodes + .AddTypedNode( + new ComfyNodeBuilder.DualCLIPLoaderGGUF + { + Name = nodes.GetUniqueName(nameof(ComfyNodeBuilder.DualCLIPLoaderGGUF)), + ClipName1 = selectedModels.Clip1Model.RelativePath, + ClipName2 = selectedModels.Clip2Model.RelativePath, + Type = "flux", + } + ) + .Output + : nodes + .AddTypedNode( + new ComfyNodeBuilder.DualCLIPLoader + { + Name = nodes.GetUniqueName(nameof(ComfyNodeBuilder.DualCLIPLoader)), + ClipName1 = selectedModels.Clip1Model.RelativePath, + ClipName2 = selectedModels.Clip2Model.RelativePath, + Type = "flux", + } + ) + .Output; // Apply LoRAs if any var (currentModel, currentClip) = ComfyWorkflowHelper.ApplyLoras( nodes, loras, unetOutput, - clipLoader.Output + clipOutput ); // 2. Encode text prompt diff --git a/StabilityMatrix.Avalonia/Services/InferenceClientManager.cs b/StabilityMatrix.Avalonia/Services/InferenceClientManager.cs index dbc2c4890..c230ba22e 100644 --- a/StabilityMatrix.Avalonia/Services/InferenceClientManager.cs +++ b/StabilityMatrix.Avalonia/Services/InferenceClientManager.cs @@ -520,11 +520,7 @@ await Client.GetRequiredNodeOptionNamesFromOptionalNodeAsync("UnetLoaderGGUF", " // Get CLIP model names from DualCLIPLoader node if (await Client.GetNodeOptionNamesAsync("DualCLIPLoader", "clip_name1") is { } clipModelNames) { - IEnumerable models = - [ - HybridModelFile.None, - .. clipModelNames.Select(HybridModelFile.FromRemote), - ]; + var remoteNames = clipModelNames.ToHashSet(); if ( await Client.GetRequiredNodeOptionNamesFromOptionalNodeAsync( @@ -534,9 +530,29 @@ await Client.GetRequiredNodeOptionNamesFromOptionalNodeAsync( { } ggufClipModelNames ) { - models = models.Concat(ggufClipModelNames.Select(HybridModelFile.FromRemote)); + remoteNames.UnionWith(ggufClipModelNames); } + // Prefer local index entries (richer metadata), and keep local files the server + // didn't report: core ComfyUI never lists .gguf text encoders (only the optional + // DualCLIPLoaderGGUF node does), but the shared TextEncoders folder is synced to + // the package, so they're loadable via the GGUF clip loaders once installed. + var localModels = modelIndexService + .FindByModelType(SharedFolderType.TextEncoders) + .Select(HybridModelFile.FromLocal) + .ToList(); + + var localIds = localModels.Select(m => m.GetId()).ToHashSet(); + + IEnumerable models = + [ + HybridModelFile.None, + .. localModels, + .. remoteNames + .Select(HybridModelFile.FromRemote) + .Where(remote => !localIds.Contains(remote.GetId())), + ]; + clipModelsSource.EditDiff(models, HybridModelFile.RemoteLocalComparer); } @@ -815,7 +831,11 @@ public async Task WriteImageToInputAsync( } [MemberNotNull(nameof(Client))] - private async Task ConnectAsyncImpl(Uri uri, CancellationToken cancellationToken = default) + private async Task ConnectAsyncImpl( + Uri uri, + PackagePair? localServerPackage = null, + CancellationToken cancellationToken = default + ) { if (IsConnected) return; @@ -830,6 +850,16 @@ private async Task ConnectAsyncImpl(Uri uri, CancellationToken cancellationToken await tempClient.ConnectAsync(cancellationToken); logger.LogDebug("Connected to {@Uri}", uri); + // Set local server paths before publishing the client as connected, so that + // consumers observing IsConnected always see a fully-populated client (e.g. + // OutputImagesDir). Otherwise a generation resuming on the IsConnected change + // could race ahead of these being set. + if (localServerPackage is not null) + { + tempClient.LocalServerPackage = localServerPackage; + tempClient.LocalServerPath = localServerPackage.InstalledPackage.FullPath!; + } + Client = tempClient; await LoadSharedPropertiesAsync(); @@ -884,7 +914,7 @@ private async Task MigrateLinksIfNeeded(PackagePair packagePair) /// public virtual Task ConnectAsync(CancellationToken cancellationToken = default) { - return ConnectAsyncImpl(new Uri("http://127.0.0.1:8188"), cancellationToken); + return ConnectAsyncImpl(new Uri("http://127.0.0.1:8188"), cancellationToken: cancellationToken); } /// @@ -927,10 +957,7 @@ public virtual async Task ConnectAsync( var uri = new UriBuilder("http", host, int.Parse(port)).Uri; - await ConnectAsyncImpl(uri, cancellationToken); - - Client.LocalServerPackage = packagePair; - Client.LocalServerPath = packagePair.InstalledPackage.FullPath!; + await ConnectAsyncImpl(uri, packagePair, cancellationToken); } public async Task CloseAsync() diff --git a/StabilityMatrix.Avalonia/Services/ModelDownloadLinkHandler.cs b/StabilityMatrix.Avalonia/Services/ModelDownloadLinkHandler.cs index ee05cf1d4..46b37c175 100644 --- a/StabilityMatrix.Avalonia/Services/ModelDownloadLinkHandler.cs +++ b/StabilityMatrix.Avalonia/Services/ModelDownloadLinkHandler.cs @@ -69,7 +69,7 @@ private void UriReceivedHandler(Uri receivedUri) var hasValidLegacyFilter = !string.IsNullOrWhiteSpace(type) && !string.IsNullOrWhiteSpace(format) - && Enum.TryParse(type, out _) + && Enum.TryParse(type.Replace(" ", ""), true, out _) && Enum.TryParse(format, out _); if ( @@ -157,7 +157,8 @@ private void UriReceivedHandler(Uri receivedUri) } else { - Enum.TryParse(type, out var civitFileType); + // File type values may contain spaces (e.g. "Diffusion Model", "Pruned Model") + Enum.TryParse(type?.Replace(" ", ""), true, out var civitFileType); Enum.TryParse(format, out var civitFormat); var possibleFiles = modelVersion.Files?.Where(x => @@ -213,9 +214,14 @@ private void UriReceivedHandler(Uri receivedUri) var rootModelsDirectory = new DirectoryPath(settingsManager.ModelsDirectory); var downloadDirectory = rootModelsDirectory.JoinDir( - selectedFile.Type == CivitFileType.VAE - ? SharedFolderType.VAE.GetStringValue() - : model.Type.ConvertTo().GetStringValue() + selectedFile + .Type.GetSharedFolderType( + model.Type, + model.BaseModelType, + selectedFile.Name, + selectedFile.Metadata?.Format + ) + .GetStringValue() ); var importTask = modelImportService.DoImport( diff --git a/StabilityMatrix.Avalonia/Services/ModelImportService.cs b/StabilityMatrix.Avalonia/Services/ModelImportService.cs index 280122fbd..ddf6fc511 100644 --- a/StabilityMatrix.Avalonia/Services/ModelImportService.cs +++ b/StabilityMatrix.Avalonia/Services/ModelImportService.cs @@ -119,8 +119,7 @@ public async Task DoImport( } // Get latest version file - var modelFile = - selectedFile ?? modelVersion.Files?.FirstOrDefault(x => x.Type == CivitFileType.Model); + var modelFile = selectedFile ?? modelVersion.Files?.FirstOrDefault(x => x.Type.IsModelWeights()); if (modelFile is null) { notificationService.Show( diff --git a/StabilityMatrix.Avalonia/Services/QwenImageEditProvider.cs b/StabilityMatrix.Avalonia/Services/QwenImageEditProvider.cs index bc98d85ed..6948092f2 100644 --- a/StabilityMatrix.Avalonia/Services/QwenImageEditProvider.cs +++ b/StabilityMatrix.Avalonia/Services/QwenImageEditProvider.cs @@ -10,8 +10,10 @@ namespace StabilityMatrix.Avalonia.Services; /// public class QwenImageEditProvider( ILogger logger, - IInferenceClientManager clientManager -) : ComfyImageGenerationProviderBase(logger, clientManager) + IInferenceClientManager clientManager, + RunningPackageService runningPackageService, + INotificationService notificationService +) : ComfyImageGenerationProviderBase(logger, clientManager, runningPackageService, notificationService) { public override string ProviderId => BananaVisionProviderIds.QwenImageEdit; public override string ProviderName => "Qwen Image Edit (Local)"; diff --git a/StabilityMatrix.Avalonia/Services/QwenImageEditWorkflowBuilder.cs b/StabilityMatrix.Avalonia/Services/QwenImageEditWorkflowBuilder.cs index d29599a19..3c99a4f15 100644 --- a/StabilityMatrix.Avalonia/Services/QwenImageEditWorkflowBuilder.cs +++ b/StabilityMatrix.Avalonia/Services/QwenImageEditWorkflowBuilder.cs @@ -90,16 +90,28 @@ public static Dictionary Build( } ); - // CLIPLoader for Qwen (type: "qwen_image") - var clipLoader = nodes.AddTypedNode( - new ComfyNodeBuilder.CLIPLoader - { - Name = nodes.GetUniqueName(nameof(ComfyNodeBuilder.CLIPLoader)), - ClipName = selectedModels.ClipModel.RelativePath, - Type = "qwen_image", - } - ); - var currentClip = clipLoader.Output; + // CLIPLoader for Qwen (type: "qwen_image"); GGUF encoders route through CLIPLoaderGGUF + var currentClip = selectedModels.ClipModel.IsGguf + ? nodes + .AddTypedNode( + new ComfyNodeBuilder.CLIPLoaderGGUF + { + Name = nodes.GetUniqueName(nameof(ComfyNodeBuilder.CLIPLoaderGGUF)), + ClipName = selectedModels.ClipModel.RelativePath, + Type = "qwen_image", + } + ) + .Output + : nodes + .AddTypedNode( + new ComfyNodeBuilder.CLIPLoader + { + Name = nodes.GetUniqueName(nameof(ComfyNodeBuilder.CLIPLoader)), + ClipName = selectedModels.ClipModel.RelativePath, + Type = "qwen_image", + } + ) + .Output; // Apply LoRAs if any (currentModel, currentClip) = ComfyWorkflowHelper.ApplyLoras(nodes, loras, currentModel, currentClip); diff --git a/StabilityMatrix.Avalonia/ViewModels/Base/InferenceGenerationViewModelBase.cs b/StabilityMatrix.Avalonia/ViewModels/Base/InferenceGenerationViewModelBase.cs index 0868417f9..a8450ab8f 100644 --- a/StabilityMatrix.Avalonia/ViewModels/Base/InferenceGenerationViewModelBase.cs +++ b/StabilityMatrix.Avalonia/ViewModels/Base/InferenceGenerationViewModelBase.cs @@ -1,6 +1,8 @@ using System; using System.Collections.Generic; using System.Collections.Immutable; +using System.Collections.Specialized; +using System.ComponentModel; using System.ComponentModel.DataAnnotations; using System.Diagnostics; using System.Diagnostics.CodeAnalysis; @@ -41,6 +43,7 @@ using StabilityMatrix.Core.Models.Inference; using StabilityMatrix.Core.Models.Notifications; using StabilityMatrix.Core.Models.PackageModification; +using StabilityMatrix.Core.Models.Packages; using StabilityMatrix.Core.Models.Packages.Extensions; using StabilityMatrix.Core.Models.Settings; using StabilityMatrix.Core.Services; @@ -640,17 +643,104 @@ private async Task GenerateImage( /// /// Shows a prompt and return false if client not connected /// - protected async Task CheckClientConnectedWithPrompt() + protected async Task CheckClientConnectedWithPrompt(CancellationToken cancellationToken = default) { if (ClientManager.IsConnected) return true; var vm = vmFactory.Get(); - await vm.CreateDialog().ShowAsync(); + var result = await vm.CreateDialog().ShowAsync(); + + if (ClientManager.IsConnected) + return true; + + // If the user chose to launch ComfyUI, the package is now starting up. The connection + // is established automatically by InferenceViewModel once startup completes, so wait for + // it here and let the generation resume instead of forcing the user to press Generate again. + if (result == ContentDialogResult.Primary && vm.IsLaunchMode) + { + return await WaitForConnectedAsync(cancellationToken); + } return ClientManager.IsConnected; } + /// + /// Waits for the ClientManager to become connected, showing indeterminate progress. + /// Used after launching ComfyUI from the connection prompt so a queued generation can + /// resume automatically once the backend is ready. Stops waiting early if ComfyUI is + /// shut down or crashes before connecting (it is removed from RunningPackages either way). + /// + private async Task WaitForConnectedAsync(CancellationToken cancellationToken) + { + if (ClientManager.IsConnected) + return true; + + // RunContinuationsAsynchronously so the await resumption (and UI updates in finally) + // don't run synchronously on the thread that raised the completing event. + var tcs = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); + + bool IsAnyComfyRunning() => + runningPackageService.RunningPackages.Values.Any(vm => vm.RunningPackage.BasePackage is ComfyUI); + + void OnPropertyChanged(object? sender, PropertyChangedEventArgs args) + { + // null/empty PropertyName means "all properties changed" per INotifyPropertyChanged + if ( + args.PropertyName is nameof(ClientManager.IsConnected) or null or "" + && ClientManager.IsConnected + ) + { + tcs.TrySetResult(); + } + } + + void OnRunningPackagesChanged(object? sender, NotifyCollectionChangedEventArgs args) + { + // ComfyUI was shut down or crashed before connecting - stop waiting + if (!IsAnyComfyRunning()) + { + tcs.TrySetResult(); + } + } + + ClientManager.PropertyChanged += OnPropertyChanged; + runningPackageService.RunningPackages.CollectionChanged += OnRunningPackagesChanged; + try + { + // Re-check in case it connected, or the package already stopped, between the + // initial checks and subscribing + if (ClientManager.IsConnected) + return true; + if (!IsAnyComfyRunning()) + return false; + + // Give up waiting after a generous timeout in case startup never completes + using var timeoutCts = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken); + timeoutCts.CancelAfter(TimeSpan.FromMinutes(5)); + + OutputProgress.IsIndeterminate = true; + OutputProgress.Text = "Waiting for ComfyUI to start..."; + + await using (timeoutCts.Token.Register(() => tcs.TrySetCanceled(timeoutCts.Token))) + { + await tcs.Task; + } + + return ClientManager.IsConnected; + } + catch (OperationCanceledException) + { + return ClientManager.IsConnected; + } + finally + { + ClientManager.PropertyChanged -= OnPropertyChanged; + runningPackageService.RunningPackages.CollectionChanged -= OnRunningPackagesChanged; + OutputProgress.ClearProgress(); + } + } + /// /// Shows a dialog and return false if prompt required extensions not installed /// @@ -680,7 +770,8 @@ private async Task CheckPromptExtensionsInstalled(NodeDictionary nodeDicti var localExtensionsByGitUrl = localExtensions .Where(ext => ext.GitRepositoryUrl is not null) - .ToDictionary(ext => ext.GitRepositoryUrl!, ext => ext); + .DistinctBy(ext => ext.GitRepositoryUrl!, StringComparer.OrdinalIgnoreCase) + .ToDictionary(ext => ext.GitRepositoryUrl!, ext => ext, StringComparer.OrdinalIgnoreCase); var requiredExtensionReferences = requiredExtensionSpecifiers .Select(specifier => specifier.Name) @@ -729,93 +820,15 @@ out var localSemVersion return true; } - var dialog = DialogHelper.CreateMarkdownDialog( - $"#### The following extensions are required for this workflow:\n" - + $"{string.Join("\n- ", missingExtensions.Select(ext => ext.Name))}" - + $"{string.Join("\n- ", outOfDateExtensions.Select(pair => $"{pair.Item1.Name} {pair.Specifier.Constraint} {pair.Specifier.Version} (Current Version: {pair.Installed.Version?.Tag})"))}", - "Install Required Extensions?" + await ComfyExtensionInstallHelper.PromptInstallAndRestartAsync( + manager, + localPackagePair, + missingExtensions, + outOfDateExtensions, + runningPackageService, + notificationService ); - dialog.IsPrimaryButtonEnabled = true; - dialog.DefaultButton = ContentDialogButton.Primary; - dialog.PrimaryButtonText = - $"{Resources.Action_Install} ({localPackagePair.InstalledPackage.DisplayName.ToRepr()} will restart)"; - dialog.CloseButtonText = Resources.Action_Cancel; - - if (await dialog.ShowAsync() == ContentDialogResult.Primary) - { - var manifestExtensionsMap = await manager.GetManifestExtensionsMapAsync( - manager.GetManifests(localPackagePair.InstalledPackage) - ); - - var steps = new List(); - - // Add install for missing extensions - foreach (var missingExtension in missingExtensions) - { - if (!manifestExtensionsMap.TryGetValue(missingExtension.Name, out var extension)) - { - Logger.Warn( - "Extension {MissingExtensionUrl} not found in manifests", - missingExtension.Name - ); - continue; - } - - steps.Add(new InstallExtensionStep(manager, localPackagePair.InstalledPackage, extension)); - } - - // Add update for out of date extensions - foreach (var (specifier, installed) in outOfDateExtensions) - { - if (!manifestExtensionsMap.TryGetValue(specifier.Name, out var extension)) - { - Logger.Warn("Extension {MissingExtensionUrl} not found in manifests", specifier.Name); - continue; - } - - steps.Add(new UpdateExtensionStep(manager, localPackagePair.InstalledPackage, installed)); - } - - var runner = new PackageModificationRunner - { - ShowDialogOnStart = true, - ModificationCompleteTitle = "Extensions Installed", - ModificationCompleteMessage = "Finished installing required extensions", - }; - EventManager.Instance.OnPackageInstallProgressAdded(runner); - - runner - .ExecuteSteps(steps) - .ContinueWith(async _ => - { - if (runner.Failed) - return; - - // Restart Package - try - { - await Dispatcher.UIThread.InvokeAsync(async () => - { - await runningPackageService.StopPackage(localPackagePair.InstalledPackage.Id); - await runningPackageService.StartPackage(localPackagePair.InstalledPackage); - }); - } - catch (Exception e) - { - Logger.Error(e, "Error while restarting package"); - - notificationService.ShowPersistent( - new AppException( - "Could not restart package", - "Please manually restart the package for extension changes to take effect" - ) - ); - } - }) - .SafeFireAndForget(); - } - return false; } diff --git a/StabilityMatrix.Avalonia/ViewModels/CheckpointBrowser/CheckpointBrowserCardViewModel.cs b/StabilityMatrix.Avalonia/ViewModels/CheckpointBrowser/CheckpointBrowserCardViewModel.cs index 5eccf5dc1..47f679977 100644 --- a/StabilityMatrix.Avalonia/ViewModels/CheckpointBrowser/CheckpointBrowserCardViewModel.cs +++ b/StabilityMatrix.Avalonia/ViewModels/CheckpointBrowser/CheckpointBrowserCardViewModel.cs @@ -140,7 +140,8 @@ private void CheckIfInstalled() var latestVersionInstalled = latestVersion.Files != null && latestVersion.Files.Any(file => - file is { Type: CivitFileType.Model, Hashes.BLAKE3: not null } + file is { Hashes.BLAKE3: not null } + && file.Type.IsModelWeights() && installedModels.Contains(file.Hashes.BLAKE3) ); @@ -150,7 +151,8 @@ private void CheckIfInstalled() || CivitModel.ModelVersions.Any(version => version.Files != null && version.Files.Any(file => - file is { Type: CivitFileType.Model, Hashes.BLAKE3: not null } + file is { Hashes.BLAKE3: not null } + && file.Type.IsModelWeights() && installedModels.Contains(file.Hashes.BLAKE3) ) ); @@ -244,8 +246,7 @@ private async Task DoImport( } // Get latest version file - var modelFile = - selectedFile ?? modelVersion.Files?.FirstOrDefault(x => x.Type == CivitFileType.Model); + var modelFile = selectedFile ?? modelVersion.Files?.FirstOrDefault(x => x.Type.IsModelWeights()); if (modelFile is null) { notificationService.Show( diff --git a/StabilityMatrix.Avalonia/ViewModels/CheckpointBrowser/CivitDetailsPageViewModel.cs b/StabilityMatrix.Avalonia/ViewModels/CheckpointBrowser/CivitDetailsPageViewModel.cs index 1f9152ac4..0d25ef06e 100644 --- a/StabilityMatrix.Avalonia/ViewModels/CheckpointBrowser/CivitDetailsPageViewModel.cs +++ b/StabilityMatrix.Avalonia/ViewModels/CheckpointBrowser/CivitDetailsPageViewModel.cs @@ -202,6 +202,18 @@ protected override async Task OnInitialLoadedAsync() { CivitModel = await civitApi.GetModelById(CivitModel.Id); } + catch (ApiException e) when (e.StatusCode == System.Net.HttpStatusCode.NotFound) + { + // Model page was deleted from CivitAI — fall through to render whatever local + // data we were navigated with instead of leaving a dead page (GH #1695). + logger.LogWarning("CivitModel {Id} no longer exists on CivitAI (404)", CivitModel.Id); + notificationService.Show( + "Model removed from CivitAI", + "This model's page no longer exists on CivitAI. Showing locally cached info. " + + "You can right-click the model card and select \"Disconnect from Source\" to stop seeing this.", + NotificationType.Warning + ); + } catch (Exception e) { logger.LogError(e, "Failed to load CivitModel {Id}", CivitModel.Id); @@ -210,7 +222,6 @@ protected override async Task OnInitialLoadedAsync() e.Message, NotificationType.Error ); - return; } } @@ -701,7 +712,8 @@ private async Task DeleteModelVersion(CivitModelVersion modelVersion) foreach (var file in modelVersion.Files) { - if (file is not { Type: CivitFileType.Model, Hashes.BLAKE3: not null }) + // Match install detection: hash-based only, so Unknown-typed files are also deleted + if (file is not { Hashes.BLAKE3: not null }) continue; var matchingModels = (await modelIndexService.FindByHashAsync(file.Hashes.BLAKE3)).ToList(); @@ -776,11 +788,35 @@ private async Task DeleteModelVersion(CivitModelVersion modelVersion) private async Task NavigateToModelByIndexOffset(int offset) { var newIndex = CurrentIndex + offset; - var modelId = ModelIdList[newIndex]; - try + while (newIndex >= 0 && newIndex < ModelIdList.Count) { - var newModel = await civitApi.GetModelById(modelId); + var modelId = ModelIdList[newIndex]; + + CivitModel newModel; + try + { + newModel = await civitApi.GetModelById(modelId); + } + catch (ApiException e) when (e.StatusCode == System.Net.HttpStatusCode.NotFound) + { + // Model page was deleted from CivitAI — skip past it in the same direction + // instead of getting stuck on an error at this index (GH #1695) + logger.LogWarning("CivitModel {Id} no longer exists on CivitAI (404); skipping", modelId); + newIndex += Math.Sign(offset); + continue; + } + catch (Exception e) + { + logger.LogError(e, "Failed to load CivitModel {Id}", modelId); + notificationService.Show( + Resources.Label_UnexpectedErrorOccurred, + e.Message, + NotificationType.Error + ); + return; + } + CivitModel = newModel; CurrentIndex = newIndex; @@ -795,16 +831,14 @@ private async Task NavigateToModelByIndexOffset(int offset) ModelVersionDescription = string.IsNullOrWhiteSpace(SelectedVersion?.ModelVersion.Description) ? string.Empty : $"""{SelectedVersion.ModelVersion.Description}"""; + return; } - catch (Exception e) - { - logger.LogError(e, "Failed to load CivitModel {Id}", modelId); - notificationService.Show( - Resources.Label_UnexpectedErrorOccurred, - e.Message, - NotificationType.Error - ); - } + + notificationService.Show( + "Model removed from CivitAI", + "The remaining models in this direction are no longer available on CivitAI.", + NotificationType.Warning + ); } private void VmOnNavigateToModelRequested(object? sender, int modelId) @@ -826,7 +860,7 @@ private bool ShouldIncludeCivitFile(CivitFile file) if (ShowTrainingData) return true; - return file.Type is CivitFileType.Model or CivitFileType.PrunedModel or CivitFileType.VAE; + return file.Type.IsDownloadableModelFile(); } partial void OnSelectedVersionChanged(ModelVersionViewModel? value) @@ -948,50 +982,19 @@ private static DirectoryPath GetSharedFolderPath( CivitModelType modelType, string? baseModelType, string? fileName = null - ) - { - if (fileType is CivitFileType.VAE) - { - return rootModelsDirectory.JoinDir(SharedFolderType.VAE.GetStringValue()); - } - - if ( - modelType is CivitModelType.Checkpoint - && ( - baseModelType == CivitBaseModelType.Flux1D.GetStringValue() - || baseModelType == CivitBaseModelType.Flux1S.GetStringValue() - || baseModelType == CivitBaseModelType.WanVideo.GetStringValue() - || baseModelType?.StartsWith("Wan", StringComparison.OrdinalIgnoreCase) is true - || baseModelType?.StartsWith("Flux", StringComparison.OrdinalIgnoreCase) is true - || baseModelType?.StartsWith("Hunyuan", StringComparison.OrdinalIgnoreCase) is true - ) - ) - { - return rootModelsDirectory.JoinDir(SharedFolderType.DiffusionModels.GetStringValue()); - } - - // GGUF checkpoints are always UNet-only, route directly to DiffusionModels - if ( - modelType is CivitModelType.Checkpoint - && fileName is not null - && Path.GetExtension(fileName).Equals(".gguf", StringComparison.OrdinalIgnoreCase) - ) - { - return rootModelsDirectory.JoinDir(SharedFolderType.DiffusionModels.GetStringValue()); - } - - return rootModelsDirectory.JoinDir(modelType.ConvertTo().GetStringValue()); - } + ) => + rootModelsDirectory.JoinDir( + (fileType ?? CivitFileType.Unknown) + .GetSharedFolderType(modelType, baseModelType, fileName) + .GetStringValue() + ); private async Task TryMoveDownloadedCheckpointToDiffusionModelsIfNeededAsync( CivitFile civitFile, DirectoryPath requestedDestinationDir ) { - if ( - civitFile.Type is not (CivitFileType.Model or CivitFileType.PrunedModel) - || CivitModel.Type is not CivitModelType.Checkpoint - ) + if (!civitFile.Type.IsModelWeights() || CivitModel.Type is not CivitModelType.Checkpoint) { return; } diff --git a/StabilityMatrix.Avalonia/ViewModels/CheckpointManager/CheckpointFileViewModel.cs b/StabilityMatrix.Avalonia/ViewModels/CheckpointManager/CheckpointFileViewModel.cs index 24b088811..2b230df9c 100644 --- a/StabilityMatrix.Avalonia/ViewModels/CheckpointManager/CheckpointFileViewModel.cs +++ b/StabilityMatrix.Avalonia/ViewModels/CheckpointManager/CheckpointFileViewModel.cs @@ -69,6 +69,12 @@ public partial class CheckpointFileViewModel : SelectableViewModelBase public bool HasEarlyAccessUpdateOnly => CheckpointFile.HasEarlyAccessUpdateOnly; public bool HasStandardUpdate => CheckpointFile.HasUpdate && !CheckpointFile.HasEarlyAccessUpdateOnly; + public bool CanDisconnect => + CheckpointFile.HasCivitMetadata + || CheckpointFile.HasOpenModelDbMetadata + || CheckpointFile.HasCivArchiveMetadata + || CheckpointFile.ConnectedModelInfo?.SourceUrl is not null; + /// public CheckpointFileViewModel( ISettingsManager settingsManager, @@ -219,6 +225,67 @@ is not IMetadataImportService importService } } + /// + /// Severs the link to the remote source (e.g. a CivitAI page that has since been deleted) + /// while keeping the local metadata (name, description, thumbnail, trigger words) as + /// custom metadata. After this, clicking the card selects it instead of opening the + /// remote details page. + /// + [RelayCommand] + private async Task DisconnectConnectedModelAsync() + { + if (CheckpointFile.ConnectedModelInfo is not { } cmInfo) + return; + + var dialog = new ContentDialog + { + Title = "Disconnect from source", + Content = + "This removes the link to the model's original page (e.g. CivitAI), so clicking the card " + + "will no longer try to open it. The local metadata — name, description, thumbnail and " + + "trigger words — is kept and stays editable via Edit Metadata.", + PrimaryButtonText = Resources.Action_Disconnect, + CloseButtonText = Resources.Action_Cancel, + DefaultButton = ContentDialogButton.Close, + }; + + if (await dialog.ShowAsync() != ContentDialogResult.Primary) + return; + + try + { + cmInfo.ModelId = null; + cmInfo.VersionId = null; + cmInfo.RemoteFileId = null; + cmInfo.SourceUrl = null; + // Click dispatch and source badges key off Source (OpenModelDb/CivArchive don't use + // integer ids), so clearing the ids alone wouldn't sever those — null means "custom". + cmInfo.Source = null; + + var modelFilePath = new FilePath( + Path.Combine(settingsManager.ModelsDirectory, CheckpointFile.RelativePath) + ); + var modelFolder = + modelFilePath.Directory + ?? Path.Combine(settingsManager.ModelsDirectory, CheckpointFile.SharedFolderType.ToString()); + + await cmInfo.SaveJsonToDirectory(modelFolder, modelFilePath.NameWithoutExtension); + + await modelIndexService.RefreshIndex(); + + notificationService.Show( + "Model disconnected", + $"\"{CheckpointFile.DisplayModelName}\" is no longer linked to its original page.", + NotificationType.Success + ); + } + catch (Exception e) + { + logger.LogError(e, "Failed to disconnect model {Name}", CheckpointFile.RelativePath); + notificationService.Show("Failed to disconnect model", e.Message, NotificationType.Error); + } + } + [RelayCommand] private async Task DeleteAsync(bool showConfirmation = true) { diff --git a/StabilityMatrix.Avalonia/ViewModels/CheckpointsPageViewModel.cs b/StabilityMatrix.Avalonia/ViewModels/CheckpointsPageViewModel.cs index 16f5764d3..244b3313f 100644 --- a/StabilityMatrix.Avalonia/ViewModels/CheckpointsPageViewModel.cs +++ b/StabilityMatrix.Avalonia/ViewModels/CheckpointsPageViewModel.cs @@ -685,7 +685,12 @@ private Task OnItemClick(CheckpointFileViewModel item) // Select item if we're in "select mode" if (NumItemsSelected > 0) item.IsSelected = !item.IsSelected; - else if (item.CheckpointFile.HasConnectedModel) + else if ( + item.CheckpointFile + is { HasCivitMetadata: true } + or { HasOpenModelDbMetadata: true } + or { HasCivArchiveMetadata: true } + ) return ShowVersionDialog(item); else item.IsSelected = !item.IsSelected; diff --git a/StabilityMatrix.Avalonia/ViewModels/Controls/PaintCanvasViewModel.Compose.cs b/StabilityMatrix.Avalonia/ViewModels/Controls/PaintCanvasViewModel.Compose.cs new file mode 100644 index 000000000..e75f9da31 --- /dev/null +++ b/StabilityMatrix.Avalonia/ViewModels/Controls/PaintCanvasViewModel.Compose.cs @@ -0,0 +1,146 @@ +using System.Collections.Immutable; +using System.Drawing; +using System.Linq; +using Microsoft.Extensions.Logging; +using SkiaSharp; +using StabilityMatrix.Avalonia.Controls.Models; + +namespace StabilityMatrix.Avalonia.ViewModels.Controls; + +public partial class PaintCanvasViewModel +{ + /// + /// An immutable snapshot of everything needed to composite the canvas, captured on the + /// UI thread. Off-screen exports compose exclusively from these inputs onto CPU surfaces + /// they own, so they never touch the persistent layer surfaces that belong to the + /// on-screen render pass (see ). + /// + private readonly record struct ComposeInputs( + int Width, + int Height, + ImmutableList BackgroundBitmaps, + ImmutableList ImagesBitmaps, + ImmutableList BrushBitmaps, + ImmutableList OverlayBitmaps, + ImmutableList Paths, + LiveStroke[] LiveStrokes, + bool ShowGrid, + int GridDivisions + ); + + /// + /// Captures the current canvas state as immutable references. The bitmap lists are + /// atomically-swapped instances, finalized paths are an + /// immutable list, and live strokes provide stable point snapshots — so the returned + /// value can be composed from without any locks. + /// + private ComposeInputs CaptureComposeInputs(bool renderBackgroundImage) + { + return new ComposeInputs( + CanvasSize.Width, + CanvasSize.Height, + renderBackgroundImage ? BackgroundLayer.Bitmaps : [], + ImagesLayer.Bitmaps, + BrushLayer.Bitmaps, + OverlayLayer.Bitmaps, + Paths, + TemporaryPaths.Values.ToArray(), + ShowGridOverlay, + GridDivisions + ); + } + + /// + /// Pure compositor: renders a snapshot of the canvas onto the target canvas. Creates its + /// own scratch CPU surface for the brush layer (so erase strokes clear brush content only, + /// matching the on-screen per-layer compositing) and never reads or mutates shared + /// surface state. + /// + private static void ComposeToCanvas(SKCanvas target, in ComposeInputs inputs) + { + target.Clear(SKColors.Transparent); + + // Background and Images layers contain only plain bitmap draws, so compositing them + // through an intermediate surface is equivalent to drawing them directly. + foreach (var bitmap in inputs.BackgroundBitmaps) + { + target.DrawBitmap(bitmap, 0, 0); + } + + foreach (var bitmap in inputs.ImagesBitmaps) + { + target.DrawBitmap(bitmap, 0, 0); + } + + // The brush layer needs isolation: erase strokes use SKBlendMode.Clear and must only + // erase brush-layer content, not the layers beneath. + using (var brushSurface = SKSurface.Create(new SKImageInfo(inputs.Width, inputs.Height))) + { + if (brushSurface is not null) + { + var brushCanvas = brushSurface.Canvas; + brushCanvas.Clear(SKColors.Transparent); + + foreach (var bitmap in inputs.BrushBitmaps) + { + brushCanvas.DrawBitmap(bitmap, 0, 0); + } + + using var paint = new SKPaint(); + + foreach (var penPath in inputs.Paths) + { + RenderPenPath(brushCanvas, penPath, paint); + } + + foreach (var stroke in inputs.LiveStrokes) + { + RenderLiveStroke(brushCanvas, stroke, paint); + } + + brushCanvas.Flush(); + target.DrawSurface(brushSurface, new SKPoint(0, 0)); + } + } + + foreach (var bitmap in inputs.OverlayBitmaps) + { + target.DrawBitmap(bitmap, 0, 0); + } + + if (inputs.ShowGrid) + { + RenderGridOverlayCore(target, inputs.Width, inputs.Height, inputs.GridDivisions); + } + + target.Flush(); + } + + /// + /// Composes a snapshot of the canvas into a new CPU-backed . + /// Safe to call from the UI thread at any time; does not interact with the on-screen + /// render pass or its surfaces. + /// + private SKImage? ComposeToNewImage(bool renderBackgroundImage) + { + if (CanvasSize == Size.Empty) + { + logger.LogWarning($"ComposeToNewImage: {nameof(CanvasSize)} is not set, returning null."); + return null; + } + + var inputs = CaptureComposeInputs(renderBackgroundImage); + + // SKSurface.Create can return null under low memory + using var surface = SKSurface.Create(new SKImageInfo(inputs.Width, inputs.Height)); + if (surface is null) + { + logger.LogWarning("ComposeToNewImage: Failed to create surface, returning null."); + return null; + } + + ComposeToCanvas(surface.Canvas, inputs); + + return surface.Snapshot(); + } +} diff --git a/StabilityMatrix.Avalonia/ViewModels/Controls/PaintCanvasViewModel.Serializer.cs b/StabilityMatrix.Avalonia/ViewModels/Controls/PaintCanvasViewModel.Serializer.cs index 35894f94c..dc5314dc5 100644 --- a/StabilityMatrix.Avalonia/ViewModels/Controls/PaintCanvasViewModel.Serializer.cs +++ b/StabilityMatrix.Avalonia/ViewModels/Controls/PaintCanvasViewModel.Serializer.cs @@ -40,7 +40,8 @@ protected PaintCanvasModel SaveState() { var model = new PaintCanvasModel { - TemporaryPaths = TemporaryPaths.ToDictionary(x => x.Key, x => x.Value), + // LiveStrokes are frozen into plain PenPaths so the serialized shape is unchanged + TemporaryPaths = TemporaryPaths.ToDictionary(x => x.Key, x => x.Value.ToPenPath()), Paths = Paths, PaintBrushColor = PaintBrushColor, PaintBrushSize = PaintBrushSize, @@ -49,7 +50,7 @@ protected PaintCanvasModel SaveState() CurrentZoom = CurrentZoom, IsPenDown = IsPenDown, SelectedTool = SelectedTool, - CanvasSize = CanvasSize + CanvasSize = CanvasSize, }; return model; @@ -60,7 +61,7 @@ protected void LoadState(PaintCanvasModel model) TemporaryPaths.Clear(); foreach (var (key, value) in model.TemporaryPaths) { - TemporaryPaths.TryAdd(key, value); + TemporaryPaths.TryAdd(key, LiveStroke.FromPenPath(value)); } Paths = model.Paths; diff --git a/StabilityMatrix.Avalonia/ViewModels/Controls/PaintCanvasViewModel.cs b/StabilityMatrix.Avalonia/ViewModels/Controls/PaintCanvasViewModel.cs index 67364ab4a..313babba8 100644 --- a/StabilityMatrix.Avalonia/ViewModels/Controls/PaintCanvasViewModel.cs +++ b/StabilityMatrix.Avalonia/ViewModels/Controls/PaintCanvasViewModel.cs @@ -28,8 +28,47 @@ public partial class PaintCanvasViewModel(ILogger logger) : LoadableViewModelBase, IDisposable { - private bool _disposed; - public ConcurrentDictionary TemporaryPaths { get; set; } = new(); + // Threading model (no locks): + // * UI thread: all mutation (strokes, undo/redo, layer bitmap swaps, exports, Dispose). + // Shared state crosses to the render thread only as immutable snapshots — ImmutableList + // swaps for Paths and SKLayer.Bitmaps, LiveStroke point-array publication. + // * Render thread (compositor): RenderToSurface only. It exclusively owns the persistent + // native objects (SKLayer.Surface, cachedPathsImage, the checkerboard shader) — they are + // created, rebuilt and disposed only inside the render pass. + // * Cross-thread disposal is deferred: bitmaps swapped out on the UI thread are queued to + // retiredLayerBitmaps and freed by the render thread after the frame completes; cache + // invalidation sets pathCacheDirty instead of disposing. + // * Dispose quiesces first: it sets _disposed (checked at render entry) and waits for + // rendersInFlight to drain before freeing render-owned resources. + private volatile bool _disposed; + + /// + /// Number of render passes currently inside . Used by + /// to wait for the in-flight frame before freeing native resources. + /// + private int rendersInFlight; + + /// + /// Set (to 1) by the UI thread when the finalized-path cache is stale; the render thread + /// consumes it with an atomic read-and-reset and disposes/rebuilds + /// on its own thread. Int rather than bool so + /// can swap it — a plain check-then-clear + /// would drop an invalidation raised between the check and the clear. + /// + private int pathCacheDirty; + + /// + /// Bitmaps swapped out of layers on the UI thread while the render thread may still be + /// drawing them. Drained (disposed) by the render thread after each frame, and by + /// . + /// + private readonly ConcurrentQueue retiredLayerBitmaps = new(); + + /// + /// Strokes currently being drawn, keyed by pointer id. Values are s: + /// the UI thread appends points while the render thread reads stable snapshots, without locks. + /// + public ConcurrentDictionary TemporaryPaths { get; set; } = new(); [ObservableProperty] [NotifyCanExecuteChangedFor(nameof(UndoCommand))] @@ -86,9 +125,6 @@ public partial class PaintCanvasViewModel(ILogger logger) [ObservableProperty] private bool isShapeStrokeOnly; - [JsonIgnore] - private SKCanvas? SourceCanvas { set; get; } - [Localizable(false)] [JsonIgnore] private OrderedDictionary Layers { get; } = @@ -125,19 +161,6 @@ public partial class PaintCanvasViewModel(ILogger logger) [JsonIgnore] private int cachedPathsCount; - /// - /// Cached surface for temporary paths during active drawing. - /// Allows incremental rendering of long strokes. - /// - [JsonIgnore] - private SKSurface? tempPathSurface; - - /// - /// Tracks how many points have been rendered to the temp path surface per pointer ID. - /// - [JsonIgnore] - private readonly ConcurrentDictionary tempPathRenderedPoints = new(); - /// /// Whether to use GPU-accelerated surfaces when available. /// @@ -262,38 +285,40 @@ public void SetLayerBitmap(string name, SKBitmap? bitmap) return; } - // Dispose old bitmaps before replacing to prevent memory leaks - lock (layer) - { - foreach (var oldBitmap in layer.Bitmaps) - { - oldBitmap.Dispose(); - } - - layer.Bitmaps = bitmap is not null ? [bitmap] : []; - } + var oldBitmaps = layer.Bitmaps; + layer.Bitmaps = bitmap is not null ? [bitmap] : []; + RetireLayerBitmaps(oldBitmaps); } - public void SetSourceCanvas(SKCanvas canvas) + public void LoadCanvasFromBitmap(SKBitmap bitmap) { - ArgumentNullException.ThrowIfNull(canvas); - SourceCanvas = canvas; + var oldBitmaps = ImagesLayer.Bitmaps; + ImagesLayer.Bitmaps = [bitmap]; + RetireLayerBitmaps(oldBitmaps); + + InvalidatePathCache(); + RefreshCanvas?.Invoke(); } - public void LoadCanvasFromBitmap(SKBitmap bitmap) + /// + /// Frees bitmaps that were swapped out of a layer. When the canvas renders on-screen + /// ( is wired), the render thread may still be drawing them, + /// so they are queued and disposed by the render thread after the frame. For export-only + /// view models that never render on-screen, they are disposed immediately. + /// + private void RetireLayerBitmaps(ImmutableList oldBitmaps) { - // Dispose old bitmaps and invalidate cache - lock (ImagesLayer) + foreach (var oldBitmap in oldBitmaps) { - foreach (var oldBitmap in ImagesLayer.Bitmaps) + if (RefreshCanvas is null) { oldBitmap.Dispose(); } - ImagesLayer.Bitmaps = [bitmap]; + else + { + retiredLayerBitmaps.Enqueue(oldBitmap); + } } - - InvalidatePathCache(); - RefreshCanvas?.Invoke(); } [RelayCommand(CanExecute = nameof(CanExecuteUndo))] @@ -340,12 +365,12 @@ public void Redo() /// /// Invalidates the cached paths image. Call when paths are modified externally. + /// The cache itself is owned by the render thread, so this only raises a flag; the + /// render thread disposes and rebuilds the cache on its own thread. /// public void InvalidatePathCache() { - cachedPathsImage?.Dispose(); - cachedPathsImage = null; - cachedPathsCount = 0; + Interlocked.Exchange(ref pathCacheDirty, 1); } /// @@ -557,7 +582,7 @@ public void UpdateShapePreview(SKPoint currentPoint) IsStrokeOnly = IsShapeStrokeOnly, StrokeWidth = (float)PaintBrushSize, }; - TemporaryPaths[ShapePointerId] = previewPath; + TemporaryPaths[ShapePointerId] = new LiveStroke { Template = previewPath }; } /// @@ -710,25 +735,23 @@ private SKBitmap GetFlattenedContentBitmap() using var canvas = new SKCanvas(bitmap); canvas.Clear(SKColors.Transparent); - // Draw all layers in order + // Draw all layers in order. Runs on the UI thread; layer.Bitmaps is an atomically-swapped + // immutable list only ever mutated from the UI thread, so a plain read is safe. foreach (var (name, layer) in Layers) { - lock (layer) + foreach (var layerBitmap in layer.Bitmaps) { - foreach (var layerBitmap in layer.Bitmaps) - { - canvas.DrawBitmap(layerBitmap, 0, 0); - } + canvas.DrawBitmap(layerBitmap, 0, 0); + } - // If this is the active brush layer, also render the active vector paths - // We render them freshly here on CPU to avoid using the GPU-backed cache from a different thread - if (name == "Brush") + // If this is the active brush layer, also render the active vector paths + // We render them freshly here on CPU to avoid using the GPU-backed cache from a different thread + if (name == "Brush") + { + using var paint = new SKPaint(); + foreach (var path in Paths) { - using var paint = new SKPaint(); - foreach (var path in Paths) - { - RenderPenPath(canvas, path, paint); - } + RenderPenPath(canvas, path, paint); } } } @@ -760,6 +783,8 @@ SKColor fillColor var queue = new Queue<(int x, int y)>(); queue.Enqueue((startX, startY)); + var hasContent = false; + // Collect horizontal spans to draw var spans = new List<(int y, int left, int right)>(); @@ -834,6 +859,7 @@ SKColor fillColor 1 + (Expand * 2), paint ); + hasContent = true; // Queue pixels above and below the span for (var i = left; i <= right; i++) @@ -862,14 +888,7 @@ SKColor fillColor } } - // Check if anything was filled (at least one visited pixel) - foreach (var v in visited) - { - if (v) - return true; - } - - return false; + return hasContent; } private static bool ColorsAreSimilar(SKColor a, SKColor b, int tolerance) @@ -892,17 +911,18 @@ private static bool ColorsAreSimilar(SKColor a, SKColor b, int tolerance) { using var _ = CodeTimer.StartDebug(); - if (CanvasSize == Size.Empty) + using var originalImage = ComposeToNewImage(renderBackgroundImage: false); + if (originalImage is null) { - logger.LogWarning($"RenderToImage: {nameof(CanvasSize)} is not set, returning null."); return null; } - using var surface = SKSurface.Create(new SKImageInfo(CanvasSize.Width, CanvasSize.Height)); - - RenderToSurface(surface); - - using var originalImage = surface.Snapshot(); + using var surface = SKSurface.Create(new SKImageInfo(originalImage.Width, originalImage.Height)); + if (surface is null) + { + logger.LogWarning("RenderToWhiteChannelImage: Failed to create surface, returning null."); + return null; + } // Replace all colors to white (255, 255, 255), keep original alpha // csharpier-ignore using var colorFilter = SKColorFilter.CreateColorMatrix( @@ -924,21 +944,16 @@ private static bool ColorsAreSimilar(SKColor a, SKColor b, int tolerance) return surface.Snapshot(); } - public SKImage? RenderToImage() + /// + /// Composes the canvas into a new CPU-backed image on the calling (UI) thread, without + /// touching the persistent surfaces owned by the on-screen render pass. + /// + /// Whether to include the background image layer. + public SKImage? RenderToImage(bool renderBackgroundImage = false) { using var _ = CodeTimer.StartDebug(); - if (CanvasSize == Size.Empty) - { - logger.LogWarning($"RenderToImage: {nameof(CanvasSize)} is not set, returning null."); - return null; - } - - using var surface = SKSurface.Create(new SKImageInfo(CanvasSize.Width, CanvasSize.Height)); - - RenderToSurface(surface); - - return surface.Snapshot(); + return ComposeToNewImage(renderBackgroundImage); } /// @@ -969,19 +984,15 @@ public Dictionary ExtractMasksByColors( var srcPixels = sourceBitmap.Pixels; // SKColor[] array - fast direct access var pixelCount = srcPixels.Length; - // Create result bitmaps for each color - var resultBitmaps = new Dictionary(); - var resultPixels = new Dictionary(); - foreach (var color in targetColors) + // Use flat arrays in the per-pixel loop to avoid dictionary lookups per pixel per color. + // default(SKColor) is transparent, so only matches need to be written. + var colorCount = targetColors.Count; + var colors = new SKColor[colorCount]; + var resultPixels = new SKColor[colorCount][]; + for (var c = 0; c < colorCount; c++) { - var bitmap = new SKBitmap( - sourceBitmap.Width, - sourceBitmap.Height, - SKColorType.Rgba8888, - SKAlphaType.Premul - ); - resultBitmaps[color] = bitmap; - resultPixels[color] = new SKColor[pixelCount]; + colors[c] = targetColors[c]; + resultPixels[c] = new SKColor[pixelCount]; } // Single pass through pixels, check all colors @@ -989,24 +1000,34 @@ public Dictionary ExtractMasksByColors( { var pixel = srcPixels[i]; - foreach (var targetColor in targetColors) + if (pixel.Alpha == 0) + continue; + + for (var c = 0; c < colorCount; c++) { - var matches = + var targetColor = colors[c]; + if ( Math.Abs(pixel.Red - targetColor.Red) <= tolerance && Math.Abs(pixel.Green - targetColor.Green) <= tolerance && Math.Abs(pixel.Blue - targetColor.Blue) <= tolerance - && pixel.Alpha > 0; - - resultPixels[targetColor][i] = matches ? SKColors.White : SKColors.Transparent; + ) + { + resultPixels[c][i] = SKColors.White; + } } } // Set pixels and convert bitmaps to images - foreach (var (color, bitmap) in resultBitmaps) + for (var c = 0; c < colorCount; c++) { - bitmap.Pixels = resultPixels[color]; - results[color] = SKImage.FromBitmap(bitmap); - bitmap.Dispose(); + using var bitmap = new SKBitmap( + sourceBitmap.Width, + sourceBitmap.Height, + SKColorType.Rgba8888, + SKAlphaType.Premul + ); + bitmap.Pixels = resultPixels[c]; + results[colors[c]] = SKImage.FromBitmap(bitmap); } return results; @@ -1143,90 +1164,125 @@ private static bool ColorMatchesWithTolerance(SKColor a, SKColor b, int toleranc && Math.Abs(a.Blue - b.Blue) <= tolerance; } + /// + /// On-screen render entry point, called by the compositor render thread each frame. + /// Tracked by so can wait for the + /// in-flight frame before freeing the native resources this pass draws with. + /// public void RenderToSurface( SKSurface surface, bool renderBackgroundFill = false, bool renderBackgroundImage = false ) { + Interlocked.Increment(ref rendersInFlight); + try + { + if (_disposed) + { + return; + } + + RenderToSurfaceCore(surface, renderBackgroundFill, renderBackgroundImage); + } + finally + { + Interlocked.Decrement(ref rendersInFlight); + } + } + + private void RenderToSurfaceCore( + SKSurface? surface, + bool renderBackgroundFill, + bool renderBackgroundImage + ) + { + // SKSurface.Create can return null under low memory or GPU context loss + if (surface is null || _disposed) + { + return; + } + + // A zero-size canvas would make SKSurface.Create return null below and NRE on layer.Surface + if (CanvasSize.Width <= 0 || CanvasSize.Height <= 0) + { + surface.Canvas.Clear(SKColors.Transparent); + return; + } + var grContext = surface.Context; var useGpu = UseGpuAcceleration && grContext != null; IsUsingGpu = useGpu; - // Initialize canvas layers + // Initialize canvas layers. The persistent layer surfaces are exclusively owned by this + // render pass (exports compose their own CPU surfaces from immutable snapshots — see + // PaintCanvasViewModel.Compose.cs), so no locking is needed. Recreate when missing, when + // the GPU context changed (device loss / GPU toggle), or on resize. foreach (var layer in Layers.Values) { - lock (layer) + var needsNewSurface = layer.Surface is null; + if (!needsNewSurface) { - var needsNewSurface = layer.Surface is null; - if (!needsNewSurface) + // Compare native handles: managed GRContext wrappers are not guaranteed unique + var expectedContextHandle = useGpu ? grContext!.Handle : IntPtr.Zero; + var layerContextHandle = layer.Surface!.Context?.Handle ?? IntPtr.Zero; + if (layerContextHandle != expectedContextHandle) { - // Recreate if the existing surface's backing doesn't match the current target. - // On-screen rendering leases a GPU surface, so the persistent layer surfaces are - // GPU-backed and tied to the render thread. Off-screen export (e.g. saving an - // annotation) composites onto a CPU surface from another thread; reusing those - // GPU surfaces there produces a blank image. Forcing a matching CPU surface fixes - // it, and the next on-screen render simply recreates the GPU surface. - var layerIsGpu = layer.Surface!.Context != null; - if (layerIsGpu != useGpu) - { - needsNewSurface = true; - } - else - { - // Check if we need to resize - var currentInfo = layer.Surface!.Canvas.DeviceClipBounds; - needsNewSurface = - currentInfo.Width != CanvasSize.Width || currentInfo.Height != CanvasSize.Height; - } + needsNewSurface = true; } - - if (needsNewSurface) + else { - // Dispose old surface if exists - layer.Surface?.Dispose(); + // Check if we need to resize + var currentInfo = layer.Surface!.Canvas.DeviceClipBounds; + needsNewSurface = + currentInfo.Width != CanvasSize.Width || currentInfo.Height != CanvasSize.Height; + } + } - var imageInfo = new SKImageInfo(CanvasSize.Width, CanvasSize.Height); + if (needsNewSurface) + { + // Dispose old surface if exists + layer.Surface?.Dispose(); - // Try GPU surface first if available - if (useGpu) - { - layer.Surface = SKSurface.Create(grContext!, budgeted: true, imageInfo); + var imageInfo = new SKImageInfo(CanvasSize.Width, CanvasSize.Height); - // Fallback to CPU if GPU surface creation failed - if (layer.Surface is null) - { - if (LogRenderingMode) - { - logger.LogWarning( - "GPU surface creation failed, falling back to CPU for layer" - ); - } - layer.Surface = SKSurface.Create(imageInfo); - } - else if (LogRenderingMode) - { - logger.LogDebug("Created GPU-accelerated surface for layer"); - } - } - else + // Try GPU surface first if available + if (useGpu) + { + layer.Surface = SKSurface.Create(grContext!, budgeted: true, imageInfo); + + // Fallback to CPU if GPU surface creation failed + if (layer.Surface is null) { - layer.Surface = SKSurface.Create(imageInfo); if (LogRenderingMode) { - logger.LogDebug("Created CPU surface for layer (GPU not available or disabled)"); + logger.LogWarning("GPU surface creation failed, falling back to CPU for layer"); } + layer.Surface = SKSurface.Create(imageInfo); + } + else if (LogRenderingMode) + { + logger.LogDebug("Created GPU-accelerated surface for layer"); } } else { - // No resize needed, just clear - layer.Surface!.Canvas.Clear(SKColors.Transparent); + layer.Surface = SKSurface.Create(imageInfo); + if (LogRenderingMode) + { + logger.LogDebug("Created CPU surface for layer (GPU not available or disabled)"); + } } } + else + { + // No resize needed, just clear + layer.Surface!.Canvas.Clear(SKColors.Transparent); + } } - // Render all layer images in order + // Render all layer images in order. layer.Bitmaps is an atomically-swapped immutable list; + // bitmaps swapped out mid-frame stay alive in retiredLayerBitmaps until the frame completes. foreach (var (layerName, layer) in Layers) { // Skip background image if not requested @@ -1235,13 +1291,10 @@ public void RenderToSurface( continue; } - lock (layer) + var layerCanvas = layer.Surface!.Canvas; + foreach (var bitmap in layer.Bitmaps) { - var layerCanvas = layer.Surface!.Canvas; - foreach (var bitmap in layer.Bitmaps) - { - layerCanvas.DrawBitmap(bitmap, new SKPoint(0, 0)); - } + layerCanvas.DrawBitmap(bitmap, new SKPoint(0, 0)); } } @@ -1263,11 +1316,8 @@ public void RenderToSurface( // Draw the layers to the main surface foreach (var layer in Layers.Values) { - lock (layer) - { - layer.Surface!.Canvas.Flush(); - surface.Canvas.DrawSurface(layer.Surface!, new SKPoint(0, 0)); - } + layer.Surface!.Canvas.Flush(); + surface.Canvas.DrawSurface(layer.Surface!, new SKPoint(0, 0)); } // Draw grid overlay if enabled @@ -1277,6 +1327,13 @@ public void RenderToSurface( } surface.Canvas.Flush(); + + // The frame is fully drawn and flushed - bitmaps retired by UI-thread layer swaps can no + // longer be referenced by this pass, so free them now, on the thread that owns rendering. + while (retiredLayerBitmaps.TryDequeue(out var retired)) + { + retired.Dispose(); + } } /// @@ -1351,7 +1408,15 @@ private static SKShader CreateCheckerboardShader() /// private void RenderGridOverlay(SKCanvas canvas) { - if (GridDivisions <= 1 || CanvasSize == Size.Empty) + if (CanvasSize == Size.Empty) + return; + + RenderGridOverlayCore(canvas, CanvasSize.Width, CanvasSize.Height, GridDivisions); + } + + private static void RenderGridOverlayCore(SKCanvas canvas, int width, int height, int gridDivisions) + { + if (gridDivisions <= 1) return; using var paint = new SKPaint @@ -1362,20 +1427,17 @@ private void RenderGridOverlay(SKCanvas canvas) StrokeWidth = 1f, }; - var width = CanvasSize.Width; - var height = CanvasSize.Height; - // Draw vertical lines - for (var i = 1; i < GridDivisions; i++) + for (var i = 1; i < gridDivisions; i++) { - var x = (float)(width * i) / GridDivisions; + var x = (float)(width * i) / gridDivisions; canvas.DrawLine(x, 0, x, height, paint); } // Draw horizontal lines - for (var i = 1; i < GridDivisions; i++) + for (var i = 1; i < gridDivisions; i++) { - var y = (float)(height * i) / GridDivisions; + var y = (float)(height * i) / gridDivisions; canvas.DrawLine(0, y, width, y, paint); } } @@ -1386,6 +1448,16 @@ private void RenderGridOverlay(SKCanvas canvas) /// private void RenderPathsWithCaching(SKCanvas paintLayerCanvas) { + // Consume a pending invalidation: the cache is owned by this (render) thread, so this is + // where the stale image actually gets disposed and reset. Atomic read-and-reset so a + // concurrent UI-thread invalidation is never lost between a check and a clear. + if (Interlocked.Exchange(ref pathCacheDirty, 0) == 1) + { + cachedPathsImage?.Dispose(); + cachedPathsImage = null; + cachedPathsCount = 0; + } + var currentPathCount = Paths.Count; var hasTemporaryPaths = !TemporaryPaths.IsEmpty; @@ -1433,208 +1505,13 @@ private void RenderPathsWithCaching(SKCanvas paintLayerCanvas) } } - // Render temporary paths directly (the batched RenderPenPath is already optimized) - foreach (var penPath in TemporaryPaths.Values) - { - RenderPenPath(paintLayerCanvas, penPath, paint); - } - } - - /// - /// Renders temporary paths with incremental caching for long strokes. - /// Only new points since last render are drawn, dramatically improving - /// performance for continuous drawing. - /// - private void RenderTemporaryPathsIncremental(SKCanvas targetCanvas, SKPaint paint) - { - if (TemporaryPaths.IsEmpty) - { - // No temporary paths - dispose surface if exists - if (tempPathSurface != null) - { - tempPathSurface.Dispose(); - tempPathSurface = null; - tempPathRenderedPoints.Clear(); - } - return; - } - - // For simplicity and reliability, use a hybrid approach: - // - Keep a cached surface for the "already rendered" portions - // - Render new points directly to target canvas (which gets composited) - - // Ensure we have a temp surface - var needNewSurface = tempPathSurface == null; - if (!needNewSurface) - { - var bounds = tempPathSurface!.Canvas.DeviceClipBounds; - needNewSurface = bounds.Width != CanvasSize.Width || bounds.Height != CanvasSize.Height; - } - - if (needNewSurface) - { - tempPathSurface?.Dispose(); - var imageInfo = new SKImageInfo(CanvasSize.Width, CanvasSize.Height); - - // Use CPU surface for temp paths to avoid GPU context threading issues - tempPathSurface = SKSurface.Create(imageInfo); - tempPathSurface?.Canvas.Clear(SKColors.Transparent); - tempPathRenderedPoints.Clear(); - } - - if (tempPathSurface == null) - { - // Fallback: render all temp paths directly - foreach (var penPath in TemporaryPaths.Values) - { - RenderPenPath(targetCanvas, penPath, paint); - } - return; - } - - var tempCanvas = tempPathSurface.Canvas; - - // Check if any paths were removed (stroke finalized) - need to clear and rebuild - var pathsRemoved = false; - foreach (var pointerId in tempPathRenderedPoints.Keys.ToArray()) - { - if (!TemporaryPaths.ContainsKey(pointerId)) - { - pathsRemoved = true; - tempPathRenderedPoints.TryRemove(pointerId, out _); - } - } - - if (pathsRemoved) - { - // A stroke was finalized - clear the temp surface - tempCanvas.Clear(SKColors.Transparent); - tempPathRenderedPoints.Clear(); - } - - // Render each temporary path - foreach (var (pointerId, penPath) in TemporaryPaths) - { - var renderedCount = tempPathRenderedPoints.GetValueOrDefault(pointerId, 0); - var totalPoints = penPath.Points.Count; - - if (totalPoints > renderedCount) - { - if (renderedCount == 0) - { - // New path - render everything to the temp surface - RenderPenPath(tempCanvas, penPath, paint); - } - else - { - // Continuing path - render new segment to temp surface - RenderPenPathSegment(tempCanvas, penPath, renderedCount, totalPoints, paint); - } - tempPathRenderedPoints[pointerId] = totalPoints; - } - } - - // Draw the temp surface to target - tempCanvas.Flush(); - using var tempImage = tempPathSurface.Snapshot(); - targetCanvas.DrawImage(tempImage, new SKPoint(0, 0)); - } - - /// - /// Renders a segment of a pen path (from startIndex to endIndex). - /// Used for incremental rendering of temporary paths. - /// - private static void RenderPenPathSegment( - SKCanvas canvas, - PenPath penPath, - int startIndex, - int endIndex, - SKPaint paint - ) - { - if (startIndex >= endIndex || penPath.Points.Count == 0) - return; - - // Apply Color - if (penPath.IsErase) - { - paint.BlendMode = SKBlendMode.Clear; - paint.Color = SKColors.Transparent; - } - else - { - paint.BlendMode = SKBlendMode.SrcOver; - paint.Color = penPath.FillColor; - } - - paint.IsDither = true; - paint.IsAntialias = true; - paint.Style = SKPaintStyle.Stroke; - paint.StrokeCap = SKStrokeCap.Round; - paint.StrokeJoin = SKStrokeJoin.Round; - - // Apply feathering (soft brush edge) using blur mask filter - if (penPath.Feathering > 0) - { - var effectiveRadiusForBlur = penPath.GetEffectiveRadius(); - var blurSigma = effectiveRadiusForBlur * penPath.Feathering * 0.5f; - if (blurSigma > 0.1f) - { - paint.MaskFilter = SKMaskFilter.CreateBlur(SKBlurStyle.Normal, blurSigma); - } - } - else - { - paint.MaskFilter = null; - } - - using var path = new SKPath(); - var started = false; - var currentThickness = 0f; - - // Start from one point before to ensure continuity - var actualStart = Math.Max(0, startIndex - 1); - - var effectiveRadius = penPath.GetEffectiveRadius(); - - for (var i = actualStart; i < endIndex && i < penPath.Points.Count; i++) - { - var point = penPath.Points[i]; - if (!point.IsPen) - continue; - - var thickness = (float)((point.Pressure ?? 1) * effectiveRadius * 2.5); - - if (!started) - { - path.MoveTo(point.X, point.Y); - currentThickness = thickness; - started = true; - } - else - { - path.LineTo(point.X, point.Y); - currentThickness = (currentThickness + thickness) / 2; - } - } - - if (started) + // Render in-progress strokes directly (the batched rendering is already optimized) + foreach (var stroke in TemporaryPaths.Values) { - paint.StrokeWidth = currentThickness; - canvas.DrawPath(path, paint); + RenderLiveStroke(paintLayerCanvas, stroke, paint); } } - /// - /// Clears the temporary path cache. Call when a stroke is finalized. - /// - public void ClearTempPathCache() - { - tempPathSurface?.Dispose(); - tempPathSurface = null; - tempPathRenderedPoints.Clear(); - } - /// /// Updates the path cache with all current completed paths. /// Uses CPU-only surfaces to avoid GPU context threading issues. @@ -1799,6 +1676,37 @@ private static void RenderBitmapPath( } } + /// + /// Renders an in-progress to a canvas using a stable point snapshot, + /// safe to call from the render thread while the UI thread keeps appending points. + /// + public static void RenderLiveStroke( + SKCanvas canvas, + LiveStroke stroke, + SKPaint paint, + SKColor? overrideColor = null + ) + { + var template = stroke.Template; + + switch (template.PathType) + { + case PenPathType.Rectangle: + case PenPathType.Ellipse: + RenderShapePath(canvas, template, paint, overrideColor); + return; + + case PenPathType.Bitmap: + RenderBitmapPath(canvas, template, paint, overrideColor); + return; + + case PenPathType.Freehand: + default: + RenderFreehandPathCore(canvas, template, stroke.GetPointsSnapshot(), paint, overrideColor); + return; + } + } + /// /// Renders freehand paths with pressure-sensitive strokes to the canvas. /// @@ -1807,10 +1715,25 @@ private static void RenderFreehandPath( PenPath penPath, SKPaint paint, SKColor? overrideColor = null + ) => RenderFreehandPathCore(canvas, penPath, penPath.Points, paint, overrideColor); + + /// + /// Shared freehand rendering over any stable point list: a finalized path's own + /// (frozen) list, or a snapshot array. + /// + private static void RenderFreehandPathCore( + SKCanvas canvas, + PenPath penPath, + IReadOnlyList points, + SKPaint paint, + SKColor? overrideColor = null ) { - // Freehand path rendering - if (penPath.Points.Count == 0) + // Freehand path rendering. The point list is always a stable snapshot here: finalized + // PenPath lists are frozen at finalize time, and LiveStroke hands out immutable arrays. + var pointCount = points.Count; + + if (pointCount == 0) { return; } @@ -1834,12 +1757,18 @@ private static void RenderFreehandPath( paint.StrokeCap = SKStrokeCap.Round; // Round caps handle endpoints paint.StrokeJoin = SKStrokeJoin.Round; + // Get effective radius (path-level, or backward-compat from the first point — + // mirrors PenPath.GetEffectiveRadius but reads the caller-supplied point list) + var effectiveRadius = + penPath.Radius > 0 ? penPath.Radius + : pointCount > 0 && points[0].Radius > 0 ? (float)points[0].Radius + : 1f; + // Apply feathering (soft brush edge) using blur mask filter if (penPath.Feathering > 0) { // Calculate blur sigma based on the effective radius and feathering amount - var effectiveRadiusForBlur = penPath.GetEffectiveRadius(); - var blurSigma = effectiveRadiusForBlur * penPath.Feathering * 0.5f; + var blurSigma = effectiveRadius * penPath.Feathering * 0.5f; if (blurSigma > 0.1f) { paint.MaskFilter = SKMaskFilter.CreateBlur(SKBlurStyle.Normal, blurSigma); @@ -1854,20 +1783,15 @@ private static void RenderFreehandPath( var penPointCount = 0; var uniformPressure = true; var firstPressure = 0.0; - var totalThickness = 0.0; var firstPenPointIndex = -1; - // Get effective radius (path-level or backward-compat from first point) - var effectiveRadius = penPath.GetEffectiveRadius(); - - for (var i = 0; i < penPath.Points.Count; i++) + for (var i = 0; i < pointCount; i++) { - var p = penPath.Points[i]; + var p = points[i]; if (!p.IsPen) continue; var pressure = p.Pressure ?? 1; - var thickness = pressure * effectiveRadius * 2.5; if (penPointCount == 0) { @@ -1879,15 +1803,14 @@ private static void RenderFreehandPath( uniformPressure = false; } - totalThickness += thickness; penPointCount++; } if (penPointCount == 0) { - // No pen points - use the ToSKPath method for mouse-based paths + // No pen points - draw a plain polyline for mouse-based paths paint.StrokeWidth = effectiveRadius * 2; - var skPath = penPath.ToSKPath(); + using var skPath = BuildSKPath(points, pointCount); canvas.DrawPath(skPath, paint); return; } @@ -1896,7 +1819,7 @@ private static void RenderFreehandPath( if (penPointCount == 1) { // Single point - draw a circle - var point = penPath.Points[firstPenPointIndex]; + var point = points[firstPenPointIndex]; var thickness = (point.Pressure ?? 1) * effectiveRadius * 2.5; paint.Style = SKPaintStyle.Fill; canvas.DrawCircle(point.X, point.Y, (float)(thickness / 2), paint); @@ -1905,16 +1828,19 @@ private static void RenderFreehandPath( if (uniformPressure) { - // All points have similar pressure - batch into single path - var avgThickness = totalThickness / penPointCount; - paint.StrokeWidth = (float)avgThickness; + // All points have similar pressure - batch into a single path. Width comes from the + // FIRST point's pressure, which never changes as the stroke grows. A running average + // here made the whole in-progress stroke re-render wider/narrower every frame as new + // points shifted the mean ("breathing" while drawing). + paint.StrokeWidth = (float)(firstPressure * effectiveRadius * 2.5); using var path = new SKPath(); var started = false; // Use plain loop instead of LINQ to avoid iterator allocation in hot path - foreach (var p in penPath.Points) + for (var i = 0; i < pointCount; i++) { + var p = points[i]; if (!p.IsPen) continue; @@ -1941,8 +1867,9 @@ private static void RenderFreehandPath( var lastPenX = 0f; var lastPenY = 0f; - foreach (var point in penPath.Points) + for (var i = 0; i < pointCount; i++) { + var point = points[i]; if (!point.IsPen) continue; @@ -1987,24 +1914,65 @@ private static void RenderFreehandPath( } /// - /// Disposes all cached resources to free memory. + /// Builds a polyline from the first entries + /// of a stable point list. Caller owns the returned path. + /// + private static SKPath BuildSKPath(IReadOnlyList points, int pointCount) + { + var skPath = new SKPath(); + + if (pointCount <= 0) + { + return skPath; + } + + skPath.MoveTo(points[0].X, points[0].Y); + + for (var i = 1; i < pointCount; i++) + { + skPath.LineTo(points[i].X, points[i].Y); + } + + return skPath; + } + + /// + /// Disposes all cached resources to free memory. Called from the UI thread. + /// Quiesces rendering first: sets (checked at render entry) and + /// waits for the in-flight render pass to exit before freeing the native resources it + /// may be drawing with. /// public void Dispose() { if (_disposed) return; + // New render passes see this at entry and return without touching resources _disposed = true; + // Wait (bounded) for an in-flight render pass to finish. Frames are short; if this + // ever times out something is badly wrong, so log and proceed rather than hang. + var waitStart = Environment.TickCount64; + while (Volatile.Read(ref rendersInFlight) > 0) + { + if (Environment.TickCount64 - waitStart > 1000) + { + logger.LogWarning( + "Dispose: timed out waiting for in-flight render pass ({Count} still active), proceeding", + Volatile.Read(ref rendersInFlight) + ); + break; + } + + // Sleep rather than yield: yielding in a tight loop busy-spins a core when no other + // thread is ready on it; frames are short so a 1ms granularity wait is plenty + Thread.Sleep(1); + } + // Dispose cached path image cachedPathsImage?.Dispose(); cachedPathsImage = null; - // Dispose temporary path surface - tempPathSurface?.Dispose(); - tempPathSurface = null; - tempPathRenderedPoints.Clear(); - // Dispose checkerboard shader cachedCheckerboardShader?.Dispose(); cachedCheckerboardShader = null; @@ -2012,17 +1980,38 @@ public void Dispose() // Dispose layer surfaces and bitmaps foreach (var layer in Layers.Values) { - lock (layer) - { - layer.Surface?.Dispose(); - layer.Surface = null; + layer.Surface?.Dispose(); + layer.Surface = null; - foreach (var bitmap in layer.Bitmaps) - { - bitmap.Dispose(); - } - layer.Bitmaps = []; + foreach (var bitmap in layer.Bitmaps) + { + bitmap.Dispose(); } + layer.Bitmaps = []; + } + + // Drain bitmaps that were retired by layer swaps but never freed by a render pass + while (retiredLayerBitmaps.TryDequeue(out var retired)) + { + retired.Dispose(); + } + + // Dispose flood-fill bitmap data owned by paths (Paths, the undo redoStack, and any + // in-progress TemporaryPaths). PenPath.BitmapData is an SKBitmap set by FloodFillAt and + // is otherwise never disposed. + foreach (var penPath in Paths) + { + penPath.BitmapData?.Dispose(); + } + + foreach (var penPath in redoStack) + { + penPath.BitmapData?.Dispose(); + } + + foreach (var stroke in TemporaryPaths.Values) + { + stroke.Template.BitmapData?.Dispose(); } // Clear paths diff --git a/StabilityMatrix.Avalonia/ViewModels/Dialogs/CivitFileViewModel.cs b/StabilityMatrix.Avalonia/ViewModels/Dialogs/CivitFileViewModel.cs index b3423d51d..923df89a0 100644 --- a/StabilityMatrix.Avalonia/ViewModels/Dialogs/CivitFileViewModel.cs +++ b/StabilityMatrix.Avalonia/ViewModels/Dialogs/CivitFileViewModel.cs @@ -53,8 +53,9 @@ public CivitFileViewModel( this.vmFactory = vmFactory; this.downloadAction = downloadAction; CivitFile = civitFile; + // Hash-based only, so files with types we don't recognize (Unknown) still show as installed IsInstalled = - CivitFile is { Type: CivitFileType.Model, Hashes.BLAKE3: not null } + CivitFile is { Hashes.BLAKE3: not null } && modelIndexService.ModelIndexBlake3Hashes.Contains(CivitFile.Hashes.BLAKE3); EventManager.Instance.ModelIndexChanged += ModelIndexChanged; @@ -95,7 +96,7 @@ private void ModelIndexChanged(object? sender, EventArgs e) Dispatcher.UIThread.Post(() => { IsInstalled = - CivitFile is { Type: CivitFileType.Model, Hashes.BLAKE3: not null } + CivitFile is { Hashes.BLAKE3: not null } && modelIndexService.ModelIndexBlake3Hashes.Contains(CivitFile.Hashes.BLAKE3); }); } diff --git a/StabilityMatrix.Avalonia/ViewModels/Dialogs/ConfirmBulkDownloadDialogViewModel.cs b/StabilityMatrix.Avalonia/ViewModels/Dialogs/ConfirmBulkDownloadDialogViewModel.cs index c502a86b5..4b9018f75 100644 --- a/StabilityMatrix.Avalonia/ViewModels/Dialogs/ConfirmBulkDownloadDialogViewModel.cs +++ b/StabilityMatrix.Avalonia/ViewModels/Dialogs/ConfirmBulkDownloadDialogViewModel.cs @@ -138,10 +138,7 @@ public override async Task OnLoadedAsync() if (fileVm.IsInstalled) return false; - return fileVm.CivitFile.Type - is CivitFileType.Model - or CivitFileType.VAE - or CivitFileType.PrunedModel; + return fileVm.CivitFile.Type.IsDownloadableModelFile(); } ); diff --git a/StabilityMatrix.Avalonia/ViewModels/Dialogs/ImageAnnotationEditorViewModel.cs b/StabilityMatrix.Avalonia/ViewModels/Dialogs/ImageAnnotationEditorViewModel.cs index e4b99435d..0e3defeec 100644 --- a/StabilityMatrix.Avalonia/ViewModels/Dialogs/ImageAnnotationEditorViewModel.cs +++ b/StabilityMatrix.Avalonia/ViewModels/Dialogs/ImageAnnotationEditorViewModel.cs @@ -118,20 +118,9 @@ public void LoadImage(Bitmap bitmap, string? sourcePath = null) /// public SKImage? RenderAnnotatedImage() { - var canvasSize = PaintCanvasViewModel.CanvasSize; - if (canvasSize.IsEmpty) - { - return null; - } - - using var surface = SKSurface.Create(new SKImageInfo(canvasSize.Width, canvasSize.Height)); - PaintCanvasViewModel.RenderToSurface( - surface, - renderBackgroundFill: false, - renderBackgroundImage: true - ); - - return surface.Snapshot(); + // Composes from an immutable snapshot onto a CPU surface owned by the call, + // without touching the on-screen render pass or its surfaces + return PaintCanvasViewModel.RenderToImage(renderBackgroundImage: true); } /// @@ -215,6 +204,7 @@ public BetterContentDialog GetDialog() public void Dispose() { + PaintCanvasViewModel.Dispose(); originalBitmap?.Dispose(); cachedAnnotatedImage?.Dispose(); GC.SuppressFinalize(this); diff --git a/StabilityMatrix.Avalonia/ViewModels/Dialogs/LayeredMaskEditorViewModel.cs b/StabilityMatrix.Avalonia/ViewModels/Dialogs/LayeredMaskEditorViewModel.cs index b62a1f104..8d99fc645 100644 --- a/StabilityMatrix.Avalonia/ViewModels/Dialogs/LayeredMaskEditorViewModel.cs +++ b/StabilityMatrix.Avalonia/ViewModels/Dialogs/LayeredMaskEditorViewModel.cs @@ -123,11 +123,20 @@ ILogger logger // Set up Move tool callback to update image layer offsets PaintCanvasViewModel.OnMoveToolDrag = (newOffsetX, newOffsetY) => { - if (SelectedLayer is { LayerType: MaskLayerType.Image }) + if (SelectedLayer is { LayerType: MaskLayerType.Image } layer) { - SelectedLayer.ImageOffsetX = newOffsetX; - SelectedLayer.ImageOffsetY = newOffsetY; - SyncSelectedLayerToCanvas(); + layer.ImageOffsetX = newOffsetX; + layer.ImageOffsetY = newOffsetY; + + // Lightweight sync: only the dragged image layer changed, so re-render just its + // bitmap instead of re-compositing every other layer via SyncSelectedLayerToCanvas + if (layer.IsVisible && layer.SourceImage is not null && CanvasSize != Size.Empty) + { + var selectedImageBitmap = RenderSingleImageLayer(layer); + PaintCanvasViewModel.SetLayerBitmap("CurrentImage", selectedImageBitmap?.Copy()); + } + + PaintCanvasViewModel.RefreshCanvas?.Invoke(); } }; diff --git a/StabilityMatrix.Avalonia/ViewModels/Dialogs/MaskEditorViewModel.cs b/StabilityMatrix.Avalonia/ViewModels/Dialogs/MaskEditorViewModel.cs index 7cd799555..d95b2fa12 100644 --- a/StabilityMatrix.Avalonia/ViewModels/Dialogs/MaskEditorViewModel.cs +++ b/StabilityMatrix.Avalonia/ViewModels/Dialogs/MaskEditorViewModel.cs @@ -246,6 +246,7 @@ public void Dispose() { _cachedMaskRenderImage?.Dispose(); _cachedMaskRenderInverseAlphaImage?.Dispose(); + PaintCanvasViewModel.Dispose(); GC.SuppressFinalize(this); } } diff --git a/StabilityMatrix.Avalonia/ViewModels/Dialogs/ModelVersionViewModel.cs b/StabilityMatrix.Avalonia/ViewModels/Dialogs/ModelVersionViewModel.cs index 9e5a97125..b162ee8ab 100644 --- a/StabilityMatrix.Avalonia/ViewModels/Dialogs/ModelVersionViewModel.cs +++ b/StabilityMatrix.Avalonia/ViewModels/Dialogs/ModelVersionViewModel.cs @@ -23,9 +23,11 @@ public ModelVersionViewModel(IModelIndexService modelIndexService, CivitModelVer ModelVersion = modelVersion; + // Install detection is purely hash-based: a file whose hash is in the local model index is + // on disk, regardless of the type Civitai reports (which may be Unknown for new types) IsInstalled = ModelVersion.Files?.Any(file => - file is { Type: CivitFileType.Model, Hashes.BLAKE3: not null } + file is { Hashes.BLAKE3: not null } && modelIndexService.ModelIndexBlake3Hashes.Contains(file.Hashes.BLAKE3) ) ?? false; @@ -36,7 +38,7 @@ public void RefreshInstallStatus() { IsInstalled = ModelVersion.Files?.Any(file => - file is { Type: CivitFileType.Model, Hashes.BLAKE3: not null } + file is { Hashes.BLAKE3: not null } && modelIndexService.ModelIndexBlake3Hashes.Contains(file.Hashes.BLAKE3) ) ?? false; } diff --git a/StabilityMatrix.Avalonia/ViewModels/Dialogs/PackageImportViewModel.cs b/StabilityMatrix.Avalonia/ViewModels/Dialogs/PackageImportViewModel.cs index 668080724..61319dd19 100644 --- a/StabilityMatrix.Avalonia/ViewModels/Dialogs/PackageImportViewModel.cs +++ b/StabilityMatrix.Avalonia/ViewModels/Dialogs/PackageImportViewModel.cs @@ -4,6 +4,7 @@ using System.Collections.ObjectModel; using System.IO; using System.Linq; +using System.Text.RegularExpressions; using System.Threading.Tasks; using AsyncAwaitBestPractices; using Avalonia.Controls; @@ -96,7 +97,7 @@ public bool IsReleaseMode public override async Task OnLoadedAsync() { - SelectedBasePackage ??= AvailablePackages[0]; + SelectedBasePackage ??= DetectPackageFromGitRemote() ?? AvailablePackages[0]; if (Design.IsDesignMode) return; @@ -338,4 +339,85 @@ await gitPackage.SetupVenv( x.Version.Major.Equals(SelectedBasePackage?.RecommendedPythonVersion.Major) && x.Version.Minor.Equals(SelectedBasePackage?.RecommendedPythonVersion.Minor) ); + + /// + /// Attempts to detect the package type of the folder being imported from its git remote origin url, + /// so the type dropdown doesn't silently default to the first (unrelated) package in the list. + /// + private BasePackage? DetectPackageFromGitRemote() + { + if (PackagePath is null) + return null; + + try + { + var gitConfigPath = Path.Combine(PackagePath, ".git", "config"); + if (!File.Exists(gitConfigPath)) + return null; + + var remoteMatch = GitConfigRemoteOriginUrlRegex().Match(File.ReadAllText(gitConfigPath)); + if (!remoteMatch.Success || GetGitHubSlug(remoteMatch.Groups[1].Value) is not { } remoteSlug) + return null; + + var candidates = AvailablePackages + .Where(p => + GetGitHubSlug(p.GithubUrl) is { } slug + && string.Equals(slug, remoteSlug, StringComparison.OrdinalIgnoreCase) + ) + .ToList(); + + if (candidates.Count > 1) + { + // Multiple packages share the same repository (e.g. Forge Classic / Neo), + // disambiguate by the currently checked-out branch + var headPath = Path.Combine(PackagePath, ".git", "HEAD"); + const string refPrefix = "ref: refs/heads/"; + if ( + File.Exists(headPath) + && File.ReadAllText(headPath).Trim() is { } head + && head.StartsWith(refPrefix, StringComparison.Ordinal) + && candidates.FirstOrDefault(p => + string.Equals( + p.MainBranch, + head[refPrefix.Length..], + StringComparison.OrdinalIgnoreCase + ) + ) + is { } branchMatch + ) + { + return branchMatch; + } + } + + if (candidates.FirstOrDefault() is { } detected) + { + Logger.Info( + "Detected package type {Package} for import path {Path} from git remote", + detected.Name, + PackagePath.Name + ); + return detected; + } + + return null; + } + catch (Exception e) + { + Logger.Warn(e, "Failed to detect package type from git remote"); + return null; + } + } + + private static string? GetGitHubSlug(string url) + { + var match = GitHubUrlSlugRegex().Match(url.Trim()); + return match.Success ? match.Groups["slug"].Value : null; + } + + [GeneratedRegex("""\[remote "origin"\][\s\S]*?url\s*=\s*(.+)""")] + private static partial Regex GitConfigRemoteOriginUrlRegex(); + + [GeneratedRegex(@"github\.com[:/](?[^/\s]+/[^/\s]+?)(?:\.git)?/?$", RegexOptions.IgnoreCase)] + private static partial Regex GitHubUrlSlugRegex(); } diff --git a/StabilityMatrix.Avalonia/ViewModels/Dialogs/RecommendedModelsViewModel.cs b/StabilityMatrix.Avalonia/ViewModels/Dialogs/RecommendedModelsViewModel.cs index 35814b773..ab1ea6c06 100644 --- a/StabilityMatrix.Avalonia/ViewModels/Dialogs/RecommendedModelsViewModel.cs +++ b/StabilityMatrix.Avalonia/ViewModels/Dialogs/RecommendedModelsViewModel.cs @@ -102,15 +102,14 @@ public override async Task OnLoadedAsync() { // Find the first non-Turbo/Lightning version, or default to the first version if none match var suitableVersion = - model.ModelVersions?.FirstOrDefault( - x => - !x.BaseModel.Contains("Turbo", StringComparison.OrdinalIgnoreCase) - && !x.BaseModel.Contains("Lightning", StringComparison.OrdinalIgnoreCase) - && x.Files != null - && x.Files.Any(f => f.Type == CivitFileType.Model) // Ensure there's a model file + model.ModelVersions?.FirstOrDefault(x => + !x.BaseModel.Contains("Turbo", StringComparison.OrdinalIgnoreCase) + && !x.BaseModel.Contains("Lightning", StringComparison.OrdinalIgnoreCase) + && x.Files != null + && x.Files.Any(f => f.Type.IsModelWeights()) // Ensure there's a model file ) - ?? model.ModelVersions?.FirstOrDefault( - x => x.Files != null && x.Files.Any(f => f.Type == CivitFileType.Model) + ?? model.ModelVersions?.FirstOrDefault(x => + x.Files != null && x.Files.Any(f => f.Type.IsModelWeights()) ); if (suitableVersion == null) @@ -127,7 +126,7 @@ public override async Task OnLoadedAsync() { ModelVersion = suitableVersion, Author = $"by {model.Creator?.Username}", - CivitModel = model + CivitModel = model, }; }) .Where(vm => vm != null); // Filter out nulls (models skipped due to no suitable version) @@ -182,9 +181,9 @@ private async Task DoImport() { // Get latest version file that is a Model type and marked primary, or fallback to first model file var modelFile = - model.ModelVersion.Files?.FirstOrDefault( - f => f is { Type: CivitFileType.Model, IsPrimary: true } - ) ?? model.ModelVersion.Files?.FirstOrDefault(f => f.Type == CivitFileType.Model); + model.ModelVersion.Files?.FirstOrDefault(f => + f is { IsPrimary: true } && f.Type.IsModelWeights() + ) ?? model.ModelVersion.Files?.FirstOrDefault(f => f is not null && f.Type.IsModelWeights()); if (modelFile is null) { diff --git a/StabilityMatrix.Avalonia/ViewModels/Dialogs/SelectModelVersionViewModel.cs b/StabilityMatrix.Avalonia/ViewModels/Dialogs/SelectModelVersionViewModel.cs index fe1099f7a..ca4b0e9ed 100644 --- a/StabilityMatrix.Avalonia/ViewModels/Dialogs/SelectModelVersionViewModel.cs +++ b/StabilityMatrix.Avalonia/ViewModels/Dialogs/SelectModelVersionViewModel.cs @@ -375,29 +375,12 @@ private static DirectoryPath GetSharedFolderPath( CivitFile? civitFile, CivitModelType modelType, string? baseModelType - ) - { - if (civitFile?.Type is CivitFileType.VAE) - { - return rootModelsDirectory.JoinDir(SharedFolderType.VAE.GetStringValue()); - } - - if ( - modelType is CivitModelType.Checkpoint - && ( - baseModelType == CivitBaseModelType.Flux1D.GetStringValue() - || baseModelType == CivitBaseModelType.Flux1S.GetStringValue() - || baseModelType == CivitBaseModelType.WanVideo.GetStringValue() - || baseModelType == CivitBaseModelType.HunyuanVideo.GetStringValue() - || civitFile?.Metadata.Format == CivitModelFormat.GGUF - ) - ) - { - return rootModelsDirectory.JoinDir(SharedFolderType.DiffusionModels.GetStringValue()); - } - - return rootModelsDirectory.JoinDir(modelType.ConvertTo().GetStringValue()); - } + ) => + rootModelsDirectory.JoinDir( + (civitFile?.Type ?? CivitFileType.Unknown) + .GetSharedFolderType(modelType, baseModelType, civitFile?.Name, civitFile?.Metadata?.Format) + .GetStringValue() + ); private void ApplySavedDownloadPreference() { diff --git a/StabilityMatrix.Avalonia/ViewModels/Inference/InferenceFluxTextToImageViewModel.cs b/StabilityMatrix.Avalonia/ViewModels/Inference/InferenceFluxTextToImageViewModel.cs index 154c732e6..fd99d1e60 100644 --- a/StabilityMatrix.Avalonia/ViewModels/Inference/InferenceFluxTextToImageViewModel.cs +++ b/StabilityMatrix.Avalonia/ViewModels/Inference/InferenceFluxTextToImageViewModel.cs @@ -180,7 +180,7 @@ CancellationToken cancellationToken } } - if (!await CheckClientConnectedWithPrompt() || !ClientManager.IsConnected) + if (!await CheckClientConnectedWithPrompt(cancellationToken) || !ClientManager.IsConnected) return; // If enabled, randomize the seed diff --git a/StabilityMatrix.Avalonia/ViewModels/Inference/InferenceImageToVideoViewModel.cs b/StabilityMatrix.Avalonia/ViewModels/Inference/InferenceImageToVideoViewModel.cs index d042573ac..17ae4ce18 100644 --- a/StabilityMatrix.Avalonia/ViewModels/Inference/InferenceImageToVideoViewModel.cs +++ b/StabilityMatrix.Avalonia/ViewModels/Inference/InferenceImageToVideoViewModel.cs @@ -76,8 +76,8 @@ RunningPackageService runningPackageService SeedCardViewModel = vmFactory.Get(); SeedCardViewModel.GenerateNewSeed(); - ModelCardViewModel = vmFactory.Get( - vm => vm.EnableModelLoaderSelection = false + ModelCardViewModel = vmFactory.Get(vm => + vm.EnableModelLoaderSelection = false ); SamplerCardViewModel = vmFactory.Get(samplerCard => @@ -120,7 +120,7 @@ protected override void BuildPrompt(BuildPromptEventArgs args) builder.Connections.Seed = args.SeedOverride switch { { } seed => Convert.ToUInt64(seed), - _ => Convert.ToUInt64(SeedCardViewModel.Seed) + _ => Convert.ToUInt64(SeedCardViewModel.Seed), }; // Load models @@ -133,7 +133,7 @@ protected override void BuildPrompt(BuildPromptEventArgs args) Name = builder.Nodes.GetUniqueName("ControlNet_LoadImage"), Image = SelectImageCardViewModel.ImageSource?.GetHashGuidFileNameCached("Inference") - ?? throw new ValidationException() + ?? throw new ValidationException(), } ); builder.Connections.Primary = imageLoad.Output1; @@ -167,7 +167,7 @@ protected override async Task GenerateImageImpl( CancellationToken cancellationToken ) { - if (!await CheckClientConnectedWithPrompt() || !ClientManager.IsConnected) + if (!await CheckClientConnectedWithPrompt(cancellationToken) || !ClientManager.IsConnected) { return; } @@ -207,13 +207,13 @@ CancellationToken cancellationToken OutputNodeNames = buildPromptArgs.Builder.Connections.OutputNodeNames.ToArray(), Parameters = SaveStateToParameters(new GenerationParameters()) with { - Seed = Convert.ToUInt64(seed) + Seed = Convert.ToUInt64(seed), }, Project = inferenceProject, FilesToTransfer = buildPromptArgs.FilesToTransfer, BatchIndex = i, // Only clear output images on the first batch - ClearOutputImages = i == 0 + ClearOutputImages = i == 0, }; batchArgs.Add(generationArgs); diff --git a/StabilityMatrix.Avalonia/ViewModels/Inference/InferenceTextToImageViewModel.cs b/StabilityMatrix.Avalonia/ViewModels/Inference/InferenceTextToImageViewModel.cs index 4f43c77d9..aaeb357a3 100644 --- a/StabilityMatrix.Avalonia/ViewModels/Inference/InferenceTextToImageViewModel.cs +++ b/StabilityMatrix.Avalonia/ViewModels/Inference/InferenceTextToImageViewModel.cs @@ -418,7 +418,7 @@ CancellationToken cancellationToken } } - if (!await CheckClientConnectedWithPrompt() || !ClientManager.IsConnected) + if (!await CheckClientConnectedWithPrompt(cancellationToken) || !ClientManager.IsConnected) return; // If enabled, randomize the seed diff --git a/StabilityMatrix.Avalonia/ViewModels/Inference/InferenceWanTextToVideoViewModel.cs b/StabilityMatrix.Avalonia/ViewModels/Inference/InferenceWanTextToVideoViewModel.cs index d6ba8743d..da7002d79 100644 --- a/StabilityMatrix.Avalonia/ViewModels/Inference/InferenceWanTextToVideoViewModel.cs +++ b/StabilityMatrix.Avalonia/ViewModels/Inference/InferenceWanTextToVideoViewModel.cs @@ -133,7 +133,7 @@ protected override async Task GenerateImageImpl( CancellationToken cancellationToken ) { - if (!await CheckClientConnectedWithPrompt() || !ClientManager.IsConnected) + if (!await CheckClientConnectedWithPrompt(cancellationToken) || !ClientManager.IsConnected) { return; } diff --git a/StabilityMatrix.Avalonia/ViewModels/Inference/ModelCardViewModel.cs b/StabilityMatrix.Avalonia/ViewModels/Inference/ModelCardViewModel.cs index 0e5429058..1409512f2 100644 --- a/StabilityMatrix.Avalonia/ViewModels/Inference/ModelCardViewModel.cs +++ b/StabilityMatrix.Avalonia/ViewModels/Inference/ModelCardViewModel.cs @@ -236,6 +236,8 @@ public HybridModelFile? SelectedUnifiedModel "sdxl", "sd3", "flux", + "flux2", + "chroma", "hunyuan_video", "hidream", "hunyuan_image", @@ -245,8 +247,15 @@ public HybridModelFile? SelectedUnifiedModel "ltxv", "newbie", "ace", - "flux2", "lumina2", + "qwen_image", + "wan", + "omnigen2", + "mochi", + "pixart", + "cosmos", + "stable_cascade", + "stable_audio", "stable_diffusion", ]; public List WorkflowProfiles { get; set; } = @@ -1006,14 +1015,7 @@ private void LoadTextEncodersFromModel(ModelCardModel model) var encoderCount = legacyNames.TakeWhile(n => n is not null).Count(); // Use at least the default count for the encoder type - var defaultCount = SelectedClipType switch - { - "flux" => 2, - "flux2" or "lumina2" or "stable_diffusion" => 1, - "sd3" => 3, - "hidream" => 4, - _ => 2, - }; + var defaultCount = DefaultEncoderCountForClipType(SelectedClipType); encoderCount = Math.Max(encoderCount, defaultCount); for (var i = 0; i < encoderCount; i++) @@ -1553,14 +1555,7 @@ private void SetDefaultEncoderCount(bool preserveUserSelections = false) return; } - var targetCount = SelectedClipType switch - { - "flux" => 2, - "flux2" or "lumina2" or "stable_diffusion" => 1, - "sd3" => 3, - "hidream" => 4, - _ => 2, // Default to 2 for unknown types - }; + var targetCount = DefaultEncoderCountForClipType(SelectedClipType); // Add or remove encoders to match target count while (TextEncoders.Count < targetCount) @@ -1577,6 +1572,31 @@ private void SetDefaultEncoderCount(bool preserveUserSelections = false) OnPropertyChanged(nameof(TextEncodersHeader)); } + /// + /// Default number of encoder slots for a clip type. Types accepted by ComfyUI's + /// DualCLIPLoader load an encoder pair (e.g. clip_l + t5xxl for flux); sd3 / hidream use + /// the triple / quadruple loaders; everything else is a single-encoder CLIPLoader type + /// (flux2, lumina2, chroma, wan, qwen_image, ...). + /// + private static int DefaultEncoderCountForClipType(string? clipType) => + clipType switch + { + "sd3" => 3, + "hidream" => 4, + "sdxl" + or "flux" + or "hunyuan_video" + or "hunyuan_image" + or "hunyuan_video_15" + or "kandinsky5" + or "kandinsky5_image" + or "ltxv" + or "newbie" + or "ace" + or null => 2, + _ => 1, + }; + /// /// Adds a new text encoder slot. /// @@ -1666,19 +1686,34 @@ private void SetupStandaloneModelLoader(ModuleApplyStepEventArgs e) if (SelectedClipType == "flux") { - // DualCLIPLoader - var clipLoader = e.Nodes.AddTypedNode( - new ComfyNodeBuilder.DualCLIPLoader - { - Name = e.Nodes.GetUniqueName(nameof(ComfyNodeBuilder.DualCLIPLoader)), - ClipName1 = - SelectedClip1?.RelativePath ?? throw new ValidationException("No Clip1 Selected"), - ClipName2 = - SelectedClip2?.RelativePath ?? throw new ValidationException("No Clip2 Selected"), - Type = SelectedClipType ?? throw new ValidationException("No Clip Type Selected"), - } - ); - e.Builder.Connections.Base.Clip = clipLoader.Output; + // DualCLIPLoader (GGUF variant can also load .safetensors, so any gguf pick routes there) + var clipName1 = SelectedClip1?.RelativePath ?? throw new ValidationException("No Clip1 Selected"); + var clipName2 = SelectedClip2?.RelativePath ?? throw new ValidationException("No Clip2 Selected"); + var clipType = SelectedClipType ?? throw new ValidationException("No Clip Type Selected"); + + e.Builder.Connections.Base.Clip = AnyClipIsGguf(SelectedClip1, SelectedClip2) + ? e + .Nodes.AddTypedNode( + new ComfyNodeBuilder.DualCLIPLoaderGGUF + { + Name = e.Nodes.GetUniqueName(nameof(ComfyNodeBuilder.DualCLIPLoaderGGUF)), + ClipName1 = clipName1, + ClipName2 = clipName2, + Type = clipType, + } + ) + .Output + : e + .Nodes.AddTypedNode( + new ComfyNodeBuilder.DualCLIPLoader + { + Name = e.Nodes.GetUniqueName(nameof(ComfyNodeBuilder.DualCLIPLoader)), + ClipName1 = clipName1, + ClipName2 = clipName2, + Type = clipType, + } + ) + .Output; } else { @@ -1756,6 +1791,8 @@ SelectedModelLoader is ModelLoader.Default private void SetupClipLoaders(ModuleApplyStepEventArgs e) { + // The GGUF loader variants can also load .safetensors encoders, so any gguf pick + // routes the whole (possibly mixed) selection through the GGUF loader if ( SelectedClip4 is { IsNone: false } && SelectedClip3 is { IsNone: false } @@ -1763,21 +1800,41 @@ private void SetupClipLoaders(ModuleApplyStepEventArgs e) && SelectedClip1 is { IsNone: false } ) { - var clipLoader = e.Nodes.AddTypedNode( - new ComfyNodeBuilder.QuadrupleCLIPLoader - { - Name = e.Nodes.GetUniqueName(nameof(ComfyNodeBuilder.QuadrupleCLIPLoader)), - ClipName1 = - SelectedClip1?.RelativePath ?? throw new ValidationException("No Clip1 Selected"), - ClipName2 = - SelectedClip2?.RelativePath ?? throw new ValidationException("No Clip2 Selected"), - ClipName3 = - SelectedClip3?.RelativePath ?? throw new ValidationException("No Clip3 Selected"), - ClipName4 = - SelectedClip4?.RelativePath ?? throw new ValidationException("No Clip4 Selected"), - } - ); - e.Builder.Connections.Base.Clip = clipLoader.Output; + var clipName1 = SelectedClip1?.RelativePath ?? throw new ValidationException("No Clip1 Selected"); + var clipName2 = SelectedClip2?.RelativePath ?? throw new ValidationException("No Clip2 Selected"); + var clipName3 = SelectedClip3?.RelativePath ?? throw new ValidationException("No Clip3 Selected"); + var clipName4 = SelectedClip4?.RelativePath ?? throw new ValidationException("No Clip4 Selected"); + + e.Builder.Connections.Base.Clip = AnyClipIsGguf( + SelectedClip1, + SelectedClip2, + SelectedClip3, + SelectedClip4 + ) + ? e + .Nodes.AddTypedNode( + new ComfyNodeBuilder.QuadrupleCLIPLoaderGGUF + { + Name = e.Nodes.GetUniqueName(nameof(ComfyNodeBuilder.QuadrupleCLIPLoaderGGUF)), + ClipName1 = clipName1, + ClipName2 = clipName2, + ClipName3 = clipName3, + ClipName4 = clipName4, + } + ) + .Output + : e + .Nodes.AddTypedNode( + new ComfyNodeBuilder.QuadrupleCLIPLoader + { + Name = e.Nodes.GetUniqueName(nameof(ComfyNodeBuilder.QuadrupleCLIPLoader)), + ClipName1 = clipName1, + ClipName2 = clipName2, + ClipName3 = clipName3, + ClipName4 = clipName4, + } + ) + .Output; } else if ( SelectedClip3 is { IsNone: false } @@ -1785,47 +1842,90 @@ private void SetupClipLoaders(ModuleApplyStepEventArgs e) && SelectedClip1 is { IsNone: false } ) { - var clipLoader = e.Nodes.AddTypedNode( - new ComfyNodeBuilder.TripleCLIPLoader - { - Name = e.Nodes.GetUniqueName(nameof(ComfyNodeBuilder.TripleCLIPLoader)), - ClipName1 = - SelectedClip1?.RelativePath ?? throw new ValidationException("No Clip1 Selected"), - ClipName2 = - SelectedClip2?.RelativePath ?? throw new ValidationException("No Clip2 Selected"), - ClipName3 = - SelectedClip3?.RelativePath ?? throw new ValidationException("No Clip3 Selected"), - } - ); - e.Builder.Connections.Base.Clip = clipLoader.Output; + var clipName1 = SelectedClip1?.RelativePath ?? throw new ValidationException("No Clip1 Selected"); + var clipName2 = SelectedClip2?.RelativePath ?? throw new ValidationException("No Clip2 Selected"); + var clipName3 = SelectedClip3?.RelativePath ?? throw new ValidationException("No Clip3 Selected"); + + e.Builder.Connections.Base.Clip = AnyClipIsGguf(SelectedClip1, SelectedClip2, SelectedClip3) + ? e + .Nodes.AddTypedNode( + new ComfyNodeBuilder.TripleCLIPLoaderGGUF + { + Name = e.Nodes.GetUniqueName(nameof(ComfyNodeBuilder.TripleCLIPLoaderGGUF)), + ClipName1 = clipName1, + ClipName2 = clipName2, + ClipName3 = clipName3, + } + ) + .Output + : e + .Nodes.AddTypedNode( + new ComfyNodeBuilder.TripleCLIPLoader + { + Name = e.Nodes.GetUniqueName(nameof(ComfyNodeBuilder.TripleCLIPLoader)), + ClipName1 = clipName1, + ClipName2 = clipName2, + ClipName3 = clipName3, + } + ) + .Output; } else if (SelectedClip2 is { IsNone: false } && SelectedClip1 is { IsNone: false }) { - var clipLoader = e.Nodes.AddTypedNode( - new ComfyNodeBuilder.DualCLIPLoader - { - Name = e.Nodes.GetUniqueName(nameof(ComfyNodeBuilder.DualCLIPLoader)), - ClipName1 = - SelectedClip1?.RelativePath ?? throw new ValidationException("No Clip1 Selected"), - ClipName2 = - SelectedClip2?.RelativePath ?? throw new ValidationException("No Clip2 Selected"), - Type = SelectedClipType ?? throw new ValidationException("No Clip Type Selected"), - } - ); - e.Builder.Connections.Base.Clip = clipLoader.Output; + var clipName1 = SelectedClip1?.RelativePath ?? throw new ValidationException("No Clip1 Selected"); + var clipName2 = SelectedClip2?.RelativePath ?? throw new ValidationException("No Clip2 Selected"); + var clipType = SelectedClipType ?? throw new ValidationException("No Clip Type Selected"); + + e.Builder.Connections.Base.Clip = AnyClipIsGguf(SelectedClip1, SelectedClip2) + ? e + .Nodes.AddTypedNode( + new ComfyNodeBuilder.DualCLIPLoaderGGUF + { + Name = e.Nodes.GetUniqueName(nameof(ComfyNodeBuilder.DualCLIPLoaderGGUF)), + ClipName1 = clipName1, + ClipName2 = clipName2, + Type = clipType, + } + ) + .Output + : e + .Nodes.AddTypedNode( + new ComfyNodeBuilder.DualCLIPLoader + { + Name = e.Nodes.GetUniqueName(nameof(ComfyNodeBuilder.DualCLIPLoader)), + ClipName1 = clipName1, + ClipName2 = clipName2, + Type = clipType, + } + ) + .Output; } else if (SelectedClip1 is { IsNone: false }) { - var clipLoader = e.Nodes.AddTypedNode( - new ComfyNodeBuilder.CLIPLoader() - { - Name = e.Nodes.GetUniqueName(nameof(ComfyNodeBuilder.CLIPLoader)), - ClipName = - SelectedClip1?.RelativePath ?? throw new ValidationException("No Clip1 Selected"), - Type = SelectedClipType ?? throw new ValidationException("No Clip Type Selected"), - } - ); - e.Builder.Connections.Base.Clip = clipLoader.Output; + var clipName1 = SelectedClip1?.RelativePath ?? throw new ValidationException("No Clip1 Selected"); + var clipType = SelectedClipType ?? throw new ValidationException("No Clip Type Selected"); + + e.Builder.Connections.Base.Clip = AnyClipIsGguf(SelectedClip1) + ? e + .Nodes.AddTypedNode( + new ComfyNodeBuilder.CLIPLoaderGGUF + { + Name = e.Nodes.GetUniqueName(nameof(ComfyNodeBuilder.CLIPLoaderGGUF)), + ClipName = clipName1, + Type = clipType, + } + ) + .Output + : e + .Nodes.AddTypedNode( + new ComfyNodeBuilder.CLIPLoader + { + Name = e.Nodes.GetUniqueName(nameof(ComfyNodeBuilder.CLIPLoader)), + ClipName = clipName1, + Type = clipType, + } + ) + .Output; } else { @@ -1836,6 +1936,9 @@ private void SetupClipLoaders(ModuleApplyStepEventArgs e) } } + private static bool AnyClipIsGguf(params HybridModelFile?[] clips) => + clips.Any(clip => clip is { IsGguf: true }); + internal class ModelCardModel { public string? SelectedModelName { get; init; } diff --git a/StabilityMatrix.Avalonia/ViewModels/Inference/UnetModelCardViewModel.cs b/StabilityMatrix.Avalonia/ViewModels/Inference/UnetModelCardViewModel.cs index b9f7e40cd..87b3c4b36 100644 --- a/StabilityMatrix.Avalonia/ViewModels/Inference/UnetModelCardViewModel.cs +++ b/StabilityMatrix.Avalonia/ViewModels/Inference/UnetModelCardViewModel.cs @@ -67,22 +67,39 @@ public void ApplyStep(ModuleApplyStepEventArgs e) new ComfyNodeBuilder.VAELoader { Name = e.Nodes.GetUniqueName(nameof(ComfyNodeBuilder.VAELoader)), - VaeName = SelectedVae?.RelativePath ?? throw new ValidationException("No VAE Selected") + VaeName = SelectedVae?.RelativePath ?? throw new ValidationException("No VAE Selected"), } ); e.Builder.Connections.Base.VAE = vaeLoader.Output; - // DualCLIPLoader - var clipLoader = e.Nodes.AddTypedNode( - new ComfyNodeBuilder.DualCLIPLoader - { - Name = e.Nodes.GetUniqueName(nameof(ComfyNodeBuilder.DualCLIPLoader)), - ClipName1 = SelectedClip1?.RelativePath ?? throw new ValidationException("No Clip1 Selected"), - ClipName2 = SelectedClip2?.RelativePath ?? throw new ValidationException("No Clip2 Selected"), - Type = "flux" - } - ); - e.Builder.Connections.Base.Clip = clipLoader.Output; + // DualCLIPLoader (GGUF variant can also load .safetensors, so any gguf pick routes there) + var clipName1 = SelectedClip1?.RelativePath ?? throw new ValidationException("No Clip1 Selected"); + var clipName2 = SelectedClip2?.RelativePath ?? throw new ValidationException("No Clip2 Selected"); + + e.Builder.Connections.Base.Clip = + SelectedClip1 is { IsGguf: true } || SelectedClip2 is { IsGguf: true } + ? e + .Nodes.AddTypedNode( + new ComfyNodeBuilder.DualCLIPLoaderGGUF + { + Name = e.Nodes.GetUniqueName(nameof(ComfyNodeBuilder.DualCLIPLoaderGGUF)), + ClipName1 = clipName1, + ClipName2 = clipName2, + Type = "flux", + } + ) + .Output + : e + .Nodes.AddTypedNode( + new ComfyNodeBuilder.DualCLIPLoader + { + Name = e.Nodes.GetUniqueName(nameof(ComfyNodeBuilder.DualCLIPLoader)), + ClipName1 = clipName1, + ClipName2 = clipName2, + Type = "flux", + } + ) + .Output; } private static ComfyTypedNodeBase GetModelLoader( @@ -96,7 +113,7 @@ string selectedDType { Name = e.Nodes.GetUniqueName(nameof(ComfyNodeBuilder.UNETLoader)), UnetName = model.RelativePath, - WeightDtype = selectedDType + WeightDtype = selectedDType, }; } @@ -109,7 +126,7 @@ public override JsonObject SaveStateToJsonObject() SelectedModelName = SelectedModel?.RelativePath, SelectedVaeName = SelectedVae?.RelativePath, SelectedClip1Name = SelectedClip1?.RelativePath, - SelectedClip2Name = SelectedClip2?.RelativePath + SelectedClip2Name = SelectedClip2?.RelativePath, } ); } @@ -157,10 +174,9 @@ public void LoadStateFromParameters(GenerationParameters parameters) // First try hash match if (parameters.ModelHash is not null) { - model = currentModels.FirstOrDefault( - m => - m.Local?.ConnectedModelInfo?.Hashes.SHA256 is { } sha256 - && sha256.StartsWith(parameters.ModelHash, StringComparison.InvariantCultureIgnoreCase) + model = currentModels.FirstOrDefault(m => + m.Local?.ConnectedModelInfo?.Hashes.SHA256 is { } sha256 + && sha256.StartsWith(parameters.ModelHash, StringComparison.InvariantCultureIgnoreCase) ); } else @@ -182,7 +198,7 @@ public GenerationParameters SaveStateToParameters(GenerationParameters parameter return parameters with { ModelName = SelectedModel?.FileName, - ModelHash = SelectedModel?.Local?.ConnectedModelInfo?.Hashes.SHA256 + ModelHash = SelectedModel?.Local?.ConnectedModelInfo?.Hashes.SHA256, }; } } diff --git a/StabilityMatrix.Avalonia/ViewModels/Inference/WanModelCardViewModel.cs b/StabilityMatrix.Avalonia/ViewModels/Inference/WanModelCardViewModel.cs index f7e33939d..4156e5dad 100644 --- a/StabilityMatrix.Avalonia/ViewModels/Inference/WanModelCardViewModel.cs +++ b/StabilityMatrix.Avalonia/ViewModels/Inference/WanModelCardViewModel.cs @@ -195,7 +195,7 @@ public void ApplyStep(ModuleApplyStepEventArgs e) } ); } - + var modelSamplingSd3 = e.Nodes.AddTypedNode( new ComfyNodeBuilder.ModelSamplingSD3 { @@ -206,19 +206,34 @@ public void ApplyStep(ModuleApplyStepEventArgs e) ); e.Builder.Connections.Base.Model = modelSamplingSd3.Output; - - var clipLoader = e.Nodes.AddTypedNode( - new ComfyNodeBuilder.CLIPLoader - { - Name = e.Nodes.GetUniqueName(nameof(ComfyNodeBuilder.CLIPLoader)), - ClipName = - SelectedClipModel?.RelativePath - ?? throw new ValidationException("No Clip Model Selected"), - Type = "wan", - } - ); - - e.Builder.Connections.Base.Clip = clipLoader.Output; + + var clipName = + SelectedClipModel?.RelativePath ?? throw new ValidationException("No Clip Model Selected"); + + // The GGUF loader variant can also load .safetensors encoders + var clipOutput = SelectedClipModel is { IsGguf: true } + ? e + .Nodes.AddTypedNode( + new ComfyNodeBuilder.CLIPLoaderGGUF + { + Name = e.Nodes.GetUniqueName(nameof(ComfyNodeBuilder.CLIPLoaderGGUF)), + ClipName = clipName, + Type = "wan", + } + ) + .Output + : e + .Nodes.AddTypedNode( + new ComfyNodeBuilder.CLIPLoader + { + Name = e.Nodes.GetUniqueName(nameof(ComfyNodeBuilder.CLIPLoader)), + ClipName = clipName, + Type = "wan", + } + ) + .Output; + + e.Builder.Connections.Base.Clip = clipOutput; var vaeLoader = e.Nodes.AddTypedNode( new ComfyNodeBuilder.VAELoader diff --git a/StabilityMatrix.Avalonia/ViewModels/OutputsPageViewModel.cs b/StabilityMatrix.Avalonia/ViewModels/OutputsPageViewModel.cs index 905ae2783..aa7deddcd 100644 --- a/StabilityMatrix.Avalonia/ViewModels/OutputsPageViewModel.cs +++ b/StabilityMatrix.Avalonia/ViewModels/OutputsPageViewModel.cs @@ -773,9 +773,22 @@ private ObservableCollection GetSubfolders(string strPath) var category = new TreeViewDirectory { Name = dirName, Path = dir }; - if (Directory.GetDirectories(dir, "*", EnumerationOptionConstants.TopLevelOnly).Length > 0) + try { - category.SubDirectories = GetSubfolders(dir); + if (Directory.GetDirectories(dir, "*", EnumerationOptionConstants.TopLevelOnly).Length > 0) + { + category.SubDirectories = GetSubfolders(dir); + } + } + catch (Exception ex) when (ex is IOException or UnauthorizedAccessException) + { + // Broken junction / symlink or inaccessible directory - skip it + logger.LogWarning( + ex, + "Skipping inaccessible directory {Dir} while building output tree", + dir + ); + continue; } subfolders.Add(category); diff --git a/StabilityMatrix.Avalonia/Views/CheckpointsPage.axaml b/StabilityMatrix.Avalonia/Views/CheckpointsPage.axaml index ad05ec584..05fbb6138 100644 --- a/StabilityMatrix.Avalonia/Views/CheckpointsPage.axaml +++ b/StabilityMatrix.Avalonia/Views/CheckpointsPage.axaml @@ -519,6 +519,12 @@ + + diff --git a/StabilityMatrix.Avalonia/Views/CivitAiBrowserPage.axaml b/StabilityMatrix.Avalonia/Views/CivitAiBrowserPage.axaml index fa75c157a..c253b8324 100644 --- a/StabilityMatrix.Avalonia/Views/CivitAiBrowserPage.axaml +++ b/StabilityMatrix.Avalonia/Views/CivitAiBrowserPage.axaml @@ -141,6 +141,7 @@ Width="330" Height="400" CornerRadius="8" + DecodeWidth="450" IsCacheEnabled="True" Source="{Binding CardImage, Converter={StaticResource CivitImageWidthConverter}, ConverterParameter=450}" Stretch="UniformToFill" /> @@ -182,20 +183,8 @@ Margin="4" HorizontalAlignment="Left" VerticalAlignment="Bottom" - BoxShadow="inset 1.2 0 80 1.8 #66000000" + Background="#99000000" CornerRadius="16"> - - - - diff --git a/StabilityMatrix.Avalonia/Views/OutputsPage.axaml b/StabilityMatrix.Avalonia/Views/OutputsPage.axaml index 6aed5367c..8bf22ee53 100644 --- a/StabilityMatrix.Avalonia/Views/OutputsPage.axaml +++ b/StabilityMatrix.Avalonia/Views/OutputsPage.axaml @@ -262,6 +262,7 @@ vendorLabs:BetterAsyncImage.ImageCache="{x:Static controls:ImageLoaders.OutputsPageImageCache}" Command="{Binding $parent[ItemsRepeater].((vm:OutputsPageViewModel)DataContext).OnImageClick}" CommandParameter="{Binding}" + DecodeWidth="600" ImageHeight="300" ImageWidth="300" IsSelected="{Binding IsSelected}" diff --git a/StabilityMatrix.Core/Converters/Json/NullToDefaultJsonConverter.cs b/StabilityMatrix.Core/Converters/Json/NullToDefaultJsonConverter.cs new file mode 100644 index 000000000..860aa57f0 --- /dev/null +++ b/StabilityMatrix.Core/Converters/Json/NullToDefaultJsonConverter.cs @@ -0,0 +1,36 @@ +using System.Text.Json; +using System.Text.Json.Serialization; + +namespace StabilityMatrix.Core.Converters.Json; + +/// +/// Reads JSON null as default(T) for non-nullable value-type properties. +/// For APIs that send null where a number is expected (e.g. CivitAI stats counts), +/// where the default serializer would throw and fail the whole response. +/// Apply per-property via — not intended for +/// , since registering it globally would +/// silently accept null for every property of type . +/// +public class NullToDefaultJsonConverter : JsonConverter + where T : struct +{ + /// + public override bool HandleNull => true; + + /// + public override T Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) + { + if (reader.TokenType is JsonTokenType.Null) + { + return default; + } + + return JsonSerializer.Deserialize(ref reader, options); + } + + /// + public override void Write(Utf8JsonWriter writer, T value, JsonSerializerOptions options) + { + JsonSerializer.Serialize(writer, value, options); + } +} diff --git a/StabilityMatrix.Core/Models/Api/CivitFileType.cs b/StabilityMatrix.Core/Models/Api/CivitFileType.cs index 314c39504..6ad9cbf75 100644 --- a/StabilityMatrix.Core/Models/Api/CivitFileType.cs +++ b/StabilityMatrix.Core/Models/Api/CivitFileType.cs @@ -1,6 +1,7 @@ -using System.Runtime.Serialization; +using System.Runtime.Serialization; using System.Text.Json.Serialization; using StabilityMatrix.Core.Converters.Json; +using StabilityMatrix.Core.Extensions; namespace StabilityMatrix.Core.Models.Api; @@ -17,5 +18,113 @@ public enum CivitFileType PrunedModel, [EnumMember(Value = "Training Data")] - TrainingData + TrainingData, + + [EnumMember(Value = "Diffusion Model")] + DiffusionModel, + + [EnumMember(Value = "Text Encoder")] + TextEncoder, + + [EnumMember(Value = "Vision Encoder")] + VisionEncoder, + + Negative, + UNet, + CLIPVision, + ControlNet, + Workflow, + Upscaler, + + [EnumMember(Value = "Enhancement LoRA")] + EnhancementLora, + + Other, +} + +public static class CivitFileTypeExtensions +{ + /// + /// True for file types that carry the primary model weights: full/pruned checkpoints and + /// UNet-only diffusion models. Used for install detection and default file selection. + /// + public static bool IsModelWeights(this CivitFileType type) => + type + is CivitFileType.Model + or CivitFileType.PrunedModel + or CivitFileType.DiffusionModel + or CivitFileType.UNet; + + /// + /// True for file types worth listing as downloadable files in the model browser — + /// model weights plus companion components (VAE, text/vision encoders, etc). + /// + public static bool IsDownloadableModelFile(this CivitFileType type) => + type.IsModelWeights() + || type + is CivitFileType.VAE + or CivitFileType.TextEncoder + or CivitFileType.VisionEncoder + or CivitFileType.CLIPVision + or CivitFileType.ControlNet + or CivitFileType.Upscaler + or CivitFileType.Negative + or CivitFileType.EnhancementLora; + + /// + /// Maps file types that unambiguously determine their destination shared folder, + /// regardless of the parent model's type. Returns null when the destination + /// depends on the model type instead (e.g. plain "Model" files). + /// + public static SharedFolderType? GetExplicitSharedFolderType(this CivitFileType type) => + type switch + { + CivitFileType.VAE => SharedFolderType.VAE, + CivitFileType.DiffusionModel or CivitFileType.UNet => SharedFolderType.DiffusionModels, + CivitFileType.TextEncoder => SharedFolderType.TextEncoders, + CivitFileType.VisionEncoder or CivitFileType.CLIPVision => SharedFolderType.ClipVision, + CivitFileType.ControlNet => SharedFolderType.ControlNet, + CivitFileType.Upscaler => SharedFolderType.ESRGAN, + _ => null, + }; + + /// + /// Resolves the destination shared folder for a file: explicit file types win; the legacy + /// fallback then guesses UNet-only checkpoints for files still typed plain "Model" + /// (base model name prefixes and GGUF format); otherwise the model type decides. + /// + public static SharedFolderType GetSharedFolderType( + this CivitFileType type, + CivitModelType modelType, + string? baseModelType, + string? fileName = null, + CivitModelFormat? format = null + ) + { + if (type.GetExplicitSharedFolderType() is { } explicitFolder) + { + return explicitFolder; + } + + if (modelType is CivitModelType.Checkpoint) + { + var isUnetOnly = + baseModelType?.StartsWith("Wan", StringComparison.OrdinalIgnoreCase) is true + || baseModelType?.StartsWith("Flux", StringComparison.OrdinalIgnoreCase) is true + || baseModelType?.StartsWith("Hunyuan", StringComparison.OrdinalIgnoreCase) is true + || baseModelType?.StartsWith("Krea", StringComparison.OrdinalIgnoreCase) is true + || format == CivitModelFormat.GGUF + || ( + fileName is not null + && Path.GetExtension(fileName).Equals(".gguf", StringComparison.OrdinalIgnoreCase) + ); + + if (isUnetOnly) + { + return SharedFolderType.DiffusionModels; + } + } + + return modelType.ConvertTo(); + } } diff --git a/StabilityMatrix.Core/Models/Api/CivitModel.cs b/StabilityMatrix.Core/Models/Api/CivitModel.cs index 26de130b7..51637a3c8 100644 --- a/StabilityMatrix.Core/Models/Api/CivitModel.cs +++ b/StabilityMatrix.Core/Models/Api/CivitModel.cs @@ -48,7 +48,7 @@ public FileSizeType FullFilesSize var latestVersion = ModelVersions?.FirstOrDefault(); if (latestVersion?.Files != null && latestVersion.Files.Any()) { - var latestModelFile = latestVersion.Files.FirstOrDefault(x => x.Type == CivitFileType.Model); + var latestModelFile = latestVersion.Files.FirstOrDefault(x => x.Type.IsModelWeights()); kbs = latestModelFile?.SizeKb ?? 0; } fullFilesSize = new FileSizeType(kbs); diff --git a/StabilityMatrix.Core/Models/Api/CivitModelStats.cs b/StabilityMatrix.Core/Models/Api/CivitModelStats.cs index 109c7585b..89a111a3f 100644 --- a/StabilityMatrix.Core/Models/Api/CivitModelStats.cs +++ b/StabilityMatrix.Core/Models/Api/CivitModelStats.cs @@ -1,15 +1,19 @@ -using System.Text.Json.Serialization; +using System.Text.Json.Serialization; +using StabilityMatrix.Core.Converters.Json; namespace StabilityMatrix.Core.Models.Api; public record CivitModelStats : CivitStats { [JsonPropertyName("favoriteCount")] + [JsonConverter(typeof(NullToDefaultJsonConverter))] public int FavoriteCount { get; set; } [JsonPropertyName("commentCount")] + [JsonConverter(typeof(NullToDefaultJsonConverter))] public int CommentCount { get; set; } [JsonPropertyName("thumbsUpCount")] + [JsonConverter(typeof(NullToDefaultJsonConverter))] public int ThumbsUpCount { get; set; } } diff --git a/StabilityMatrix.Core/Models/Api/CivitStats.cs b/StabilityMatrix.Core/Models/Api/CivitStats.cs index 932a9c4aa..5d3d1c91e 100644 --- a/StabilityMatrix.Core/Models/Api/CivitStats.cs +++ b/StabilityMatrix.Core/Models/Api/CivitStats.cs @@ -1,15 +1,19 @@ -using System.Text.Json.Serialization; +using System.Text.Json.Serialization; +using StabilityMatrix.Core.Converters.Json; namespace StabilityMatrix.Core.Models.Api; public record CivitStats { [JsonPropertyName("downloadCount")] + [JsonConverter(typeof(NullToDefaultJsonConverter))] public int DownloadCount { get; set; } [JsonPropertyName("ratingCount")] + [JsonConverter(typeof(NullToDefaultJsonConverter))] public int RatingCount { get; set; } [JsonPropertyName("rating")] + [JsonConverter(typeof(NullToDefaultJsonConverter))] public double Rating { get; set; } } diff --git a/StabilityMatrix.Core/Models/Api/Comfy/Nodes/ComfyNodeBuilder.cs b/StabilityMatrix.Core/Models/Api/Comfy/Nodes/ComfyNodeBuilder.cs index 317c23250..f08125648 100644 --- a/StabilityMatrix.Core/Models/Api/Comfy/Nodes/ComfyNodeBuilder.cs +++ b/StabilityMatrix.Core/Models/Api/Comfy/Nodes/ComfyNodeBuilder.cs @@ -761,6 +761,65 @@ public record UnetLoaderGGUF : ComfyTypedNodeBase public required string UnetName { get; init; } } + // The GGUF CLIP loaders below can also load .safetensors encoders, so a mixed + // gguf + safetensors selection can be routed entirely through the GGUF variant + + [TypedNodeOptions( + Name = "CLIPLoaderGGUF", + RequiredExtensions = ["https://github.com/city96/ComfyUI-GGUF"] + )] + public record CLIPLoaderGGUF : ComfyTypedNodeBase + { + public required string ClipName { get; init; } + + /// + /// Same values as + /// + public required string Type { get; init; } + } + + [TypedNodeOptions( + Name = "DualCLIPLoaderGGUF", + RequiredExtensions = ["https://github.com/city96/ComfyUI-GGUF"] + )] + public record DualCLIPLoaderGGUF : ComfyTypedNodeBase + { + public required string ClipName1 { get; init; } + public required string ClipName2 { get; init; } + + /// + /// Same values as + /// + public required string Type { get; init; } + } + + [TypedNodeOptions( + Name = "TripleCLIPLoaderGGUF", + RequiredExtensions = ["https://github.com/city96/ComfyUI-GGUF"] + )] + public record TripleCLIPLoaderGGUF : ComfyTypedNodeBase + { + public required string ClipName1 { get; init; } + public required string ClipName2 { get; init; } + public required string ClipName3 { get; init; } + + // no type input, like TripleCLIPLoader + } + + [TypedNodeOptions( + Name = "QuadrupleCLIPLoaderGGUF", + RequiredExtensions = ["https://github.com/city96/ComfyUI-GGUF"] + )] + public record QuadrupleCLIPLoaderGGUF : ComfyTypedNodeBase + { + public required string ClipName1 { get; init; } + public required string ClipName2 { get; init; } + public required string ClipName3 { get; init; } + public required string ClipName4 { get; init; } + + // no type input, like QuadrupleCLIPLoader + } + [TypedNodeOptions( Name = "Inference_Core_PromptExpansion", RequiredExtensions = ["https://github.com/LykosAI/ComfyUI-Inference-Core-Nodes >= 0.2.0"] diff --git a/StabilityMatrix.Core/Models/HybridModelFile.cs b/StabilityMatrix.Core/Models/HybridModelFile.cs index 03464db8d..4a49a5d03 100644 --- a/StabilityMatrix.Core/Models/HybridModelFile.cs +++ b/StabilityMatrix.Core/Models/HybridModelFile.cs @@ -61,6 +61,14 @@ public record HybridModelFile : ISearchText, IDownloadableResource [JsonIgnore] public string FileName => Path.GetFileName(RelativePath); + /// + /// Whether this file is a GGUF-quantized model (by file extension). + /// + [JsonIgnore] + public bool IsGguf => + Type is not HybridModelType.None + && RelativePath.EndsWith(".gguf", StringComparison.OrdinalIgnoreCase); + [JsonIgnore] public string ShortDisplayName { diff --git a/StabilityMatrix.Core/Models/Packages/Reforge.cs b/StabilityMatrix.Core/Models/Packages/Reforge.cs index 6bf79fb8c..f31cb1dd8 100644 --- a/StabilityMatrix.Core/Models/Packages/Reforge.cs +++ b/StabilityMatrix.Core/Models/Packages/Reforge.cs @@ -159,6 +159,10 @@ await rocmPackageHelper // unpinned to match. Forge (the SDWebForge base) keeps its own default. protected override string TorchVersionSpec => "==2.9.0"; + // The rocm7.2 index has no torch 2.9.x, so with the ==2.9.0 pin pip would fall back to the + // CUDA wheel from PyPI on Linux AMD installs (#1669). rocm6.4 hosts 2.9.0+rocm6.4. + protected override string RocmIndexName => "rocm6.4"; + protected override ImmutableDictionary GetEnvVars( ImmutableDictionary env, InstalledPackage installedPackage diff --git a/StabilityMatrix.Core/Models/Packages/SDWebForge.cs b/StabilityMatrix.Core/Models/Packages/SDWebForge.cs index 517427ad1..481d6913c 100644 --- a/StabilityMatrix.Core/Models/Packages/SDWebForge.cs +++ b/StabilityMatrix.Core/Models/Packages/SDWebForge.cs @@ -54,6 +54,12 @@ IPipWheelService pipWheelService /// protected virtual string TorchVersionSpec => " "; + /// + /// ROCm torch index used on Linux installs. Subclasses that pin torch must pick an index that + /// actually hosts that version, or pip falls back to the CUDA wheel from PyPI. + /// + protected virtual string RocmIndexName => "rocm7.2"; + public override List LaunchOptions => [ new() @@ -191,7 +197,7 @@ torchIndex is TorchIndex.Cuda TorchVersion = TorchVersionSpec, TorchvisionVersion = " ", CudaIndex = isLegacyNvidia ? "cu126" : "cu128", - RocmIndex = "rocm7.2", + RocmIndex = RocmIndexName, ExtraPipArgs = [ "https://github.com/openai/CLIP/archive/d50d76daa670286dd6cacf3bcd80b5e4823fc8e1.zip", diff --git a/StabilityMatrix.Core/Services/ModelIndexService.cs b/StabilityMatrix.Core/Services/ModelIndexService.cs index ea69d302f..c7c8633fc 100644 --- a/StabilityMatrix.Core/Services/ModelIndexService.cs +++ b/StabilityMatrix.Core/Services/ModelIndexService.cs @@ -671,7 +671,7 @@ await liteDbContext.LocalModelFiles.FindAllAsync().ConfigureAwait(false) ?? [] } var latestHashes = latestVersionFiles - .Where(f => f.Type == CivitFileType.Model) + .Where(f => f.Type.IsModelWeights()) .Select(f => f.Hashes.BLAKE3) .Where(hash => hash is not null) .ToList(); diff --git a/StabilityMatrix.Tests/Core/CivitFileTypeTests.cs b/StabilityMatrix.Tests/Core/CivitFileTypeTests.cs new file mode 100644 index 000000000..8806f7766 --- /dev/null +++ b/StabilityMatrix.Tests/Core/CivitFileTypeTests.cs @@ -0,0 +1,84 @@ +using System.Text.Json; +using StabilityMatrix.Core.Models; +using StabilityMatrix.Core.Models.Api; + +namespace StabilityMatrix.Tests.Core; + +[TestClass] +public class CivitFileTypeTests +{ + /// + /// The canonical file type list from Civitai's constants.modelFileTypes + /// (src/server/common/constants.ts). A type missing from our enum deserializes as + /// Unknown, which hides the file (and its download link) in the model browser — + /// keep in sync with this list. + /// + private static readonly string[] CivitaiModelFileTypes = + [ + "Model", + "Text Encoder", + "Vision Encoder", + "Pruned Model", + "Negative", + "Training Data", + "VAE", + "Config", + "Archive", + "UNet", + "Diffusion Model", + "CLIPVision", + "ControlNet", + "Workflow", + "Upscaler", + "Enhancement LoRA", + "Other", + ]; + + [TestMethod] + public void AllCivitaiFileTypes_ShouldDeserializeToKnownValues() + { + foreach (var typeString in CivitaiModelFileTypes) + { + var result = JsonSerializer.Deserialize($"\"{typeString}\""); + + Assert.AreNotEqual( + CivitFileType.Unknown, + result, + $"'{typeString}' deserialized to Unknown - add a member (with EnumMember for spaced values) to CivitFileType" + ); + } + } + + [TestMethod] + public void DiffusionModelFileTypes_ShouldCountAsModelWeights() + { + Assert.IsTrue(CivitFileType.Model.IsModelWeights()); + Assert.IsTrue(CivitFileType.PrunedModel.IsModelWeights()); + Assert.IsTrue(CivitFileType.DiffusionModel.IsModelWeights()); + Assert.IsTrue(CivitFileType.UNet.IsModelWeights()); + + Assert.IsFalse(CivitFileType.VAE.IsModelWeights()); + Assert.IsFalse(CivitFileType.TrainingData.IsModelWeights()); + Assert.IsFalse(CivitFileType.Unknown.IsModelWeights()); + } + + [TestMethod] + public void ExplicitlyTypedComponentFiles_ShouldMapToSharedFolders() + { + Assert.AreEqual( + SharedFolderType.DiffusionModels, + CivitFileType.DiffusionModel.GetExplicitSharedFolderType() + ); + Assert.AreEqual(SharedFolderType.DiffusionModels, CivitFileType.UNet.GetExplicitSharedFolderType()); + Assert.AreEqual(SharedFolderType.VAE, CivitFileType.VAE.GetExplicitSharedFolderType()); + Assert.AreEqual( + SharedFolderType.TextEncoders, + CivitFileType.TextEncoder.GetExplicitSharedFolderType() + ); + Assert.AreEqual(SharedFolderType.ClipVision, CivitFileType.CLIPVision.GetExplicitSharedFolderType()); + + // Plain "Model" files depend on the parent model type, not the file type + Assert.IsNull(CivitFileType.Model.GetExplicitSharedFolderType()); + Assert.IsNull(CivitFileType.PrunedModel.GetExplicitSharedFolderType()); + } +} diff --git a/StabilityMatrix.Tests/Core/NullToDefaultJsonConverterTests.cs b/StabilityMatrix.Tests/Core/NullToDefaultJsonConverterTests.cs new file mode 100644 index 000000000..f26bcdd71 --- /dev/null +++ b/StabilityMatrix.Tests/Core/NullToDefaultJsonConverterTests.cs @@ -0,0 +1,78 @@ +using System.Text.Json; +using StabilityMatrix.Core.Models.Api; + +namespace StabilityMatrix.Tests.Core; + +[TestClass] +public class NullToDefaultJsonConverterTests +{ + [TestMethod] + public void TestDeserialize_CivitStatsWithNulls_ShouldDefaultToZero() + { + // CivitAI started sending null for numeric stats fields (observed live 2026-08-01) + const string json = """ + { + "downloadCount": null, + "ratingCount": null, + "rating": null, + "favoriteCount": null, + "commentCount": null, + "thumbsUpCount": null + } + """; + + var result = JsonSerializer.Deserialize(json); + + Assert.IsNotNull(result); + Assert.AreEqual(0, result.DownloadCount); + Assert.AreEqual(0, result.RatingCount); + Assert.AreEqual(0d, result.Rating); + Assert.AreEqual(0, result.FavoriteCount); + Assert.AreEqual(0, result.CommentCount); + Assert.AreEqual(0, result.ThumbsUpCount); + } + + [TestMethod] + public void TestDeserialize_CivitStatsWithValues_ShouldReadValues() + { + const string json = """ + { + "downloadCount": 1234, + "ratingCount": 56, + "rating": 4.5, + "favoriteCount": 7, + "commentCount": 8, + "thumbsUpCount": 90 + } + """; + + var result = JsonSerializer.Deserialize(json); + + Assert.IsNotNull(result); + Assert.AreEqual(1234, result.DownloadCount); + Assert.AreEqual(56, result.RatingCount); + Assert.AreEqual(4.5, result.Rating); + Assert.AreEqual(7, result.FavoriteCount); + Assert.AreEqual(8, result.CommentCount); + Assert.AreEqual(90, result.ThumbsUpCount); + } + + [TestMethod] + public void TestSerialize_CivitStats_ShouldRoundTrip() + { + var stats = new CivitModelStats + { + DownloadCount = 42, + Rating = 3.5, + ThumbsUpCount = 5, + }; + + var json = JsonSerializer.Serialize(stats); + var result = JsonSerializer.Deserialize(json); + + Assert.IsNotNull(result); + Assert.AreEqual(42, result.DownloadCount); + Assert.AreEqual(3.5, result.Rating); + Assert.AreEqual(5, result.ThumbsUpCount); + } +} diff --git a/StabilityMatrix.UITests/PaintCanvasConcurrencyTests.cs b/StabilityMatrix.UITests/PaintCanvasConcurrencyTests.cs new file mode 100644 index 000000000..9cee205cd --- /dev/null +++ b/StabilityMatrix.UITests/PaintCanvasConcurrencyTests.cs @@ -0,0 +1,259 @@ +using System; +using System.Collections.Generic; +using System.Collections.Immutable; +using System.Diagnostics; +using System.Threading; +using System.Threading.Tasks; +using SkiaSharp; +using StabilityMatrix.Avalonia.Controls.Models; +using StabilityMatrix.Avalonia.ViewModels.Controls; + +namespace StabilityMatrix.UITests; + +/// +/// Phase 0 stress scaffold for the races that the threading redesign will fix. These tests are +/// implemented fully but SKIPPED so CI stays green while the current code still races. They are +/// intended to be enabled in Phase 5 (after the redesign) to prove the races are gone. +/// +/// One-time characterization (Phase 0, run manually with the Skip removed, GPU disabled / CPU-only +/// headless Skia) against the CURRENT code — 3 runs each: +/// * Test A (live-stroke append vs ToSKPath/index reads): PASSED all 3 runs, no throw. The writer +/// completes 100k appends almost immediately (~11ms total), so in this configuration the reader +/// rarely overlaps a backing-array resize. The race is real in principle (List<PenPoint> is +/// not thread-safe and ToSKPath/RenderFreehandPath only snapshot Count, not the array reference), +/// but this harness does NOT reliably reproduce it. Phase 5 should consider a slower/paced writer +/// or a longer-lived overlap to actually surface it. +/// * Test B (export vs mutate): PASSED all 3 runs (full 2s window), no throw, no AccessViolation, +/// host exit code 0. The headless export path composites onto CPU surfaces guarded by renderLock +/// + per-layer locks, which is sufficient here. The AccessViolation-class crashes the VM comments +/// describe happen on the GPU compositor render thread (on-screen leased GPU surfaces tied to that +/// thread) — a path this off-screen export harness does not exercise. Phase 5 may need an +/// on-screen / GPU-backed scenario to characterize the real crash. +/// +public class PaintCanvasConcurrencyTests +{ + // Both stress tests are live: Test A was enabled in Phase 3 (LiveStroke), Test B in Phase 5 + // (lock-free ownership model). The characterization notes in the class doc describe how the + // PRE-redesign code behaved when these were first written. + + // ==== Test A: live stroke append vs concurrent snapshot reads ==== + // Enabled as of Phase 3: LiveStroke publishes copy-on-append snapshots, so concurrent + // append-while-render is structurally safe (a captured array is never mutated). + + [AvaloniaFact] + public async Task LiveStroke_AppendWhileReading_DoesNotThrow() + { + await RunLiveStrokeStress(); + } + + private static async Task RunLiveStrokeStress() + { + var stroke = new LiveStroke + { + Template = new PenPath + { + FillColor = SKColors.Red, + Radius = 3f, + PathType = PenPathType.Freehand, + }, + }; + + using var cts = new CancellationTokenSource(); + Exception? failure = null; + + // Writer: publish points in small batches (mirrors HandlePointerMoved's per-event batch), + // yielding occasionally so the reader gets real overlap with array growth. + var writer = Task.Run(() => + { + try + { + var batch = new PenPoint[8]; + for (var i = 0; i < 100_000; i += batch.Length) + { + for (var j = 0; j < batch.Length; j++) + { + var n = i + j; + batch[j] = new PenPoint((ulong)(n % 64), (ulong)((n / 64) % 64)) + { + IsPen = true, + Pressure = (n % 100) / 100.0, + }; + } + + stroke.AddPoints(batch); + + if (i % 1024 == 0) + { + Thread.Yield(); + } + } + } + catch (Exception ex) + { + failure = ex; + } + finally + { + cts.Cancel(); + } + }); + + // Reader: render snapshots to a CPU canvas and verify prefix consistency — the number of + // observed points must never decrease, and every snapshot must be fully readable. + var reader = Task.Run(() => + { + try + { + using var surface = SKSurface.Create(new SKImageInfo(64, 64)); + using var paint = new SKPaint(); + var lastCount = 0; + + while (!cts.IsCancellationRequested) + { + var snapshot = stroke.GetPointsSnapshot(); + + Assert.True( + snapshot.Length >= lastCount, + $"Snapshot shrank: {snapshot.Length} < {lastCount}" + ); + lastCount = snapshot.Length; + + foreach (var p in snapshot) + { + _ = p.X + p.Y + (p.Pressure ?? 1) + (p.IsPen ? 1 : 0); + } + + PaintCanvasViewModel.RenderLiveStroke(surface.Canvas, stroke, paint); + } + } + catch (Exception ex) + { + failure = ex; + } + }); + + await Task.WhenAll(writer, reader); + Assert.Null(failure); + Assert.Equal(100_000, stroke.GetPointsSnapshot().Length); + } + + // ==== Test B: render thread vs UI-thread mutation + export ==== + // Enabled as of Phase 5 (lock-free ownership model). Mirrors the app's REAL threading + // contract: one thread plays the compositor render thread (RenderToSurface in a loop), one + // thread plays the UI thread (mutations and exports are serialized on it, as they are in the + // app via the dispatcher). This is exactly the interleaving that used to cause the native + // use-after-free crash class: the UI thread swapping/disposing layer bitmaps and invalidating + // the path cache while a frame was mid-render. + + [AvaloniaFact] + public async Task Render_WhileUiThreadMutatesAndExports_DoesNotThrow() + { + await RunRenderMutateStress(TimeSpan.FromSeconds(2)); + } + + private static async Task RunRenderMutateStress(TimeSpan duration) + { + var vm = TestHelpers.CreatePaintCanvasViewModel(); + vm.CanvasSize = new System.Drawing.Size(64, 64); + // Signal that this canvas renders on-screen so swapped-out layer bitmaps take the + // deferred-dispose path (drained by the render loop) instead of synchronous disposal. + vm.RefreshCanvas = () => { }; + vm.Paths = ImmutableList.Create( + TestHelpers.BuildPenStroke(SKColors.Red), + TestHelpers.BuildMouseStroke(SKColors.Blue), + TestHelpers.BuildRectangle(SKColors.Green, new SKRect(20, 20, 50, 50)) + ); + + using var cts = new CancellationTokenSource(duration); + var token = cts.Token; + var failures = new List(); + var failuresLock = new object(); + + void Record(Exception ex) + { + lock (failuresLock) + { + failures.Add(ex); + } + } + + // Render thread: RenderToSurface onto a locally created CPU surface, every "frame". + var renderThread = Task.Run(() => + { + try + { + using var surface = SKSurface.Create(new SKImageInfo(64, 64)); + Assert.NotNull(surface); + + while (!token.IsCancellationRequested) + { + vm.RenderToSurface(surface!, renderBackgroundFill: true, renderBackgroundImage: true); + } + } + catch (Exception ex) + { + Record(ex); + } + }); + + // Simulated UI thread: mutations AND exports, serialized with each other (as the + // dispatcher serializes them in the app) but fully concurrent with the render thread. + var uiThread = Task.Run(() => + { + try + { + var i = 0; + while (!token.IsCancellationRequested) + { + switch (i++ % 6) + { + case 0: + vm.Paths = vm.Paths.Add(TestHelpers.BuildPenStroke(SKColors.Yellow)); + vm.ClearRedoStack(); + break; + case 1: + vm.Undo(); + break; + case 2: + vm.Redo(); + break; + case 3: + using (var bmp = new SKBitmap(64, 64, SKColorType.Rgba8888, SKAlphaType.Premul)) + { + // Give SetLayerBitmap ownership of a fresh copy each time. + vm.SetLayerBitmap("Images", bmp.Copy()); + } + break; + case 4: + using (var image = vm.RenderToImage()) + { + Assert.NotNull(image); + } + using (var mask = vm.RenderToWhiteChannelImage()) + { + Assert.NotNull(mask); + } + break; + case 5: + vm.ClearCanvas(); + vm.Paths = ImmutableList.Create( + TestHelpers.BuildRectangle(SKColors.Green, new SKRect(10, 10, 40, 40)) + ); + break; + } + } + } + catch (Exception ex) + { + Record(ex); + } + }); + + await Task.WhenAll(renderThread, uiThread); + + // Dispose while conceptually "just after" rendering stopped — exercises the quiescence gate. + vm.Dispose(); + + Assert.Empty(failures); + } +} diff --git a/StabilityMatrix.UITests/PaintCanvasRenderTests.cs b/StabilityMatrix.UITests/PaintCanvasRenderTests.cs new file mode 100644 index 000000000..2e302f85e --- /dev/null +++ b/StabilityMatrix.UITests/PaintCanvasRenderTests.cs @@ -0,0 +1,306 @@ +using System; +using System.Collections.Immutable; +using SkiaSharp; +using StabilityMatrix.Avalonia.Controls.Models; +using StabilityMatrix.Avalonia.ViewModels.Controls; +using Xunit.Abstractions; + +namespace StabilityMatrix.UITests; + +/// +/// Phase 0 golden characterization of paint-canvas export rendering. Later refactor phases +/// (the threading redesign) must not silently change the exported image. Assertions are robust to +/// anti-aliasing: we do NOT hash whole images. Instead we pin the non-transparent pixel count within +/// a tolerance band, the painted bounding box within a few px, and exact colors at a handful of +/// interior sample points chosen after observing the current (correct-by-definition) output. +/// +public class PaintCanvasRenderTests +{ + private readonly ITestOutputHelper output; + + public PaintCanvasRenderTests(ITestOutputHelper output) + { + this.output = output; + } + + private const int CanvasWidth = 64; + private const int CanvasHeight = 64; + + private static readonly SKColor PenColor = new(255, 0, 0, 255); // red + private static readonly SKColor MouseColor = new(0, 0, 255, 255); // blue + private static readonly SKColor RectColor = new(0, 200, 0, 255); // green + + private static PaintCanvasViewModel BuildScene() + { + var vm = TestHelpers.CreatePaintCanvasViewModel(); + vm.CanvasSize = new System.Drawing.Size(CanvasWidth, CanvasHeight); + + // Deterministic scene: pen stroke (upper band), mouse stroke (lower band), + // rectangle (mid), erase stroke crossing them. + vm.Paths = ImmutableList.Create( + TestHelpers.BuildPenStroke(PenColor), + TestHelpers.BuildMouseStroke(MouseColor), + TestHelpers.BuildRectangle(RectColor, new SKRect(30, 24, 58, 40)), + TestHelpers.BuildEraseStroke() + ); + + return vm; + } + + private static SKBitmap RenderToBitmap(SKImage image) + { + var bitmap = new SKBitmap( + new SKImageInfo(image.Width, image.Height, SKColorType.Rgba8888, SKAlphaType.Unpremul) + ); + Assert.True(image.ReadPixels(bitmap.Info, bitmap.GetPixels(), bitmap.RowBytes, 0, 0)); + return bitmap; + } + + private static int CountNonTransparent(SKBitmap bitmap) + { + var count = 0; + for (var y = 0; y < bitmap.Height; y++) + { + for (var x = 0; x < bitmap.Width; x++) + { + if (bitmap.GetPixel(x, y).Alpha > 8) + count++; + } + } + + return count; + } + + private static SKRectI PaintedBounds(SKBitmap bitmap) + { + int minX = bitmap.Width, + minY = bitmap.Height, + maxX = -1, + maxY = -1; + for (var y = 0; y < bitmap.Height; y++) + { + for (var x = 0; x < bitmap.Width; x++) + { + if (bitmap.GetPixel(x, y).Alpha <= 8) + continue; + minX = Math.Min(minX, x); + minY = Math.Min(minY, y); + maxX = Math.Max(maxX, x); + maxY = Math.Max(maxY, y); + } + } + + return new SKRectI(minX, minY, maxX, maxY); + } + + /// + /// Diagnostic dump used once to derive the characterization constants below. Kept (skipped) so a + /// future maintainer can re-derive expected values after an intentional rendering change. + /// + [AvaloniaFact(Skip = "Diagnostic only; run manually to re-derive characterization constants")] + public void DumpCharacterization() + { + var vm = BuildScene(); + + using var image = vm.RenderToImage()!; + using var bitmap = RenderToBitmap(image); + var bounds = PaintedBounds(bitmap); + + output.WriteLine($"RenderToImage non-transparent count = {CountNonTransparent(bitmap)}"); + output.WriteLine($"RenderToImage painted bounds = {bounds}"); + + // Sample a grid of interior coordinates so we can pick stable ones. + for (var y = 8; y < CanvasHeight; y += 8) + { + for (var x = 8; x < CanvasWidth; x += 8) + { + output.WriteLine($" px ({x},{y}) = {bitmap.GetPixel(x, y)}"); + } + } + + using var whiteImage = vm.RenderToWhiteChannelImage()!; + using var whiteBitmap = RenderToBitmap(whiteImage); + output.WriteLine( + $"RenderToWhiteChannelImage non-transparent count = {CountNonTransparent(whiteBitmap)}" + ); + output.WriteLine($"White painted bounds = {PaintedBounds(whiteBitmap)}"); + for (var y = 8; y < CanvasHeight; y += 8) + { + for (var x = 8; x < CanvasWidth; x += 8) + { + var p = whiteBitmap.GetPixel(x, y); + if (p.Alpha > 8) + output.WriteLine($" white px ({x},{y}) = {p}"); + } + } + } + + // ==== Characterization constants (derived from the current, correct-by-definition output) ==== + // Filled in after running DumpCharacterization once. See method above to regenerate. + + private const int ExpectedColorNonTransparent = 696; + private const int ColorCountTolerance = 40; + + [AvaloniaFact] + public void RenderToImage_NonTransparentPixelCount_WithinBand() + { + var vm = BuildScene(); + using var image = vm.RenderToImage()!; + Assert.NotNull(image); + using var bitmap = RenderToBitmap(image); + + var count = CountNonTransparent(bitmap); + Assert.InRange( + count, + ExpectedColorNonTransparent - ColorCountTolerance, + ExpectedColorNonTransparent + ColorCountTolerance + ); + } + + [AvaloniaFact] + public void RenderToImage_PaintedBounds_WithinTolerance() + { + var vm = BuildScene(); + using var image = vm.RenderToImage()!; + using var bitmap = RenderToBitmap(image); + var bounds = PaintedBounds(bitmap); + + AssertClose(bounds.Left, ExpectedBoundsLeft); + AssertClose(bounds.Top, ExpectedBoundsTop); + AssertClose(bounds.Right, ExpectedBoundsRight); + AssertClose(bounds.Bottom, ExpectedBoundsBottom); + } + + private const int ExpectedBoundsLeft = 3; + private const int ExpectedBoundsTop = 12; + private const int ExpectedBoundsRight = 57; + private const int ExpectedBoundsBottom = 48; + + private static void AssertClose(int actual, int expected, int tolerance = 2) + { + Assert.InRange(actual, expected - tolerance, expected + tolerance); + } + + [AvaloniaFact] + public void RenderToImage_SampledColors_MatchGolden() + { + var vm = BuildScene(); + using var image = vm.RenderToImage()!; + using var bitmap = RenderToBitmap(image); + + foreach (var (x, y, expected) in ColorSamples) + { + var actual = bitmap.GetPixel(x, y); + Assert.True( + ColorsClose(actual, expected), + $"pixel ({x},{y}) expected ~{expected} but was {actual}" + ); + } + } + + // Sample points chosen from the diagnostic dump: stroke centers, erased region, empty region. + private static readonly (int X, int Y, SKColor Expected)[] ColorSamples = + [ + (16, 16, new SKColor(255, 0, 0, 255)), // pen stroke (red) + (40, 16, new SKColor(255, 0, 0, 255)), // pen stroke (red) + (48, 32, new SKColor(0, 200, 0, 255)), // rectangle interior (green) + (48, 24, new SKColor(0, 200, 0, 255)), // rectangle interior (green) + (16, 48, new SKColor(0, 0, 255, 255)), // mouse stroke (blue) + (40, 48, SKColors.Transparent), // erase stroke cleared this part of the mouse stroke + (8, 8, SKColors.Transparent), // empty region + ]; + + private static bool ColorsClose(SKColor a, SKColor b, int tolerance = 20) + { + // When both are (near-)transparent, RGB is meaningless — Skia zeroes it to #00000000 + // while SKColors.Transparent is #00FFFFFF. Compare by alpha only in that case. + if (a.Alpha <= tolerance && b.Alpha <= tolerance) + return true; + + return Math.Abs(a.Red - b.Red) <= tolerance + && Math.Abs(a.Green - b.Green) <= tolerance + && Math.Abs(a.Blue - b.Blue) <= tolerance + && Math.Abs(a.Alpha - b.Alpha) <= tolerance; + } + + [AvaloniaFact] + public void RenderToWhiteChannelImage_PaintsWhereColorImageDoes() + { + var vm = BuildScene(); + using var colorImage = vm.RenderToImage()!; + using var colorBitmap = RenderToBitmap(colorImage); + + using var whiteImage = vm.RenderToWhiteChannelImage()!; + Assert.NotNull(whiteImage); + using var whiteBitmap = RenderToBitmap(whiteImage); + + // White-channel keeps original alpha but forces RGB to white; the painted footprint should + // closely match the color render's footprint. + var colorCount = CountNonTransparent(colorBitmap); + var whiteCount = CountNonTransparent(whiteBitmap); + Assert.InRange(whiteCount, colorCount - ColorCountTolerance, colorCount + ColorCountTolerance); + + // Every opaque white-channel pixel must be white (RGB). + for (var y = 0; y < whiteBitmap.Height; y++) + { + for (var x = 0; x < whiteBitmap.Width; x++) + { + var p = whiteBitmap.GetPixel(x, y); + if (p.Alpha <= 200) + continue; + Assert.True( + p is { Red: >= 235, Green: >= 235, Blue: >= 235 }, + $"white-channel pixel ({x},{y}) not white: {p}" + ); + } + } + } + + [AvaloniaFact] + public void Dispose_CalledTwice_DoesNotThrow() + { + var vm = BuildScene(); + // Force layer surfaces / caches to be allocated so Dispose has something to free. + using (var image = vm.RenderToImage()) + { + Assert.NotNull(image); + } + + vm.Dispose(); + var exception = Record.Exception(() => vm.Dispose()); + Assert.Null(exception); + } + + /// + /// A flood-fill path carries an owned in + /// (set by FloodFillAt) that is never otherwise disposed. Phase 1 of the threading redesign + /// makes free it. SKBitmap's underlying native handle + /// (Handle, public on SKObject) is reset to once the wrapped native + /// resource is released, so we use that to verify the bitmap was actually disposed rather than + /// merely asserting Dispose doesn't throw. (SKObject also exposes IsDisposed, but it's not + /// publicly accessible on the concrete SKBitmap type in this SkiaSharp 3.0 preview.) + /// + [AvaloniaFact] + public void Dispose_DisposesFloodFillBitmapData() + { + var vm = TestHelpers.CreatePaintCanvasViewModel(); + vm.CanvasSize = new System.Drawing.Size(CanvasWidth, CanvasHeight); + + var bitmapData = new SKBitmap(CanvasWidth, CanvasHeight, SKColorType.Rgba8888, SKAlphaType.Premul); + var bitmapPath = new PenPath + { + PathType = PenPathType.Bitmap, + FillColor = SKColors.Magenta, + BitmapData = bitmapData, + Bounds = new SKRect(0, 0, CanvasWidth, CanvasHeight), + }; + + vm.Paths = ImmutableList.Create(bitmapPath); + + Assert.NotEqual(IntPtr.Zero, bitmapData.Handle); + + vm.Dispose(); + + Assert.Equal(IntPtr.Zero, bitmapData.Handle); + } +} diff --git a/StabilityMatrix.UITests/PaintCanvasSerializationTests.cs b/StabilityMatrix.UITests/PaintCanvasSerializationTests.cs new file mode 100644 index 000000000..bdaf7e49e --- /dev/null +++ b/StabilityMatrix.UITests/PaintCanvasSerializationTests.cs @@ -0,0 +1,394 @@ +using System.Collections.Generic; +using System.Collections.Immutable; +using System.Linq; +using System.Text.Json; +using System.Text.Json.Nodes; +using SkiaSharp; +using StabilityMatrix.Avalonia.Controls.Models; + +namespace StabilityMatrix.UITests; + +/// +/// Phase 0 characterization tests guarding the JSON contract for / +/// . These are the persisted representation of paint-canvas strokes and +/// mask layers; every later threading-redesign phase must keep this contract byte-stable so old +/// projects keep loading. Two independent serialization paths are exercised: +/// +/// Path A: the source-gen serializer used by +/// PaintCanvasViewModel.SaveStateToJsonObject (via PaintCanvasModelSerializerContext). +/// Path B: the reflection path used by MaskLayer.SaveStateToJsonObject +/// (JsonSerializer.SerializeToNode(List<PenPath>) then Deserialize<List<PenPath>>). +/// +/// +/// Members intentionally NOT persisted (see [JsonIgnore] on the models): +/// +/// — legacy per-point radius; new paths carry radius on . +/// — not persisted directly, but inferred on decompress (Phase 4b): +/// a mouse point (null Pressure, IsPen false) is written with a -1 pressure sentinel and round-trips as a +/// mouse point; any written pressure in [0, 1] round-trips as a pen point. +/// — the property itself is [JsonIgnore]; points are serialized by the +/// custom converter as the compressed points string, so they DO round-trip (just not as a JSON array). +/// +/// +/// Contract quirks that these tests pin (current behavior = correct-by-definition): +/// +/// / are ulong; the point is compressed as a +/// float then read back and clamped to non-negative. So X/Y round-trip exactly for small integers. +/// is stored as a float; on decompress a value is only kept when it lies +/// in [0, 1], otherwise the point becomes a mouse point (null Pressure, IsPen false). A pen point's null +/// pressure is written as 1.0 and comes back as 1.0; a mouse point's null is written as -1 and comes back +/// as null. +/// +/// +public class PaintCanvasSerializationTests +{ + private static List BuildVariedPoints() + { + // ~10 points, varied Pressure (including null and an out-of-range value that must clamp to null), + // varied Radius (a NOT-persisted member), IsPen true/false (also NOT persisted). + return + [ + new PenPoint(0, 0) + { + Pressure = 0.0, + Radius = 3, + IsPen = true, + }, + new PenPoint(10, 5) + { + Pressure = 0.25, + Radius = 4, + IsPen = false, + }, + new PenPoint(20, 12) + { + Pressure = 0.5, + Radius = 5, + IsPen = true, + }, + new PenPoint(30, 20) + { + Pressure = null, + Radius = 6, + IsPen = true, + }, + new PenPoint(45, 33) + { + Pressure = 0.75, + Radius = 2, + IsPen = false, + }, + new PenPoint(60, 40) + { + Pressure = 1.0, + Radius = 7, + IsPen = true, + }, + new PenPoint(75, 55) + { + Pressure = null, + Radius = 1, + IsPen = false, + }, + new PenPoint(90, 70) + { + Pressure = 0.33, + Radius = 8, + IsPen = true, + }, + new PenPoint(120, 90) + { + Pressure = 0.9, + Radius = 9, + IsPen = true, + }, + new PenPoint(150, 110) + { + Pressure = 0.15, + Radius = 10, + IsPen = false, + }, + ]; + } + + private static PenPath BuildFreehandPath() + { + return new PenPath + { + Points = BuildVariedPoints(), + FillColor = new SKColor(12, 34, 56, 200), + IsErase = true, + Feathering = 0.4f, + StrokeWidth = 7.5f, + Radius = 6.25f, + IsStrokeOnly = true, + PathType = PenPathType.Freehand, + }; + } + + private static PenPath BuildRectanglePath() + { + return new PenPath + { + FillColor = new SKColor(255, 0, 0, 255), + PathType = PenPathType.Rectangle, + Bounds = new SKRect(4, 8, 40, 60), + IsStrokeOnly = false, + StrokeWidth = 5f, + }; + } + + private static PenPath BuildEllipsePath() + { + return new PenPath + { + FillColor = new SKColor(0, 128, 255, 128), + PathType = PenPathType.Ellipse, + Bounds = new SKRect(10, 10, 50, 30), + IsStrokeOnly = true, + StrokeWidth = 3f, + }; + } + + private static PenPath BuildBitmapPath() + { + // A small deterministic bitmap so the base64 PNG round-trip is exercised. + var bitmap = new SKBitmap(4, 4, SKColorType.Rgba8888, SKAlphaType.Premul); + for (var y = 0; y < 4; y++) + { + for (var x = 0; x < 4; x++) + { + bitmap.SetPixel(x, y, (x + y) % 2 == 0 ? SKColors.Lime : SKColors.Transparent); + } + } + + return new PenPath + { + FillColor = new SKColor(0, 255, 0, 255), + PathType = PenPathType.Bitmap, + BitmapData = bitmap, + Bounds = new SKRect(0, 0, 4, 4), + }; + } + + private static List BuildAllVariants() => + [BuildFreehandPath(), BuildRectanglePath(), BuildEllipsePath(), BuildBitmapPath()]; + + /// + /// Computes the expected post-round-trip (Pressure, IsPen) for a point, mirroring the + /// compress/decompress contract: written pressure = Pressure ?? (IsPen ? 1.0 : -1.0); + /// on read, values in [0, 1] are pen points, anything else is a mouse point with null pressure. + /// + private static (double? Pressure, bool IsPen) ExpectedPointRoundTrip(PenPoint point) + { + var written = point.Pressure ?? (point.IsPen ? 1.0 : -1.0); + var isPen = written is >= 0 and <= 1; + return (isPen ? written : null, isPen); + } + + /// + /// Asserts full value equality of the persisted members after a round-trip. + /// Explicitly does NOT compare PenPoint.Radius (not persisted). + /// + private static void AssertPathEqual(PenPath expected, PenPath actual) + { + Assert.Equal(expected.FillColor, actual.FillColor); + Assert.Equal(expected.IsErase, actual.IsErase); + Assert.Equal(expected.PathType, actual.PathType); + Assert.Equal(expected.Bounds, actual.Bounds); + Assert.Equal(expected.IsStrokeOnly, actual.IsStrokeOnly); + Assert.Equal(expected.StrokeWidth, actual.StrokeWidth); + Assert.Equal(expected.Radius, actual.Radius); + Assert.Equal(expected.Feathering, actual.Feathering); + + // Points: compare via the compressed/persisted representation. + // X/Y are ulong and round-trip exactly for our small integer coordinates. + Assert.Equal(expected.Points.Count, actual.Points.Count); + for (var i = 0; i < expected.Points.Count; i++) + { + var e = expected.Points[i]; + var a = actual.Points[i]; + Assert.Equal(e.X, a.X); + Assert.Equal(e.Y, a.Y); + + var (expectedPressure, expectedIsPen) = ExpectedPointRoundTrip(e); + Assert.Equal(expectedIsPen, a.IsPen); + + if (expectedPressure is { } pressure) + { + // Pressure is compressed as a float, so a double like 0.33 comes back as the nearest + // float (0.3300000131...). Compare through a float cast to characterize that precision loss. + Assert.NotNull(a.Pressure); + Assert.Equal((float)pressure, (float)a.Pressure!.Value); + } + else + { + Assert.Null(a.Pressure); + } + } + + // Bitmap data: compare dimensions and a sampling of pixels if present. + if (expected.BitmapData is { } expectedBitmap) + { + Assert.NotNull(actual.BitmapData); + Assert.Equal(expectedBitmap.Width, actual.BitmapData!.Width); + Assert.Equal(expectedBitmap.Height, actual.BitmapData.Height); + for (var y = 0; y < expectedBitmap.Height; y++) + { + for (var x = 0; x < expectedBitmap.Width; x++) + { + Assert.Equal(expectedBitmap.GetPixel(x, y), actual.BitmapData.GetPixel(x, y)); + } + } + } + else + { + Assert.Null(actual.BitmapData); + } + } + + // ---- Path B: reflection path used by MaskLayer.cs (SerializeToNode(List) -> Deserialize) ---- + + [AvaloniaFact] + public void MaskLayerReflectionPath_RoundTripsAllVariants() + { + var original = BuildAllVariants(); + + // Mirror of MaskLayer.SaveStateToJsonObject line ~417 / LoadStateFromJsonObject line ~381. + var node = JsonSerializer.SerializeToNode(original); + Assert.NotNull(node); + Assert.IsType(node); + + var roundTripped = node.Deserialize>(); + Assert.NotNull(roundTripped); + Assert.Equal(original.Count, roundTripped!.Count); + + for (var i = 0; i < original.Count; i++) + { + AssertPathEqual(original[i], roundTripped[i]); + } + } + + [AvaloniaFact] + public void MaskLayerReflectionPath_DoesNotPersistIgnoredPenPointMembers() + { + // PenPoint.Radius and PenPoint.IsPen are [JsonIgnore] and must not survive as-authored. + var original = new List { BuildFreehandPath() }; + var node = JsonSerializer.SerializeToNode(original); + var roundTripped = node.Deserialize>()!; + + var points = roundTripped[0].Points; + var originalPoints = original[0].Points; + + // Radius is not persisted; the decompress path always reconstructs with the default (1). + Assert.All(points, p => Assert.Equal(1d, p.Radius)); + + // IsPen is inferred from the written pressure (Phase 4b): mouse points (null pressure, + // IsPen false) round-trip as mouse points via the -1 sentinel; everything else is a pen point. + for (var i = 0; i < points.Count; i++) + { + Assert.Equal(ExpectedPointRoundTrip(originalPoints[i]).IsPen, points[i].IsPen); + } + } + + // ---- Path A: source-gen serializer used by PaintCanvasViewModel.Serializer.cs ---- + + [AvaloniaFact] + public void PaintCanvasViewModelState_RoundTripsPaths() + { + var original = BuildAllVariants(); + + var save = TestHelpers.CreatePaintCanvasViewModel(); + save.Paths = original.ToImmutableList(); + + // SaveStateToJsonObject serializes through PaintCanvasModelSerializerContext (source-gen). + var state = save.SaveStateToJsonObject(); + + var load = TestHelpers.CreatePaintCanvasViewModel(); + load.LoadStateFromJsonObject(state); + + Assert.Equal(original.Count, load.Paths.Count); + for (var i = 0; i < original.Count; i++) + { + AssertPathEqual(original[i], load.Paths[i]); + } + } + + [AvaloniaFact] + public void PaintCanvasViewModelState_PreservesScalarState() + { + var save = TestHelpers.CreatePaintCanvasViewModel(); + save.CanvasSize = new System.Drawing.Size(128, 96); + save.PaintBrushSize = 21; + save.PaintBrushAlpha = 0.5; + save.SelectedTool = StabilityMatrix.Avalonia.Models.PaintCanvasTool.Eraser; + + var state = save.SaveStateToJsonObject(); + + var load = TestHelpers.CreatePaintCanvasViewModel(); + load.LoadStateFromJsonObject(state); + + Assert.Equal(new System.Drawing.Size(128, 96), load.CanvasSize); + Assert.Equal(21, load.PaintBrushSize); + Assert.Equal(0.5, load.PaintBrushAlpha); + Assert.Equal(StabilityMatrix.Avalonia.Models.PaintCanvasTool.Eraser, load.SelectedTool); + } + + // ---- Byte-stability of the compressed points string ---- + + [AvaloniaFact] + public void CompressedPoints_AreByteStableAcrossSerializations() + { + var points = BuildVariedPoints(); + + // The compressed string is what actually lands in the persisted JSON (the "points" member). + var first = PenPath.CompressPointsPublic(points); + var second = PenPath.CompressPointsPublic(points); + + Assert.NotNull(first); + Assert.Equal(first, second); + } + + [AvaloniaFact] + public void SerializedPath_PointsMemberIsByteStable() + { + var path = BuildFreehandPath(); + + var firstNode = (JsonObject)JsonSerializer.SerializeToNode(path)!; + var secondNode = (JsonObject)JsonSerializer.SerializeToNode(path)!; + + var firstPoints = firstNode["points"]!.GetValue(); + var secondPoints = secondNode["points"]!.GetValue(); + + Assert.Equal(firstPoints, secondPoints); + + // And the whole serialized object is stable too (guards the full converter output, not just points). + Assert.Equal(firstNode.ToJsonString(), secondNode.ToJsonString()); + } + + // ---- Stroke finalization: LiveStroke -> PenPath ---- + + /// + /// is used at stroke-finalize time. The finalized path must + /// carry an equal, fully independent copy of the live points so later appends to the live + /// stroke cannot leak into the path that was moved into the immutable Paths collection. + /// + [AvaloniaFact] + public void LiveStrokeToPenPath_ProducesIndependentPointsList() + { + var template = BuildFreehandPath() with { Points = [] }; + var stroke = new LiveStroke { Template = template }; + stroke.AddPoints(BuildVariedPoints().ToArray()); + + var finalized = stroke.ToPenPath(); + + Assert.Equal(stroke.GetPointsSnapshot().Length, finalized.Points.Count); + Assert.Equal(BuildVariedPoints(), finalized.Points); + + // Appending to the live stroke (as the pointer handler does) must not affect the + // finalized copy. + stroke.AddPoints([new PenPoint(999, 999) { Pressure = 1.0, IsPen = true }]); + + Assert.NotEqual(stroke.GetPointsSnapshot().Length, finalized.Points.Count); + } +} diff --git a/StabilityMatrix.UITests/PaintCanvasTestHelpers.cs b/StabilityMatrix.UITests/PaintCanvasTestHelpers.cs new file mode 100644 index 000000000..7b52cb7b2 --- /dev/null +++ b/StabilityMatrix.UITests/PaintCanvasTestHelpers.cs @@ -0,0 +1,112 @@ +using System.Collections.Generic; +using Microsoft.Extensions.Logging.Abstractions; +using SkiaSharp; +using StabilityMatrix.Avalonia.Controls.Models; +using StabilityMatrix.Avalonia.ViewModels.Controls; + +namespace StabilityMatrix.UITests; + +/// +/// Shared construction helpers for the paint-canvas Phase 0 characterization tests. +/// +/// VM construction note (relevant to later phases): 's only ctor +/// dependency is ILogger<PaintCanvasViewModel>, so it can be new'd directly with a +/// — no DI container / DialogFactory needed (DesignData.cs:1541 uses +/// DialogFactory.Get<PaintCanvasViewModel>() only because that's the app's ambient pattern). +/// GPU acceleration is disabled here so rendering is deterministic CPU-only Skia; the headless test +/// app is built with UseHeadlessDrawing = false + real Skia, so RenderToImage() works. +/// +public static class TestHelpers +{ + public static PaintCanvasViewModel CreatePaintCanvasViewModel() + { + return new PaintCanvasViewModel(NullLogger.Instance) + { + // Deterministic CPU rendering; also avoids leaning on a GPU context that + // the headless runner may not provide. + UseGpuAcceleration = false, + // The checkerboard is only painted when renderBackgroundFill is requested, but keep it + // off so nothing bleeds into export characterization. + ShowCheckerboardBackground = false, + }; + } + + /// + /// A freehand pen stroke with varied pressure, running left-to-right across the upper band. + /// + public static PenPath BuildPenStroke(SKColor color) + { + var points = new List(); + for (var i = 0; i < 12; i++) + { + var x = (ulong)(6 + i * 4); + var y = (ulong)(16 + (i % 3)); + var pressure = 0.3 + (i % 5) * 0.15; // varies 0.3 .. 0.9 + points.Add(new PenPoint(x, y) { Pressure = pressure, IsPen = true }); + } + + return new PenPath + { + Points = points, + FillColor = color, + Radius = 3f, + PathType = PenPathType.Freehand, + }; + } + + /// + /// A mouse stroke (IsPen = false, no pressure) running along the lower band. + /// + public static PenPath BuildMouseStroke(SKColor color) + { + var points = new List(); + for (var i = 0; i < 12; i++) + { + var x = (ulong)(6 + i * 4); + var y = (ulong)46; + points.Add(new PenPoint(x, y) { IsPen = false }); + } + + return new PenPath + { + Points = points, + FillColor = color, + Radius = 3f, + PathType = PenPathType.Freehand, + }; + } + + public static PenPath BuildRectangle(SKColor color, SKRect bounds) + { + return new PenPath + { + FillColor = color, + PathType = PenPathType.Rectangle, + Bounds = bounds, + IsStrokeOnly = false, + }; + } + + /// + /// An erase stroke crossing the other content. + /// + public static PenPath BuildEraseStroke() + { + var points = new List(); + for (var i = 0; i < 10; i++) + { + var x = (ulong)(20 + i * 2); + var y = (ulong)(10 + i * 4); + points.Add(new PenPoint(x, y) { IsPen = true, Pressure = 1.0 }); + } + + return new PenPath + { + Points = points, + FillColor = SKColors.Transparent, + IsErase = true, + Radius = 4f, + PathType = PenPathType.Freehand, + }; + } +} diff --git a/docs/.vitepress/config.mts b/docs/.vitepress/config.mts index c86b01a0e..54674375f 100644 --- a/docs/.vitepress/config.mts +++ b/docs/.vitepress/config.mts @@ -43,6 +43,17 @@ export default defineConfig({ appearance: 'dark', + // Apply the saved layout-width preference (nav-bar toggle, see + // theme/LayoutWidthToggle.vue) before first paint so the page doesn't + // flash at the wrong width. Full-width is the default. + head: [ + [ + 'script', + {}, + `(function () { try { if (localStorage.getItem('sm-docs-layout') !== 'centered') document.documentElement.classList.add('sm-fluid') } catch (e) { document.documentElement.classList.add('sm-fluid') } })()` + ] + ], + // Requires full git history at build time (fetch-depth: 0 in the deploy job). lastUpdated: true, @@ -74,7 +85,8 @@ export default defineConfig({ { text: 'Inference', link: '/stability-matrix/inference/overview' }, { text: 'Advanced', link: '/stability-matrix/advanced/overview' }, { text: 'Tips and Tricks', link: '/stability-matrix/tips/overview' }, - { text: 'Troubleshooting', link: '/stability-matrix/troubleshooting/common-issues' } + { text: 'Troubleshooting', link: '/stability-matrix/troubleshooting/common-issues' }, + { text: 'Release Notes', link: '/stability-matrix/release-notes/2.16.2' } ], sidebar: { @@ -134,6 +146,14 @@ export default defineConfig({ { text: 'Common Issues', link: '/stability-matrix/troubleshooting/common-issues' } ] } + ], + '/stability-matrix/release-notes/': [ + { + text: 'Release Notes', + items: [ + { text: 'v2.16.2', link: '/stability-matrix/release-notes/2.16.2' } + ] + } ] }, diff --git a/docs/.vitepress/theme/LayoutWidthToggle.vue b/docs/.vitepress/theme/LayoutWidthToggle.vue new file mode 100644 index 000000000..e18a5fa53 --- /dev/null +++ b/docs/.vitepress/theme/LayoutWidthToggle.vue @@ -0,0 +1,101 @@ + + + + + diff --git a/docs/.vitepress/theme/custom.css b/docs/.vitepress/theme/custom.css new file mode 100644 index 000000000..801e2f8db --- /dev/null +++ b/docs/.vitepress/theme/custom.css @@ -0,0 +1,29 @@ +/* + * Fluid layout, toggleable from the nav bar (LayoutWidthToggle.vue). + * + * The default theme fixes the page shell at 1440px and the prose column at + * ~688px, which leaves most of a large monitor empty and forces wide tables + * (e.g. the environment-variables reference) into cramped horizontal + * scrolling. When carries the `sm-fluid` class (the default; users + * can opt out via the toggle, persisted in localStorage as + * `sm-docs-layout`), the layout tracks the window width instead. + * + * --vp-layout-max-width drives the nav bar, sidebar positioning, and doc + * container. The per-column overrides below lift the default theme's inner + * caps; !important is required because those caps live in Vue scoped styles + * (`.content-container[data-v-…]`), which outrank any plain selector here. + */ +html.sm-fluid { + --vp-layout-max-width: 100%; +} + +/* Pages with the right-hand "On this page" aside (most doc pages). */ +html.sm-fluid .VPDoc.has-aside .content-container { + max-width: none !important; +} + +/* Pages without a sidebar/aside (e.g. plain content pages). */ +html.sm-fluid .VPDoc:not(.has-sidebar) .container, +html.sm-fluid .VPDoc:not(.has-sidebar) .content { + max-width: none !important; +} diff --git a/docs/.vitepress/theme/index.ts b/docs/.vitepress/theme/index.ts new file mode 100644 index 000000000..356c508cd --- /dev/null +++ b/docs/.vitepress/theme/index.ts @@ -0,0 +1,12 @@ +import { h } from 'vue' +import DefaultTheme from 'vitepress/theme' +import LayoutWidthToggle from './LayoutWidthToggle.vue' +import './custom.css' + +export default { + extends: DefaultTheme, + Layout: () => + h(DefaultTheme.Layout, null, { + 'nav-bar-content-after': () => h(LayoutWidthToggle) + }) +} diff --git a/docs/release-notes/2.16.2.md b/docs/release-notes/2.16.2.md new file mode 100644 index 000000000..d4ecfdaa6 --- /dev/null +++ b/docs/release-notes/2.16.2.md @@ -0,0 +1,76 @@ +# v2.16.2 Technical Notes + +Extended technical notes for the v2.16.2 release — the deeper "how and why" behind each change. For the user-facing summary, see the [CHANGELOG](https://github.com/LykosAI/StabilityMatrix/blob/main/CHANGELOG.md). + +[`Home`](../README.md) + +## Table of Contents + +- [Mask Editor and Annotation Editor Stability](#mask-editor-and-annotation-editor-stability) +- [CivitAI File Types and Install Detection](#civitai-file-types-and-install-detection) +- [GGUF Text Encoders](#gguf-text-encoders) +- [Chroma and Single-Encoder CLIP Types](#chroma-and-single-encoder-clip-types) +- [Required-Extension Checks](#required-extension-checks) +- [Package Manager Fixes](#package-manager-fixes) +- [Linux AppImage](#linux-appimage) +- [Windows ROCm](#windows-rocm) + +## Mask Editor and Annotation Editor Stability + +**Random crashes while drawing or saving.** Canvas rendering runs on a separate render thread, while undo/redo, layer operations, exporting, and closing the editor could free the graphics resources a frame was still drawing with — occasionally crashing the app mid-stroke or while saving. The canvas threading model was redesigned so this can't happen structurally: the render thread now exclusively owns the on-screen graphics resources, exports composite from immutable snapshots on their own surfaces, in-progress strokes hand the renderer stable point snapshots, and closing the editor waits for the in-flight frame before freeing anything. + +**Pen pressure.** Pressing harder mid-stroke re-widened the entire stroke while drawing (the width was a running average recomputed every frame), and after saving and reopening a project, mouse-drawn strokes came back roughly 25% thicker as full-pressure pen strokes. Pressure now stays per-segment while drawing, and mouse strokes keep their original width across save/load. Existing project files load exactly as before. + +**Graphics memory leaks.** Neither editor released its paint canvas on close, and paint-bucket fill results were never freed at all — each fill pinned a full-canvas bitmap until app exit. + +**Performance.** Dragging an image layer with the Move tool now re-renders only the dragged layer instead of re-compositing every layer on each pointer move. Color-mask extraction for regional prompting was rewritten — a 1024×1024 canvas with six regions previously did over six million dictionary lookups per extraction. A small native leak (one path object per frame while drawing with the mouse) was fixed, and a redundant full-canvas scan after every paint-bucket fill was removed. + +## CivitAI File Types and Install Detection + +**Empty Files sections.** CivitAI expanded its file type list; files with the newer labels (Diffusion Model, UNet, Text Encoder, and others) deserialized to an internal `Unknown` type, and the browser filtered file lists by `Type == Model` — so official releases like Krea 2 Turbo and Z-Image showed an empty Files pane with no download links. All current CivitAI file types are now recognized, and the canonical type list is pinned by a test so a future addition fails the build loudly instead of silently hiding files. + +**"Installed" detection for Unknown-typed files** ([#1679](https://github.com/LykosAI/StabilityMatrix/issues/1679)). The install-state checks gated on the file type being recognized model weights before ever looking at the hash, so Unknown-typed files never showed the Installed label or version checkmark despite being downloaded and indexed. Detection is now purely blake3-hash-based: a file whose hash is in the local model index is on disk, regardless of what type CivitAI reports. Version delete uses the same rule, so Unknown-typed files are cleaned up too. + +**Download destinations.** Downloads now route by the file's declared type first (Diffusion Model/UNet → DiffusionModels, Text Encoder → TextEncoders, CLIP Vision → ClipVision, ControlNet → ControlNet, Upscaler → upscalers), with the older name/base-model heuristics kept only for files still typed plain "Model". External `stabilitymatrix://` download links use the same logic as the in-app browser. + +## GGUF Text Encoders + +**Loading** — the Text Encoder dropdowns listed `.gguf` files, but every CLIP call site wired them into the standard `CLIPLoader`/`DualCLIPLoader` nodes, which ComfyUI rejects. Typed nodes for the ComfyUI-GGUF loaders (`CLIPLoaderGGUF`, `DualCLIPLoaderGGUF`, `TripleCLIPLoaderGGUF`, `QuadrupleCLIPLoaderGGUF`) were added — verified against the extension's source: the single and dual variants take a `type` input, the triple and quadruple do not. Any selection containing a `.gguf` encoder routes through the GGUF loader; since those loaders also read `.safetensors`, mixed GGUF + safetensors selections work. This covers all Inference workflows (single through quadruple encoder setups, Wan, custom UNet) and the Image Lab providers. + +**Dropdown visibility while connected** — the connected-mode text encoder list was replaced wholesale by the names ComfyUI reports, and core ComfyUI never lists `.gguf` files in the text encoder folder (only the GGUF extension's loader nodes do). So a GGUF encoder in the shared TextEncoders folder vanished from the dropdown the moment ComfyUI connected. The list is now built like the checkpoint list: local index entries first (they carry richer metadata), then server-reported names not already covered. + +**HuggingFace browser destinations** — the folder guesser had a blanket ".gguf → DiffusionModels" rule, so text encoder GGUFs were suggested into the UNet folder. It now recognizes more encoder name patterns (`byt5`/`mt5`/`llama`/`gemma`/`qwen2` prefixes, `text-encoder`/`enconly` names) before that fallback. + +## Chroma and Single-Encoder CLIP Types + +ComfyUI's single `CLIPLoader` and `DualCLIPLoader` accept different type lists — notably, `flux` is dual-only (it loads a clip_l + t5xxl pair), while `chroma` is the single-encoder type for Chroma's lone t5xxl. The Inference encoder Type dropdown was missing `chroma` and several other single-encoder types (`wan`, `qwen_image`, `omnigen2`, `mochi`, `pixart`, `cosmos`, `stable_cascade`, `stable_audio`), so the closest available option demanded a second encoder these models don't use. The missing types are now listed, and the default encoder slot count comes from one shared mapping: dual-loader types get a pair, `sd3`/`hidream` get three/four, and every other type gets a single slot. + +## Required-Extension Checks + +**FaceDetailer duplicate-key error** ([#1681](https://github.com/LykosAI/StabilityMatrix/issues/1681)). The pre-generation extension check builds a dictionary of installed ComfyUI extensions keyed by git remote URL. Two `custom_nodes` folders reporting the same remote — for example a stray ComfyUI clone inside `custom_nodes` — made that dictionary construction throw "An item with the same key has already been added" before generation started. FaceDetailer merely activates this code path via its required-extension declarations; the fix de-duplicates by URL (case-insensitively, since git hosting URLs are case-insensitive). + +**Image Lab** ([#1672](https://github.com/LykosAI/StabilityMatrix/issues/1672)). Image Lab queued workflows without checking their declared required extensions, so a GGUF model without ComfyUI-GGUF installed surfaced as a generic "ComfyUI rejected the workflow" 400. It now performs the same pre-queue check as Inference and offers the same one-click install-and-restart dialog; the flow is shared between both features. + +## Package Manager Fixes + +**Package import misclassification** ([#1667](https://github.com/LykosAI/StabilityMatrix/issues/1667)). The import dialog performed no package-type detection — it silently pre-selected the first entry in the difficulty-sorted list, a Forge variant whose launch command is `launch.py`. Re-importing packages after settings loss could therefore record everything (including ComfyUI) as Forge. The dialog now reads the folder's `.git` remote origin URL and matches it against known packages, with the checked-out branch as a tie-break for packages sharing one repository (Forge Classic vs Neo). + +**reForge CUDA-on-AMD** ([#1669](https://github.com/LykosAI/StabilityMatrix/issues/1669)). reForge pins `torch==2.9.0` to match upstream, but the `rocm7.2` wheel index hosts no torch 2.9.x at all — and because the index is passed via `--extra-index-url`, pip kept PyPI in play and quietly resolved the CUDA wheel instead. reForge now installs from `rocm6.4`, which hosts `2.9.0+rocm6.4` (verified against the index); Forge itself keeps `rocm7.2`. + +**Output Browser crash** ([#1668](https://github.com/LykosAI/StabilityMatrix/issues/1668)). A broken junction or symlink under an output folder — a reparse point whose target no longer exists — threw `DirectoryNotFoundException` mid-enumeration while building the folder tree, which is why manually re-creating the missing folder didn't help. Inaccessible entries are now skipped with a logged warning. + +**Interpreter probe hardening.** The `sitecustomize.py` shipped into package venvs now routes its error messages to stderr, so interpreter probes and `pip list` output parsing can't be corrupted by them (follow-up to the [#1620](https://github.com/LykosAI/StabilityMatrix/issues/1620)/[#1643](https://github.com/LykosAI/StabilityMatrix/issues/1643) startup hardening; the underlying trigger is security software such as ESET interfering with the embedded Python). + +## Linux AppImage + +**FUSE3 runtime** ([#1682](https://github.com/LykosAI/StabilityMatrix/issues/1682)). The old build pipeline embedded a FUSE2-based AppImage runtime requiring `libfuse.so.2`, which modern distros (Ubuntu 24.04+, Fedora 40+) no longer ship. The build was upgraded to PupNet 1.9.1 with the modern type2-runtime, which is statically linked against libfuse3, accepts either `fusermount` or `fusermount3` at runtime, and falls back to `--appimage-extract-and-run` when FUSE isn't available at all. Most desktop users need nothing; minimal systems may need `sudo apt install fuse3` or equivalent. Contributed by [@NeuralFault](https://github.com/NeuralFault). + +**Desktop entry** ([#1666](https://github.com/LykosAI/StabilityMatrix/issues/1666)). AppImage runs previously wrote a `.desktop` entry with `NoDisplay=true` and no icon, so Stability Matrix never appeared in the application launcher, and the file was rewritten (reverting manual edits) on every exit. Runs now install a correct entry with the extracted app icon and report a matching `WM_CLASS`, so the running window shows the proper icon in the dock/taskbar. Only applies to AppImage runs; deb/rpm/flatpak installs keep their package-managed entries. + +**Deep links.** `stabilitymatrix://` URIs (for example CivitAI's "Download with Stability Matrix" buttons) were ignored on Linux: Windows registers the scheme handler with a `--uri` argument, but the Linux `.desktop` handler passes the URI as a bare positional argument (`%u`). Both forms are now accepted and forwarded to the running instance. + +## Windows ROCm + +**bitsandbytes wheel.** The Windows ROCm helper's bundled bitsandbytes wheel was updated to a build compatible with ROCm 7.13–7.15, keeping pace with AMD's ROCm Technical Preview builds. Contributed by [@NeuralFault](https://github.com/NeuralFault). + +**OneTrainer.** OneTrainer joined the Windows ROCm helper: new installs on supported AMD hardware get the ROCm Technical Preview PyTorch build via the shared install path, a ROCm-aware bitsandbytes wheel plus triton-windows, and the helper's hardware-appropriate launch environment applied automatically. Upstream OneTrainer also renamed its UI script (`train_ui.py` → `train_ui_ctk.py`) — the launch command was updated to match, and new installs default to Python 3.12 on all platforms. Contributed by [@NeuralFault](https://github.com/NeuralFault).