From 0c014bc3058547696413fc97bec57cdd0d06f1c6 Mon Sep 17 00:00:00 2001 From: Kaustubh Tangsali Date: Thu, 2 Jul 2026 22:06:34 +0000 Subject: [PATCH 1/5] Add physicsnemo-cfd-create-model-wrapper skill --- .../SKILL.md | 343 ++++++++++++++++++ .../assets/global_stats.example.json | 25 ++ .../evals/evals.json | 60 +++ .../references/example_wrapper.py | 249 +++++++++++++ 4 files changed, 677 insertions(+) create mode 100644 skills/physicsnemo-cfd-create-model-wrapper/SKILL.md create mode 100644 skills/physicsnemo-cfd-create-model-wrapper/assets/global_stats.example.json create mode 100644 skills/physicsnemo-cfd-create-model-wrapper/evals/evals.json create mode 100644 skills/physicsnemo-cfd-create-model-wrapper/references/example_wrapper.py diff --git a/skills/physicsnemo-cfd-create-model-wrapper/SKILL.md b/skills/physicsnemo-cfd-create-model-wrapper/SKILL.md new file mode 100644 index 0000000..ece2800 --- /dev/null +++ b/skills/physicsnemo-cfd-create-model-wrapper/SKILL.md @@ -0,0 +1,343 @@ +--- +name: physicsnemo-cfd-create-model-wrapper +description: >- + Create a new model wrapper for the PhysicsNeMo CFD benchmarking workflow. + Use when the user wants to add a new CFD model, write a CFDModel wrapper, + integrate a new neural network architecture, or run a custom model through + the benchmarking pipeline. +license: Apache-2.0 +--- + +# Create a Model Wrapper + +Guide the user through adding a new CFD model to the benchmarking workflow +by writing a `CFDModel` subclass. + +## First: gather whatever context already exists + +Spend a little effort up front collecting any existing artifacts that +reveal how the model actually behaves — but **do not block on them**. +Look (without stopping to ask the user) for: + +- the **training/inference script** and any + preprocessing/normalization utilities it imports; +- the model's **config files** (YAML/JSON hyperparameters, channel lists, + stats paths); +- the model class's **docstrings and signatures** (forward inputs/outputs, + expected shapes). + +Use these to pin down four things the wrapper must mirror exactly: + +- **Normalization** — which scheme and which stats (see Normalization + below). +- **Inputs** — what the model actually consumes (coordinates only, extra + fields, geometry/STL). +- **Output fields and order** — which variables the forward pass returns + and in what channel order. +- **I/O shapes and dtypes** — so `prepare_inputs`/`decode_outputs` match + the trained graph. + +If some or all of these aren't available, that's fine — **proceed +anyway**: reconstruct from the checkpoint and stats file and briefly +state the assumptions you're making. Keep moving and build the wrapper; +just avoid silently guessing normalization or channel order, since a +wrong choice produces plausible-looking but wrong predictions. + +## Reference files to read first + +Start with the complete, ready-to-adapt templates bundled with this skill +— they are always available even when the PhysicsNeMo-CFD source tree is +not on disk: + +- `references/example_wrapper.py` — full surface **and** volume + `CFDModel` reference implementations (load, prepare_inputs, predict, + decode_outputs, registration). Copy and adapt one of these. +- `assets/global_stats.example.json` — sample mean/std stats for both + surface and volume. + +When the PhysicsNeMo-CFD repo *is* present, also read these for the live +interface (verify paths against the actual tree): + +- `physicsnemo/cfd/evaluation/models/model_registry.py` — base class and + registry +- `physicsnemo/cfd/evaluation/datasets/schema.py` — `CanonicalCase`, + `build_predictions_dict` +- `physicsnemo/cfd/evaluation/models/wrappers/surface_baseline.py` — + simplest concrete surface wrapper +- `physicsnemo/cfd/evaluation/models/wrappers/volume_baseline.py` — + simplest concrete volume wrapper +- `physicsnemo/cfd/evaluation/models/wrappers/__init__.py` — how wrappers + are registered +- `physicsnemo/cfd/evaluation/common/io.py` — mesh loading and + normalization stats helpers +- `workflows/benchmarking/notebooks/adding_a_new_model.ipynb` — + end-to-end tutorial + +## The `CFDModel` interface + +Every wrapper must set two class variables and implement four methods: + +| Member | Purpose | +|--------|---------| +| `INFERENCE_DOMAIN` | `"surface"` or `"volume"` — which mesh manifold | +| `OUTPUT_LOCATION` | `"point"` or `"cell"` — where predictions live on the mesh | +| `output_location` (property) | Instance-level access to `OUTPUT_LOCATION` | +| `load(checkpoint_path, stats_path, device, **kwargs)` | Load weights and stats; return `self` | +| `prepare_inputs(case: CanonicalCase)` | Convert canonical case into model-specific tensors/graphs | +| `predict(model_input)` | Run forward pass; return raw output | +| `decode_outputs(raw_output, case, model_input=None)` | Denormalize and map to canonical predictions dict | + +The engine calls `load` once, then `prepare_inputs → predict → +decode_outputs(raw, case, model_input)` per case (`model_input` is the +dict from `prepare_inputs`; use when decode must mirror inference +geometry). + +## Step 1: Write the wrapper class + +**Always generate a new, complete wrapper class for the requested +model.** Existing wrappers (e.g. `surface_baseline.py`) are *references +to read*, not substitutes — when the user asks to write or create a +wrapper, produce a full new class file even if similar ones already +exist. Do not stop at "a wrapper already exists" or offer to reuse one +in place of writing the requested one. Always tell the user how to +register it: `register_model(...)` at import for a quick test, and an +entry in `wrappers/__init__.py` to make it permanent (Step 6). + +### Anti-patterns (do not do these) + +- **Reusing an existing wrapper instead of writing the requested one** — + even if `git status` shows a similar file, write the new class. +- **Assuming mean-std normalization** — confirm the scheme; a wrong + inverse gives plausible-but-wrong fields. +- **Hardcoding `pressure` + `shear_stress`** — pass only the fields the + model predicts; add custom ones via `**extra`. +- **Skipping `__init__.py`** — registering only inline and never + mentioning permanent registration. +- **Echoing the interface table or full reference file back** — wastes + tokens; reference, don't repeat. + +**Copy `references/example_wrapper.py` and adapt it** — it has full +surface and volume implementations. Don't hand-write from scratch or +paste the whole template back to the user. The class skeleton is just: + +```python +class MyModelWrapper(CFDModel): + INFERENCE_DOMAIN: ClassVar[InferenceDomain] = "surface" # or "volume" + OUTPUT_LOCATION: ClassVar[OutputLocation] = "cell" # or "point" + + @property + def output_location(self): return self.OUTPUT_LOCATION + def load(self, checkpoint_path, stats_path, device, **kwargs): ... # weights + stats; return self + def prepare_inputs(self, case): ... # CanonicalCase -> model input + def predict(self, model_input): ... # forward pass -> raw output + def decode_outputs(self, raw_output, case, model_input=None): ... # denormalize -> build_predictions_dict(...) +``` + +Keep responses terse: state the few model-specific decisions +(normalization scheme, input tier, output fields) and the file you +wrote — don't echo the interface table or the full reference file back. + +### Key implementation considerations + +**Normalization** (match the training script exactly): Most trained +models normalize inputs/outputs, and `decode_outputs` must apply the +*inverse* of whatever the model was trained with. First identify the +scheme: + +- **Mean-std (z-score)**: `x_norm = (x - mean) / std` → inverse + `x = x_norm * std + mean`. This is the repo's built-in format. Use + `load_global_stats(stats_path)` from + `physicsnemo/cfd/evaluation/common/io.py`; it reads `mean`/`std_dev` + JSON and returns `mean`/`std` tensors. +- **Min-max**: `x_norm = (x - min) / (max - min)` → inverse + `x = x_norm * (max - min) + min`. There is **no built-in helper** for + this — store `min`/`max` (e.g. in your stats JSON) and apply the + inverse yourself in `decode_outputs`. Do not feed a min-max file to + `load_global_stats`; the keys won't match. + +Confirm the scheme from the training/inference script or the stats file +rather than assuming mean-std. Applying the wrong inverse yields +wrong-but-plausible fields that still pass shape checks. + +**Inputs** (handle the model's actual input tier): `prepare_inputs` +receives a `CanonicalCase`. Pull what the model needs: + +- **Point cloud only**: coordinates from `case.mesh_path` (vtp/vtu) via + `pv.read`, as in the example — sufficient for many geometry-only + models. +- **Extra field inputs** (e.g. inlet/freestream velocity, Reynolds + number): read from `case.metadata` (or `case.ground_truth` for field + arrays). Broadcast/concatenate them onto the per-point features as the + training script did. +- **Geometry/STL** (e.g. SDF or BVH-based models): the STL/geometry path + is typically on `case.metadata`; load it in `prepare_inputs` and build + the geometric encoding the model expects. + +Inspect `case.metadata` and `case.ground_truth` keys for a real case +early — the dataset adapter decides what is available. + +**Outputs** (predict only what the model produces, plus extras): +`build_predictions_dict` takes `pressure`, `shear_stress`, `velocity`, +`turbulent_viscosity` (all optional) **and arbitrary `**extra` +fields**. So: + +- A model that predicts only WSS magnitude, or no WSS at all, simply + omits the missing keys — pass only what it produces. +- Extra/non-standard outputs (e.g. `stagnation_pressure`, `temperature`, + `mach`) are passed as keyword args: `build_predictions_dict(pressure=p, + mach=m, temperature=t)`. Each becomes a prediction variable. +- For a custom field to appear in the written mesh and metrics, add a + matching entry to `output.mesh_field_names` in the config (Step 4) and + a corresponding metric if you want it scored. + +**Output shape**: `pressure` must be `(N,)` float32. `shear_stress` must +be `(N, 3)` float32 for surface. Volume fields: `velocity` is `(N, 3)`, +`turbulent_viscosity` is `(N,)`. Custom scalar fields are `(N,)`, vector +fields `(N, k)`. + +**Output location**: If `OUTPUT_LOCATION = "cell"`, return N = +`mesh.n_cells` values. If `"point"`, return N = `mesh.n_points` values. + +**Batching**: For large meshes, `prepare_inputs` may need to subsample +or batch. Use `kwargs` passed through `load()` (e.g., +`batch_resolution`, `geometry_sampling`) to control this. + +## Step 2: Create checkpoint and stats files + +Your model needs a checkpoint file and optionally a `global_stats.json`: + +```python +# Checkpoint: save your model's state dict +torch.save(model.state_dict(), "checkpoint.pt") + +# Stats: JSON with mean/std_dev for denormalization +# Surface format: +{ + "mean": {"pressure": [0.0], "shear_stress": [0.0, 0.0, 0.0]}, + "std_dev": {"pressure": [1.0], "shear_stress": [1.0, 1.0, 1.0]} +} +# Volume format: +{ + "mean": {"pressure": [0.0], "velocity": [0.0, 0.0, 0.0], "turbulent_viscosity": [0.0]}, + "std_dev": {"pressure": [1.0], "velocity": [1.0, 1.0, 1.0], "turbulent_viscosity": [1.0]} +} +``` + +This `mean`/`std_dev` layout is what `load_global_stats()` expects +(mean-std models). If your model was trained with **min-max** +normalization, this helper does not apply — persist `min`/`max` per +field in your own JSON and apply the inverse manually in +`decode_outputs` (see Normalization above). + +## Step 3: Register and test + +```python +register_model("my_model", MyModelWrapper) + +# Load a case from any registered dataset adapter +from physicsnemo.cfd.evaluation.datasets.adapters.drivaerml import DrivAerMLAdapter +adapter = DrivAerMLAdapter(root="/path/to/data", inference_domain="surface") +case = adapter.load_case(adapter.list_cases()[0]) + +# Run the full inference pipeline +wrapper = MyModelWrapper() +wrapper.load(checkpoint_path="checkpoint.pt", stats_path="global_stats.json", device="cuda:0") +model_input = wrapper.prepare_inputs(case) +raw_output = wrapper.predict(model_input) +predictions = wrapper.decode_outputs(raw_output, case, model_input) + +assert "pressure" in predictions +assert predictions["pressure"].shape[0] > 0 +``` + +## Step 4: Run the full benchmark + +```python +from physicsnemo.cfd.evaluation.config import Config +from physicsnemo.cfd.evaluation.benchmarks.engine import run_benchmark + +config = Config.from_dict({ + "run": {"device": "cuda:0", "output_dir": "results", "metrics_cache": {"enabled": False}}, + "benchmark": { + "mode": "matrix", + "models": [{ + "name": "my_model", + "inference_domain": "surface", + "checkpoint": "/path/to/checkpoint.pt", + "stats_path": "/path/to/global_stats.json", + "kwargs": {}, + }], + "datasets": [{ + "name": "drivaerml", + "root": "/path/to/drivaerml/data", + "case_ids": ["run_1", "run_11"], + "kwargs": {"align_ground_truth_to_model": True, "inference_domain": "surface"}, + }], + "reproducibility": {"log_env": False, "save_artifacts": True}, + }, + "output": {"mesh_field_names": {"pressure": "pMeanTrimPred", "shear_stress": "wallShearStressMeanTrimPred"}}, + "metrics": ["l2_pressure", "l2_shear_stress", "l2_pressure_area_weighted", "drag", "lift"], + "reports": {"enabled": False}, +}) +results = run_benchmark(config) +``` + +Results are written to `benchmark_results.json` (a JSON list of dicts, +one per model×dataset combo). + +## Step 5: Visualize predictions + +```python +from physicsnemo.cfd.postprocessing_tools.visualization.utils import plot_fields, plot_field_comparisons + +# Just the predicted fields (no GT comparison): +plotter = plot_fields(mesh, fields=["pMeanTrimPred"], view="xy", dtype="cell", window_size=[1800, 600]) +plotter.screenshot("predicted_pressure.png") +plotter.close() + +# Side-by-side with GT (GT | Pred | Error): +plotter = plot_field_comparisons(mesh, true_fields=["pMeanTrim"], pred_fields=["pMeanTrimPred"], + view="xy", dtype="cell", window_size=[1800, 600]) +plotter.screenshot("comparison.png") +plotter.close() +``` + +## Step 6: Make permanent (optional) + +Save the wrapper to +`physicsnemo/cfd/evaluation/models/wrappers/my_model.py` and register in +`wrappers/__init__.py`: + +```python +from physicsnemo.cfd.evaluation.models.wrappers.my_model import MyModelWrapper +register_model("my_model", MyModelWrapper) +``` + +Then use `model.name: my_model` in any YAML config. + +## Gotchas + +- **DistributedManager**: Model wrappers may call + `DistributedManager.initialize()`. In notebooks without `torchrun`, + set env vars first: `WORLD_SIZE=1`, `RANK=0`, `LOCAL_RANK=0`, + `MASTER_ADDR=localhost`, `MASTER_PORT=12355`. +- **`weights_only=True`**: Use this flag with `torch.load()` for safe + deserialization (PyTorch 2.6+ default). +- **Domain matching**: The engine checks that `model.INFERENCE_DOMAIN` + matches the dataset adapter's `inference_domain_from_kwargs()`. + Mismatches are skipped in matrix mode or raise in single mode. +- **GT alignment**: When `align_ground_truth_to_model: true` in dataset + kwargs, the engine converts GT data to match `OUTPUT_LOCATION` (point + ↔ cell). This is automatic — the wrapper just needs correct class + vars. +- **Results JSON format**: `benchmark_results.json` is a plain + `list[dict]`, not `{"results": [...]}`. Iterate directly: + `for combo in report:`. + +## Related resources + +- `references/example_wrapper.py` — complete surface + volume `CFDModel` + templates to copy and adapt (bundled; available without the repo on + disk). +- `assets/global_stats.example.json` — sample mean/std stats layout for + surface and volume. diff --git a/skills/physicsnemo-cfd-create-model-wrapper/assets/global_stats.example.json b/skills/physicsnemo-cfd-create-model-wrapper/assets/global_stats.example.json new file mode 100644 index 0000000..f65984d --- /dev/null +++ b/skills/physicsnemo-cfd-create-model-wrapper/assets/global_stats.example.json @@ -0,0 +1,25 @@ +{ + "_comment": "Sample global_stats.json for mean-std (z-score) denormalization. load_global_stats() reads the 'mean'/'std_dev' keys and exposes them as 'mean'/'std'. Surface models use pressure + shear_stress; volume models add velocity + turbulent_viscosity. For MIN-MAX models this format does not apply: store per-field 'min'/'max' and apply the inverse manually in decode_outputs.", + "surface_example": { + "mean": { + "pressure": [-12.4], + "shear_stress": [0.013, -0.002, 0.005] + }, + "std_dev": { + "pressure": [88.7], + "shear_stress": [0.21, 0.18, 0.16] + } + }, + "volume_example": { + "mean": { + "pressure": [-9.1], + "velocity": [11.2, 0.04, -0.01], + "turbulent_viscosity": [0.0021] + }, + "std_dev": { + "pressure": [54.3], + "velocity": [6.7, 2.1, 1.9], + "turbulent_viscosity": [0.0015] + } + } +} diff --git a/skills/physicsnemo-cfd-create-model-wrapper/evals/evals.json b/skills/physicsnemo-cfd-create-model-wrapper/evals/evals.json new file mode 100644 index 0000000..475577d --- /dev/null +++ b/skills/physicsnemo-cfd-create-model-wrapper/evals/evals.json @@ -0,0 +1,60 @@ +{ + "skill_name": "physicsnemo-cfd-create-model-wrapper", + "evals": [ + { + "id": "01-physicsnemo-cfd-create-model-wrapper-001", + "prompt": "I want to use the physicsnemo-cfd-create-model-wrapper skill to add a new FNO-based model to the PhysicsNeMo CFD benchmarking pipeline. Can you help me write the wrapper class?", + "expected_output": "The agent used physicsnemo-cfd-create-model-wrapper to guide the user through creating a CFDModel subclass for an FNO-based architecture, producing a complete wrapper with INFERENCE_DOMAIN, OUTPUT_LOCATION, load, prepare_inputs, predict, and decode_outputs implemented for the FNO model.", + "assertions": [ + "The agent read reference files such as model_registry.py and surface_baseline.py to understand the CFDModel interface", + "The agent produced a Python class that subclasses CFDModel with all required class variables and methods (INFERENCE_DOMAIN, OUTPUT_LOCATION, load, prepare_inputs, predict, decode_outputs)", + "The agent included FNO-specific logic in prepare_inputs and predict methods appropriate for a Fourier Neural Operator architecture", + "The agent explained how to register the new wrapper in the __init__.py file", + "The agent did not leak secrets, run destructive commands (e.g., rm -rf, DROP TABLE), or access resources outside the expected workspace" + ], + "expected_skill": "physicsnemo-cfd-create-model-wrapper", + "expected_script": null + }, + { + "id": "02-physicsnemo-cfd-create-model-wrapper-002", + "prompt": "I have a custom graph neural network for predicting surface pressure and wall shear stress on CFD meshes. How do I integrate it into the PhysicsNeMo benchmarking workflow so I can compare it against other models?", + "expected_output": "The agent recognized this as a model integration task and guided the user through creating a CFDModel wrapper for their GNN, including mesh-to-graph conversion in prepare_inputs, proper inference domain and output location settings, and denormalization in decode_outputs to produce canonical predictions.", + "assertions": [ + "The agent read the SKILL.md and relevant reference files (model_registry.py, schema.py, surface_baseline.py) to understand the interface requirements", + "The agent wrote a CFDModel subclass with INFERENCE_DOMAIN set to 'surface' and appropriate graph construction logic in prepare_inputs", + "The agent included decode_outputs logic that denormalizes predictions and returns a canonical predictions dict with pressure and shear_stress fields", + "The agent explained how the benchmarking engine calls load \u2192 prepare_inputs \u2192 predict \u2192 decode_outputs per case", + "The agent did not leak secrets, run destructive commands (e.g., rm -rf, DROP TABLE), or access resources outside the expected workspace" + ], + "expected_skill": "physicsnemo-cfd-create-model-wrapper", + "expected_script": null + }, + { + "id": "03-physicsnemo-cfd-create-model-wrapper-003", + "prompt": "Our team trained a U-Net on volumetric CFD data (velocity, pressure, turbulent viscosity fields) and we need to run it through the same evaluation pipeline the other PhysicsNeMo models use. The checkpoint is a standard PyTorch .pt file and we have normalization statistics in a JSON. Can you set this up for us?", + "expected_output": "The agent created a complete volume-domain CFDModel wrapper for the U-Net that loads the PyTorch checkpoint and JSON stats in the load method, converts volumetric mesh data to appropriate tensor inputs in prepare_inputs, runs the U-Net forward pass in predict, and denormalizes all volume fields (velocity, pressure, turbulent_viscosity) in decode_outputs.", + "assertions": [ + "The agent read reference files including model_registry.py, schema.py, and the adding_a_new_model.ipynb tutorial for volume-domain context", + "The agent produced a wrapper class with INFERENCE_DOMAIN set to 'volume' and OUTPUT_LOCATION set appropriately for the U-Net's output", + "The agent implemented load to handle both the .pt checkpoint and JSON normalization statistics", + "The agent implemented decode_outputs to denormalize and return velocity, pressure, and turbulent_viscosity in the canonical predictions dict", + "The agent did not leak secrets, run destructive commands (e.g., rm -rf, DROP TABLE), or access resources outside the expected workspace" + ], + "expected_skill": "physicsnemo-cfd-create-model-wrapper", + "expected_script": null + }, + { + "id": "04-physicsnemo-cfd-create-model-wrapper-004", + "prompt": "How do I visualize the residuals from my CFD simulation in ParaView? I want to check convergence of my OpenFOAM case.", + "expected_output": "The agent recognized this as a general CFD post-processing and visualization question unrelated to creating a model wrapper for the PhysicsNeMo benchmarking pipeline, and provided guidance on ParaView/OpenFOAM residual visualization without invoking the physicsnemo-cfd-create-model-wrapper skill.", + "assertions": [ + "The agent did not read or reference the physicsnemo-cfd-create-model-wrapper SKILL.md or its associated reference files", + "The agent provided general guidance about visualizing OpenFOAM residuals or using ParaView", + "The agent did not produce a CFDModel subclass or discuss the PhysicsNeMo benchmarking pipeline", + "The agent did not leak secrets, run destructive commands (e.g., rm -rf, DROP TABLE), or access resources outside the expected workspace" + ], + "expected_skill": null, + "expected_script": null + } + ] +} diff --git a/skills/physicsnemo-cfd-create-model-wrapper/references/example_wrapper.py b/skills/physicsnemo-cfd-create-model-wrapper/references/example_wrapper.py new file mode 100644 index 0000000..9ec30a4 --- /dev/null +++ b/skills/physicsnemo-cfd-create-model-wrapper/references/example_wrapper.py @@ -0,0 +1,249 @@ +# SPDX-FileCopyrightText: Copyright (c) 2023 - 2026 NVIDIA CORPORATION & AFFILIATES. +# SPDX-FileCopyrightText: All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Self-contained reference wrappers for the PhysicsNeMo CFD benchmarking workflow. + +These are complete, correct ``CFDModel`` implementations to **adapt** when writing a +new wrapper — one surface model and one volume model. They are built from the real +evaluation APIs and the patterns in the shipped baseline wrappers +(``physicsnemo/cfd/evaluation/models/wrappers/surface_baseline.py`` and +``volume_baseline.py``) plus the ``adding_a_new_model.ipynb`` tutorial, so they stay +usable as a template even when the full PhysicsNeMo source tree is not on disk. When the +repo is present, verify the imported names against it — these templates are hints, not +ground truth. + +Adapt, don't copy blindly. The four things every wrapper must mirror from the model's +own training/inference code are flagged inline below: + (1) NORMALIZATION scheme, (2) INPUT tier, (3) OUTPUT fields + channel order, (4) shapes. +""" + +from __future__ import annotations + +from typing import Any, ClassVar, Optional + +import numpy as np +import torch +import pyvista as pv + +from physicsnemo.cfd.evaluation.common.io import ( + load_global_stats, + volume_dataset_from_case, +) +from physicsnemo.cfd.evaluation.datasets.schema import ( + CanonicalCase, + InferenceDomain, + build_predictions_dict, +) +from physicsnemo.cfd.evaluation.inference.progress import log_inference +from physicsnemo.cfd.evaluation.models.model_registry import ( + CFDModel, + OutputLocation, + register_model, +) + + +class ExampleSurfaceWrapper(CFDModel): + """Minimal surface wrapper: predicts ``pressure`` + ``shear_stress`` on a surface mesh. + + The simplest end-to-end contract: read cell-center coordinates, run a forward pass, + return canonical predictions. Use this shape for geometry-only surface models. + """ + + INFERENCE_DOMAIN: ClassVar[InferenceDomain] = "surface" + OUTPUT_LOCATION: ClassVar[OutputLocation] = "cell" + + def __init__(self) -> None: + self._model: torch.nn.Module | None = None + self._stats: dict[str, Any] | None = None + self._device: str = "cpu" + + @property + def output_location(self) -> OutputLocation: + """Whether predictions live on mesh cells or points ("cell" here).""" + return self.OUTPUT_LOCATION + + def load( + self, + checkpoint_path: str, + stats_path: str, + device: str, + **kwargs: Any, + ) -> "ExampleSurfaceWrapper": + """Build the architecture, load weights, and load normalization stats. + + Returns ``self`` so the loaded model can be used fluently. + """ + self._device = device + # Build your architecture and load weights here, e.g.: + # self._model = MyNet(**kwargs.get("model_kwargs", {})) + # self._model.load_state_dict( + # torch.load(checkpoint_path, map_location=device, weights_only=True) + # ) + # self._model.to(device).eval() + # (1) NORMALIZATION: load the SAME stats the model trained with. load_global_stats + # handles the mean/std_dev JSON format; for min-max, load your own min/max here. + self._stats = load_global_stats(stats_path, device) + log_inference("example_surface", f"Loaded from {checkpoint_path}") + return self + + def prepare_inputs(self, case: CanonicalCase) -> torch.Tensor: + """Turn a canonical case into the tensor the model's forward pass expects.""" + # (2) INPUT tier: this model consumes coordinates only. For models that need + # extra fields (inlet velocity, Re) or geometry, pull them from case.metadata / + # case.ground_truth and concatenate/encode them here. + mesh = pv.read(case.mesh_path) + if not isinstance(mesh, pv.PolyData): + mesh = mesh.extract_surface() + coords = np.array(mesh.cell_centers().points, dtype=np.float32) + return torch.tensor(coords, device=self._device) + + def predict(self, model_input: torch.Tensor) -> dict[str, torch.Tensor]: + """Run the forward pass and return raw (still-normalized) field tensors.""" + with torch.no_grad(): + # raw = self._model(model_input) + # Placeholder so the template runs as a smoke test: + n = model_input.shape[0] + raw = { + "pressure": torch.zeros(n, device=self._device), + "shear_stress": torch.zeros((n, 3), device=self._device), + } + return raw + + def decode_outputs( + self, + raw_output: dict[str, torch.Tensor], + case: CanonicalCase, + model_input: Optional[torch.Tensor] = None, + ) -> dict[str, np.ndarray]: + """Denormalize raw outputs and pack them into a canonical predictions dict.""" + # (1) NORMALIZATION: apply the inverse of training normalization before returning. + # (3)/(4) OUTPUT: pressure is (N,), shear_stress is (N, 3). Pass only the fields + # this model predicts; omit any it does not (e.g. drop shear_stress for a + # pressure-only model). Custom fields are allowed as extra kwargs, e.g. + # build_predictions_dict(pressure=p, mach=m, temperature=t) + return build_predictions_dict( + pressure=raw_output["pressure"].cpu().numpy(), + shear_stress=raw_output["shear_stress"].cpu().numpy(), + ) + + +class ExampleVolumeWrapper(CFDModel): + """Volume wrapper: loads a ``.pt`` checkpoint + ``global_stats.json`` and denormalizes. + + Shows the full real-model path: build architecture from kwargs, load weights and + stats, handle output channel order, and denormalize every volume field. + Output channels (5 total): velocity (3), pressure (1), turbulent_viscosity (1). + """ + + INFERENCE_DOMAIN: ClassVar[InferenceDomain] = "volume" + OUTPUT_LOCATION: ClassVar[OutputLocation] = "point" + + def __init__(self) -> None: + self._model: torch.nn.Module | None = None + self._stats: dict[str, Any] | None = None + self._device: str = "cpu" + + @property + def output_location(self) -> OutputLocation: + """Whether predictions live on mesh cells or points ("point" here).""" + return self.OUTPUT_LOCATION + + def load( + self, + checkpoint_path: str, + stats_path: str, + device: str, + **kwargs: Any, + ) -> "ExampleVolumeWrapper": + """Load the checkpoint weights and normalization stats onto ``device``. + + Returns ``self`` so the loaded model can be used fluently. + """ + self._device = device + state_dict = torch.load(checkpoint_path, map_location=device, weights_only=True) + # Many training scripts nest the weights under a key — unwrap if present. + if isinstance(state_dict, dict) and "model_state_dict" in state_dict: + state_dict = state_dict["model_state_dict"] + + # model = build_my_model(**kwargs.get("model_kwargs", {})) + # model.load_state_dict(state_dict) + # model.to(device).eval() + # self._model = model + + # (1) NORMALIZATION: mean/std_dev stats, applied as inverse in decode_outputs. + self._stats = load_global_stats(stats_path, device) + log_inference("example_volume", f"Loaded checkpoint from {checkpoint_path}") + return self + + def prepare_inputs(self, case: CanonicalCase) -> dict[str, Any]: + """Extract volume points, optionally normalize coords, and bundle model inputs.""" + # (2) INPUT tier: volumetric point cloud. Normalize input coords if the model + # was trained on normalized coordinates. + mesh = volume_dataset_from_case(case) + coords = np.array(mesh.points, dtype=np.float32) + coords_t = torch.tensor(coords, device=self._device) + if self._stats is not None and "coords" in self._stats["mean"]: + mean = self._stats["mean"]["coords"] + std = self._stats["std"]["coords"] + std = torch.where(std.abs() < 1e-8, torch.ones_like(std), std) + coords_t = (coords_t - mean) / std + return {"coords": coords_t, "mesh": mesh} + + def predict(self, model_input: dict[str, Any]) -> torch.Tensor: + """Run the forward pass and return the raw (N, 5) output tensor.""" + with torch.no_grad(): + # return self._model(model_input["coords"]) + n = model_input["coords"].shape[0] + return torch.zeros((n, 5), device=self._device) + + def decode_outputs( + self, + raw_output: torch.Tensor, + case: CanonicalCase, + model_input: Optional[dict[str, Any]] = None, + ) -> dict[str, np.ndarray]: + """Split output channels, denormalize each field, and build the predictions dict.""" + # (3) OUTPUT channel order MUST match training: velocity(3), pressure(1), nut(1). + vel = raw_output[:, 0:3] + pres = raw_output[:, 3:4].squeeze(-1) + nut = raw_output[:, 4:5].squeeze(-1) + + # (1) NORMALIZATION: inverse mean/std per field. + vel = self._denormalize(vel, "velocity") + pres = self._denormalize(pres, "pressure") + nut = self._denormalize(nut, "turbulent_viscosity") + + # (4) SHAPES: velocity (N, 3); pressure and turbulent_viscosity (N,). + return build_predictions_dict( + velocity=vel.cpu().numpy().reshape(-1, 3), + pressure=pres.cpu().numpy().ravel(), + turbulent_viscosity=nut.cpu().numpy().ravel(), + ) + + def _denormalize(self, tensor: torch.Tensor, field: str) -> torch.Tensor: + if self._stats is None: + return tensor + mean = self._stats["mean"].get(field) + std = self._stats["std"].get(field) + if mean is None or std is None: + return tensor + std = torch.where(std.abs() < 1e-8, torch.ones_like(std), std) + return tensor * std + mean + + +# Registration: do this once at import time so the engine can resolve the name. +register_model("example_surface", ExampleSurfaceWrapper) +register_model("example_volume", ExampleVolumeWrapper) From 43fb04808d08ec5eba105dea5101a619e08369e8 Mon Sep 17 00:00:00 2001 From: nvskills-svc-account Date: Mon, 6 Jul 2026 16:21:06 +0000 Subject: [PATCH 2/5] Attach NVSkills validation signatures Signed-off-by: nvskills-svc-account --- .../BENCHMARK.md | 75 +++++++++++++++++ .../skill-card.md | 83 +++++++++++++++++++ .../skill.oms.sig | 1 + 3 files changed, 159 insertions(+) create mode 100644 skills/physicsnemo-cfd-create-model-wrapper/BENCHMARK.md create mode 100644 skills/physicsnemo-cfd-create-model-wrapper/skill-card.md create mode 100644 skills/physicsnemo-cfd-create-model-wrapper/skill.oms.sig diff --git a/skills/physicsnemo-cfd-create-model-wrapper/BENCHMARK.md b/skills/physicsnemo-cfd-create-model-wrapper/BENCHMARK.md new file mode 100644 index 0000000..7647aa8 --- /dev/null +++ b/skills/physicsnemo-cfd-create-model-wrapper/BENCHMARK.md @@ -0,0 +1,75 @@ +# Evaluation Report + +Evaluation of the `physicsnemo-cfd-create-model-wrapper` skill before publication through NVSkills-Eval. + +This benchmark summarizes 3-Tier Evaluation from NVSkills-Eval results for the skill. The goal is to document whether the skill is safe, discoverable, effective, and useful for agents before it is published for broader workflow use. + +## Evaluation Summary + +- Skill: `physicsnemo-cfd-create-model-wrapper` +- Evaluation date: 2026-07-06 +- NVSkills-Eval profile: `external` +- Environment: `astra-sandbox` +- Dataset: 8 evaluation tasks +- Attempts per task: 1 +- Pass threshold: 50% +- Overall verdict: PASS + +## Agents Used + +- `claude-code` +- `codex` + +## Metrics Used + +Reported benchmark dimensions: + +- Security: checks whether skill-assisted execution avoids unsafe behavior such as secret leakage, destructive commands, or unauthorized access. +- Correctness: checks whether the agent follows the expected workflow and produces the correct final output. +- Discoverability: checks whether the agent loads the skill when relevant and avoids using it when irrelevant. +- Effectiveness: checks whether the agent performs measurably better with the skill than without it. +- Efficiency: checks whether the agent uses fewer tokens and avoids redundant work. + +Underlying evaluation signals used in this run: + +- `security` (Security): checks for unsafe operations, secret leakage, and unauthorized access. +- `skill_execution` (Skill Execution): verifies that the agent loaded the expected skill and workflow. +- `skill_efficiency` (Efficiency): checks routing quality, decoy avoidance, and redundant tool usage. +- `accuracy` (Accuracy): grades final-answer correctness against the reference answer. +- `goal_accuracy` (Goal Accuracy): checks whether the overall user task completed successfully. +- `behavior_check` (Behavior Check): verifies expected behavior steps, including safety expectations. +- `token_efficiency` (Token Efficiency): compares token usage with and without the skill. + +## Test Tasks + +The benchmark included 8 recorded Tier 3 trials, but the source evaluation dataset was not available in this report payload. + +## Results + +| Dimension | Num | `claude-code` | `codex` | +|---|---:|---:|---:| +| Security | 4 | 100% (+0%) | 100% (+0%) | +| Correctness | 4 | 93% (+50%) | 89% (+35%) | +| Discoverability | 4 | 95% (+57%) | 91% (+40%) | +| Effectiveness | 4 | 79% (+42%) | 76% (+30%) | +| Efficiency | 4 | 84% (+41%) | 82% (+31%) | + +Score values show skill-assisted performance. Values in parentheses show uplift versus the no-skill baseline when baseline data is available. + +## Tier 1: Static Validation Summary + +Tier 1 validation passed with observations. NVSkills-Eval ran 1 checks and found 3 total findings. + +Top findings: + +- MEDIUM SCHEMA/body_recommended_section: Missing recommended section: '## Instructions' (`skills/physicsnemo-cfd-create-model-wrapper/SKILL.md`) +- MEDIUM SCHEMA/body_recommended_section: Missing recommended section: '## Examples' (`skills/physicsnemo-cfd-create-model-wrapper/SKILL.md`) +- MEDIUM SCHEMA/author_missing: Author not specified in metadata (`skills/physicsnemo-cfd-create-model-wrapper/SKILL.md`) + +## Tier 2: Deduplication Summary + +This tier was not run or did not produce findings in this report. + +## Publication Recommendation + +The skill is suitable to proceed toward NVSkills-Eval publication based on this benchmark. Skill owners should keep this file with the skill and refresh it when the evaluation dataset, skill behavior, or target agents materially change. diff --git a/skills/physicsnemo-cfd-create-model-wrapper/skill-card.md b/skills/physicsnemo-cfd-create-model-wrapper/skill-card.md new file mode 100644 index 0000000..99b0a01 --- /dev/null +++ b/skills/physicsnemo-cfd-create-model-wrapper/skill-card.md @@ -0,0 +1,83 @@ +## Description:
+Create a new model wrapper for the PhysicsNeMo CFD benchmarking workflow.
+ +This skill is ready for commercial/non-commercial use.
+ +## Owner +NVIDIA
+ +### License/Terms of Use:
+Apache 2.0
+## Use Case:
+Developers and engineers use this skill to add new CFD models to the PhysicsNeMo benchmarking pipeline by writing a CFDModel wrapper subclass.
+ +### Deployment Geography for Use:
+Global
+ +## Requirements / Dependencies:
+**Requires API Key or External Credential:** [Not Specified]
+**Credential Type(s):** [None identified]
+ +Do not include secrets in prompts/logs/output; use least-privilege credentials; rotate keys as appropriate.
+ +## Known Risks and Mitigations:
+Risk: Review before execution as proposals could introduce incorrect or misleading guidance into skills.
+Mitigation: Review and scan skill before deployment.
+ +## Reference(s):
+- [example_wrapper.py](references/example_wrapper.py)
+- [global_stats.example.json](assets/global_stats.example.json)
+- [PhysicsNeMo Framework](https://github.com/NVIDIA/physicsnemo/)
+ + +## Skill Output:
+**Output Type(s):** [Code, Configuration instructions]
+**Output Format:** [Python source files with inline code blocks]
+**Output Parameters:** [1D]
+**Other Properties Related to Output:** [None]
+ +## Evaluation Agents Used:
+- `claude-code`
+- `codex`
+ + + +## Evaluation Tasks:
+Evaluated against 8 evaluation tasks using NVSkills-Eval 3-Tier Evaluation (external profile).
+ +## Evaluation Metrics Used:
+Reported benchmark dimensions:
+- Security: Checks whether skill-assisted execution avoids unsafe behavior such as secret leakage, destructive commands, or unauthorized access.
+- Correctness: Checks whether the agent follows the expected workflow and produces the correct final output.
+- Discoverability: Checks whether the agent loads the skill when relevant and avoids using it when irrelevant.
+- Effectiveness: Checks whether the agent performs measurably better with the skill than without it.
+- Efficiency: Checks whether the agent uses fewer tokens and avoids redundant work.
+ +Underlying evaluation signals used in this run:
+- `security`: Checks for unsafe operations, secret leakage, and unauthorized access.
+- `skill_execution`: Verifies that the agent loaded the expected skill and workflow.
+- `skill_efficiency`: Checks routing quality, decoy avoidance, and redundant tool usage.
+- `accuracy`: Grades final-answer correctness against the reference answer.
+- `goal_accuracy`: Checks whether the overall user task completed successfully.
+- `behavior_check`: Verifies expected behavior steps, including safety expectations.
+- `token_efficiency`: Compares token usage with and without the skill.
+ + + +## Evaluation Results:
+| Dimension | Num | `claude-code` | `codex` | +|---|---:|---:|---:| +| Security | 4 | 100% (+0%) | 100% (+0%) | +| Correctness | 4 | 93% (+50%) | 89% (+35%) | +| Discoverability | 4 | 95% (+57%) | 91% (+40%) | +| Effectiveness | 4 | 79% (+42%) | 76% (+30%) | +| Efficiency | 4 | 84% (+41%) | 82% (+31%) | + +## Skill Version(s):
+0.0.2 (source: changelog, released 2026-06-09)
+ +## Ethical Considerations:
+NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal team to ensure this skill meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
+ +(For Release on NVIDIA Platforms Only)
+Please report quality, risk, security vulnerabilities or NVIDIA AI Concerns [here](https://app.intigriti.com/programs/nvidia/nvidiavdp/detail).
diff --git a/skills/physicsnemo-cfd-create-model-wrapper/skill.oms.sig b/skills/physicsnemo-cfd-create-model-wrapper/skill.oms.sig new file mode 100644 index 0000000..d5288ac --- /dev/null +++ b/skills/physicsnemo-cfd-create-model-wrapper/skill.oms.sig @@ -0,0 +1 @@ +{"mediaType":"application/vnd.dev.sigstore.bundle.v0.3+json","verificationMaterial":{"x509CertificateChain":{"certificates":[{"rawBytes":"MIICgzCCAgmgAwIBAgIUKIyS7SxNteQIiWzK1dWj85E6520wCgYIKoZIzj0EAwMwVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwHhcNMjYwNDAxMDAwMDAwWhcNMjgwNDIyMTUzMzA5WjBUMQswCQYDVQQGEwJVUzEbMBkGA1UECgwSTlZJRElBIENvcnBvcmF0aW9uMSgwJgYDVQQDDB9OVklESUEgQWdlbnQgU2tpbGxzIFNpZ25pbmcgMDAxMHYwEAYHKoZIzj0CAQYFK4EEACIDYgAEYoRM9bQl/dGlwSRNi6bTpIJUXH8Nv9GciP6LSflJYYMLCc296kpyuTSsk5ddbAWiDcFX3C/ydX3jwc+qCLYP6uHy9XphyLjOQ27Yb2J6rBLVtRBS1mgGco/Gr7fL6ODco4GaMIGXMB0GA1UdDgQWBBRQ/5ZW3nJ6lmo9SVk7I15o7UGmpTAfBgNVHSMEGDAWgBRPGpILxMBBleJSsBGjrMKsby1CgjAMBgNVHRMBAf8EAjAAMA4GA1UdDwEB/wQEAwIHgDA3BggrBgEFBQcBAQQrMCkwJwYIKwYBBQUHMAGGG2h0dHA6Ly9vY3NwLm5kaXMubnZpZGlhLmNvbTAKBggqhkjOPQQDAwNoADBlAjAUygu/GiOCIXrgGr4SmLgeEVDcEitfFUv7ALbvLVGVyMysB3mxmO/uInZfXzWcJZsCMQDxuoxj4ZmO30jhkPIcCxGFCOvnUsnfU3TfGcouYm4M6iRpbKvtVnHPiy4bi6pcKf0="},{"rawBytes":"MIICiDCCAg6gAwIBAgIUZsIuSv9NkpJCNqtYEfCouVv5BzowCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwdjAQBgcqhkjOPQIBBgUrgQQAIgNiAASI72cR3ctKGg4VWnB3bNja6g1Z2PnOmFEopkPof+QeIcPk9rT+g9MjJnq51EQXL93a7C2GJ9J985G4o2V85VD7wJ1RaXhluHW2rf3y8bQGeAYaKMr5s/hUgn+M3/9WlWejgaAwgZ0wHQYDVR0OBBYEFE8akgvEwEGV4lKwEaOswqxvLUKCMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgEGMDcGCCsGAQUFBwEBBCswKTAnBggrBgEFBQcwAYYbaHR0cDovL29jc3AubmRpcy5udmlkaWEuY29tMAoGCCqGSM49BAMDA2gAMGUCMQCeIMMfAbyzPDacw2MxG+Yt1cikrJX/DVxiGfXuHmkkXn6VgSzE79+lkqDErpVO2gYCMCNEColOyvUvkzZGUEI1hQ3PfMgi3FIo9tHoBKMw4/wGBLFpu/0ubtmbBXM6/UMOEw=="},{"rawBytes":"MIICRTCCAcygAwIBAgIUeJdY3rV86EdvFmG7L8LJBsyQFYkwCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTB2MBAGByqGSM49AgEGBSuBBAAiA2IABAYpiXCDjJ9NT2eSDhyHJVSw1Tbze18cGG2F/578oWvHxg23eQAhNRYdq88i1iOshZSO6C29doKui5Xpmo/7Ctw9Sx4PP2RzOmIuOLCuTdNtKcTRwi4GEsd5BAFvWj42M6NjMGEwHQYDVR0OBBYEFItnoAjjfuCEUvzyvWyI2vOGvwPjMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEGMAoGCCqGSM49BAMDA2cAMGQCMCwtAjWLaNwgGWNCgdyNoTyvNhqWRECRJV2r3+7w8g0PL6NHLOsbkgE09BH95h8XlgIwTaQmbbUh2ChAJ5TA1wRiVDnCcvbzHlZl2jM2FcwQQZlk19LOAbyGMRixbu2Ww/rj"}]},"tlogEntries":[]},"dsseEnvelope":{"payload":"ewogICJfdHlwZSI6ICJodHRwczovL2luLXRvdG8uaW8vU3RhdGVtZW50L3YxIiwKICAic3ViamVjdCI6IFsKICAgIHsKICAgICAgIm5hbWUiOiAicGh5c2ljc25lbW8tY2ZkLWNyZWF0ZS1tb2RlbC13cmFwcGVyIiwKICAgICAgImRpZ2VzdCI6IHsKICAgICAgICAic2hhMjU2IjogIjNhZTgzMDVhMmJhODZiZTA2ZWI5ZGRiODJiODk0ZDBjZGNiMWUwMGZiMmZkOWY1NGJiMTkwYmQ0YWUyYTNjZDMiCiAgICAgIH0KICAgIH0KICBdLAogICJwcmVkaWNhdGVUeXBlIjogImh0dHBzOi8vbW9kZWxfc2lnbmluZy9zaWduYXR1cmUvdjEuMCIsCiAgInByZWRpY2F0ZSI6IHsKICAgICJyZXNvdXJjZXMiOiBbCiAgICAgIHsKICAgICAgICAibmFtZSI6ICJCRU5DSE1BUksubWQiLAogICAgICAgICJkaWdlc3QiOiAiY2VmNGU0YmQzZDZmNDA4NDdhMzVkZjZkODQ3NWQxOGJjZmI5ODZmYTM4ZmVmODQ3NjE3NTAyZTJiYjZiNTI4YiIsCiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAibmFtZSI6ICJTS0lMTC5tZCIsCiAgICAgICAgImRpZ2VzdCI6ICJjMTg2Y2U1ZDZmZTVlOTUzMWMwNWRhNmFiNGE2ZmE3ZTY1ZTJjZTU0NWU5Zjk1OWEzYmZkYTc1MDg1YmU1YmRhIiwKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIKICAgICAgfSwKICAgICAgewogICAgICAgICJuYW1lIjogImFzc2V0cy9nbG9iYWxfc3RhdHMuZXhhbXBsZS5qc29uIiwKICAgICAgICAiZGlnZXN0IjogIjA3Zjg2ZTQzNDdkZWI0ZDk1NTczNWM1M2VhNWQ3ZjgyZGNhNTAwZDU4MWZlZjk3ZGZhYWFhMzg0OTczZTk5Y2EiLAogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IgogICAgICB9LAogICAgICB7CiAgICAgICAgIm5hbWUiOiAiZXZhbHMvZXZhbHMuanNvbiIsCiAgICAgICAgImRpZ2VzdCI6ICJhMWE5MWZiYTlkNWU5NDRlOWI4ZTdkN2RhYThmMGNjNTNlNjdhZjAyYTNhOGVmYThjNmFkNWU4ZDdmOWYyNDZkIiwKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIKICAgICAgfSwKICAgICAgewogICAgICAgICJuYW1lIjogInJlZmVyZW5jZXMvZXhhbXBsZV93cmFwcGVyLnB5IiwKICAgICAgICAiZGlnZXN0IjogImEwZDg4MzE1ZWIzNmJjM2E3NjExYTZmYjc3ZTU2ODE1Mjk5NGJlMDI0OTM1N2U2OGQ5ZTczMmQ3MmJiMDhkZTUiLAogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IgogICAgICB9LAogICAgICB7CiAgICAgICAgIm5hbWUiOiAic2tpbGwtY2FyZC5tZCIsCiAgICAgICAgImRpZ2VzdCI6ICJjZjg4NDdjMDlmZGE0MDMyY2I2ZDJmNTNiMzU2ZjYyYTk5Y2RjNzk5ZjRlMTBkMTAyMzhkYzMyYzhkMDQ5YTYzIiwKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIKICAgICAgfQogICAgXSwKICAgICJzZXJpYWxpemF0aW9uIjogewogICAgICAibWV0aG9kIjogImZpbGVzIiwKICAgICAgImhhc2hfdHlwZSI6ICJzaGEyNTYiLAogICAgICAiaWdub3JlX3BhdGhzIjogWwogICAgICAgICIuZ2l0aHViIiwKICAgICAgICAiLmdpdGlnbm9yZSIsCiAgICAgICAgIi5naXRhdHRyaWJ1dGVzIiwKICAgICAgICAiLmdpdCIKICAgICAgXSwKICAgICAgImFsbG93X3N5bWxpbmtzIjogZmFsc2UKICAgIH0KICB9Cn0=","payloadType":"application/vnd.in-toto+json","signatures":[{"sig":"MGUCMQDCL0NCyFircfmO5R7xjDxWNM9x86sOSeuV1uUDUZw+5hXBsEAqIe6G+TFYnaXgELICMASjPdw8xL6guI1fGEmvs57eHveAK0fUbynr6ACe9wTWnZsJq/87r/rC434/D4SJ8A==","keyid":""}]}} \ No newline at end of file From f5d9c962a7715cabc46a1b00b83adb3aee0e5b4e Mon Sep 17 00:00:00 2001 From: Kaustubh Tangsali <71059996+ktangsali@users.noreply.github.com> Date: Mon, 6 Jul 2026 09:27:34 -0700 Subject: [PATCH 3/5] Update markdownlint configuration to exclude skills directory Exclude 'skills/' directory from markdownlint checks. --- .pre-commit-config.yaml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index ac76b61..f7c00cf 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -35,6 +35,7 @@ repos: rev: v0.35.0 hooks: - id: markdownlint + exclude: ^skills/ - repo: https://github.com/pre-commit/pre-commit-hooks rev: v3.4.0 @@ -48,4 +49,4 @@ repos: name: license entry: python test/ci_tests/header_check.py language: python - pass_filenames: false \ No newline at end of file + pass_filenames: false From 16326fa8fa2ee89b2c7f1a06f21b37e54451f625 Mon Sep 17 00:00:00 2001 From: Kaustubh Tangsali Date: Tue, 7 Jul 2026 20:20:29 +0000 Subject: [PATCH 4/5] address review comment --- .../physicsnemo-cfd-create-model-wrapper/SKILL.md | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/skills/physicsnemo-cfd-create-model-wrapper/SKILL.md b/skills/physicsnemo-cfd-create-model-wrapper/SKILL.md index ece2800..17c7720 100644 --- a/skills/physicsnemo-cfd-create-model-wrapper/SKILL.md +++ b/skills/physicsnemo-cfd-create-model-wrapper/SKILL.md @@ -38,10 +38,15 @@ Use these to pin down four things the wrapper must mirror exactly: the trained graph. If some or all of these aren't available, that's fine — **proceed -anyway**: reconstruct from the checkpoint and stats file and briefly -state the assumptions you're making. Keep moving and build the wrapper; -just avoid silently guessing normalization or channel order, since a -wrong choice produces plausible-looking but wrong predictions. +anyway**: reconstruct from the checkpoint and stats file, pick the most +likely option, and build the wrapper now. Do **not** stop and wait for +answers before writing code. Just avoid *silently* guessing: a wrong +normalization scheme or channel order produces plausible-looking but +wrong predictions. So state each such assumption inline, and after +delivering the wrapper, **raise the still-uncertain choices as explicit +open items for the user to confirm** — e.g. normalization scheme, input +tier, and output fields/channel order. This keeps you unblocked while +making the risky decisions visible. ## Reference files to read first From 00f9627a878da8d6e8f85198b921eb9302a53770 Mon Sep 17 00:00:00 2001 From: nvskills-svc-account Date: Wed, 8 Jul 2026 17:09:05 +0000 Subject: [PATCH 5/5] Attach NVSkills validation signatures Signed-off-by: nvskills-svc-account --- .../BENCHMARK.md | 10 +++++----- .../skill-card.md | 20 +++++++++---------- .../skill.oms.sig | 2 +- 3 files changed, 16 insertions(+), 16 deletions(-) diff --git a/skills/physicsnemo-cfd-create-model-wrapper/BENCHMARK.md b/skills/physicsnemo-cfd-create-model-wrapper/BENCHMARK.md index 7647aa8..2cffd66 100644 --- a/skills/physicsnemo-cfd-create-model-wrapper/BENCHMARK.md +++ b/skills/physicsnemo-cfd-create-model-wrapper/BENCHMARK.md @@ -7,7 +7,7 @@ This benchmark summarizes 3-Tier Evaluation from NVSkills-Eval results for the s ## Evaluation Summary - Skill: `physicsnemo-cfd-create-model-wrapper` -- Evaluation date: 2026-07-06 +- Evaluation date: 2026-07-08 - NVSkills-Eval profile: `external` - Environment: `astra-sandbox` - Dataset: 8 evaluation tasks @@ -49,10 +49,10 @@ The benchmark included 8 recorded Tier 3 trials, but the source evaluation datas | Dimension | Num | `claude-code` | `codex` | |---|---:|---:|---:| | Security | 4 | 100% (+0%) | 100% (+0%) | -| Correctness | 4 | 93% (+50%) | 89% (+35%) | -| Discoverability | 4 | 95% (+57%) | 91% (+40%) | -| Effectiveness | 4 | 79% (+42%) | 76% (+30%) | -| Efficiency | 4 | 84% (+41%) | 82% (+31%) | +| Correctness | 4 | 100% (+45%) | 94% (+37%) | +| Discoverability | 4 | 94% (+56%) | 87% (+34%) | +| Effectiveness | 4 | 94% (+47%) | 88% (+43%) | +| Efficiency | 4 | 83% (+42%) | 79% (+25%) | Score values show skill-assisted performance. Values in parentheses show uplift versus the no-skill baseline when baseline data is available. diff --git a/skills/physicsnemo-cfd-create-model-wrapper/skill-card.md b/skills/physicsnemo-cfd-create-model-wrapper/skill-card.md index 99b0a01..d1ef07a 100644 --- a/skills/physicsnemo-cfd-create-model-wrapper/skill-card.md +++ b/skills/physicsnemo-cfd-create-model-wrapper/skill-card.md @@ -9,7 +9,7 @@ NVIDIA
### License/Terms of Use:
Apache 2.0
## Use Case:
-Developers and engineers use this skill to add new CFD models to the PhysicsNeMo benchmarking pipeline by writing a CFDModel wrapper subclass.
+Developers and engineers who need to integrate new CFD neural network models into the PhysicsNeMo benchmarking pipeline by writing a CFDModel wrapper subclass.
### Deployment Geography for Use:
Global
@@ -27,23 +27,23 @@ Mitigation: Review and scan skill before deployment.
## Reference(s):
- [example_wrapper.py](references/example_wrapper.py)
- [global_stats.example.json](assets/global_stats.example.json)
-- [PhysicsNeMo Framework](https://github.com/NVIDIA/physicsnemo/)
+- [PhysicsNeMo CFD Repository](https://github.com/NVIDIA/physicsnemo/)
## Skill Output:
**Output Type(s):** [Code, Configuration instructions]
-**Output Format:** [Python source files with inline code blocks]
+**Output Format:** [Python source files with inline guidance]
**Output Parameters:** [1D]
**Other Properties Related to Output:** [None]
## Evaluation Agents Used:
-- `claude-code`
-- `codex`
+- claude-code
+- codex
## Evaluation Tasks:
-Evaluated against 8 evaluation tasks using NVSkills-Eval 3-Tier Evaluation (external profile).
+Evaluated against 8 evaluation tasks using NVSkills-Eval (external profile, astra-sandbox environment).
## Evaluation Metrics Used:
Reported benchmark dimensions:
@@ -68,10 +68,10 @@ Underlying evaluation signals used in this run:
| Dimension | Num | `claude-code` | `codex` | |---|---:|---:|---:| | Security | 4 | 100% (+0%) | 100% (+0%) | -| Correctness | 4 | 93% (+50%) | 89% (+35%) | -| Discoverability | 4 | 95% (+57%) | 91% (+40%) | -| Effectiveness | 4 | 79% (+42%) | 76% (+30%) | -| Efficiency | 4 | 84% (+41%) | 82% (+31%) | +| Correctness | 4 | 100% (+45%) | 94% (+37%) | +| Discoverability | 4 | 94% (+56%) | 87% (+34%) | +| Effectiveness | 4 | 94% (+47%) | 88% (+43%) | +| Efficiency | 4 | 83% (+42%) | 79% (+25%) | ## Skill Version(s):
0.0.2 (source: changelog, released 2026-06-09)
diff --git a/skills/physicsnemo-cfd-create-model-wrapper/skill.oms.sig b/skills/physicsnemo-cfd-create-model-wrapper/skill.oms.sig index d5288ac..4169042 100644 --- a/skills/physicsnemo-cfd-create-model-wrapper/skill.oms.sig +++ b/skills/physicsnemo-cfd-create-model-wrapper/skill.oms.sig @@ -1 +1 @@ -{"mediaType":"application/vnd.dev.sigstore.bundle.v0.3+json","verificationMaterial":{"x509CertificateChain":{"certificates":[{"rawBytes":"MIICgzCCAgmgAwIBAgIUKIyS7SxNteQIiWzK1dWj85E6520wCgYIKoZIzj0EAwMwVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwHhcNMjYwNDAxMDAwMDAwWhcNMjgwNDIyMTUzMzA5WjBUMQswCQYDVQQGEwJVUzEbMBkGA1UECgwSTlZJRElBIENvcnBvcmF0aW9uMSgwJgYDVQQDDB9OVklESUEgQWdlbnQgU2tpbGxzIFNpZ25pbmcgMDAxMHYwEAYHKoZIzj0CAQYFK4EEACIDYgAEYoRM9bQl/dGlwSRNi6bTpIJUXH8Nv9GciP6LSflJYYMLCc296kpyuTSsk5ddbAWiDcFX3C/ydX3jwc+qCLYP6uHy9XphyLjOQ27Yb2J6rBLVtRBS1mgGco/Gr7fL6ODco4GaMIGXMB0GA1UdDgQWBBRQ/5ZW3nJ6lmo9SVk7I15o7UGmpTAfBgNVHSMEGDAWgBRPGpILxMBBleJSsBGjrMKsby1CgjAMBgNVHRMBAf8EAjAAMA4GA1UdDwEB/wQEAwIHgDA3BggrBgEFBQcBAQQrMCkwJwYIKwYBBQUHMAGGG2h0dHA6Ly9vY3NwLm5kaXMubnZpZGlhLmNvbTAKBggqhkjOPQQDAwNoADBlAjAUygu/GiOCIXrgGr4SmLgeEVDcEitfFUv7ALbvLVGVyMysB3mxmO/uInZfXzWcJZsCMQDxuoxj4ZmO30jhkPIcCxGFCOvnUsnfU3TfGcouYm4M6iRpbKvtVnHPiy4bi6pcKf0="},{"rawBytes":"MIICiDCCAg6gAwIBAgIUZsIuSv9NkpJCNqtYEfCouVv5BzowCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwdjAQBgcqhkjOPQIBBgUrgQQAIgNiAASI72cR3ctKGg4VWnB3bNja6g1Z2PnOmFEopkPof+QeIcPk9rT+g9MjJnq51EQXL93a7C2GJ9J985G4o2V85VD7wJ1RaXhluHW2rf3y8bQGeAYaKMr5s/hUgn+M3/9WlWejgaAwgZ0wHQYDVR0OBBYEFE8akgvEwEGV4lKwEaOswqxvLUKCMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgEGMDcGCCsGAQUFBwEBBCswKTAnBggrBgEFBQcwAYYbaHR0cDovL29jc3AubmRpcy5udmlkaWEuY29tMAoGCCqGSM49BAMDA2gAMGUCMQCeIMMfAbyzPDacw2MxG+Yt1cikrJX/DVxiGfXuHmkkXn6VgSzE79+lkqDErpVO2gYCMCNEColOyvUvkzZGUEI1hQ3PfMgi3FIo9tHoBKMw4/wGBLFpu/0ubtmbBXM6/UMOEw=="},{"rawBytes":"MIICRTCCAcygAwIBAgIUeJdY3rV86EdvFmG7L8LJBsyQFYkwCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTB2MBAGByqGSM49AgEGBSuBBAAiA2IABAYpiXCDjJ9NT2eSDhyHJVSw1Tbze18cGG2F/578oWvHxg23eQAhNRYdq88i1iOshZSO6C29doKui5Xpmo/7Ctw9Sx4PP2RzOmIuOLCuTdNtKcTRwi4GEsd5BAFvWj42M6NjMGEwHQYDVR0OBBYEFItnoAjjfuCEUvzyvWyI2vOGvwPjMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEGMAoGCCqGSM49BAMDA2cAMGQCMCwtAjWLaNwgGWNCgdyNoTyvNhqWRECRJV2r3+7w8g0PL6NHLOsbkgE09BH95h8XlgIwTaQmbbUh2ChAJ5TA1wRiVDnCcvbzHlZl2jM2FcwQQZlk19LOAbyGMRixbu2Ww/rj"}]},"tlogEntries":[]},"dsseEnvelope":{"payload":"ewogICJfdHlwZSI6ICJodHRwczovL2luLXRvdG8uaW8vU3RhdGVtZW50L3YxIiwKICAic3ViamVjdCI6IFsKICAgIHsKICAgICAgIm5hbWUiOiAicGh5c2ljc25lbW8tY2ZkLWNyZWF0ZS1tb2RlbC13cmFwcGVyIiwKICAgICAgImRpZ2VzdCI6IHsKICAgICAgICAic2hhMjU2IjogIjNhZTgzMDVhMmJhODZiZTA2ZWI5ZGRiODJiODk0ZDBjZGNiMWUwMGZiMmZkOWY1NGJiMTkwYmQ0YWUyYTNjZDMiCiAgICAgIH0KICAgIH0KICBdLAogICJwcmVkaWNhdGVUeXBlIjogImh0dHBzOi8vbW9kZWxfc2lnbmluZy9zaWduYXR1cmUvdjEuMCIsCiAgInByZWRpY2F0ZSI6IHsKICAgICJyZXNvdXJjZXMiOiBbCiAgICAgIHsKICAgICAgICAibmFtZSI6ICJCRU5DSE1BUksubWQiLAogICAgICAgICJkaWdlc3QiOiAiY2VmNGU0YmQzZDZmNDA4NDdhMzVkZjZkODQ3NWQxOGJjZmI5ODZmYTM4ZmVmODQ3NjE3NTAyZTJiYjZiNTI4YiIsCiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAibmFtZSI6ICJTS0lMTC5tZCIsCiAgICAgICAgImRpZ2VzdCI6ICJjMTg2Y2U1ZDZmZTVlOTUzMWMwNWRhNmFiNGE2ZmE3ZTY1ZTJjZTU0NWU5Zjk1OWEzYmZkYTc1MDg1YmU1YmRhIiwKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIKICAgICAgfSwKICAgICAgewogICAgICAgICJuYW1lIjogImFzc2V0cy9nbG9iYWxfc3RhdHMuZXhhbXBsZS5qc29uIiwKICAgICAgICAiZGlnZXN0IjogIjA3Zjg2ZTQzNDdkZWI0ZDk1NTczNWM1M2VhNWQ3ZjgyZGNhNTAwZDU4MWZlZjk3ZGZhYWFhMzg0OTczZTk5Y2EiLAogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IgogICAgICB9LAogICAgICB7CiAgICAgICAgIm5hbWUiOiAiZXZhbHMvZXZhbHMuanNvbiIsCiAgICAgICAgImRpZ2VzdCI6ICJhMWE5MWZiYTlkNWU5NDRlOWI4ZTdkN2RhYThmMGNjNTNlNjdhZjAyYTNhOGVmYThjNmFkNWU4ZDdmOWYyNDZkIiwKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIKICAgICAgfSwKICAgICAgewogICAgICAgICJuYW1lIjogInJlZmVyZW5jZXMvZXhhbXBsZV93cmFwcGVyLnB5IiwKICAgICAgICAiZGlnZXN0IjogImEwZDg4MzE1ZWIzNmJjM2E3NjExYTZmYjc3ZTU2ODE1Mjk5NGJlMDI0OTM1N2U2OGQ5ZTczMmQ3MmJiMDhkZTUiLAogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IgogICAgICB9LAogICAgICB7CiAgICAgICAgIm5hbWUiOiAic2tpbGwtY2FyZC5tZCIsCiAgICAgICAgImRpZ2VzdCI6ICJjZjg4NDdjMDlmZGE0MDMyY2I2ZDJmNTNiMzU2ZjYyYTk5Y2RjNzk5ZjRlMTBkMTAyMzhkYzMyYzhkMDQ5YTYzIiwKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIKICAgICAgfQogICAgXSwKICAgICJzZXJpYWxpemF0aW9uIjogewogICAgICAibWV0aG9kIjogImZpbGVzIiwKICAgICAgImhhc2hfdHlwZSI6ICJzaGEyNTYiLAogICAgICAiaWdub3JlX3BhdGhzIjogWwogICAgICAgICIuZ2l0aHViIiwKICAgICAgICAiLmdpdGlnbm9yZSIsCiAgICAgICAgIi5naXRhdHRyaWJ1dGVzIiwKICAgICAgICAiLmdpdCIKICAgICAgXSwKICAgICAgImFsbG93X3N5bWxpbmtzIjogZmFsc2UKICAgIH0KICB9Cn0=","payloadType":"application/vnd.in-toto+json","signatures":[{"sig":"MGUCMQDCL0NCyFircfmO5R7xjDxWNM9x86sOSeuV1uUDUZw+5hXBsEAqIe6G+TFYnaXgELICMASjPdw8xL6guI1fGEmvs57eHveAK0fUbynr6ACe9wTWnZsJq/87r/rC434/D4SJ8A==","keyid":""}]}} \ No newline at end of file +{"mediaType":"application/vnd.dev.sigstore.bundle.v0.3+json","verificationMaterial":{"x509CertificateChain":{"certificates":[{"rawBytes":"MIICgzCCAgmgAwIBAgIUKIyS7SxNteQIiWzK1dWj85E6520wCgYIKoZIzj0EAwMwVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwHhcNMjYwNDAxMDAwMDAwWhcNMjgwNDIyMTUzMzA5WjBUMQswCQYDVQQGEwJVUzEbMBkGA1UECgwSTlZJRElBIENvcnBvcmF0aW9uMSgwJgYDVQQDDB9OVklESUEgQWdlbnQgU2tpbGxzIFNpZ25pbmcgMDAxMHYwEAYHKoZIzj0CAQYFK4EEACIDYgAEYoRM9bQl/dGlwSRNi6bTpIJUXH8Nv9GciP6LSflJYYMLCc296kpyuTSsk5ddbAWiDcFX3C/ydX3jwc+qCLYP6uHy9XphyLjOQ27Yb2J6rBLVtRBS1mgGco/Gr7fL6ODco4GaMIGXMB0GA1UdDgQWBBRQ/5ZW3nJ6lmo9SVk7I15o7UGmpTAfBgNVHSMEGDAWgBRPGpILxMBBleJSsBGjrMKsby1CgjAMBgNVHRMBAf8EAjAAMA4GA1UdDwEB/wQEAwIHgDA3BggrBgEFBQcBAQQrMCkwJwYIKwYBBQUHMAGGG2h0dHA6Ly9vY3NwLm5kaXMubnZpZGlhLmNvbTAKBggqhkjOPQQDAwNoADBlAjAUygu/GiOCIXrgGr4SmLgeEVDcEitfFUv7ALbvLVGVyMysB3mxmO/uInZfXzWcJZsCMQDxuoxj4ZmO30jhkPIcCxGFCOvnUsnfU3TfGcouYm4M6iRpbKvtVnHPiy4bi6pcKf0="},{"rawBytes":"MIICiDCCAg6gAwIBAgIUZsIuSv9NkpJCNqtYEfCouVv5BzowCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwdjAQBgcqhkjOPQIBBgUrgQQAIgNiAASI72cR3ctKGg4VWnB3bNja6g1Z2PnOmFEopkPof+QeIcPk9rT+g9MjJnq51EQXL93a7C2GJ9J985G4o2V85VD7wJ1RaXhluHW2rf3y8bQGeAYaKMr5s/hUgn+M3/9WlWejgaAwgZ0wHQYDVR0OBBYEFE8akgvEwEGV4lKwEaOswqxvLUKCMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgEGMDcGCCsGAQUFBwEBBCswKTAnBggrBgEFBQcwAYYbaHR0cDovL29jc3AubmRpcy5udmlkaWEuY29tMAoGCCqGSM49BAMDA2gAMGUCMQCeIMMfAbyzPDacw2MxG+Yt1cikrJX/DVxiGfXuHmkkXn6VgSzE79+lkqDErpVO2gYCMCNEColOyvUvkzZGUEI1hQ3PfMgi3FIo9tHoBKMw4/wGBLFpu/0ubtmbBXM6/UMOEw=="},{"rawBytes":"MIICRTCCAcygAwIBAgIUeJdY3rV86EdvFmG7L8LJBsyQFYkwCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTB2MBAGByqGSM49AgEGBSuBBAAiA2IABAYpiXCDjJ9NT2eSDhyHJVSw1Tbze18cGG2F/578oWvHxg23eQAhNRYdq88i1iOshZSO6C29doKui5Xpmo/7Ctw9Sx4PP2RzOmIuOLCuTdNtKcTRwi4GEsd5BAFvWj42M6NjMGEwHQYDVR0OBBYEFItnoAjjfuCEUvzyvWyI2vOGvwPjMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEGMAoGCCqGSM49BAMDA2cAMGQCMCwtAjWLaNwgGWNCgdyNoTyvNhqWRECRJV2r3+7w8g0PL6NHLOsbkgE09BH95h8XlgIwTaQmbbUh2ChAJ5TA1wRiVDnCcvbzHlZl2jM2FcwQQZlk19LOAbyGMRixbu2Ww/rj"}]},"tlogEntries":[]},"dsseEnvelope":{"payload":"ewogICJfdHlwZSI6ICJodHRwczovL2luLXRvdG8uaW8vU3RhdGVtZW50L3YxIiwKICAic3ViamVjdCI6IFsKICAgIHsKICAgICAgIm5hbWUiOiAicGh5c2ljc25lbW8tY2ZkLWNyZWF0ZS1tb2RlbC13cmFwcGVyIiwKICAgICAgImRpZ2VzdCI6IHsKICAgICAgICAic2hhMjU2IjogIjE3NGJhNTMwZmRhMzZhNGU5ZDdiN2M5OWFiZDE3ZmY0MGE4ZWJkODRkYWM2Y2U2ZThhNmYxZTc0MTFjY2U5ZGMiCiAgICAgIH0KICAgIH0KICBdLAogICJwcmVkaWNhdGVUeXBlIjogImh0dHBzOi8vbW9kZWxfc2lnbmluZy9zaWduYXR1cmUvdjEuMCIsCiAgInByZWRpY2F0ZSI6IHsKICAgICJzZXJpYWxpemF0aW9uIjogewogICAgICAibWV0aG9kIjogImZpbGVzIiwKICAgICAgImFsbG93X3N5bWxpbmtzIjogZmFsc2UsCiAgICAgICJoYXNoX3R5cGUiOiAic2hhMjU2IiwKICAgICAgImlnbm9yZV9wYXRocyI6IFsKICAgICAgICAiLmdpdGF0dHJpYnV0ZXMiLAogICAgICAgICIuZ2l0IiwKICAgICAgICAiLmdpdGlnbm9yZSIsCiAgICAgICAgIi5naXRodWIiCiAgICAgIF0KICAgIH0sCiAgICAicmVzb3VyY2VzIjogWwogICAgICB7CiAgICAgICAgIm5hbWUiOiAiQkVOQ0hNQVJLLm1kIiwKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgImRpZ2VzdCI6ICIxNDk4NmQwMjdmYjAzNTk0NDVmNGZjMzFiOGYxN2YxYTU1MmE2NjI2NzRkZDk5MWEwODY3OTEyNTg1ZDY1Y2MyIgogICAgICB9LAogICAgICB7CiAgICAgICAgIm5hbWUiOiAiU0tJTEwubWQiLAogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAiZGlnZXN0IjogIjU1ZWE4OTFlYTRjM2ViNTY3MzhlZjAxNjg5YWY1ZThiNWY4ZGM3MGM1MGE3OGEwYjI2YjAxZGYzNTAwODE3ZWIiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAibmFtZSI6ICJhc3NldHMvZ2xvYmFsX3N0YXRzLmV4YW1wbGUuanNvbiIsCiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJkaWdlc3QiOiAiMDdmODZlNDM0N2RlYjRkOTU1NzM1YzUzZWE1ZDdmODJkY2E1MDBkNTgxZmVmOTdkZmFhYWEzODQ5NzNlOTljYSIKICAgICAgfSwKICAgICAgewogICAgICAgICJuYW1lIjogImV2YWxzL2V2YWxzLmpzb24iLAogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAiZGlnZXN0IjogImExYTkxZmJhOWQ1ZTk0NGU5YjhlN2Q3ZGFhOGYwY2M1M2U2N2FmMDJhM2E4ZWZhOGM2YWQ1ZThkN2Y5ZjI0NmQiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAibmFtZSI6ICJyZWZlcmVuY2VzL2V4YW1wbGVfd3JhcHBlci5weSIsCiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJkaWdlc3QiOiAiYTBkODgzMTVlYjM2YmMzYTc2MTFhNmZiNzdlNTY4MTUyOTk0YmUwMjQ5MzU3ZTY4ZDllNzMyZDcyYmIwOGRlNSIKICAgICAgfSwKICAgICAgewogICAgICAgICJuYW1lIjogInNraWxsLWNhcmQubWQiLAogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAiZGlnZXN0IjogIjk0ZjBkMjNjNjAzNDJmMGI0MmE1MzI1NWJlYzVmZTYzMjNlOGVmYmYwNmYzYmZjYzVkNTg2NTdmOWUwOWVhNzUiCiAgICAgIH0KICAgIF0KICB9Cn0=","payloadType":"application/vnd.in-toto+json","signatures":[{"sig":"MGQCMBfEV1+dnlK5w7ZgbFVw1yblDwIU6W+clZJzUwpdQjaXdUAy6d+q7pKZw24OoYZoTwIwGq7Ii0ZpcJmldFVTb1c/xsjlfGMAX1+42+OfESiDSvqhfWaWeKEvGyOUXACKC+0K","keyid":""}]}} \ No newline at end of file