From b258dc59da58ac78f559d60203582e57db256a85 Mon Sep 17 00:00:00 2001 From: dylan Date: Tue, 25 Aug 2026 11:40:49 -0500 Subject: [PATCH] docs: remove obsolete ONNX submission guide --- README.md | 2 +- docs/Discriminative-Mining.md | 2 +- docs/ONNX.md | 180 ---------------------------------- gas/cli.py | 6 +- 4 files changed, 5 insertions(+), 185 deletions(-) delete mode 100644 docs/ONNX.md diff --git a/README.md b/README.md index 994ce496..1e51d44a 100644 --- a/README.md +++ b/README.md @@ -37,7 +37,7 @@ GAS runs two parallel competition tracks on Bittensor Subnet 34: **Key facts:** - **Three modalities**: Image, video, and audio detection are all scored independently - **Cloud-evaluated**: Discriminator models are benchmarked on cloud infrastructure -- no GPU hosting required -- **Model format**: Safetensors only (ONNX is deprecated) +- **Model format**: Safetensors only (ONNX submissions are not accepted) - **Datasets refresh weekly** with fresh GAS-Station data alongside static benchmarks - **One model per modality per hotkey** for discriminative miners diff --git a/docs/Discriminative-Mining.md b/docs/Discriminative-Mining.md index 1b67d11b..d7984e58 100644 --- a/docs/Discriminative-Mining.md +++ b/docs/Discriminative-Mining.md @@ -167,7 +167,7 @@ Before your model is ever scored on the network, it must pass an **entrance exam - Your model must achieve **≥ 80% accuracy** averaged across all submitted modalities to pass - The exam has a **maximum wall-clock timeout of 1 hour 25 minutes** (5,100 seconds); models that exceed this are treated as failed - The exam runs in an **isolated cloud sandbox** — your code has no network access and cannot interact with the host environment -- ONNX models are additionally scanned for cheat patterns (embedded lookup tables or memorization artifacts) before the exam runs; detection results in an immediate block +- Submissions are statically analyzed and executed in an isolated sandbox; prohibited code or imports result in rejection **Model status during the exam:** diff --git a/docs/ONNX.md b/docs/ONNX.md deleted file mode 100644 index e38425b3..00000000 --- a/docs/ONNX.md +++ /dev/null @@ -1,180 +0,0 @@ -# ONNX Model Creation Guide - -> **⚠️ DEPRECATED**: ONNX format is no longer accepted for Subnet 34 competition submissions. -> Please use **safetensors format** instead. See the [Safetensors Guide](https://github.com/bitmind-ai/gasbench/blob/main/docs/Safetensors.md) for requirements. - ---- - -This guide explains how to create ONNX models for discriminative mining using the example scripts in `neurons/discriminator/onnx_examples`. - -## Key Requirements - -**⚠️ IMPORTANT**: Your ONNX models must meet these requirements: - -1. **Input Shape**: Specify fixed spatial dimensions for optimal batching - - Image models: `['batch_size', 3, H, W]` where H and W are fixed (e.g., 224) - - Video models: `['batch_size', 'frames', 3, H, W]` where H and W are fixed - - ⚠️ If you use dynamic H/W axes, gasbench will default to 224x224 - -2. **Pixel Range**: Accept raw pixel values in range `[0-255]` - - Gasbench handles preprocessing (shortest-edge resize, center crop, augmentations) - - Your model wrapper should only normalize to [0, 1] and apply model-specific normalization - -3. **Output Format**: Historical ONNX artifacts must follow the same current class order as safetensors submissions: - - Image: 3 logits `[real, synthetic, semisynthetic]` - - Video: 4 logits `[real, synthetic, semisynthetic, rendered]` after temporal aggregation - - Audio: 2 logits `[real, synthetic]` - -This page is retained only for legacy artifact maintenance. New competition submissions must follow the [Safetensors Model Specification](https://github.com/bitmind-ai/gasbench/blob/main/docs/Safetensors.md). - - -## Example Scripts - -The `neurons/discriminator/onnx_examples` directory contains working examples: - -### PyTorch Models -- `pytorch_models/image_model.py` - Custom image classification model -- `pytorch_models/video_model.py` - Custom video classification model - -### HuggingFace Models -- `huggingface_models/image_model.py` - Convert HuggingFace image models (e.g., ResNet50) -- `huggingface_models/video_model.py` - Convert HuggingFace video models (e.g., VideoMAE) - -## Quick Start - -1. **Navigate to the examples directory:** - ```bash - cd neurons/discriminator/onnx_examples - ``` - -2. **Run an example script:** - ```bash - # For custom PyTorch models - python pytorch_models/image_model.py - python pytorch_models/video_model.py - - # For HuggingFace models - python huggingface_models/image_model.py - python huggingface_models/video_model.py - ``` - -3. **Check the output:** - ```bash - ls models/ - # Should see: image_detector.onnx, video_detector.onnx - ``` - -## Preprocessing Pipeline - -Gasbench performs the following preprocessing before passing data to your model: -1. **Resize shortest edge** to target size (preserving aspect ratio) -2. **Center crop** to exact target size (H x W from your model spec) -3. **Random augmentations** (rotation, flip, crop, color jitter, etc.) -4. **Batching** with configurable batch size - -Your ONNX wrapper should only handle: -- Normalization: `[0-255]` → `[0-1]` -- Model-specific transforms (e.g., ImageNet mean/std) -- **Video models**: Temporal aggregation - -## Custom Models - -To create your own model: - -1. **Inherit from `nn.Module`** and implement your architecture -2. **Wrap with preprocessing** (normalize, model-specific transforms, temporal aggregation) -3. **Export with fixed spatial dimensions** as shown in the examples -4. **Test with batched uint8 inputs** to ensure compatibility - -## Example Export Code - -### Image Model -```python -# Create dummy input with raw pixel values (fixed spatial dims) -dummy_input = torch.randint(0, 256, (1, 3, 224, 224), dtype=torch.uint8) - -# Export with fixed spatial dimensions for batching -torch.onnx.export( - wrapped_model, - dummy_input, - "models/image_detector.onnx", - input_names=['input'], - output_names=['logits'], - dynamic_axes={ - 'input': {0: 'batch_size'}, # Only batch_size is dynamic - 'logits': {0: 'batch_size'} - }, - opset_version=11, - do_constant_folding=True, - export_params=True, - keep_initializers_as_inputs=False -) -``` - -### Video Model -```python -# Create dummy input (B, T, C, H, W) with fixed spatial dims -dummy_input = torch.randint(0, 256, (1, 8, 3, 224, 224), dtype=torch.uint8) - -# Export with dynamic batch and frames, fixed spatial dims -torch.onnx.export( - wrapped_model, - dummy_input, - "models/video_detector.onnx", - input_names=['input'], - output_names=['logits'], - dynamic_axes={ - 'input': {0: 'batch_size', 1: 'frames'}, # H, W are fixed - 'logits': {0: 'batch_size'} - }, - opset_version=11, - do_constant_folding=True, - export_params=True, - keep_initializers_as_inputs=False -) -``` - -## Model Preprocessing Wrapper - -To perform input preprocessing or ouptut postprocessing, you can wrap your model: - -```python -class PreprocessingWrapper(nn.Module): - def __init__(self, model, is_video=False): - super().__init__() - self.model = model - - def forward(self, x): - # Input x: (B, T, C, H, W) for video, (B, C, H, W) for image - # Values in range [0, 255] - - # Normalize to [0, 1] - x = x.float() / 255.0 - - # Apply model - outputs = self.model(x) - - # Any necessary output postprocessing - - return outputs -``` - -The temporal aggregation prevents single-frame anomalies from dominating predictions. - -## Packaging Your Models - -After creating your ONNX models, you need to package them into zip files before pushing to the network (keeps the system flexible in case we need to add supplemental files in the future): - -```bash -# Package image model -zip image_detector.zip image_detector.onnx - -# Package video model -zip video_detector.zip video_detector.onnx -``` - -Each zip file should contain only the corresponding ONNX model file. - -## Next Steps - -Once you have your ONNX files packaged as zip files, follow the [Discriminative Mining Guide](Discriminative-Mining.md) to push them to the network. diff --git a/gas/cli.py b/gas/cli.py index c00599d1..ba995744 100644 --- a/gas/cli.py +++ b/gas/cli.py @@ -680,9 +680,9 @@ def _f(v): @discriminator.command(name="benchmark", context_settings={"ignore_unknown_options": True, "allow_extra_args": True}) -@click.option("--image-model", help="Path to image detector ONNX model or zip file") -@click.option("--video-model", help="Path to video detector ONNX model or zip file") -@click.option("--audio-model", help="Path to audio detector ONNX model or zip file") +@click.option("--image-model", help="Path to image detector safetensors zip file") +@click.option("--video-model", help="Path to video detector safetensors zip file") +@click.option("--audio-model", help="Path to audio detector safetensors zip file") @click.pass_context def benchmark(ctx, image_model, video_model, audio_model): """Run image/video/audio benchmarks for provided detector models using gasbench.