Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 5 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,6 @@ GAS runs two parallel competition tracks on Bittensor Subnet 34:

**Key facts:**
- **Three modalities**: Image, video, and audio detection are all scored independently
- **Cloud-evaluated**: Discriminator models are benchmarked on cloud infrastructure -- no GPU hosting required
- **Model format**: Safetensors only (ONNX submissions are not accepted)
- **Datasets refresh weekly** with fresh GAS-Station data alongside static benchmarks
- **One model per hotkey** for discriminative miners
Expand Down Expand Up @@ -71,17 +70,10 @@ gascli validator start
# Miners: Start or restart generative miner
gascli generator start

# Miners: Push discriminator models (all three modalities at once)
gascli d push \
--image-model image_detector.zip \
--video-model video_detector.zip \
--audio-model audio_detector.zip \
# Miners: Push one discriminator model per hotkey
gascli d push --image-model image_detector.zip \
--wallet-name default --wallet-hotkey default

# Or push one model at a time
gascli d push --image-model image_detector.zip
gascli d push --video-model video_detector.zip
gascli d push --audio-model audio_detector.zip
# Video or audio: use --video-model or --audio-model on a different hotkey

# Miners: Check your benchmark performance (epistula-authenticated)
gascli d perf --wallet-name default --wallet-hotkey default
Expand All @@ -104,12 +96,10 @@ pm2 start validator.config.js
# Miners: Start or restart generative miner
pm2 start gen_miner.config.js

# Miners: Push discriminator models
# Miners: Push one discriminator model per hotkey
source .venv/bin/activate
python neurons/discriminator/push_model.py \
--image-model image_detector.zip \
--video-model video_detector.zip \
--audio-model audio_detector.zip \
--wallet-name default --wallet-hotkey default
```
For detailed installation and usage instructions, see [Installation Guide](docs/Installation.md).
Expand All @@ -120,7 +110,7 @@ For detailed installation and usage instructions, see [Installation Guide](docs/
> This documentation assumes basic familiarity with [Bittensor concepts](https://docs.bittensor.com/learn/bittensor-building-blocks).

#### Discriminative Miners [[docs](docs/Discriminative-Mining.md)]
Discriminative miners submit detection models for evaluation against a wide variety of real and synthetic media across **image, video, and audio** modalities. Models are evaluated on cloud infrastructure and rewarded based on their accuracy and calibration. This significantly reduces the capital required to mine compared to previous versions that required GPU hosting, and allows the subnet to more reliably identify unique models and reward novel contributions proportionally to their accuracy.
Discriminative miners submit detection models for evaluation against a wide variety of real and synthetic media across **image, video, and audio** modalities. Models are rewarded based on their accuracy and calibration. This significantly reduces the capital required to mine compared to previous versions that required GPU hosting, and allows the subnet to more reliably identify unique models and reward novel contributions proportionally to their accuracy.

#### Generative Miners [[docs](docs/Generative-Mining.md)]

Expand Down
30 changes: 12 additions & 18 deletions docs/Discriminative-Mining.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Follow the [Installation Guide](Installation.md) to set up your environment befo
- Miners submit media-provenance classifiers across three modalities: **image**, **video**, and **audio**.
- Image models classify `[real, synthetic, semisynthetic]`; video models classify `[real, synthetic, semisynthetic, rendered]`; audio remains `[real, synthetic]`.
- The visual taxonomy is experimental. Semisynthetic media retains materially captured visual content alongside spatially localized generated or replaced content. Fully synthesized output remains synthetic even when captured media conditions generation.
- Models are evaluated on cloud infrastructure -- miners do not need to host hardware for inference.
- Miners do not need to host hardware for inference.

Class order is part of the submission contract:

Expand All @@ -31,34 +31,31 @@ Discriminative miners must submit models in **safetensors format**:

**📖 [Safetensors Model Specification](https://github.com/bitmind-ai/gasbench/blob/main/docs/Safetensors.md)** - Requirements for model submission

You can submit models for any combination of modalities:
Each hotkey can submit **one** model, in one of these modalities:
- `image_detector.zip` - Image classification model
- `video_detector.zip` - Video classification model
- `video_detector.zip` - Video classification model
- `audio_detector.zip` - Audio classification model

A second modality needs a second registered hotkey.

## Pushing Your Model

First, activate the virtual environment:
```bash
source .venv/bin/activate
```

Push your models to the network using the `push` command:
Push one model per hotkey using the `push` command:

```bash
# Upload all three models
gascli d push \
--image-model image_detector.zip \
--video-model video_detector.zip \
--audio-model audio_detector.zip \
--wallet-name your_wallet_name \
--wallet-hotkey your_hotkey_name

# Or upload individual models
gascli d push \
--image-model image_detector.zip \
--wallet-name your_wallet_name \
--wallet-hotkey your_hotkey_name
# Video or audio on a different hotkey:
# gascli d push --video-model video_detector.zip --wallet-hotkey video_key
# gascli d push --audio-model audio_detector.zip --wallet-hotkey audio_key
```

### Command Options
Expand All @@ -68,8 +65,6 @@ The `push` command accepts several parameters:
```bash
gascli d push \
--image-model image_detector.zip \
--video-model video_detector.zip \
--audio-model audio_detector.zip \
--wallet-name your_wallet_name \
--wallet-hotkey your_hotkey_name \
--netuid 34 \
Expand All @@ -87,7 +82,7 @@ gascli d push \
- `--chain-endpoint`: Subtensor network endpoint (default: "wss://entrypoint-finney.opentensor.ai:443/")
- `--retry-delay`: Retry delay in seconds (default: 60)

At least one model (image, video, or audio) must be provided.
Provide exactly one of `--image-model`, `--video-model`, or `--audio-model`. A second model needs a different hotkey.

## Submission Limits

Expand Down Expand Up @@ -123,7 +118,6 @@ For the complete list of allowed and blocked imports, see the [Safetensors Model

### Evaluation

- Models are benchmarked on cloud infrastructure (not miner hardware)
- Evaluation runs against a diverse dataset of image samples, video samples, and audio samples per benchmark cycle
- Datasets are refreshed weekly with new GAS-Station data alongside static benchmark datasets

Expand Down Expand Up @@ -158,7 +152,7 @@ gascli d push --image-model my_detector.zip
### What Happens During Push

1. **Model Validation**: The system checks that the zip files are present and valid
2. **Model Upload**: Your model zip files are uploaded to the cloud inference system
2. **Model Upload**: Your model zip files are uploaded for evaluation
3. **Blockchain Registration**: Model metadata is registered on the Bittensor blockchain
4. **Verification**: The system verifies the registration was successful

Expand All @@ -175,7 +169,7 @@ Before your model is ever scored on the network, it must pass an **entrance exam
- Internally this runs `gasbench run --small`, which downloads one archive per dataset and evaluates roughly 100 samples per dataset
- Your model must achieve **≥ 80% accuracy** averaged across all submitted modalities to pass
- The exam has a **maximum wall-clock timeout of 1 hour 25 minutes** (5,100 seconds); models that exceed this are treated as failed
- The exam runs in an **isolated cloud sandbox** — your code has no network access and cannot interact with the host environment
- The exam runs in an **isolated sandbox** — your code has no network access and cannot interact with the host environment
- Submissions are statically analyzed and executed in an isolated sandbox; prohibited code or imports result in rejection

**Model status during the exam:**
Expand Down
2 changes: 1 addition & 1 deletion docs/Incentive.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Incentive Mechanism

## Benchmark Runs
Submitted discriminator miners are evaluated against a subset of the data sources listed below. Models are evaluated on cloud infrastructure -- miners do not need to host hardware for inference. A portion of the evaluation data comes from generative miners, who are rewarded based on their ability to submit data that both pass validator sanity checks (prompt alignment, etc.) and fool discriminators in benchmark runs.
Submitted discriminator miners are evaluated against a subset of the data sources listed below. Miners do not need to host hardware for inference. A portion of the evaluation data comes from generative miners, who are rewarded based on their ability to submit data that both pass validator sanity checks (prompt alignment, etc.) and fool discriminators in benchmark runs.

Each modality (image, video, audio) is scored independently using the `sn34_score` metric, which combines classification performance (MCC) with probability calibration (Brier score). The active round selects binary or multiclass scoring per modality.

Expand Down
2 changes: 1 addition & 1 deletion docs/Mining.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
GAS supports two types of miners that work together in an adversarial loop:

## [Discriminative Mining](Discriminative-Mining.md) 📖
Miners submit classifiers that detect AI-generated content across **image, video, and audio** modalities. Models are evaluated on cloud infrastructure against diverse benchmark datasets and scored using the `sn34_score` metric (accuracy + calibration). Each hotkey gets one counted submission (any modality); see [Discriminative Mining](Discriminative-Mining.md#submission-limits).
Miners submit classifiers that detect AI-generated content across **image, video, and audio** modalities. Models are evaluated against diverse benchmark datasets and scored using the `sn34_score` metric (accuracy + calibration). Each hotkey gets one counted submission (any modality); see [Discriminative Mining](Discriminative-Mining.md#submission-limits).

## [Generative Mining](Generative-Mining.md) 🎨
Miners create synthetic media (images and videos) that challenges the discriminators. They generate increasingly realistic content to test and improve detection capabilities, and are rewarded based on validation pass rate and adversarial performance.
Expand Down
2 changes: 1 addition & 1 deletion gas/koth_weights.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@


def kings_by_modality(payload: Optional[dict]) -> Dict[str, str]:
"""Map modality -> hotkey from a /current-kings response."""
"""Map modality -> hotkey from a /kings response."""
out: Dict[str, str] = {}
if not payload:
return out
Expand Down
6 changes: 3 additions & 3 deletions gas/protocol/validator_requests.py
Original file line number Diff line number Diff line change
Expand Up @@ -183,12 +183,12 @@ async def get_current_kings(
hotkey,
base_url: str = "https://gas.bitmind.ai",
) -> Optional[Dict[str, Any]]:
"""Fetch current KOTH kings from gas-api /validator/current-kings."""
"""Fetch current KOTH kings from gas-api /validator/kings."""
try:
bt.logging.info(f"Fetching current kings from {base_url}/api/v1/validator/current-kings")
bt.logging.info(f"Fetching current kings from {base_url}/api/v1/validator/kings")
timeout = aiohttp.ClientTimeout(total=30)
async with aiohttp.ClientSession(timeout=timeout) as session:
url = f"{base_url}/api/v1/validator/current-kings"
url = f"{base_url}/api/v1/validator/kings"
epistula_headers = generate_header(hotkey, b"", None)
async with session.get(url, headers=epistula_headers) as response:
if response.status == 200:
Expand Down
4 changes: 2 additions & 2 deletions neurons/validator/validator.py
Original file line number Diff line number Diff line change
Expand Up @@ -203,11 +203,11 @@ async def set_weights(self, block):
if kings_payload is not None:
self.kings_state.payload = kings_payload
elif self.kings_state.payload is not None:
bt.logging.warning("current-kings API unavailable; using last known kings")
bt.logging.warning("kings API unavailable; using last known kings")
kings_payload = self.kings_state.payload
else:
bt.logging.warning(
"current-kings API unavailable and no cached kings; "
"kings API unavailable and no cached kings; "
"discriminator shares will burn"
)
kings_payload = {"kings": []}
Expand Down
Loading