Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ GAS runs two parallel competition tracks on Bittensor Subnet 34:
- **Cloud-evaluated**: Discriminator models are benchmarked on cloud infrastructure -- no GPU hosting required
- **Model format**: Safetensors only (ONNX submissions are not accepted)
- **Datasets refresh weekly** with fresh GAS-Station data alongside static benchmarks
- **One model per modality per hotkey** for discriminative miners
- **One model per hotkey** for discriminative miners

See [Incentive Mechanism](docs/Incentive.md) for full scoring details.

Expand Down
13 changes: 11 additions & 2 deletions docs/Discriminative-Mining.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,15 @@ gascli d push \

At least one model (image, video, or audio) must be provided.

## Submission Limits

Each registered hotkey gets **one counted submission** (image, video, or audio — not one of each).

- Exam failures and incomplete uploads do not consume the slot. You can retry on the same key until a model is successfully uploaded and not later marked exam-failed.
- A confirmed or superseded model **does** consume the slot for the life of that hotkey, for every modality.
- A new benchmark version does **not** refill the slot.
- To submit another model, register a new miner hotkey.

---

## Competition Rules and Constraints
Expand All @@ -104,7 +113,7 @@ The normalized terms apply exponents $1.2$ to MCC performance and $1.8$ to Brier
### Model Requirements

- **Format**: Safetensors only (ONNX is no longer accepted)
- **Three model per modality per hotkey**: You can submit up to three image, three video, and three audio model per registered hotkey
- **Submission cap**: one counted model per hotkey (see [Submission Limits](#submission-limits))

### Sandbox and Import Restrictions

Expand Down Expand Up @@ -194,7 +203,7 @@ Models that pass the entrance exam are benchmarked against the **complete datase
- **Private holdout datasets** — curated datasets not visible to miners, used to prevent overfitting to the public benchmark set
- Refreshed weekly with new data from the GAS-Station pipeline

The full benchmark has a **maximum wall-clock timeout of 5 hours** (18,000 seconds) per modality. The benchmark score from this stage determines your **TAO emissions** on Subnet 34. The active round configuration selects provenance weighting, multiclass scoring, and augmentation robustness parameters; see [Incentive Mechanism](Incentive.md).
The full benchmark has a **maximum wall-clock timeout of 5 hours** (18,000 seconds) per modality. This `sn34_score` is what the King of the Hill competition uses: a high enough score can take or keep a lane, and emissions then follow the 85/10/5 split on the current king plus the previous two. The active round configuration selects provenance weighting, multiclass scoring, and augmentation robustness parameters; see [Incentive Mechanism](Incentive.md).

You can simulate a full benchmark run locally (without holdouts) to get a sense of your model's performance:

Expand Down
2 changes: 1 addition & 1 deletion docs/Incentive.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,7 @@ The normative implementation details and complete metric field glossary live in

### King of the Hill

Discriminator emission is King of the Hill. Each modality has one reigning model. Validators set that lane's weight on registered hotkeys every tempo — not on an escrow wallet.
Discriminator emission is King of the Hill. Each modality has one reigning model. Validators set that lane's weight on registered hotkeys every tempo — not on an escrow wallet. Each hotkey may land **one counted submission** for the life of that registration (any modality; exam failures do not count; a new model needs a new key).

Current split:

Expand Down
2 changes: 1 addition & 1 deletion docs/Mining.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
GAS supports two types of miners that work together in an adversarial loop:

## [Discriminative Mining](Discriminative-Mining.md) 📖
Miners submit classifiers that detect AI-generated content across **image, video, and audio** modalities. Models are evaluated on cloud infrastructure against diverse benchmark datasets and scored using the `sn34_score` metric (accuracy + calibration).
Miners submit classifiers that detect AI-generated content across **image, video, and audio** modalities. Models are evaluated on cloud infrastructure against diverse benchmark datasets and scored using the `sn34_score` metric (accuracy + calibration). Each hotkey gets one counted submission (any modality); see [Discriminative Mining](Discriminative-Mining.md#submission-limits).

## [Generative Mining](Generative-Mining.md) 🎨
Miners create synthetic media (images and videos) that challenges the discriminators. They generate increasingly realistic content to test and improve detection capabilities, and are rewarded based on validation pass rate and adversarial performance.
Expand Down
50 changes: 0 additions & 50 deletions docs/Validating-New.md

This file was deleted.

167 changes: 24 additions & 143 deletions docs/Validating.md
Original file line number Diff line number Diff line change
@@ -1,169 +1,50 @@
# Validator Guide

## Required .env.validator variables (PM2 and Docker)
Run the SN34 validator (validator + generator + data services). Two ways: **PM2** (native) or **Docker**.

You **must** set these in `.env.validator` before starting; the rest have defaults in the template.

| Variable | Required? | Notes |
|----------|-----------|--------|
| `WALLET_NAME` | **Yes** | Your Bittensor wallet name (e.g. `default` or a name you created). |
| `WALLET_HOTKEY` | **Yes** | Hotkey for this validator (e.g. `default`). Must be registered on the subnet. |
| `HUGGINGFACE_HUB_TOKEN` | **Yes** | Needed to download models and upload data. Create at [huggingface.co/settings/tokens](https://huggingface.co/settings/tokens). Reach out to a member of the BitMind team to have your account added to the `gasstation` org |
| `CHAIN_ENDPOINT` | No | Default is mainnet. Set to testnet URL if needed. |
| `WANDB_API_KEY` | No | Optional; for Weights & Biases logging. |
| Cache / Docker paths | No | `SN34_CACHE_DIR`, `HF_HOME`, `WALLET_PATH`, `BT_LOGGING_LOGGING_DIR` have defaults; override if you use different paths. |

See [.env.validator.template](../.env.validator.template) for every option.
Prerequisites: [Installation Guide](Installation.md), Bittensor wallet, GPU for generation.

---

## PM2 setup

Follow the [Installation Guide](Installation.md) to set up your environment before proceeding with validator setup.

> Create a `.env.validator` file and **set the required variables above** (wallet, API keys). Then:

```bash
$ cp .env.validator.template .env.validator
# Edit .env.validator: fill in WALLET_NAME, WALLET_HOTKEY, HUGGINGFACE_HUB_TOKEN at minimum
```

See [.env.validator.template](../.env.validator.template) for all options. Then activate the virtual environment and start your validator processes:

```bash
$ source .venv/bin/activate
$ gascli validator start
```
The above command will create 3 pm2 processes:
```bash
┌────┬───────────────────┬─────────────┬─────────┬─────────┬──────────┬────────┬──────┬───────────┬──────────┬──────────┬──────────┬──────────┐
│ id │ name │ namespace │ version │ mode │ pid │ uptime │ ↺ │ status │ cpu │ mem │ user │ watching │
├────┼───────────────────┼─────────────┼─────────┼─────────┼──────────┼────────┼──────┼───────────┼──────────┼──────────┼──────────┼──────────┤
│ 2 │ sn34-data │ default │ N/A │ fork │ 4032914 │ 2s │ 72 │ online │ 100% │ 529.0mb │ user │ disabled │
│ 1 │ sn34-generator │ default │ N/A │ fork │ 4032936 │ 2s │ 72 │ online │ 100% │ 448.5mb │ user │ disabled │
│ 0 │ sn34-validator │ default │ N/A │ fork │ 4032918 │ 2s │ 72 │ online │ 100% │ 504.0mb │ user │ disabled │
└────┴───────────────────┴─────────────┴─────────┴─────────┴──────────┴────────┴──────┴───────────┴──────────┴──────────┴──────────┴──────────┘
```
- **sn34-data**: Handles data downloads
- **sn34-generator**: Responsible for generating prompts, synthetic media, and validating miner-generated data
- **sn34-validator**: Core validator logic. Challenges, scoring, weight setting.

## Quick start: PM2

### Validator Operations

First, activate the virtual environment:
```bash
cp .env.validator.template .env.validator
# Edit .env.validator: WALLET_NAME, WALLET_HOTKEY, API keys, CHAIN_ENDPOINT
source .venv/bin/activate
```

Then run validator commands:
```bash
# Start validator services
gascli validator start
gascli v start # Using alias
gascli v stop
gascli v status
gascli v logs
gascli v --help
```

That starts three PM2 processes: `sn34-validator`, `sn34-generator`, `sn34-data`. Use `gascli v stop|status|logs|--help` as needed.

## Docker Deployment

As an alternative to the PM2-based setup above, you can run the validator stack in Docker. Three containers (validator, generator, data) run one process each and share bind-mounted cache and wallet; the validator container uses `network_mode: host` so its FastAPI callback is on the host port miners are told to hit.

### Prerequisites

- **Docker** (with Docker Compose v2)
- **NVIDIA Container Toolkit** for GPU passthrough. Install guide: https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html
- **Bittensor wallet** files already created on the host (typically at `~/.bittensor/wallets/`)

### Quick Start

1. **Clone the repository** (if not already done):
```bash
git clone <repository-url>
cd bitmind-subnet
```

2. **Create your `.env.validator`** and set the **required variables** (same as PM2):
```bash
cp .env.validator.template .env.validator
```
You **must** set: `WALLET_NAME`, `WALLET_HOTKEY`, `HUGGINGFACE_HUB_TOKEN`, `OPEN_ROUTER_API_KEY`. See [Required .env.validator variables](#required-envvalidator-variables-pm2-and-docker) and [.env.validator.template](../.env.validator.template) for the full list.

3. **Build and start** (use `--env-file .env.validator` so the same file drives both container env and Compose options like `WALLET_PATH` and `CALLBACK_PORT`):
```bash
docker compose --env-file .env.validator up -d --build
```

4. **View logs** (per service):
```bash
docker compose logs -f validator # or generator, data
```


### Updating
---

**Automatic (recommended):** add a crontab entry so `docker/autoupdate.sh` runs periodically (it checks VERSION, then pulls, runs `down --remove-orphans`, rebuilds, and brings the stack up):
## Quick start: Docker

```bash
*/5 * * * * /path/to/bitmind-subnet/docker/autoupdate.sh >> /var/log/bitmind-docker-update.log 2>&1
```

**Manual:** rebuild and recreate all services:
Three containers (validator, generator, data) share one image and volumes; one process per container.

```bash
git pull
docker compose --env-file .env.validator down --remove-orphans
docker compose --env-file .env.validator build
docker compose --env-file .env.validator up -d
cp .env.validator.template .env.validator
# Edit .env.validator (same as above; WALLET_PATH is for host wallet bind-mount)
docker compose --env-file .env.validator up -d --build
docker compose logs -f validator # or generator, data
```

### Configuration
Always use `--env-file .env.validator`. First run downloads 100+ GB of models into the `hf-cache` volume.

| Variable | Docker behavior |
|---|---|
| `SN34_CACHE_DIR` | Bind-mounted into the container; same path as PM2 so cache is shared when you switch. Set in `.env.validator`; use `--env-file .env.validator`. |
| `HF_HOME` | Bind-mounted into the container; same path as PM2 so model cache is shared. Set in `.env.validator`. |
| `AUTO_UPDATE` | In-container autoupdate is disabled. Use manual rebuild or the cron-based `docker/autoupdate.sh` (see Updating). |
| `WALLET_PATH` | Host base path for wallets. Only the directory `WALLET_PATH/WALLET_NAME` is mounted (not the whole wallets folder). Set in `.env.validator`. |
| `WALLET_NAME` | Which wallet dir to mount; with `WALLET_PATH` only that wallet is visible to the container. |
| `BT_LOGGING_LOGGING_DIR` | Base path for validator state (scores, challenge tasks) and bittensor logs. State lives under `BT_LOGGING_LOGGING_DIR`/`WALLET_NAME`/`WALLET_HOTKEY`/…; bind-mounted so it persists. Default `~/.bittensor`. |
| `NETUID` | Auto-derived from `CHAIN_ENDPOINT`. Set explicitly if using a custom endpoint. |
**Update:** Manual: `git pull && docker compose --env-file .env.validator build && docker compose --env-file .env.validator up -d`. Automatic: cron with `docker/autoupdate.sh` (see [.env.validator.template](../.env.validator.template) comments for crontab example).

### Wallet, cache, and validator state

- **Wallet:** Only the configured wallet directory (`WALLET_PATH`/`WALLET_NAME`) is bind-mounted, not the entire wallets folder.
- **Cache:** `SN34_CACHE_DIR` and `HF_HOME` are bind-mounted so PM2 and Docker share the same data (no re-download when switching).
- **Validator state:** Scores and challenge tasks are saved under bittensor’s logging path (`BT_LOGGING_LOGGING_DIR`/`WALLET_NAME`/`WALLET_HOTKEY`/…). That path is bind-mounted so state persists across container restarts.

> **Note**: The first startup will download 100+ GB of ML models into the HF cache directory. Subsequent restarts reuse the cached models.

### Common Operations

Use `--env-file .env.validator` so Compose reads `WALLET_PATH`, `WALLET_NAME`, `CALLBACK_PORT`, and cache paths from your config:

```bash
# Start all three services (validator, generator, data)
docker compose --env-file .env.validator up -d

# Stop all
docker compose down
---

# View logs (per service)
docker compose logs -f validator # or generator, data
## Config

# Rebuild after code changes
docker compose --env-file .env.validator down --remove-orphans
docker compose --env-file .env.validator build && docker compose --env-file .env.validator up -d
One file for both PM2 and Docker: `.env.validator`. Copy from [.env.validator.template](../.env.validator.template) and fill in wallet, API keys, `CHAIN_ENDPOINT`. The template lists every option and Docker-specific notes (e.g. `WALLET_PATH`, cache paths, autoupdate cron).

# Restart one service
docker compose restart validator # or generator, data
---

# Container status
docker compose ps
## Reference

# Shell into a container
docker compose exec validator bash # or generator, data
```
| Path | Start | Stop / logs |
|--------|-------|-------------|
| PM2 | `gascli validator start` | `gascli v stop` / `gascli v logs` |
| Docker | `docker compose --env-file .env.validator up -d` | `docker compose down` / `docker compose logs -f validator` (or `generator`, `data`) |
1 change: 1 addition & 0 deletions gas/protocol/miner_requests.py
Original file line number Diff line number Diff line change
Expand Up @@ -279,6 +279,7 @@ def extract_error(result: dict) -> str:
"error": extract_error(presigned_result),
"response": presigned_result['response'],
"already_uploaded": True,
"file_hash": file_hash,
}
return {
"success": False,
Expand Down
25 changes: 12 additions & 13 deletions neurons/discriminator/push_model.py
Original file line number Diff line number Diff line change
Expand Up @@ -107,16 +107,10 @@ def _accept_already_uploaded(modality: str, result: dict, skip_chain: bool) -> b
print_warning(
f"{modality.capitalize()} model already uploaded — server already has this hash."
)
if skip_chain:
print_success(
f"{modality.capitalize()} upload already accepted; nothing more to do (--skip-chain)."
)
return True
print_error(
f"Cannot retrieve r2_key for already-uploaded {modality} model. "
"Re-run with the original r2_key, or pass --skip-chain if you only needed the upload."
print_success(
f"{modality.capitalize()} upload already accepted; exam will still run."
)
return False
return True
print_error(f"{modality.capitalize()} model upload failed at step: {result.get('step', 'unknown')}")
print_error(f"Error: {result.get('error', 'Unknown error')}")
if result.get("response"):
Expand Down Expand Up @@ -250,8 +244,12 @@ async def push_separate_models(
for modality, result in results.items():
model_key = result.get("r2_key", "")
if not model_key:
print_error(f"{modality.capitalize()} model key not provided in upload response")
return False
print_warning(
f"Skipping {modality} chain registration — no r2_key in the upload "
"response. The upload already succeeded and the model will still be examined."
)
chain_ok = False
continue

# Create hash for this specific model
model_hash = hashlib.sha256(
Expand Down Expand Up @@ -303,7 +301,8 @@ async def push_separate_models(
continue
print_warning(
f"Giving up chain registration after {attempt} attempt(s). "
"The upload already succeeded and the model will still be examined."
"The upload already succeeded and the model will still be examined. "
f"r2_key={model_key}"
)
chain_ok = False
break
Expand All @@ -315,7 +314,7 @@ async def push_separate_models(
print_success("All models registered successfully!")
else:
print_warning("Upload succeeded; one or more chain registrations did not complete.")
return chain_ok
return True


def main():
Expand Down
Loading
Loading